An instance of BTG is already running

[dspruell-s01]

I started a job in BTG which output errors:

$ ~/venv.d/btg/bin/btg zief.pl
    ____ ____________
   / __ )_  __/ ____/
  / __  |/ / / / __  
 / /_/ // / / /_/ /  
/_____//_/  \____/ v2.2

Every IOCs have been enqueued, BTG is processing ...

Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/run_worker.py", line 19, in <module>
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/run_worker.py", line 19, in <module>
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/run_worker.py", line 19, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/run_worker.py", line 19, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/poller.py", line 21, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/hypervisor.py", line 21, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'

(redis module is installed, with pip install -r requirements.txt).

I did a Ctrl-C to terminate that query.

Any future query then returns:

$ ~/venv.d/btg/bin/btg 8.8.8.8
    ____ ____________
   / __ )_  __/ ____/
  / __  |/ / / / __  
 / /_/ // / / /_/ /  
/_____//_/  \____/ v2.2

An instance of BTG is already running, we will wait 30s or until its completion
[MAIN][FATAL_ERROR] We have reached maximum waiting time, BTG is closing ...

BTG seems unusable now, as this is the same result hours later. Is a statefile or lockfile of some kind orphaned? Any solution?

[IrootGeek]

Hello,

BTG work with python3, the problem here is that you use the pip and not the pip3.

The solution is to install the requirements with pip3 & make sure that the server redis-server is installed on the machine and the service is running.

[dspruell-s01]

Confirming:

• Redis server is running:

    $ ps uaxwww |grep redi[s]
    darrenspruell     1195   0.0  0.0  5005312    356   ??  S    Sat04PM   2:18.81 /usr/local/opt/redis/bin/redis-server 127.0.0.1:6379
  

• BTG is installed in a Python 3 virtual environment and requirements are installed as well.

$ ~/venv.d/btg/bin/python -V
Python 3.7.4

$ ~/venv.d/btg/bin/pip list
Package         Version  
--------------- ---------
aiohttp         3.6.1    
async-timeout   3.0.1    
attrs           19.2.0   
BTG             2.2.1    
certifi         2019.9.11
chardet         3.0.4    
Click           7.0      
decorator       4.4.0    
Deprecated      1.2.6    
dnspython       1.16.0   
idna            2.8      
jsonschema      3.0.2    
multidict       4.5.2    
netaddr         0.7.19   
OTXv2           1.5.3    
pip             19.0.3   
pymisp          2.4.114  
pyrsistent      0.15.4   
python-dateutil 2.8.0    
pytz            2019.3   
redis           3.3.8    
requests        2.22.0   
requests-cache  0.5.2    
requests-file   1.4.3    
rq              1.1.0    
setuptools      40.8.0   
six             1.12.0   
tldextract      2.2.1    
urllib3         1.25.6   
validators      0.14.0   
wrapt           1.11.2   
yarl            1.3.0

$ echo $(cat ~/devel/external/BTG/requirements.txt)
validators>=0.12.0 pymisp dnspython requests-cache netaddr rq redis tldextract otxv2 aiohttp

Can import the installed redis module fine in that environment:

$ ~/venv.d/btg/bin/python3 -c 'import redis; print("ok")'
ok

The Python environment appears to be fine, and it began to execute the first time I tried it as I showed in first post, but subsequent attempts failed. It's been a few days and I've since rebooted, and attempting to run BTG again does the first thing I showed:

$ ~/venv.d/btg/bin/btg 8.8.8.8
    ____ ____________
   / __ )_  __/ ____/
  / __  |/ / / / __  
 / /_/ // / / /_/ /  
/_____//_/  \____/ v2.2

Every IOCs have been enqueued, BTG is processing ...

Traceback (most recent call last):
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/run_worker.py", line 19, in <module>
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/run_worker.py", line 19, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/run_worker.py", line 19, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/run_worker.py", line 19, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/poller.py", line 21, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'
Traceback (most recent call last):
  File "/Users/darrenspruell/venv.d/btg/lib/python3.7/site-packages/BTG/lib/hypervisor.py", line 21, in <module>
    from redis import Redis
ModuleNotFoundError: No module named 'redis'

The command just sits there and never exits. As before, if I Ctrl-C to terminate the running process and attempt to requery afterword, I get the same message "An instance of BTG is already running, we will wait 30s or until its completion".

[dspruell-s01]

To recreate the above on macOS:

brew install python redis
brew services start redis

python3 -mvenv ~/venv.d/btg_py3
~/venv.d/btg_py3/bin/pip3 install -r requirements.txt
~/venv.d/btg_py3/bin/pip3 install .
~/venv.d/btg_py3/bin/btg 8.8.8.8

[secusoc]

Hi Conix-security,

Do you have a solution for this problem:
"An instance of BTG is already running, we will wait 30s or until its completion"

Installation on Ubuntu20 and Ubuntu16
-Env: python3
-BTG install on pip3
-redis server up and running
This error message is strange because impossible to find an instance of BTG running.
Kind regards
(Even if trouble for me: Thanks for the tool because great idea!)