Race condition bug in sfscan.py and sf.py

[guylando]

Spiderfoot has a race condition bug that sfscan.py in the end of __startScan first changes status to FINISHED and then does few more things and on the other hand sf.py in the end of start_scan checks every 1 second if status is FINISHED and if it is then exits without waiting for the sfscan.py to complete.

This causes some of sfscan.py logic to be quit before being done such as not printing some of the expected output to stdout.

Therefore to fix this in sf.py before the sys.exit(0) need to wait for the scan process to quit. To fix the problem and also prevent infinite wait you can add "p.join(5)" before "sys.exit(0)" (around line 449 in sf.py).

[bcoles]

Thanks @guylando. Well spotted.

PR in #1818 is a simple fix by using a timeout of 60 seconds. Unfortunately, the logic for passing scan state needs further review, as correlations on large scans may take longer than 60 seconds to complete.