Race condition when read token file

[anthonybu]

When multiple threads creates BWProject, system may fail to open the token file. This exception is explicitly discarded (bwproject.py#L104), which in turn causing an UnboundLocalError at run time.

[anthonybu]

Example traceback:

[2018-03-20 17:21:35,409: ERROR/ForkPoolWorker-18] Task external_api.tasks.run_scheduled_task[797ad0a9-0318-43fd-bb69-bbc8272c8529] raised unexpected: UnboundLocalError("local variable 'token' referenced before assignment",)
Traceback (most recent call last):
  File "/opt/Conversus/venv/local/lib/python3.6/site-packages/celery/app/trace.py", line 374, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/Conversus/venv/local/lib/python3.6/site-packages/celery/app/trace.py", line 629, in __protected_call__
    return self.run(*args, **kwargs)
  File "/opt/Conversus/external_api/tasks.py", line 216, in run_scheduled_task
    scheduled_task[0].run()
  File "/opt/Conversus/external_api/models.py", line 269, in run
    self._run_span()
  File "/opt/Conversus/external_api/models.py", line 315, in _run_span
    'data_source_info': import_options})
  File "/opt/Conversus/external_api/data_targets/brandwatch.py", line 46, in push_data
    self.push_subscription_results(**kwargs)
  File "/opt/Conversus/external_api/data_targets/brandwatch.py", line 93, in push_subscription_results
    self.login(data_source_info['credential'], data_source_info)
  File "/opt/Conversus/external_api/data_targets/brandwatch.py", line 31, in login
    client_id="partner-api-client")
  File "/opt/Conversus/venv/local/lib/python3.6/site-packages/bwproject.py", line 230, in __init__
    super().__init__(token=token, token_path=token_path, username=username, password=password, grant_type=grant_type, client_id=client_id, apiurl=apiurl)
  File "/opt/Conversus/venv/local/lib/python3.6/site-packages/bwproject.py", line 51, in __init__
    self._write_auth(token_path)
  File "/opt/Conversus/venv/local/lib/python3.6/site-packages/bwproject.py", line 91, in _write_auth
    user_tokens = self._read_auth_file(token_path)
  File "/opt/Conversus/venv/local/lib/python3.6/site-packages/bwproject.py", line 106, in _read_auth_file
    user_tokens[user.lower()] = token
UnboundLocalError: local variable 'token' referenced before assignment

[anthonybu]

@amybarker

[jessicabowden]

Hey @anthonybu if you are still having issues with this, could you provide some example code in addition to the stack trace? Thanks