Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

timesketch_importer error when importing plaso file #3124

Open
hasamba opened this issue Jul 15, 2024 · 9 comments
Open

timesketch_importer error when importing plaso file #3124

hasamba opened this issue Jul 15, 2024 · 9 comments
Labels

Comments

@hasamba
Copy link

hasamba commented Jul 15, 2024

while importing a plaso file, getting the following error:

[2024-07-15 11:42:34,046] timesketch_importer.importer_frontend/INFO Using cached credentials.
[2024-07-15 11:42:34,108] timesketch_importer.importer_frontend/INFO Creating a client.
[2024-07-15 11:42:34,447] timesketch_importer.importer_frontend/INFO Client created.
[2024-07-15 11:42:34,447] timesketch_importer.importer_frontend/INFO Saving TS config.
[2024-07-15 11:42:34,456] timesketch_api.client/WARNING Unable to json decode the Timesketch API response!
Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/error.py", line 97, in get_response_json
    return response.json()
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)



The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/bin/timesketch_importer", line 8, in <module>
    sys.exit(main())
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/tools/timesketch_importer.py", line 593, in main
    my_sketch = ts_client.create_sketch(sketch_name)
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/client.py", line 442, in create_sketch
    response_dict = error.get_response_json(response, logger)
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/error.py", line 100, in get_response_json
    raise ValueError("Unable to json decode the Timesketch API response!") from e
ValueError: Unable to json decode the Timesketch API response!

Timesketch version: 20240508.1
API Client Version: 20240215
Importer Client Version: 20230721

Importing the same plaso file on another machine (both ubuntu server 22.04+python VENV 3.10.12) with same timesketch+timesketch_importer versions works!

what can be the problem?
thanks

@hasamba hasamba added the Bug label Jul 15, 2024
@jkppr
Copy link
Collaborator

jkppr commented Jul 15, 2024

On the same machine where this error happens, can you upload the Plaso file using the web UI or dftimewolf? Does it result in the same errors?

Are there any related errors on the server side in the timesketch/logs/wsgi_errors.log or worker.log files?

@hasamba
Copy link
Author

hasamba commented Jul 15, 2024

wsgi_errors.log

forgot to mention that we tried upload the same plaso file through the UI and it worked.

could not find wsgi_errors.log

worker.log last 20 lines:

[2024-07-14 08:57:51,429] celery.apps.worker/WARNING Please run `celery upgrade settings path/to/settings.py` to avoid these warnings and to allow a smoother upgrade to Celery 6.0.
[2024-07-14 08:57:53,361] celery.worker.consumer.connection/INFO Connected to redis://redis:6379//
[2024-07-14 08:57:53,384] celery.worker.consumer.mingle/INFO mingle: searching for neighbors
[2024-07-14 08:57:54,411] celery.worker.consumer.mingle/INFO mingle: all alone
[2024-07-14 08:57:54,445] celery.apps.worker/INFO celery@b16ba86fb63f ready.
[2024-07-14 12:08:10,220] py.warnings/WARNING /usr/local/lib/python3.10/dist-packages/celery/app/utils.py:204: CDeprecationWarning:
    The 'CELERY_RESULT_BACKEND' setting is deprecated and scheduled for removal in
    version 6.0.0. Use the result_backend instead

  deprecated.warn(description=f'The {setting!r} setting',

[2024-07-14 12:08:10,221] celery.apps.worker/WARNING Please run `celery upgrade settings path/to/settings.py` to avoid these warnings and to allow a smoother upgrade to Celery 6.0.
[2024-07-14 12:08:11,161] celery.worker.consumer.connection/INFO Connected to redis://redis:6379//
[2024-07-14 12:08:11,165] celery.worker.consumer.mingle/INFO mingle: searching for neighbors
[2024-07-14 12:08:12,181] celery.worker.consumer.mingle/INFO mingle: all alone
[2024-07-14 12:08:12,197] celery.apps.worker/INFO celery@b16ba86fb63f ready.
[2024-07-15 07:33:08,227] celery.worker.strategy/INFO Task timesketch.lib.tasks.run_plaso[ce70e9e801c947f085e90f77b8b80e9d] received
[2024-07-15 07:33:08,483] timesketch.tasks/INFO Index timeline [artifacts] to index [d1762398f7804957a38d8349c91346e8] (source: plaso)
[2024-07-15 07:33:10,580] celery.app.trace/INFO Task timesketch.lib.tasks.run_plaso[ce70e9e801c947f085e90f77b8b80e9d] succeeded in 2.3513273459975608s: 'd1762398f7804957a38d8349c91346e8'

@jkppr
Copy link
Collaborator

jkppr commented Jul 15, 2024

If you have a docker deployment, the wsgi_error.log should be in the same folder as the worker.log (e.g. /opt/timesketch/logs/wsgi_error.log).

Worker log looks good.

  • Is the issue persistent? (Ruling out a potential temporary network issue)
    • The error message indicates that the issue is with the first character in the response, which can mean it is empty or malformed. This would point to a network issue.
  • Can you upload other Plaso or csv files using the importer_client? (Ruling out problems with the specific file)
  • If the issue is persistent, can you capture and post the API response?
    • e.g. by patching the API client at this section with a logging or print command that prints you response.json or response.text.

@hasamba
Copy link
Author

hasamba commented Jul 15, 2024

wsgi_error

  return io.open(fd, mode, buffering, encoding, *args, **kwargs)
[2024-07-14 12:08:04 +0000] [8] [INFO] Booting worker with pid: 8
[2024-07-14 12:08:04 +0000] [9] [INFO] Booting worker with pid: 9
[2024-07-14 12:08:05 +0000] [10] [INFO] Booting worker with pid: 10
[2024-07-14 12:08:05 +0000] [11] [INFO] Booting worker with pid: 11
[2024-07-14 12:08:06 +0000] [7] [INFO] Starting gunicorn 19.10.0
[2024-07-14 12:08:06 +0000] [7] [INFO] Listening at: http://0.0.0.0:5000 (7)
[2024-07-14 12:08:06 +0000] [7] [INFO] Using worker: sync
/usr/lib/python3.10/os.py:1030: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  return io.open(fd, mode, buffering, encoding, *args, **kwargs)
[2024-07-14 12:08:06 +0000] [8] [INFO] Booting worker with pid: 8
[2024-07-14 12:08:06 +0000] [9] [INFO] Booting worker with pid: 9
[2024-07-14 12:08:06 +0000] [10] [INFO] Booting worker with pid: 10
[2024-07-14 12:08:06 +0000] [11] [INFO] Booting worker with pid: 11
[2024-07-15 07:17:33,750] flask_wtf.csrf/INFO The CSRF token is missing.
[2024-07-15 07:19:17,789] flask_wtf.csrf/INFO The CSRF token is missing.
[2024-07-15 07:19:49,444] flask_wtf.csrf/INFO The CSRF token is missing.
[2024-07-15 07:20:58,221] flask_wtf.csrf/INFO The CSRF token is missing.
[2024-07-15 07:21:26,652] flask_wtf.csrf/INFO The CSRF token is missing.
[2024-07-15 07:33:02,308] timesketch.analyzers.hashlookup/ERROR Hashlookup conf not found
  • we tested multiple times with same results,
  • we also tested with a different plaso file with same results

API Response

<!DOCTYPE html><html lang=en><head><meta name=csrf-token content="ImQwMzM1ODEwNzgyZDc4Mjk1NmQ4MDA2NzA4ODA0YWFiYWIwZDk5ZjEi.ZpYkzA.txxqdZ5gv_5mD1uekpjcQkO--yA"><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><link rel=icon href=/legacy/dist/favicon.ico><title>Timesketch</title><style>html,
        body {
            display: flex;
            height: 100%;
            width: 100%;
            align-items: center;
            justify-content: center;
        }
        .card.is-wide {
            width: 520px;
            height: 350px;
            padding-top:30px;
        }
        .card.has-text-centered,
        .card-content {
            justify-content: center;
            align-items: center;
        }</style><link href=/legacy/dist/css/chunk-common.673b3577.css rel=preload as=style><link href=/legacy/dist/css/chunk-vendors.96a655cf.css rel=preload as=style><link href=/legacy/dist/js/chunk-common.d12c964e.js rel=preload as=script><link href=/legacy/dist/js/chunk-vendors.91a8ae5d.js rel=preload as=script><link href=/legacy/dist/js/login.10bd00b4.js rel=preload as=script><link href=/legacy/dist/css/chunk-vendors.96a655cf.css rel=stylesheet><link href=/legacy/dist/css/chunk-common.673b3577.css rel=stylesheet></head><body class=login-page><div class=columns><div class=column><div class="card is-wide has-text-centered"><div class=card-content><div class=content><div><img src=/legacy/dist/timesketch-color.png style=width:40px;><div style="font-size: 1.5em;">time<b>sketch</b></div><div style="font-size: 0.8em;">Digital Forensic Timeline Analysis</div></div><br><form method=post style="width:50%;margin-left: auto; margin-right: auto;"><div class=field><div class=control><input type=text class=input name=username placeholder=Username style=text-align:center;></div></div><div class=field><div class=control><input type=password class=input name=password placeholder=Password style=text-align:center;></div></div><div class=field><div class="control has-text-centered"><button type=submit class="button is-rounded is-wide is-info" style=width:100%;>Sign in</button></div></div><input id="csrf_token" name="csrf_token" type="hidden" value="ImQwMzM1ODEwNzgyZDc4Mjk1NmQ4MDA2NzA4ODA0YWFiYWIwZDk5ZjEi.ZpYkzA.txxqdZ5gv_5mD1uekpjcQkO--yA"></form></div></div></div></div></div><script src=/legacy/dist/js/chunk-vendors.91a8ae5d.js></script><script src=/legacy/dist/js/chunk-common.d12c964e.js></script><script src=/legacy/dist/js/login.10bd00b4.js></script></body></html>

@jkppr
Copy link
Collaborator

jkppr commented Jul 17, 2024

The API response you have sheared is actually the code of the login page. It points to an issue with the authentication.

Can you please remove the ~/.timesketchrc and ~/.timesketch.token files on your system and try again? (You will be asked to re-authenticate to Timesketch)

Alternatively you can also try to pass the authentication in the cli: timesketch_importer.py --host <URL> -u <USERNAME> -p <PASSWORD> <PATH>

@hasamba
Copy link
Author

hasamba commented Jul 21, 2024

we always send the full syntax with host and user/pwd because we are planning to run it inside a script.
anyway i deleted the files, same results.

can it be related to the fact that we are trying to run it in a virtual environment?
no python is installed on the host, only portable python inside the venv

@hasamba
Copy link
Author

hasamba commented Jul 22, 2024

something very strange, it didnt work for something like a week, and we tried alot of times each day.
now i tried with on the same machine with the same plaso file and it working.
nothing was changed (except the date/time) or installed

timesketch_importer --host http://localhost:5000 -u import -p import /tmp/artifacts.plaso
[2024-07-22 10:53:00,127] timesketch_importer.importer_frontend/INFO Saving Credentials.
[2024-07-22 10:53:00,246] timesketch_importer.importer_frontend/INFO Creating a client.
[2024-07-22 10:53:00,616] timesketch_importer.importer_frontend/INFO Client created.
[2024-07-22 10:53:00,616] timesketch_importer.importer_frontend/INFO Saving TS config.
timesketch_importer [response]:{"meta":{},"objects":[{"active_timelines":[],"all_permissions":"{\"user/import\": [\"read\", \"write\", \"delete\"], \"is_public\": false}","created_at":"2024-07-22T10:53:00.621622","description":"New Sketch From Importer CLI","graphs":[],"id":2,"label_string":"","my_permissions":"[\"read\", \"write\", \"delete\"]","name":"New Sketch From Importer CLI","status":[{"created_at":"2024-07-22T10:53:00.621622","id":2,"status":"new","updated_at":"2024-07-22T10:53:00.621622"}],"timelines":[],"updated_at":"2024-07-22T10:53:00.621622","user":{"active":true,"admin":false,"email":null,"groups":[],"id":2,"name":"import","profile":{"picture_filename":null,"picture_url":null},"username":"import"}}]}

[2024-07-22 10:53:00,750] timesketch_importer.importer_frontend/INFO New sketch created: [2] New Sketch From Importer CLI
What is the timeline name [artifacts]:
[2024-07-22 10:53:10,827] timesketch_importer.importer_frontend/INFO Uploading file.
[2024-07-22 10:53:10,836] timesketch_importer.importer_frontend/INFO About to upload file.
[2024-07-22 10:53:27,227] timesketch_importer.importer_frontend/INFO File upload completed.

@hasamba
Copy link
Author

hasamba commented Jul 24, 2024

i think i found the bug, seems like there is a problem with my 'admin' user and also the user in the command line (-u) is not taking into consideration if the timesketch.token/rc is already available :

  1. delete both .timesketch files
  2. running with user admin failed:
timesketch_importer --host http://localhost:5000 -u admin -p admin /tmp/artifacts.plaso
[2024-07-24 06:42:00,436] timesketch_api.config_assistance/WARNING No timesketch section in the config
[2024-07-24 06:42:00,436] timesketch_importer.importer_frontend/INFO Saving Credentials.
[2024-07-24 06:42:00,553] timesketch_importer.importer_frontend/INFO Creating a client.
[2024-07-24 06:42:00,885] timesketch_importer.importer_frontend/INFO Client created.
[2024-07-24 06:42:00,886] timesketch_importer.importer_frontend/INFO Saving TS config.
timesketch_importer [response]:<!DOCTYPE html><html lang=en><head><meta name=csrf-token content="ImExYTg5MzBmMWUzM2QwYTAyMWYyNWU2MjM5ZjU3NGI4N2IyNzg1NDUi.ZqCiOA.YCkbe1Cd_5aFVbehfhN7nCbXTlU"><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><link rel=icon href=/legacy/dist/favicon.ico><title>Timesketch</title><style>html,
        body {
            display: flex;
            height: 100%;
            width: 100%;
            align-items: center;
            justify-content: center;
        }
        .card.is-wide {
            width: 520px;
            height: 350px;
            padding-top:30px;
        }
        .card.has-text-centered,
        .card-content {
            justify-content: center;
            align-items: center;
        }</style><link href=/legacy/dist/css/chunk-common.673b3577.css rel=preload as=style><link href=/legacy/dist/css/chunk-vendors.96a655cf.css rel=preload as=style><link href=/legacy/dist/js/chunk-common.d12c964e.js rel=preload as=script><link href=/legacy/dist/js/chunk-vendors.91a8ae5d.js rel=preload as=script><link href=/legacy/dist/js/login.10bd00b4.js rel=preload as=script><link href=/legacy/dist/css/chunk-vendors.96a655cf.css rel=stylesheet><link href=/legacy/dist/css/chunk-common.673b3577.css rel=stylesheet></head><body class=login-page><div class=columns><div class=column><div class="card is-wide has-text-centered"><div class=card-content><div class=content><div><img src=/legacy/dist/timesketch-color.png style=width:40px;><div style="font-size: 1.5em;">time<b>sketch</b></div><div style="font-size: 0.8em;">Digital Forensic Timeline Analysis</div></div><br><form method=post style="width:50%;margin-left: auto; margin-right: auto;"><div class=field><div class=control><input type=text class=input name=username placeholder=Username style=text-align:center;></div></div><div class=field><div class=control><input type=password class=input name=password placeholder=Password style=text-align:center;></div></div><div class=field><div class="control has-text-centered"><button type=submit class="button is-rounded is-wide is-info" style=width:100%;>Sign in</button></div></div><input id="csrf_token" name="csrf_token" type="hidden" value="ImExYTg5MzBmMWUzM2QwYTAyMWYyNWU2MjM5ZjU3NGI4N2IyNzg1NDUi.ZqCiOA.YCkbe1Cd_5aFVbehfhN7nCbXTlU"></form></div></div></div></div></div><script src=/legacy/dist/js/chunk-vendors.91a8ae5d.js></script><script src=/legacy/dist/js/chunk-common.d12c964e.js></script><script src=/legacy/dist/js/login.10bd00b4.js></script></body></html>
[2024-07-24 06:42:00,895] timesketch_api.client/WARNING Unable to json decode the Timesketch API response!
Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/requests/models.py", line 974, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/error.py", line 97, in get_response_json
    return response.json()
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/requests/models.py", line 978, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/bin/timesketch_importer", line 8, in <module>
    sys.exit(main())
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/tools/timesketch_importer.py", line 593, in main
    my_sketch = ts_client.create_sketch(sketch_name)
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/client.py", line 443, in create_sketch
    response_dict = error.get_response_json(response, logger)
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/error.py", line 100, in get_response_json
    raise ValueError("Unable to json decode the Timesketch API response!") from e
ValueError: Unable to json decode the Timesketch API response!

  1. running with different user (import) failed
timesketch_importer --host http://localhost:5000 -u import -p import /tmp/artifacts.plaso
[2024-07-24 06:44:26,632] timesketch_importer.importer_frontend/INFO Using cached credentials.
[2024-07-24 06:44:26,691] timesketch_importer.importer_frontend/INFO Creating a client.
[2024-07-24 06:44:27,024] timesketch_importer.importer_frontend/INFO Client created.
[2024-07-24 06:44:27,024] timesketch_importer.importer_frontend/INFO Saving TS config.
timesketch_importer [response]:<!DOCTYPE html><html lang=en><head><meta name=csrf-token content="Ijk2ZTAxZTQ3YWJlYjJhMGNkMGJmOTIyMmQwZmQzYjhlY2NkZWNkYTYi.ZqCiyw.uWgrVlY9wA50nLdWid7sDTfrIKo"><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><link rel=icon href=/legacy/dist/favicon.ico><title>Timesketch</title><style>html,
        body {
            display: flex;
            height: 100%;
            width: 100%;
            align-items: center;
            justify-content: center;
        }
        .card.is-wide {
            width: 520px;
            height: 350px;
            padding-top:30px;
        }
        .card.has-text-centered,
        .card-content {
            justify-content: center;
            align-items: center;
        }</style><link href=/legacy/dist/css/chunk-common.673b3577.css rel=preload as=style><link href=/legacy/dist/css/chunk-vendors.96a655cf.css rel=preload as=style><link href=/legacy/dist/js/chunk-common.d12c964e.js rel=preload as=script><link href=/legacy/dist/js/chunk-vendors.91a8ae5d.js rel=preload as=script><link href=/legacy/dist/js/login.10bd00b4.js rel=preload as=script><link href=/legacy/dist/css/chunk-vendors.96a655cf.css rel=stylesheet><link href=/legacy/dist/css/chunk-common.673b3577.css rel=stylesheet></head><body class=login-page><div class=columns><div class=column><div class="card is-wide has-text-centered"><div class=card-content><div class=content><div><img src=/legacy/dist/timesketch-color.png style=width:40px;><div style="font-size: 1.5em;">time<b>sketch</b></div><div style="font-size: 0.8em;">Digital Forensic Timeline Analysis</div></div><br><form method=post style="width:50%;margin-left: auto; margin-right: auto;"><div class=field><div class=control><input type=text class=input name=username placeholder=Username style=text-align:center;></div></div><div class=field><div class=control><input type=password class=input name=password placeholder=Password style=text-align:center;></div></div><div class=field><div class="control has-text-centered"><button type=submit class="button is-rounded is-wide is-info" style=width:100%;>Sign in</button></div></div><input id="csrf_token" name="csrf_token" type="hidden" value="Ijk2ZTAxZTQ3YWJlYjJhMGNkMGJmOTIyMmQwZmQzYjhlY2NkZWNkYTYi.ZqCiyw.uWgrVlY9wA50nLdWid7sDTfrIKo"></form></div></div></div></div></div><script src=/legacy/dist/js/chunk-vendors.91a8ae5d.js></script><script src=/legacy/dist/js/chunk-common.d12c964e.js></script><script src=/legacy/dist/js/login.10bd00b4.js></script></body></html>
[2024-07-24 06:44:27,034] timesketch_api.client/WARNING Unable to json decode the Timesketch API response!
Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/requests/models.py", line 974, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/error.py", line 97, in get_response_json
    return response.json()
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/requests/models.py", line 978, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/bin/timesketch_importer", line 8, in <module>
    sys.exit(main())
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/tools/timesketch_importer.py", line 593, in main
    my_sketch = ts_client.create_sketch(sketch_name)
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/client.py", line 443, in create_sketch
    response_dict = error.get_response_json(response, logger)
  File "/home/Bacteria5570/mssp/risx-mssp-python-script/mssp_env/lib/python3.10/site-packages/timesketch_api_client/error.py", line 100, in get_response_json
    raise ValueError("Unable to json decode the Timesketch API response!") from e
ValueError: Unable to json decode the Timesketch API response!
  1. delete both .timesketch files
  2. running again with import user success
timesketch_importer --host http://localhost:5000 -u import -p import /tmp/artifacts.plaso
[2024-07-24 06:45:32,316] timesketch_api.config_assistance/WARNING No timesketch section in the config
[2024-07-24 06:45:32,317] timesketch_importer.importer_frontend/INFO Saving Credentials.
[2024-07-24 06:45:32,435] timesketch_importer.importer_frontend/INFO Creating a client.
[2024-07-24 06:45:32,777] timesketch_importer.importer_frontend/INFO Client created.
[2024-07-24 06:45:32,777] timesketch_importer.importer_frontend/INFO Saving TS config.
timesketch_importer [response]:{"meta":{},"objects":[{"active_timelines":[],"all_permissions":"{\"user/import\": [\"read\", \"write\", \"delete\"], \"is_public\": false}","created_at":"2024-07-24T06:45:32.780496","description":"New Sketch From Importer CLI","graphs":[],"id":4,"label_string":"","my_permissions":"[\"read\", \"write\", \"delete\"]","name":"New Sketch From Importer CLI","status":[{"created_at":"2024-07-24T06:45:32.780496","id":4,"status":"new","updated_at":"2024-07-24T06:45:32.780496"}],"timelines":[],"updated_at":"2024-07-24T06:45:32.780496","user":{"active":true,"admin":false,"email":null,"groups":[],"id":2,"name":"import","profile":{"picture_filename":null,"picture_url":null},"username":"import"}}]}

[2024-07-24 06:45:32,898] timesketch_importer.importer_frontend/INFO New sketch created: [4] New Sketch From Importer CLI

  1. running again with user admin successed but it seems it runs as import
timesketch_importer --host http://localhost:5000 -u admin -p admin /tmp/artifacts.plaso
[2024-07-24 06:46:13,784] timesketch_importer.importer_frontend/INFO Using cached credentials.
[2024-07-24 06:46:13,843] timesketch_importer.importer_frontend/INFO Creating a client.
[2024-07-24 06:46:14,183] timesketch_importer.importer_frontend/INFO Client created.
[2024-07-24 06:46:14,183] timesketch_importer.importer_frontend/INFO Saving TS config.
timesketch_importer [response]:{"meta":{},"objects":[{"active_timelines":[],"all_permissions":"{\"user/import\": [\"read\", \"write\", \"delete\"], \"is_public\": false}","created_at":"2024-07-24T06:46:14.186401","description":"New Sketch From Importer CLI","graphs":[],"id":5,"label_string":"","my_permissions":"[\"read\", \"write\", \"delete\"]","name":"New Sketch From Importer CLI","status":[{"created_at":"2024-07-24T06:46:14.186401","id":5,"status":"new","updated_at":"2024-07-24T06:46:14.186401"}],"timelines":[],"updated_at":"2024-07-24T06:46:14.186401","user":{"active":true,"admin":false,"email":null,"groups":[],"id":2,"name":"import","profile":{"picture_filename":null,"picture_url":null},"username":"import"}}]}

[2024-07-24 06:46:14,281] timesketch_importer.importer_frontend/INFO New sketch created: [5] New Sketch From Importer CLI

@jkppr
Copy link
Collaborator

jkppr commented Aug 13, 2024

Thanks for going the extra mile to debug this @hasamba . We will have to take a look at how the script handles authentication and how to permanently fix this. I guess, if the login is provided via parameters, the stored creds should be ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants