Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task sometimes process block very slowly #2

Open
0237h opened this issue Oct 25, 2022 · 1 comment
Open

Task sometimes process block very slowly #2

0237h opened this issue Oct 25, 2022 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@0237h
Copy link
Collaborator

0237h commented Oct 25, 2022

Sometimes during the extraction process, one task (no evidence of this occuring for multiple tasks yet) will run considerably slower than the others, making the whole script run about 30% slower.

slow_task

@0237h 0237h added the bug Something isn't working label Oct 25, 2022
@0237h
Copy link
Collaborator Author

0237h commented Nov 29, 2022

Still noticing significant differences for the Firehose BP extraction action:

2022-11-22T12:33:31.1609449Z bash -- Extracting data for 'eosio.bpay eosio.vpay' between 2022-11-21T00:00:00+00:00 and 2022-11-22T00:00:00+00:00
2022-11-22T12:33:31.1611590Z scripts/producerpay.sh: line 16: .venv/bin/activate: No such file or directory
2022-11-22T12:33:40.9367610Z 2022-11-22 12:33:40:T+9739 [*] Streaming 171787 blocks on EOS chain (running 25 workers)...
2022-11-22T12:46:36.2212105Z 2022-11-22 12:46:36:T+785024 [*] Block streaming done !
2022-11-22T12:46:38.6565831Z 2022-11-22 12:46:38:T+787459 [*] Finished block processing, parsed 84 rows of data [SUCCESS]
2022-11-22T12:46:38.8457953Z 2022-11-22 12:46:38:T+787648 [*] Wrote 84 rows of data to jsonl/producerpay/2022-11-21.jsonl [SUCCESS]
2022-11-22T12:46:38.9693567Z 
2022-11-22T12:46:38.9694342Z real	13m7.808s
2022-11-22T12:46:38.9694795Z user	0m28.889s
2022-11-22T12:46:38.9695136Z sys	0m9.337s

VS

 2022-11-21T12:34:02.8216000Z bash -- Extracting data for 'eosio.bpay eosio.vpay' between 2022-11-20T00:00:00+00:00 and 2022-11-21T00:00:00+00:00
2022-11-21T12:34:02.8216461Z scripts/producerpay.sh: line 16: .venv/bin/activate: No such file or directory
2022-11-21T12:34:05.4028557Z 2022-11-21 12:34:05:T+2551 [*] Streaming 172730 blocks on EOS chain (running 25 workers)...
2022-11-21T13:00:24.3667829Z 2022-11-21 13:00:24:T+1581515 [*] Block streaming done !
2022-11-21T13:00:26.1907693Z 2022-11-21 13:00:26:T+1583339 [*] Finished block processing, parsed 84 rows of data [SUCCESS]
2022-11-21T13:00:26.3078298Z 2022-11-21 13:00:26:T+1583456 [*] Wrote 84 rows of data to jsonl/producerpay/2022-11-20.jsonl [SUCCESS]
2022-11-21T13:00:26.3777243Z 
2022-11-21T13:00:26.3780595Z real	26m23.556s
2022-11-21T13:00:26.3797141Z user	0m21.566s
2022-11-21T13:00:26.3797720Z sys	0m8.750s

@0237h 0237h self-assigned this Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant