Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log parser crash when there are few CF #25

Open
erez-speedb opened this issue Aug 28, 2023 · 3 comments
Open

Log parser crash when there are few CF #25

erez-speedb opened this issue Aug 28, 2023 · 3 comments
Assignees

Comments

@erez-speedb
Copy link

ubuntu@zIP-9-151-1-21:~/log-parser$ git branch -av
* main                                                                                            d600b0e Discard (not assert / fail) when compaction stats lines are badly formed (#22)
  remotes/origin/18-display-the-relative-path-of-the-csv-s-in-the-json-rather-than-absolute-paths 80a8445 Create validation.yml (#3)
  remotes/origin/19-assertionerror-dbwidestatsmngris_start_line                                   72b28a6 Support non-contiguous 'DUMPING STATS' and 'DB Stats' entries
  remotes/origin/21-indexerror-parsing-a-log-file-with-cropped-compaction-stats-header-line       766da36 Discard (not assert / fail) when compaction stats lines are badly formed
  remotes/origin/24-assertion-error-block-cache-parsing                                           b560e96 Discard (rather than assert) badly formed Block cache stats lines (#24)
  remotes/origin/HEAD                                                                             -> origin/main
  remotes/origin/main                                                                             d600b0e Discard (not assert / fail) when compaction stats lines are badly formed (#22)
ubuntu@zIP-9-151-1-21:~/log-parser$ python3 log_parser.py -l /data/LOG.old.1693224943334021
Traceback (most recent call last):
  File "log_parser.py", line 352, in <module>
    main()
  File "log_parser.py", line 292, in main
    parsed_log = parse_log(log_file_path)
  File "log_parser.py", line 73, in parse_log
    return ParsedLog(log_file_path, log_lines,
  File "/home/ubuntu/log-parser/log_file.py", line 184, in __init__
    self.parse_rest_of_log()
  File "/home/ubuntu/log-parser/log_file.py", line 470, in parse_rest_of_log
    if self.try_parse_as_stats_entries():
  File "/home/ubuntu/log-parser/log_file.py", line 412, in try_parse_as_stats_entries
    self.stats_mngr.try_adding_entries(self.log_entries,
  File "/home/ubuntu/log-parser/stats_mngr.py", line 1139, in try_adding_entries
    self.parse_next_db_stats_entry_lines(
  File "/home/ubuntu/log-parser/stats_mngr.py", line 1059, in parse_next_db_stats_entry_lines
    self.block_cache_stats_mngr.add_lines(time, cf_name,
  File "/home/ubuntu/log-parser/stats_mngr.py", line 864, in add_lines
    self.parse_global_entry_stats_line(time, cache_id, db_stats_lines[1])
  File "/home/ubuntu/log-parser/stats_mngr.py", line 887, in parse_global_entry_stats_line
    roles, roles_stats = BlockCacheStatsMngr.parse_entry_stats_line(
  File "/home/ubuntu/log-parser/stats_mngr.py", line 939, in parse_entry_stats_line
    assert False, str(ParsingAssertion(
AssertionError: Error Parsing block cache stats line. roles:['FilterBlock', 'IndexBlock', 'WriteBuffer', 'FilterConstruction', 'BlockBasedTableReader', 'FileMetadata'], roles_stats:['4126,2.71 GB,67.7071%', '4144,911.92 MB,22.2637%', '993,248.25 MB,6.06079%', '14,3.50 MB,0.0854492%', '12,3.00 MB,0.0732422%'][File:? (line#:?)]
FilterBlock(4126,2.71 GB,67.7071%) IndexBlock(4144,911.92 MB,22.2637%) WriteBuffer(993,248.25 MB,6.06079%) FilterConstruction(14,3.50 MB,0.0854492%) BlockBasedTableReader(12,3.00 MB,0.0732422%) FileMetadata(12,3.00 MB,0.07324
@erez-speedb
Copy link
Author

Attaching the log
LOG.old.1693224943334021.log.zip

@udi-speedb
Copy link
Contributor

I have pushed some fixes to the branch 24-assertion-error-block-cache-parsing following this issue: #24
I have just parsed your attached file using that branch and it is currently parsed with that branch

@erez-speedb
Copy link
Author

Confirmed on the other problematic logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants