Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented multi-threaded parsing #10

Merged
merged 1 commit into from
Nov 6, 2024

Conversation

DavidBakerEffendi
Copy link
Collaborator

This benefit is not felt on MRI Ruby, as it is locked to a single thread, however JRuby benefits from this immensely.

# Native (MRI) Ruby: Single threaded before PR
bundle exec exe/ruby_ast_gen -i ~/Workspace/test-repos/gitlab  125.88s user 8.24s system 93% cpu 2:23.46 total
# Native (MRI) Ruby: Multi-threaded after PR
bundle exec exe/ruby_ast_gen -i ~/Workspace/test-repos/gitlab  134.01s user 11.44s system 98% cpu 2:28.05 total


# JRuby: This PR (max 10 threads 1 file per thread)
java -jar jruby.jar -S vendor/bundle/jruby/3.1.0/bin/bundle exec  -i   539.63s user 20.85s system 530% cpu 1:45.64 total
# JRuby: Batches of 10 files per thread
java -jar jruby.jar -S vendor/bundle/jruby/3.1.0/bin/bundle exec  -i   544.80s user 21.30s system 495% cpu 1:54.19 total

@DavidBakerEffendi DavidBakerEffendi self-assigned this Nov 6, 2024
@DavidBakerEffendi DavidBakerEffendi merged commit 3e4a56c into main Nov 6, 2024
1 check passed
@DavidBakerEffendi DavidBakerEffendi deleted the dave/multi-threaded-parsing branch November 6, 2024 10:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant