Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixit is incredibly slow with many files. #458

Open
jarshwah opened this issue May 19, 2024 · 0 comments
Open

Fixit is incredibly slow with many files. #458

jarshwah opened this issue May 19, 2024 · 0 comments

Comments

@jarshwah
Copy link

jarshwah commented May 19, 2024

Related to #457 I was doing some profiling on our custom fixit rules, and determined that the number of files being linted was the primary determinate in fixit runtime.

To that end, I've setup a repository that demonstrates the performance issues. It contains a large number of empty __init__.py files in a tree of modules, about 41,000 files in total. It has a single fixit rule with a visit_Module that does nothing except return. Running fixit against this tree pegs all of my cores to 100% and takes about 40 seconds to complete.

https://github.com/jarshwah/experiments-fixit is the repository which I'm using to document my findings. So far I've only looked at the file-finding, but plan to dig in further where time permits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant