Make OpamSwitchState load files lazily (makes lock 4x faster) #194
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Profiling
opam monorepo lock
reveals that it is spending a lot of time parsingfiles (in particular, lexing them) and in
OpamPp
combinators.We cannot do a lot about these, but this dominates the profile because opam
parses so many files, tens of thousands of them. It turns out that it parses all
files from all the repositories it knows about.
This PR changes the internals of
OpamRepositoryState
andOpamSwitchState
sothat opam files are parsed only lazily. The name and version of each package is
already known just with the file path, and the contents are often not necessary
at all so they can be hidden behind a
lazy
.Some code is left stubbed out (it does not seem to matter to opam-monorepo at
least), but the performance is better: a lock on opam-monorepo itself now takes
1.5s (from 6.5s on
main
).