Fix up CLI and FathomFox dependencies to make fathom train run again
#329
+13,029
−7,284
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tensorboardX 1.9 isn't compatible with NumPy >= 1.24.0 (pytorch/pytorch#91516), because it uses a deprecated comparison that throws a `TypeError` in newer versions.
Selenium 4 removes the deprecated signatures and methods still used by the Vectorizer (mozilla#308), and Selenium 3 is incompatible with newer versions of Firefox, so we can't pin to the older version.
Flake8 3.x uses a deprecated interface that was removed in `importlib-metadata` 5 (python/importlib_metadata#406).
Yarn fails to install its dependencies during the "building FathomFox with your ruleset" phase, reporting a syntax error in the cached FathomFox's `yarn.lock`. Regenerating `yarn.lock` in the repo's copy of FathomFox fixes the syntax errors, but then Yarn fails to resolve package URLs during the same phase. I think this is because the Vectorizer's invocation of `npm install yarn` in the cached FathomFox directory itself alters `package.json`, and writes a `package-lock.json` that confuses Yarn. Yarn specifically warns about `package-lock.json` causing resolution inconsistencies. npm's performance has improved quite a bit since Yarn was added, and its lockfile also supports hashes. Instead of debugging further, I opted to use npm to manage FathomFox's dependencies, and replaced `yarn install` with `npm install`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi! 👋🏼 I'm doing some prototyping with Fathom, and ran into a few dependency-related snags getting
fathom trainto work. This PR is an attempt to fix all of them up—with this patch stack,fathom trainworks for me, and prints metrics and results! 🎉I haven't worked with (or on) Fathom before, but happy to revert or fix up any commits. The individual commit messages have some more details about the versions I chose, but here's a quick summary:
yarn.lock. I think it's unhappy thatnpm installnow saves topackage.jsonby default, and generates its ownpackage-lock.jsonlockfile that interferes with Yarn's. After a bit of debugging, I just replaced Yarn with npm.I also had to use Python 3.9.13. It looks like 3.11 is too new for the version of PyTorch that Fathom depends on—but I wasn't sure about bumping that dependency just yet.
Thanks!
/cc @gleonard-m @DimiDL