Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check requirements against requirements of all dependencies #1113

Open
jayvdb opened this issue Dec 10, 2016 · 9 comments · May be fixed by #2935
Open

Check requirements against requirements of all dependencies #1113

jayvdb opened this issue Dec 10, 2016 · 9 comments · May be fixed by #2935

Comments

@jayvdb
Copy link
Member

jayvdb commented Dec 10, 2016

Due to the way pip works, without a proper dependency graph, the version installed is not deterministic unless our versions are identical to the versions of our dependencies.

All requirements files should be in sync. A bear needs to check this.

@jayvdb
Copy link
Member Author

jayvdb commented Dec 10, 2016

See coala/coala#3200 and coala/coala#3183 for the types of problems that should be avoidable.

@jayvdb
Copy link
Member Author

jayvdb commented Jan 1, 2017

coala/coala#3444 is another example.

@jayvdb
Copy link
Member Author

jayvdb commented Jan 12, 2017

https://pypi.python.org/pypi/PinRequirementsBear by @Adrianzatreanu looks interesting.

https://github.com/landscapeio/requirements-detector looks like it only goes one level deep, and doesnt catch indirect dependencies. However it has some very useful logic, which can help if recusing through dependencies.

@jayvdb
Copy link
Member Author

jayvdb commented Jan 12, 2017

Note I would be suspicious of any solution which installs many sets of requirements and then uses pip freeze or similar to determine the dependencies. Something like https://pypi.python.org/pypi/freeze-requirements will stablise the system somewhat, given that it is pinning all dependencies and also has a sorted list (sorted introduces stable order, which pip needs). But that means we loose the ability to accept patched and minor versions.

pip will happily install a conflicting set of packages in some conditions, and those conflicts are only visible when pkg_resources is used (such as via setuptools hooks).

The most mature approach to solving this problem appears to be https://pypi.python.org/pypi/openstack-requirements . It may look like overkill for coala ; while coala is much smaller, we have dependencies on a lot of foreign distinct python projects, and some of those projects (especially python linters) have inter-dependencies meaning the correct usable set is actually quite constrained by forces beyond our control.

One way we could release some of this pressure is to make some bears incompatible with other bears, when we know their requirements are not reconcilable. E.g. currently we have to hold back Radon, otherwise we cant have pyflakes 1.4 . #1242

@jayvdb
Copy link
Member Author

jayvdb commented Jan 12, 2017

Another problem which has been hotfixed: #1282 (and is probably the reason coala/coala#3532 was created)

@pratyushprakash
Copy link
Contributor

@jayvdb Just to be clear, the bear is required to report any mismatch between the dependencies listed in the requirements files and the dependencies of the dependencies listed in said requirements files?

@jayvdb
Copy link
Member Author

jayvdb commented Jan 22, 2017 via email

@bkhanale
Copy link
Member

This issue is reserved under the GSoC project Improve Generic Bear Quality.

@Makman2 Makman2 assigned bkhanale and unassigned pratyushprakash May 14, 2019
@jayvdb
Copy link
Member Author

jayvdb commented Jun 12, 2019

One way to approach this is to collect them all into a single requirements file, find any conflicts within the file (this bit still needs analysis), and then that file can be used as a pip 'constraints' file for any use of pip to avoid wrong versions being installed.

bkhanale added a commit to bkhanale/coala-bears that referenced this issue Jul 11, 2019
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Jul 21, 2019
@bkhanale bkhanale linked a pull request Jul 21, 2019 that will close this issue
2 tasks
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Jul 21, 2019
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Jul 21, 2019
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Jul 21, 2019
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Jul 29, 2019
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Jul 29, 2019
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Jul 29, 2019
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Aug 4, 2019
bkhanale added a commit to bkhanale/coala-bears that referenced this issue Aug 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

6 participants