Skip to content

Commit

Permalink
Don't prefetch unreachable packages
Browse files Browse the repository at this point in the history
When batch prefetching we can fetch versions we know that are incompatible. In the following example, we were prefetching sentry-kafka-schemas below version 1.50.0.

```
python-rapidjson<=1.20,>=1.4
sentry-kafka-schemas<=0.1.113,>=0.1.50
```

Using a new pubgrub interface from astral-sh/pubgrub#32, we can avoid those prefetches by asking for incompatibilities that won't change anymore (those with root).
  • Loading branch information
konstin committed Oct 16, 2024
1 parent e71b1d0 commit ffd1871
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 2 deletions.
16 changes: 14 additions & 2 deletions crates/uv-resolver/src/resolver/batch_prefetch.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use std::cmp::min;

use itertools::Itertools;
use pubgrub::Range;
use pubgrub::{Range, Term};
use rustc_hash::FxHashMap;
use tokio::sync::mpsc::Sender;
use tracing::{debug, trace};
Expand Down Expand Up @@ -49,6 +49,7 @@ impl BatchPrefetcher {
index: Option<&IndexUrl>,
version: &Version,
current_range: &Range<Version>,
unchangeable_constraints: Option<&Term<Range<Version>>>,
python_requirement: &PythonRequirement,
request_sink: &Sender<Request>,
in_memory: &InMemoryIndex,
Expand Down Expand Up @@ -119,11 +120,22 @@ impl BatchPrefetcher {
}
}
BatchPrefetchStrategy::InOrder { previous } => {
let range = if selector.use_highest_version(name) {
let mut range = if selector.use_highest_version(name) {
Range::strictly_lower_than(previous)
} else {
Range::strictly_higher_than(previous)
};
// If we have constraints from root, don't go beyond those. Example: We are
// prefetching for foo 1.60 and have a dependency for `foo>=1.50`, so we should
// only prefetch 1.60 to 1.50, knowing 1.49 will always be rejected.
if let Some(unchangeable_constraints) = unchangeable_constraints {
range = match unchangeable_constraints {
Term::Positive(constraints) => range.intersection(constraints),
Term::Negative(negative_constraints) => {
range.intersection(&negative_constraints.complement())
}
};
}
if let Some(candidate) =
selector.select_no_preference(name, &range, version_map, markers)
{
Expand Down
4 changes: 4 additions & 0 deletions crates/uv-resolver/src/resolver/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -475,6 +475,10 @@ impl<InstalledPackages: InstalledPackagesProvider> ResolverState<InstalledPackag
index,
&version,
term_intersection.unwrap_positive(),
state
.pubgrub
.partial_solution
.unchanging_term_for_package(&state.next),
&state.python_requirement,
&request_sink,
&self.index,
Expand Down

0 comments on commit ffd1871

Please sign in to comment.