Spliterators for sorted collections cause inefficient stream operations when natural ordering is used #6187
Comments
Spliterators returned by CollectSpliterators.indexed and ImmutableSortedSet.spliterator violated the Spliterator API by not returning null in getComparator when the source items are naturally sorted. The effect was that sort operations on Streams backed by these Spliterators were not optimized away, resulting in additional unnecessary sorting. Fixes google#6187.
|
Nice find, and thanks for the deep links. It's of course generally impossible to know whether a We should search for a JDK issue; given that it introduced that |
|
Oh funny, I didn't bother mentioning it could special-case |
Spliterators returned by CollectSpliterators.indexed and ImmutableSortedSet.spliterator violated the Spliterator API by not returning null in getComparator when the source items are naturally sorted. The effect was that sort operations on Streams backed by these Spliterators were not optimized away, resulting in additional unnecessary sorting. Fixes google#6187.
|
I agree that the JDK ought to handle this check itself, although it wouldn't help Guava at the moment due to |
This has always struck me as a weird feature, but people use it, and it "works" in the sense of "does not cause NPE" (though it may cause CCE...). I see both calls to `sort(null)` and calls like `sort(priorityQueue.comparator())` (which _might_ be `null`; [example](https://github.com/google/guava/blob/e82e2a2c07c68108f318958ee0355cc835c97743/guava-testlib/src/com/google/common/collect/testing/testers/SortedSetNavigationTester.java#L57)). And while I prefer a world in which methods like `comparator()` never return `null`, as we arranged for [in `SortedMultiset`](https://guava.dev/SortedMultiset#comparator()), there are apparently [downsides](google/guava#6187) to using a `Comparator` that implements natural order rather than using `null`. Is any of that convincing? :) This is definitely a case in which I can see how eisop would want to stay on its own path. My main motivation for doing this now is that I hear that the current signature is causing us trouble during Java->Kotlin transpilation. I can get more details.
* Annotate the parameter of `List.sort` as `@Nullable`. This has always struck me as a weird feature, but people use it, and it "works" in the sense of "does not cause NPE" (though it may cause CCE...). I see both calls to `sort(null)` and calls like `sort(priorityQueue.comparator())` (which _might_ be `null`; [example](https://github.com/google/guava/blob/e82e2a2c07c68108f318958ee0355cc835c97743/guava-testlib/src/com/google/common/collect/testing/testers/SortedSetNavigationTester.java#L57)). And while I prefer a world in which methods like `comparator()` never return `null`, as we arranged for [in `SortedMultiset`](https://guava.dev/SortedMultiset#comparator()), there are apparently [downsides](google/guava#6187) to using a `Comparator` that implements natural order rather than using `null`. Also, Werner points us to `Arrays.sort`. (#13)
The Spliterators returned by sorted collections such as
ImmutableSortedSetandRegularImmutableAsListresult in unnecessary sorting occurring when a sort operation is performed on their associated Stream. The root cause is that theSpliterators'
getComparator()implementations always return aComparatorinstance, even when natural ordering is used.According to the Javadoc,
nullshould be returned bygetComparator()to indicate the source items are sorted in natural order.You can see here that the
SORTEDflag gets unset when a spliterator has theSORTEDcharacteristic but does not return anullcomparator. And here is where the missed optimization happens due to that.The issue is trivially reproducible by doing the following, and stepping through the code in SortedOps where the expected no-op / optimization would ideally happen:
The text was updated successfully, but these errors were encountered: