Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Apache Lucene 9.11.0-snapshot-fb97840 #13195

Merged
merged 1 commit into from
Apr 15, 2024
Merged

Update to Apache Lucene 9.11.0-snapshot-fb97840 #13195

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion buildSrc/version.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
opensearch = 3.0.0
lucene = 9.11.0-snapshot-8a555eb
lucene = 9.11.0-snapshot-fb97840

bundled_jdk_vendor = adoptium
bundled_jdk = 21.0.2+13
Expand Down
1 change: 0 additions & 1 deletion libs/core/licenses/lucene-core-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions libs/core/licenses/lucene-core-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
53a828e3e88f55c83979cd3df0704617cc9edb9a
1 change: 0 additions & 1 deletion modules/lang-expression/licenses/lucene-expressions-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions modules/lang-expression/licenses/lucene-expressions-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ab914b48665f484315b78e4b6787aa42f5966bb6
1 change: 0 additions & 1 deletion plugins/analysis-icu/licenses/lucene-analysis-icu-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions plugins/analysis-icu/licenses/lucene-analysis-icu-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
f9cd7bec33c8cf3b891976cb674ffe9c97f8747f
1 change: 0 additions & 1 deletion plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions plugins/analysis-kuromoji/licenses/lucene-analysis-kuromoji-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c244a56bf7cd171a19379c96f1d20c477a34578d
1 change: 0 additions & 1 deletion plugins/analysis-nori/licenses/lucene-analysis-nori-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions plugins/analysis-nori/licenses/lucene-analysis-nori-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
da26df43f2b0d7c2dfecbf208cae0772a5e382ca
1 change: 0 additions & 1 deletion plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions plugins/analysis-phonetic/licenses/lucene-analysis-phonetic-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
f752ffa5ee4697b04643214236138f3defdee2f4
1 change: 0 additions & 1 deletion plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions plugins/analysis-smartcn/licenses/lucene-analysis-smartcn-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
73fe44fe755aef72e7293b2ffdd934beb631429d
1 change: 0 additions & 1 deletion plugins/analysis-stempel/licenses/lucene-analysis-stempel-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions plugins/analysis-stempel/licenses/lucene-analysis-stempel-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c2b48831b25e1c7e8f683a63d1505c2d133256d3
1 change: 0 additions & 1 deletion ...s/analysis-ukrainian/licenses/lucene-analysis-morfologik-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions ...s/analysis-ukrainian/licenses/lucene-analysis-morfologik-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
757f8b29f103f82b6fb6948634e93dd497c9d7a8
19 changes: 13 additions & 6 deletions server/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -411,14 +411,21 @@ tasks.register("downloadSnapshot", Copy) {
def groupId = "org.opensearch"
def artifactId = "opensearch"

repositories {
maven {
url mavenSnapshotRepoUrl
}
}
def repos = project.getRepositories();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reta Why did we need to make these changes? It looks like it caused issues in the download workflow [1] for other PRs.

./gradlew :server:japicmp
...
FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':server:downloadSnapshot'.
> Could not resolve all task dependencies for configuration ':server:snapshotArtifact'.
   > Could not find org.apache.lucene:lucene-core:9.11.0-snapshot-fb[97](https://github.com/opensearch-project/OpenSearch/actions/runs/8693672822/job/23846998626?pr=12812#step:4:100)840.
     Searched in the following locations:
       - https://repo.maven.apache.org/maven2/org/apache/lucene/lucene-core/9.11.0-snapshot-fb97840/lucene-core-9.11.0-snapshot-fb97840.pom
       - https://aws.oss.sonatype.org/content/repositories/snapshots/org/apache/lucene/lucene-core/9.11.0-snapshot-fb97840/lucene-core-9.11.0-snapshot-fb97840.pom

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@peternied the change was supposed to fix exactly that: the issue I run into was that that snapshot tasks tries to download opensearch deps from the https://aws.oss.sonatype.org/ whereas it should not since it has to use another repo. The change adds the https://aws.oss.sonatype.org/ as exclusive for org.opensearch so it seemed to help but not entirely, I will take a look shortly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I'm curious if the source for the artifacts changed. We have a myriad of changes [1] with hard to predict downstream effects, but this seemed like a new source or at least a source we haven't seen create friction before.

Copy link
Collaborator Author

@reta reta Apr 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I'm curious if the source for the artifacts changed

The Apache Lucene snapshots are coming from different place (RepositoriesSetupPlugin): https://ci.opensearch.org/ci/dbc/snapshots/lucene/

, but this seemed like a new source or at least a source we haven't seen create friction before.

Correct, but I think it is scoped only to japicmp supporting tasks we've added (since we've been modifying repositories on the fly).

MavenArtifactRepository opensearchRepo = repos.maven(repo -> {
repo.setName("opensearch-snapshots");
repo.setUrl(mavenSnapshotRepoUrl);
});

repos.exclusiveContent(exclusiveRepo -> {
exclusiveRepo.filter(descriptor -> descriptor.includeGroup(groupId));
exclusiveRepo.forRepositories(opensearchRepo);
});

configurations {
snapshotArtifact
snapshotArtifact {
exclude group: 'org.apache.lucene'
}
}

dependencies {
Expand Down
1 change: 0 additions & 1 deletion server/licenses/lucene-analysis-common-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-analysis-common-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
cd0afb5da5dcb4c7498bd1ee7f7bab0e289404b8
1 change: 0 additions & 1 deletion server/licenses/lucene-backward-codecs-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-backward-codecs-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
88888315cd60e565960ae2e6fed2af0df077a2a2
1 change: 0 additions & 1 deletion server/licenses/lucene-core-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-core-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
53a828e3e88f55c83979cd3df0704617cc9edb9a
1 change: 0 additions & 1 deletion server/licenses/lucene-grouping-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-grouping-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
d1f54a816c9d85e890a862a2dffdc734ece2770c
1 change: 0 additions & 1 deletion server/licenses/lucene-highlighter-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-highlighter-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1e7c9336fa86fb866fcd76ea5d6283c804b4d580
1 change: 0 additions & 1 deletion server/licenses/lucene-join-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-join-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
17be4fc1f9feca0dac84a37d54dca4b32df4c619
1 change: 0 additions & 1 deletion server/licenses/lucene-memory-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-memory-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
7350675a2cf386c0f003b667b61db614f03bb984
1 change: 0 additions & 1 deletion server/licenses/lucene-misc-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-misc-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c5c8bd120d5985ab6bd4e5f89efe08c226c0a323
1 change: 0 additions & 1 deletion server/licenses/lucene-queries-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-queries-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
2e1975ac26e9172722f734bf0f5583317e5eb16a
1 change: 0 additions & 1 deletion server/licenses/lucene-queryparser-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-queryparser-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c8cf3d5dd4d0538b38e4e88bb865bc59d835d887
1 change: 0 additions & 1 deletion server/licenses/lucene-sandbox-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-sandbox-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5a3a7a138ff4978f3ddb186d9786e6cb4793b291
1 change: 0 additions & 1 deletion server/licenses/lucene-spatial-extras-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-spatial-extras-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
dc71e0125c66d29a1bffc1ddeab4b96526e737c8
1 change: 0 additions & 1 deletion server/licenses/lucene-spatial3d-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-spatial3d-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
aef4c04d353092a438eee302521fe34188b7c4df
1 change: 0 additions & 1 deletion server/licenses/lucene-suggest-9.11.0-snapshot-8a555eb.jar.sha1
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions server/licenses/lucene-suggest-9.11.0-snapshot-fb97840.jar.sha1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
86f68cacd85f99b4ddcda3aff7c873349ba59381
3 changes: 3 additions & 0 deletions test/framework/src/main/java/org/opensearch/indices/analysis/AnalysisFactoryTestCase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,9 @@ public abstract class AnalysisFactoryTestCase extends OpenSearchTestCase {
.put("persianstem", Void.class)
// https://github.com/apache/lucene/pull/12169
.put("word2vecsynonym", Void.class)
// https://github.com/apache/lucene/pull/12915
.put("japanesehiraganauppercase", Void.class)
.put("japanesekatakanauppercase", Void.class)
.immutableMap();

static final Map<String, Class<?>> KNOWN_CHARFILTERS = new MapBuilder<String, Class<?>>()
Expand Down
Loading