diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 47ae31be6..dccc260df 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -11,10 +11,12 @@ - [Run Single-node Cluster Locally](#run-single-node-cluster-locally) - [Run Multi-node Cluster Locally](#run-multi-node-cluster-locally) - [Debugging](#debugging) + - [Major Dependencies](#major-dependencies) - [Backwards Compatibility Testing](#backwards-compatibility-testing) - [Adding new tests](#adding-new-tests) - [Supported configurations](#supported-configurations) - [Submitting Changes](#submitting-changes) + - [Building On Lucene Version Updates](#building-on-lucene-version-updates) # Developer Guide @@ -88,12 +90,22 @@ Please follow these formatting guidelines: OpenSearch neural-search uses a [Gradle](https://docs.gradle.org/6.6.1/userguide/userguide.html) wrapper for its build. Run `gradlew` on Unix systems. -Build OpenSearch neural-search using `gradlew build` +Build OpenSearch neural-search using `gradlew build`. This command will +also run Integration Tests and Unit Tests. ``` ./gradlew build ``` +## Run Unit Tests +If you want to strictly test that your unit tests are passing +you can run the following. + +``` +./gradlew test +``` + + ## Run OpenSearch neural-search ### Run Single-node Cluster Locally @@ -227,6 +239,12 @@ Additionally, it is possible to attach one debugger to the cluster JVM and anoth ./gradlew :integTest -Dtest.debug=1 -Dcluster.debug=1 ``` +#### Major Dependencies +Currently, the major dependencies that Neural Search depends on are [ML-Commons](https://github.com/opensearch-project/ml-commons) and [K-NN](https://github.com/opensearch-project/k-NN). +Make sure to check on them when you observe a failure that affects Neural Search. +See [Building on Lucene Version updates](#building-on-lucene-version-updates) as an example where K-NN caused a build failure. +Also, please note that it may take time for developers to create a fix for your current dependency issue. + ## Backwards Compatibility Testing The purpose of Backwards Compatibility Testing and different types of BWC tests are explained [here](https://github.com/opensearch-project/opensearch-plugins/blob/main/TESTING.md#backwards-compatibility-testing). The BWC tests (i.e. Restart-Upgrade, Mixed-Cluster and Rolling-Upgrade scenarios) should be added with any new feature being added to Neural Search. @@ -292,3 +310,31 @@ original PR with an appropriate label `backport ` is merge run successfully on the PR. For example, if a PR on main needs to be backported to `2.x` branch, add a label `backport 2.x` to the PR and make sure the backport workflow runs on the PR along with other checks. Once this PR is merged to main, the workflow will create a backport PR to the `2.x` branch. + +## Building On Lucene Version Updates +There may be a Lucene version update that can affect your workflow causing errors like +`java.lang.NoClassDefFoundError: org/apache/lucene/codecs/lucene99/Lucene99Codec` or +`Provider org.opensearch.knn.index.codec.KNN910Codec.KNN910Codec could not be instantiated`. In this case +we can observe there may be an issue with a dependency with [K-NN](https://github.com/opensearch-project/k-NN). +This results in having issues with not being able to do `./gradlew run` or `./gradlew build`. + +You can check this [K-NN PR](https://github.com/opensearch-project/k-NN/pull/2195) as an example of this event happening or this [Neural Search PR](https://github.com/opensearch-project/neural-search/pull/913#issuecomment-2400189329) that shows a developer going +through the same build issue. + +**Follow the steps to remedy the gradle run issue.** +1. From your cloned neural search repo root directory `rm -rf build .gradle` +2. Clear the following directories from your gradle folder located in your root directory + 1. `cd ~/.gradle` + 2. `rm -rf caches workers wrapper daemon` + 3. `cd -` switch back the previous directory (i.e. the neural search repo root directory) +3. Finally run `./gradlew run` + +**Follow the steps to remedy the gradle build issue** + +**PREREQ:** Make sure you have OpenSearch repo cloned locally + +1. From your cloned neural search repo root directory `rm -rf build .gradle` +2. Delete the .gradle folder and .m2 folder. `rm -rf ~/.gradle ~/.m2` +3. Head over to your OpenSearch cloned repo root directory + 1. `./gradlew publisToMavenLocal` +4. Finally run `./gradlew build` from the neural search repo