Add integration test framework #687

ryanbogan · 2023-04-19T21:36:56Z

Description

Adds an integration test framework, utilizing the changes made in opensearch-project/OpenSearch#7235. The integration test task starts the helloWorld extension, spins up a test single-node cluster to initialize the extensions, and then kills the cluster and the extension process running in the background. There are currently no tests written for this framework, so the task will fail with an error temporarily.

Note: Must be merged after opensearch-project/OpenSearch#7235

Issues Resolved

#589

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Ryan Bogan <[email protected]>

dblock · 2023-04-25T18:18:41Z

Is this the first time a consumer of opensearch-sdk-java would be bringing in the gradle tasks from OpenSearch core? That feels like undesirable. The alternative would be to (re)implement a simpler version of the functionality of test cluster setup in the SDK.

Signed-off-by: Ryan Bogan <[email protected]>

saratvemulapalli · 2023-05-02T19:02:39Z

Is this the first time a consumer of opensearch-sdk-java would be bringing in the gradle tasks from OpenSearch core? That feels like undesirable. The alternative would be to (re)implement a simpler version of the functionality of test cluster setup in the SDK.

We definitely could implement one in SDK. We've taken this path to:

Not duplicate code for test cluster setup
Plugins migrating to extensions have all the features from the framework
Seamless integration test support with opensearch-build

I definitely understand we'd like to keep the SDK as independent as possible, we'd definitely take it as a follow up.

saratvemulapalli · 2023-05-08T17:21:08Z

@ryanbogan whats needed here to get this moving?

ryanbogan · 2023-05-08T17:21:57Z

@saratvemulapalli Working on fixing merge conflicts now. I also want to test it out with the new protobuf additions but I'll have everything finalized by EOD

Signed-off-by: Ryan Bogan <[email protected]>

ryanbogan · 2023-05-10T20:13:37Z

I'm not sure why gradle check is failing for a dependency conflict. We have the same version as OpenSearch and everything is passing on my local.

build.gradle

Signed-off-by: Ryan Bogan <[email protected]>

build.gradle

src/main/resources/sample/helloworld-settings.yml

Signed-off-by: Ryan Bogan <[email protected]>

src/main/java/org/opensearch/sdk/sample/helloworld/HelloWorldExtension.java

Signed-off-by: Ryan Bogan <[email protected]>

build.gradle

joshpalis

LGTM, thanks for the changes Ryan.

dbwiddis

LGTM! A few config changes outside the scope of this that I'll save for another PR/issue.

owaiskazi19 · 2023-05-31T21:27:25Z

The path for the integTest folder should be under a directory called test -> src/test/java/org/opensearch/sdk/test/integTest/TransportCommunicationIT.java.

* Add integration test task that uses the OpenSearch gradle plugin Signed-off-by: Ryan Bogan <[email protected]> * Added closeTestExtension task to kill testExtension process Signed-off-by: Ryan Bogan <[email protected]> * Fix spotless Signed-off-by: Ryan Bogan <[email protected]> * Implement resolution strategy to fix dependency conflict Signed-off-by: Ryan Bogan <[email protected]> * Minor adjustment Signed-off-by: Ryan Bogan <[email protected]> * Change jackson version Signed-off-by: Ryan Bogan <[email protected]> * Change jackson dataformat version Signed-off-by: Ryan Bogan <[email protected]> * Revert jackson version change Signed-off-by: Ryan Bogan <[email protected]> * Add resolution strategy to fix jackson dependency conflict Signed-off-by: Ryan Bogan <[email protected]> * Address PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Add new implementation to fix error from separate merge Signed-off-by: Ryan Bogan <[email protected]> * Address PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Address PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Revert minor change Signed-off-by: Ryan Bogan <[email protected]> * Address PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Add reading from yaml for testExtensionCreation Signed-off-by: Ryan Bogan <[email protected]> * Spotless Signed-off-by: Ryan Bogan <[email protected]> * Minor change Signed-off-by: Ryan Bogan <[email protected]> * Debug test commit Signed-off-by: Ryan Bogan <[email protected]> * Minor changes Signed-off-by: Ryan Bogan <[email protected]> * Remove extra test class Signed-off-by: Ryan Bogan <[email protected]> * Add index Signed-off-by: Ryan Bogan <[email protected]> * Fix inadvertent change Signed-off-by: Ryan Bogan <[email protected]> * Minor change Signed-off-by: Ryan Bogan <[email protected]> * Add log4j resolution strategy Signed-off-by: Ryan Bogan <[email protected]> * Remove testing println Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Co-authored-by: Sarat Vemulapalli <[email protected]> Co-authored-by: Owais Kazi <[email protected]> (cherry picked from commit 04d9eab) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

This reverts commit 04d9eab.

This reverts commit 04d9eab. Signed-off-by: owaiskazi19 <[email protected]>

* Add integration test framework (#687) * Add integration test task that uses the OpenSearch gradle plugin Signed-off-by: Ryan Bogan <[email protected]> * Added closeTestExtension task to kill testExtension process Signed-off-by: Ryan Bogan <[email protected]> * Fix spotless Signed-off-by: Ryan Bogan <[email protected]> * Implement resolution strategy to fix dependency conflict Signed-off-by: Ryan Bogan <[email protected]> * Minor adjustment Signed-off-by: Ryan Bogan <[email protected]> * Change jackson version Signed-off-by: Ryan Bogan <[email protected]> * Change jackson dataformat version Signed-off-by: Ryan Bogan <[email protected]> * Revert jackson version change Signed-off-by: Ryan Bogan <[email protected]> * Add resolution strategy to fix jackson dependency conflict Signed-off-by: Ryan Bogan <[email protected]> * Address PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Add new implementation to fix error from separate merge Signed-off-by: Ryan Bogan <[email protected]> * Address PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Address PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Revert minor change Signed-off-by: Ryan Bogan <[email protected]> * Address PR Comments Signed-off-by: Ryan Bogan <[email protected]> * Add reading from yaml for testExtensionCreation Signed-off-by: Ryan Bogan <[email protected]> * Spotless Signed-off-by: Ryan Bogan <[email protected]> * Minor change Signed-off-by: Ryan Bogan <[email protected]> * Debug test commit Signed-off-by: Ryan Bogan <[email protected]> * Minor changes Signed-off-by: Ryan Bogan <[email protected]> * Remove extra test class Signed-off-by: Ryan Bogan <[email protected]> * Add index Signed-off-by: Ryan Bogan <[email protected]> * Fix inadvertent change Signed-off-by: Ryan Bogan <[email protected]> * Minor change Signed-off-by: Ryan Bogan <[email protected]> * Add log4j resolution strategy Signed-off-by: Ryan Bogan <[email protected]> * Remove testing println Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Co-authored-by: Sarat Vemulapalli <[email protected]> Co-authored-by: Owais Kazi <[email protected]> (cherry picked from commit 04d9eab) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Change versions and resolve conflicts from original PR Signed-off-by: Ryan Bogan <[email protected]> * Fix dependency conflicts Signed-off-by: Ryan Bogan <[email protected]> * Fix snapshot version Signed-off-by: Ryan Bogan <[email protected]> * Combined commit for backporting 'Fix Publishing' PR's Signed-off-by: Ryan Bogan <[email protected]> --------- Signed-off-by: Ryan Bogan <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Sarat Vemulapalli <[email protected]> Co-authored-by: Owais Kazi <[email protected]> Co-authored-by: Ryan Bogan <[email protected]>

ryanbogan added 3 commits April 18, 2023 18:41

Add integration test task that uses the OpenSearch gradle plugin

6a7629a

Signed-off-by: Ryan Bogan <[email protected]>

Merge main into add_integ_test_framework

7c5cf1f

Signed-off-by: Ryan Bogan <[email protected]>

Added closeTestExtension task to kill testExtension process

52dda1c

Signed-off-by: Ryan Bogan <[email protected]>

ryanbogan requested review from joshpalis, owaiskazi19, saratvemulapalli and dbwiddis as code owners April 19, 2023 21:36

github-actions bot added seasoned-contributor backport 1.x labels Apr 19, 2023

ryanbogan added 2 commits April 19, 2023 21:59

Fix spotless

17be5bf

Signed-off-by: Ryan Bogan <[email protected]>

Merge branch 'main' into add_integ_test_framework

a45ac35

owaiskazi19 mentioned this pull request Apr 28, 2023

Setup extension TLS #619

Merged

Merge main into add_integ_test_framework

53bbf1d

Signed-off-by: Ryan Bogan <[email protected]>

ryanbogan and others added 8 commits May 8, 2023 18:11

Implement resolution strategy to fix dependency conflict

3d96e32

Signed-off-by: Ryan Bogan <[email protected]>

Merge branch 'main' into add_integ_test_framework

afba241

Minor adjustment

af6904e

Signed-off-by: Ryan Bogan <[email protected]>

Merge branch 'main' into add_integ_test_framework

ea2f1f0

Merge branch 'main' into add_integ_test_framework

b0d742a

Change jackson version

d0c9ed8

Signed-off-by: Ryan Bogan <[email protected]>

Change jackson dataformat version

78a0c3b

Signed-off-by: Ryan Bogan <[email protected]>

Revert jackson version change

476263e

Signed-off-by: Ryan Bogan <[email protected]>

vibrantvarun reviewed May 10, 2023

View reviewed changes

build.gradle Show resolved Hide resolved

build.gradle Outdated Show resolved Hide resolved

ryanbogan and others added 2 commits May 11, 2023 10:58

Merge branch 'main' into add_integ_test_framework

f60262b

Add resolution strategy to fix jackson dependency conflict

1b602a5

Signed-off-by: Ryan Bogan <[email protected]>

joshpalis reviewed May 12, 2023

View reviewed changes

build.gradle Outdated Show resolved Hide resolved

build.gradle Outdated Show resolved Hide resolved

joshpalis reviewed May 24, 2023

View reviewed changes

src/main/resources/sample/helloworld-settings.yml Outdated Show resolved Hide resolved

ryanbogan added 4 commits May 24, 2023 17:54

Fix inadvertent change

342caa1

Signed-off-by: Ryan Bogan <[email protected]>

Minor change

7f0fd39

Signed-off-by: Ryan Bogan <[email protected]>

Merge main into add_integ_test_framework

80c1fc2

Signed-off-by: Ryan Bogan <[email protected]>

Add log4j resolution strategy

e7540d8

Signed-off-by: Ryan Bogan <[email protected]>

ryanbogan requested a review from joshpalis May 25, 2023 16:13

joshpalis reviewed May 25, 2023

View reviewed changes

src/main/java/org/opensearch/sdk/sample/helloworld/HelloWorldExtension.java Outdated Show resolved Hide resolved

Remove testing println

bdadfa7

Signed-off-by: Ryan Bogan <[email protected]>

owaiskazi19 reviewed May 25, 2023

View reviewed changes

build.gradle Show resolved Hide resolved

ryanbogan mentioned this pull request May 26, 2023

[BUG] Gradle run not working #789

Open

joshpalis approved these changes May 26, 2023

View reviewed changes

Merge branch 'main' into add_integ_test_framework

2db7e35

ryanbogan requested a review from owaiskazi19 May 30, 2023 16:02

Merge branch 'main' into add_integ_test_framework

bc8654c

dbwiddis approved these changes May 31, 2023

View reviewed changes

owaiskazi19 approved these changes May 31, 2023

View reviewed changes

owaiskazi19 merged commit 04d9eab into opensearch-project:main May 31, 2023

ryanbogan deleted the add_integ_test_framework branch May 31, 2023 21:31

opensearch-trigger-bot bot mentioned this pull request May 31, 2023

[Backport 1.x] Add integration test framework #796

Merged

dbwiddis mentioned this pull request Jun 1, 2023

[BUG] Build publishing broken due to duplicate artifacts #798

Closed

owaiskazi19 added a commit that referenced this pull request Jun 1, 2023

Revert "Add integration test framework (#687)"

6925331

This reverts commit 04d9eab.

owaiskazi19 mentioned this pull request Jun 1, 2023

Revert "Add integration test framework" #799

Closed

owaiskazi19 added a commit that referenced this pull request Jun 1, 2023

Revert "Add integration test framework (#687)"

c4f27c7

This reverts commit 04d9eab. Signed-off-by: owaiskazi19 <[email protected]>

This was referenced Jun 1, 2023

Fix publishing #800

Merged

Fix publishing to sonatype #804

Merged

Fix Maven Local Publishing #807

Merged

Add changes made in 1.x for manual backport #809

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add integration test framework #687

Add integration test framework #687

ryanbogan commented Apr 19, 2023

dblock commented Apr 25, 2023

saratvemulapalli commented May 2, 2023

saratvemulapalli commented May 8, 2023

ryanbogan commented May 8, 2023

ryanbogan commented May 10, 2023

joshpalis left a comment

dbwiddis left a comment

owaiskazi19 commented May 31, 2023

Add integration test framework #687

Add integration test framework #687

Conversation

ryanbogan commented Apr 19, 2023

Description

Issues Resolved

dblock commented Apr 25, 2023

saratvemulapalli commented May 2, 2023

saratvemulapalli commented May 8, 2023

ryanbogan commented May 8, 2023

ryanbogan commented May 10, 2023

joshpalis left a comment

Choose a reason for hiding this comment

dbwiddis left a comment

Choose a reason for hiding this comment

owaiskazi19 commented May 31, 2023