[PROPOSAL] Add a developer guide for integration tests and usage of abstractions library

[Yury-Fridlyand]

Abstractions library used in integration tests and it is responsible for downloading, unpacking and installation of OpenSearch cluster.
Updating integration test infrastructure usually requires updates in abstractions, the document to be added should describe how a developer can test and debug abstractions. Default behaviour getting abstractions will be to pull down Abstractions from Nuget however a developer method is still desired.

There are few options to test abstractions locally or in GitHub Actions (GHA) environment. The document should describe at least one of them.

1. Add abstraction binaries to client’s repo as it happens now (as of client 1.0 release).

1. Compile abstractions
2. Build packages
3. Remove old packages
4. Deploy to the directory in .net client repo
5. Update references

Pros

• Easy to deploy
• Can be used to run tests locally and in GHA both

Cons

• Not automated process - human error may occur
• Storing compiled binaries is a bad practice
• No strict link between stored packages and source code on the repository or a local branch
• Due to limitations of abstractions versioning it is possible to get different package with the same version

2. Copy abstractions code into clients repo for R&D purposes

1. Replace package references by project references
2. (Optional) Commit entire abstractions lib (sources) into client's repo to have tests run in GHA
3. Apply changes done in abstractions to the abstractions repo

Pros

• No local deployment required
• Easy to debug

Cons

• Copy entire repository into another one is a bad practice even for debugging
• It is possible to have a mistake on stage of copying back and forth

3. Checkout abstractions and build in the workflow during GHA

There are 2 ways of running test with this method.
Using package reference (like method 1 above):

1. Checkout abstractions on the given branch or commit hash
2. Compile it
3. Deploy it to client's repo
4. Run tests

Using project reference (like method 2 above):

1. Checkout abstractions on the given branch or commit hash
2. Run tests

Pros

• Fully automated process
• Strict link between test results and source code

Cons

• This might require a GH token or something else like this - OpenSearch denied this
• Debugging is impossible
• Require R&D of GHA workflow

4. Have an easy way to do unofficial releases, not visible/available for users

1. Make a debug of abstractions
2. Publish it to CI nuget repository
3. Use it in .net client

Pros

• Fully automated process
• Strict link between test results and source code
• Tests like a real nuget package
• Could be used for debug and for GHA

Cons

• Infrastructure team involved
• No rapid integration
• For security reasons only maintainers will be able to use this method

[wbeckler]

For Option 2, does it make sense to jettison the concept of a separate repo, and then there is no more "copying" across code? Without 2 repos, the whole problem goes away, right?

[Yury-Fridlyand]

Right, it is one of the options.

[MaxKsyunz]

For Option 2, does it make sense to jettison the concept of a separate repo, and then there is no more "copying" across code? Without 2 repos, the whole problem goes away, right?

This makes sense to me -- abstractions is a set of libraries used for integration tests. There's no need to publish them and it's simple enough to do if necessary.

[Yury-Fridlyand]

According to the second method it was decided to copy abstractions in .net repo.
Closing, since #92 merged.