Skip to content

Commit

Permalink
Add smoke tests in README file from test workflow
Browse files Browse the repository at this point in the history
Signed-off-by: Zelin Hao <[email protected]>
  • Loading branch information
zelinh committed Jan 15, 2025
1 parent 76e3491 commit 5bf740d
Showing 1 changed file with 18 additions and 6 deletions.
24 changes: 18 additions & 6 deletions src/test_workflow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,19 +127,23 @@ See [anomaly-detection#766](https://github.com/opensearch-project/anomaly-detect
### Smoke Tests
Run Smoke tests invoking `run_smoke_test.py` in each component from a distribution manifest.

Smoke tests
The smoke tests are designed to verify basic functionality of a distribution bundle with all plugins installed.
Each plugin being tested has its own test specification YAML file, located in the [OpenSearch Build repository](https://github.com/opensearch-project/opensearch-build/tree/main/src/test_workflow/smoke_test/smoke_tests_spec), organized by major version. These test specification files list all testing API cases. The smoke test workflow executes each API case and compares the results with the API specifications from the [OpenSearch API Specification repository](https://github.com/opensearch-project/opensearch-api-specification) to perform basic validation and testing.

Usage:
```aiignore
To execute the smoke tests, use the following command:
```bash
./test.sh smoke-test <test-manifest-path> <target>
```

For example, building and assembling the artifacts locally and run smoke tests:
```aiignore
Examples
Running smoke tests locally with artifacts built and assembled:
```bash
./test.sh smoke-test manifests/2.19.0/opensearch-2.19.0-test.yml --paths opensearch=/workspace/opensearch-build/tar
```

To run smoke tests against existing build from CI:
```aiignore
Running smoke tests against an existing build from CI:
```bash
./test.sh smoke-test manifests/2.19.0/opensearch-2.19.0-test.yml --paths opensearch=https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.19.0/10545/linux/x64/tar/
```

Expand Down Expand Up @@ -311,6 +315,14 @@ After the performance test completes, it will report back the test results as we

The development is tracked by meta issue [#126](https://github.com/opensearch-project/opensearch-build/issues/126)

#### smokeTest job

It is a Jenkins job designed to run smoke tests on bundle artifacts. It retrieves the bundle artifacts using the provided manifest URL parameter and spins up a cluster with all plugins installed. The cluster is deployed using the default configuration without any customizations. Smoke tests for all components are executed sequentially, and the results are returned at the end.

See the smoke test [configuration file](jenkins/opensearch/smoke-test.jenkinsfile) and related [jenkins job](https://build.ci.opensearch.org/job/smoke-test/)

The development is tracked by meta issue [#5223](https://github.com/opensearch-project/opensearch-build/issues/5223)

## Manifest Files

Manifest files are configurations for a particular bundle. `test-workflow` uses three types of manifest files to run test suites.
Expand Down

0 comments on commit 5bf740d

Please sign in to comment.