Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support correctness validation in OSB workloads #367

Open
jainankitk opened this issue Aug 13, 2024 · 0 comments
Open

[FEATURE] Support correctness validation in OSB workloads #367

jainankitk opened this issue Aug 13, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@jainankitk
Copy link

Is your feature request related to a problem?

The OSB workloads do not validate response of the search requests. Any 200 OK response work well from the benchmark perspective.

What solution would you like?

Given we have these workloads running nightly, having correctness validation also will prevent regressions like opensearch-project/OpenSearch#15169. And I believe, we should be able to validate correctness with minimal effort without impacting the performance numbers. We can have static file containing expected response OR validate the correctness between baseline and candidate in addition to performance numbers.

What alternatives have you considered?

Ideally, integration and unit tests cover most of these scenarios, but it is impractical to have such large amounts of data as part of such tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Todo
Development

No branches or pull requests

3 participants