You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The OSB workloads do not validate response of the search requests. Any 200 OK response work well from the benchmark perspective.
What solution would you like?
Given we have these workloads running nightly, having correctness validation also will prevent regressions like opensearch-project/OpenSearch#15169. And I believe, we should be able to validate correctness with minimal effort without impacting the performance numbers. We can have static file containing expected response OR validate the correctness between baseline and candidate in addition to performance numbers.
What alternatives have you considered?
Ideally, integration and unit tests cover most of these scenarios, but it is impractical to have such large amounts of data as part of such tests
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem?
The OSB workloads do not validate response of the search requests. Any 200 OK response work well from the benchmark perspective.
What solution would you like?
Given we have these workloads running nightly, having correctness validation also will prevent regressions like opensearch-project/OpenSearch#15169. And I believe, we should be able to validate correctness with minimal effort without impacting the performance numbers. We can have static file containing expected response OR validate the correctness between baseline and candidate in addition to performance numbers.
What alternatives have you considered?
Ideally, integration and unit tests cover most of these scenarios, but it is impractical to have such large amounts of data as part of such tests
The text was updated successfully, but these errors were encountered: