From 7b5c70cd0cef4beb04b7436d70a788b4454e0a70 Mon Sep 17 00:00:00 2001 From: "Theo N. Truong" Date: Tue, 10 Sep 2024 07:34:33 -0600 Subject: [PATCH] Corrected Search Body Sort (#563) --- CHANGELOG.md | 1 + DEVELOPER_GUIDE.md | 20 ++- TESTING_GUIDE.md | 46 ++--- spec/_global_parameters.yaml | 2 +- spec/schemas/_common.yaml | 5 + tests/default/_core/search/sort_by_field.yaml | 163 ++++++++++++++++++ 6 files changed, 206 insertions(+), 31 deletions(-) create mode 100644 tests/default/_core/search/sort_by_field.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ced75260..a0707cc98 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -107,6 +107,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - Split test suite ([#472])(https://github.com/opensearch-project/opensearch-api-specification/pull/472) - Changed `WriteResponseBase`'s `_primary_term`, `_seq_no` & `_version` to have `int64` format ([#530](https://github.com/opensearch-project/opensearch-api-specification/pull/530)) - Adjusted indices, shards cat API to test against unassigned indices ([#551](https://github.com/opensearch-project/opensearch-api-specification/pull/551)) +- Corrected Sort schema for `/_search`'s request body ([#529](https://github.com/opensearch-project/opensearch-api-specification/issues/529)) - Rename `Bytes` component to `StorageType` ([#552](https://github.com/opensearch-project/opensearch-api-specification/pull/552)) - Rename `ByteSize` to `StorageSize` ([#552](https://github.com/opensearch-project/opensearch-api-specification/pull/552)) diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 43c30d7f3..8314eb633 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -17,15 +17,15 @@ * [Spec Linter](#spec-linter) * [Arguments](#arguments-1) * [Example](#example-1) + * [Spec Tester](#spec-tester) * [Dump Cluster Spec](#dump-cluster-spec) * [Arguments](#arguments-2) * [Example](#example-2) * [Coverage](#coverage) * [Arguments](#arguments-3) * [Example](#example-3) - * [Testing](#testing) - * [Tests](#tests) - * [Lints](#lints) + * [Tools Testing](#tools-testing) + * [Tools Linting](#tools-linting) * [Workflows](#workflows) * [Analyze PR Changes](#analyze-pr-changes) * [Build](#build) @@ -211,6 +211,14 @@ We can take advantage of the default values and simply lint the specification vi npm run lint:spec ``` +### Spec Tester + +```bash +npm run test:spec -- --help +``` + +The spec test framework validates the OpenSearch spec against a running OpenSearch cluster. As you modify the spec, you should add or update the spec test stories in the [./tests](tests) directory. For information on this topic, see [TESTING_GUIDE.md](TESTING_GUIDE.md). + ### [Dump Cluster Spec](tools/src/dump-cluster-spec) ```bash @@ -306,9 +314,7 @@ The output file `build/coverage.json` will now contain data of like below: } ``` -### Testing - -#### Tests +### Tools Testing All tools should have tests added in [tools/tests](tools/tests), tests are implemented using [Jest](https://jestjs.io/). They can be run via: ```bash @@ -322,7 +328,7 @@ npm run jest -- tools/tests/linter/lint.test.ts The test suite contains unit tests and integration tests. Integration tests, such as [these](tools/tests/tester/integ/), require a local instance of OpenSearch and are placed into a folder named `integ`. Unit tests are run in parallel and integration tests are run sequentially using `--runInBand`. You can run unit tests with `npm run test:unit` separately from integration tests with `npm run test:integ`. -#### Lints +### Tools Linting All TypeScript code and YAML files are linted using [ESLint](https://eslint.org/), [typescript-eslint](https://typescript-eslint.io/), and [eslint-plugin-yml](https://ota-meshi.github.io/eslint-plugin-yml/). Linting can be run via: ```bash diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 91b211686..65312657e 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -1,27 +1,27 @@ -- [Spec Testing Guide](#spec-testing-guide) - - [Running Spec Tests](#running-spec-tests) - - [Running Spec Tests Locally](#running-spec-tests-locally) - - [Prerequisites](#prerequisites) - - [OpenSearch Cluster](#opensearch-cluster) - - [Run Tests](#run-tests) - - [Running Spec Tests with Amazon OpenSearch](#running-spec-tests-with-amazon-opensearch) - - [Common Errors](#common-errors) - - [401 Unauthorized](#401-unauthorized) - - [FORBIDDEN/10/cluster create-index blocked (api)](#forbidden10cluster-create-index-blocked-api) - - [FAILED Cat with a json response (from security-analytics).](#failed--cat-with-a-json-response-from-security-analytics) - - [Writing Spec Tests](#writing-spec-tests) - - [Simple Test Story](#simple-test-story) - - [Using Output from Previous Chapters](#using-output-from-previous-chapters) - - [Managing Versions](#managing-versions) - - [Managing Distributions](#managing-distributions) - - [Waiting for Tasks](#waiting-for-tasks) - - [Warnings](#warnings) - - [multiple-paths-detected](#multiple-paths-detected) - - [Suppressing Warnings](#suppressing-warnings) - - [Collecting Test Coverage](#collecting-test-coverage) - - [Coverage Summary](#coverage-summary) - - [Coverage Report](#coverage-report) +* [Spec Testing Guide](#spec-testing-guide) + * [Running Spec Tests](#running-spec-tests) + * [Running Spec Tests Locally](#running-spec-tests-locally) + * [Prerequisites](#prerequisites-) + * [OpenSearch Cluster](#opensearch-cluster-) + * [Run Tests](#run-tests) + * [Running Spec Tests with Amazon OpenSearch](#running-spec-tests-with-amazon-opensearch) + * [Common Errors](#common-errors) + * [401 Unauthorized](#401-unauthorized) + * [FORBIDDEN/10/cluster create-index blocked (api)](#forbidden10cluster-create-index-blocked-api) + * [FAILED Cat with a json response (from security-analytics).](#failed--cat-with-a-json-response-from-security-analytics) + * [Writing Spec Tests](#writing-spec-tests) + * [Simple Test Story](#simple-test-story) + * [Using Output from Previous Chapters](#using-output-from-previous-chapters) + * [Managing Versions](#managing-versions) + * [Managing Distributions](#managing-distributions) + * [Waiting for Tasks](#waiting-for-tasks) + * [Warnings](#warnings) + * [multiple-paths-detected](#multiple-paths-detected) + * [Suppressing Warnings](#suppressing-warnings) + * [Collecting Test Coverage](#collecting-test-coverage) + * [Coverage Summary](#coverage-summary) + * [Coverage Report](#coverage-report) # Spec Testing Guide diff --git a/spec/_global_parameters.yaml b/spec/_global_parameters.yaml index d3e7f2806..91ddbfe8f 100644 --- a/spec/_global_parameters.yaml +++ b/spec/_global_parameters.yaml @@ -34,7 +34,7 @@ components: filter_path: name: filter_path in: query - description: Comma-separated list of filters used to reduce the response. + description: Used to reduce the response. This parameter takes a comma-separated list of filters. It supports using wildcards to match any field or part of a field’s name. You can also exclude fields with "-". schema: oneOf: - type: string diff --git a/spec/schemas/_common.yaml b/spec/schemas/_common.yaml index 22c70dbfa..a33d64dd8 100644 --- a/spec/schemas/_common.yaml +++ b/spec/schemas/_common.yaml @@ -496,7 +496,12 @@ components: SortCombinations: oneOf: - $ref: '#/components/schemas/Field' + - $ref: '#/components/schemas/FieldWithOrder' - $ref: '#/components/schemas/SortOptions' + FieldWithOrder: + type: object + additionalProperties: + $ref: '#/components/schemas/ScoreSort' SortOptions: type: object properties: diff --git a/tests/default/_core/search/sort_by_field.yaml b/tests/default/_core/search/sort_by_field.yaml new file mode 100644 index 000000000..4256929ee --- /dev/null +++ b/tests/default/_core/search/sort_by_field.yaml @@ -0,0 +1,163 @@ +$schema: ../../../../json_schemas/test_story.schema.yaml + +description: Test different ways to sort by field. + +epilogues: + - path: /sorted_movies + method: DELETE + status: [200] + +prologues: + - path: /sorted_movies + method: PUT + request: + payload: + mappings: + properties: + title: + type: keyword + year: + type: integer + + - path: /sorted_movies/_doc + method: POST + parameters: + refresh: true + request: + payload: + title: The Lion King + year: 1994 + status: [201] + - path: /sorted_movies/_doc + method: POST + parameters: + refresh: true + request: + payload: + title: The Lion King + year: 2019 + status: [201] + - path: /sorted_movies/_doc + method: POST + parameters: + refresh: true + request: + payload: + title: Frozen + year: 2013 + status: [201] + +chapters: + - synopsis: QueryString Sort by year. + path: /{index}/_search + parameters: + index: sorted_movies + sort: year + method: GET + response: + status: 200 + payload: + hits: + hits: + - _source: + title: The Lion King + year: 1994 + - _source: + title: Frozen + year: 2013 + - _source: + title: The Lion King + year: 2019 + + - synopsis: QueryString Sort by title:desc. + path: /{index}/_search + parameters: + index: sorted_movies + sort: title:desc + method: GET + response: + status: 200 + payload: + hits: + hits: + - _source: + title: The Lion King + year: 1994 + - _source: + title: The Lion King + year: 2019 + - _source: + title: Frozen + year: 2013 + + - synopsis: QueryString Sort by title:desc,year:desc. + path: /{index}/_search + parameters: + index: sorted_movies + sort: title:desc,year:desc + method: GET + response: + status: 200 + payload: + hits: + hits: + - _source: + title: The Lion King + year: 2019 + - _source: + title: The Lion King + year: 1994 + - _source: + title: Frozen + year: 2013 + + - synopsis: Body Sort by [title, year]. + path: /{index}/_search + parameters: + index: sorted_movies + method: GET + request: + payload: + sort: + - title + - year + response: + status: 200 + payload: + hits: + hits: + - _source: + title: Frozen + year: 2013 + - _source: + title: The Lion King + year: 1994 + - _source: + title: The Lion King + year: 2019 + + - synopsis: Body Sort by year:asc,title:desc. + path: /{index}/_search + parameters: + index: sorted_movies + method: GET + request: + payload: + sort: + - title: {order: asc} + - year: {order: desc} + response: + status: 200 + payload: + hits: + hits: + - _source: + title: Frozen + year: 2013 + - _source: + title: The Lion King + year: 2019 + - _source: + title: The Lion King + year: 1994 +