Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test framework/guide #359

Merged
merged 2 commits into from
Jun 25, 2024
Merged

Conversation

nhtruong
Copy link
Collaborator

@nhtruong nhtruong commented Jun 24, 2024

  • Reduced noise when running spec tests
  • --verbose now provides better context for non-2XX responses.
  • Moved linter/utils to _utils because it's also being used by the tester tool (To avoid misleading stacktrace when an error appears in utils)
  • Added SPEC_TESTING.md

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@nhtruong nhtruong force-pushed the test_framework/guide branch from a493867 to 89f31e1 Compare June 24, 2024 23:42
@nhtruong nhtruong marked this pull request as ready for review June 24, 2024 23:43
Copy link
Contributor

github-actions bot commented Jun 24, 2024

Changes Analysis

Commit SHA: c8aaf54
Comparing To SHA: feb3ea0

API Changes

Summary

NO CHANGES

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/9666746566/artifacts/1637069617

API Coverage

Before After Δ
Covered (%) 483 (47.31 %) 483 (47.31 %) 0 (0 %)
Uncovered (%) 538 (52.69 %) 538 (52.69 %) 0 (0 %)
Unknown 24 24 0

SPEC_TESTING.md Outdated Show resolved Hide resolved
SPEC_TESTING.md Outdated Show resolved Hide resolved
Comment on lines -13 to +14
import { SchemaVisitor } from './utils/SpecificationVisitor'
import { is_ref, type MaybeRef, SpecificationContext } from './utils'
import { SchemaVisitor } from '../_utils/SpecificationVisitor'
import { is_ref, type MaybeRef, SpecificationContext } from '../_utils'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for the leading underscore on _utils?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To differentiate it from other folders like tester linter merger where they are tools while _utils is not a tool.

} from '../OpenSearchHttpClient'
import ChapterReader from './ChapterReader'
import ChapterEvaluator from './ChapterEvaluator'
import OperationLocator from './OperationLocator'
import SchemaValidator from './SchemaValidator'
import StoryEvaluator from './StoryEvaluator'
import { ConsoleResultLogger } from './ResultLogger'
import {ConsoleResultLogger} from './ResultLogger'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised the linter isn't complaining about this as it should be enforcing spacing around the braces (or at least it was)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I setup my IDE to run --fix on save. I thought the linter wanted space too.

Copy link
Member

@dblock dblock Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks suspicious. Other files have a space. We definitely want 1 rule for spaces before/after {} - Something is off with the eslint rule file being used?

https://github.com/opensearch-project/opensearch-api-specification/blob/main/tools/src/coverage/coverage.ts#L12

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed it. Didn't dig too deep but object-curly-spacing rule was deprecated in ESLint v8.53.0 and moved to a different plugin. Installing the plugin and enabled that rule solved it.

- Reduced noise when running spec tests
- `--verbose` now provides better context for non-2XX responses.
- Moved `linter/utils` to `_utils` because it's also being used by the tester tool (To avoid misleading stacktrace when an error appears in utils)

Signed-off-by: Theo Truong <[email protected]>
@nhtruong nhtruong force-pushed the test_framework/guide branch from dde1822 to 0547647 Compare June 25, 2024 14:46
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nits + let's resolve this linter problem.

Feel free to break up the PR, the doc looks good.

SPECIFICATION_TESTING.md Show resolved Hide resolved
} from '../OpenSearchHttpClient'
import ChapterReader from './ChapterReader'
import ChapterEvaluator from './ChapterEvaluator'
import OperationLocator from './OperationLocator'
import SchemaValidator from './SchemaValidator'
import StoryEvaluator from './StoryEvaluator'
import { ConsoleResultLogger } from './ResultLogger'
import {ConsoleResultLogger} from './ResultLogger'
Copy link
Member

@dblock dblock Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks suspicious. Other files have a space. We definitely want 1 rule for spaces before/after {} - Something is off with the eslint rule file being used?

https://github.com/opensearch-project/opensearch-api-specification/blob/main/tools/src/coverage/coverage.ts#L12

Signed-off-by: Theo Truong <[email protected]>
@dblock dblock requested a review from Xtansia June 25, 2024 18:39
@Xtansia Xtansia merged commit 35f3a5c into opensearch-project:main Jun 25, 2024
9 of 10 checks passed
@nhtruong nhtruong deleted the test_framework/guide branch June 25, 2024 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants