Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

F3411-19 requirements listed in F3411-22a test suite #352

Closed
BenjaminPelletier opened this issue Nov 16, 2023 · 2 comments
Closed

F3411-19 requirements listed in F3411-22a test suite #352

BenjaminPelletier opened this issue Nov 16, 2023 · 2 comments
Assignees
Labels
automated-testing Related to automated testing tools bug Something isn't working P2 Normal priority test-scenario-behavior

Comments

@BenjaminPelletier
Copy link
Member

report.json is not attached because
No need for a report, the issue is in the test suite documentation for F3411-22a.

Test check
Multiple; see below

Difference from expected behavior
The F3411-22a test suite should only test requirements from the -22a standard version, not any from the -19 standard version. But, we have a few -19 requirements listed as tested by the test suite. This is likely due to copy/paste from -19 not being adjusted in the -22a documentation.

We should update the documentation as appropriate so that the F3411-22a test suite does not list any -19 requirements as being potentially checked by that test suite. Test suite documentation is autogenerated using make format and ultimately uses the test scenario documentation to identified potentially-checked requirements.

@BenjaminPelletier BenjaminPelletier added bug Something isn't working automated-testing Related to automated testing tools P2 Normal priority test-scenario-behavior labels Nov 16, 2023
@Shastick
Copy link
Contributor

Will fix.

Note to self: I'll check if I can add a simple check to the linting target(s), eg something like find ./scenarios/astm/netrid/v22a/ -name "*.md" -exec grep -l "v19" {} + as a sanity check, as I don't believe there is any valid reason for a version of the requirement to point to the other one.

@Shastick
Copy link
Contributor

Should be fixed by #354, which should also make this unlikely to impossible to reoccur.

github-actions bot added a commit that referenced this issue Nov 18, 2023
…v22a or vice-versa (and fix #352) (#354)

[uss_qualifier] Make target that checks if a v19 doc file references v22a or vice-versa 190def8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automated-testing Related to automated testing tools bug Something isn't working P2 Normal priority test-scenario-behavior
Projects
None yet
Development

No branches or pull requests

2 participants