Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improper invalid data incorrectly attributed to mock_uss provider #416

Open
BenjaminPelletier opened this issue Dec 17, 2023 · 0 comments
Open
Labels
automated-testing Related to automated testing tools P3 Lower priority test-scenario-behavior

Comments

@BenjaminPelletier
Copy link
Member

report.json is not attached because
An instance of this problem has not yet been observed at runtime -- rather, it can be deduced via inspection at design time.

Test check
Invalid data in Operational intent details shared by Mock USS for negative test check

Difference from expected behavior
This check is asserting that mock_uss (the control_uss resource in Data Validation of GET operational intents by USS test scenario) should have provided invalid operational intent data -- if mock_uss provides valid data, this check fails and the failure is attributed to the participant providing the mock_uss instance. However, mock_uss actually has no requirement to provide invalid data here -- instead, mock_uss's obligation is merely to respond with the data that the client (uss_qualifier) instructed it to respond with. If uss_qualifier or the test designer instructed mock_uss to respond with valid data, then the failure was caused by uss_qualifier or the test designer and not mock_uss. However, currently the failure will be incorrectly attributed to mock_uss instead.

The best way to disentangle the two responsibilities (providing invalid data, and sharing whatever data was specified) should be to change the expectation from expect_shared_with_invalid_data to expect_shared_with_specified_data (the actual requirement for mock_uss).

Additional context
This issue was raised in this thread but accepted as tech debt for #376.

@BenjaminPelletier BenjaminPelletier added automated-testing Related to automated testing tools P3 Lower priority test-scenario-behavior labels Dec 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automated-testing Related to automated testing tools P3 Lower priority test-scenario-behavior
Projects
None yet
Development

No branches or pull requests

1 participant