[uss_qualifier] Organize DSS tests #218
Conversation
There was a problem hiding this comment.
Side note: depending on which gets merged first between this PR and #204, the service_area resource in dss_probing.yaml will need to be updated.
Side note bis: this test framework is really becoming powerful and in a mostly generic way, have you ever considered factoring it out in a project of its own at some point? :)
monitoring/uss_qualifier/suites/astm/netrid/f3411_19/dss_probing.yaml
Outdated
Show resolved
Hide resolved
| @@ -73,6 +73,15 @@ def __init__( | |||
| if local_debug is not None: | |||
| self.local_debug = local_debug | |||
|
|
|||
| def is_same_as(self, other: DSSInstance) -> bool: | |||
There was a problem hiding this comment.
nit: Shouldn't comparing the participant IDs be enough? It looks like if there are different DSSs declared with the same participant ID, of the same DSSs declared multiple times with different configuration, it should error out before.
There was a problem hiding this comment.
It should be relatively common for a single participant to provide multiple instances -- InterUSS's DSS needs an odd number of instances and some redundancy is good practice, so the baseline starting instance count for a normal deployment is 3. If there are fewer than 3 USSs in the ecosystem who want to help host the DSS pool, then one participant will necessarily provide multiple instances. I think it should be fine to provide multiple DSS instances from the same participant (both conceptually and in the current implementation)? If not, I'll follow up with a fix in another PR.
There was a problem hiding this comment.
Ohh I didn't think at all about the 1 participant having multiple instances, that makes sense yes.
This PR establishes a new test suite to validate the InterUSS DSS implementation which implements DSS requirements across three standards (though we only currently have dedicated uss_qualifier DSS tests for F3411-19 and F3411-22a) and makes the resource definitions in the configuration explicit rather than $ref'd so the full content is easily accessible.
It also regroups the DSS testing for F3411-19 and F3411-22a to run the interoperability test for all participants acting as primary DSS -- this allows a number of requirements to be verified for all participants that previously weren't checked when only one participant's DSS acted as primary DSS. In order to accomplish this, an optional resource specifying all the other DSS instances is declared in each of the standard's dss_probing test suites and used for the interoperability scenario if present. To support cases where someone wants to test only a single DSS instance, support is added for skipping test suite actions that don't have the necessary resources specified (see suite README.md).
Finally, some test suite documentation_urls are populated and an off-by-one error corrected.