[uss_qualifier] DSS0030 port ISA-subscriptions interactions test

[Shastick]

This ports the existing test_subscription_isa_interaction.py prober test to a generic ISASubscriptionInteractions scenario.

This also adds a delete_any_subscription function to monitoring/uss_qualifier/scenarios/astm/netrid/common/dss/utils.py

A few API methods have been added to requirements/interuss/f3411/dss_endpoints.md, as this scenario depends on implementation details specified in the A4.1 annex that are not mentioned in the standard.

[Shastick]

Two notes/questions here:

• I'm not checking the notification query to our own subscription because it fails, as it assumes https is available. As we are querying ourselves anyway, failure or success on that query does not give us any additional information about the DSS we are testing. Happy to dig further into this if required. (see log excerpt below)

• subscriber_url is a bad stand-in for identifying a failing participant here: it will cause the qualifier to fail when writing test reports, when it attempts to find a directory that has the faulty URL in its path (I kept things as-is for now as this is done this way in every place that notifies ISA subscribers and deserves a separate PR to properly fix it)

For reference, here's a warning we get when failing to notify ourselves:

2023-10-26 16:32:44.610 | WARNING | monitoring.monitorlib.fetch:query_and_describe:361 - query_and_describe attempt 1 from PID 1 to POST https://uss_qualifier.test.utm/dummy_base_url/v1/uss/identification_service_areas/00000172-e36d-40be-d38b-eca6ca350000 failed with non-retryable RequestException ConnectionError: HTTPSConnectionPool(host='uss_qualifier.test.utm', port=443): Max retries exceeded with url: /dummy_base_url/v1/uss/identification_service_areas/00000172-e36d-40be-d38b-eca6ca350000 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xffff4ab2a130>: Failed to establish a new connection: [Errno -2] Name or service not known'))

And here's the exception the qualifier fails with at the end when we use a URL as a participant_id:

Traceback (most recent call last):
  File "main.py", line 299, in <module>
    sys.exit(main())
  File "main.py", line 284, in main
    run_config(
  File "main.py", line 231, in run_config
    generate_tested_requirements(redacted_report, tested_reqs_config, path)
  File "/app/monitoring/uss_qualifier/reports/tested_requirements.py", line 243, in generate_tested_requirements
    with open(participant_file, "w") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'output/netrid_v19/requirements/https://uss_qualifier.test.utm/dummy_base_url/v1/uss/identification_service_areas.html'

[BenjaminPelletier]

I don't think we'd want to create new ParticipantIDs on the fly -- they're supposed to refer to participants in the test rather than, for instance, bystanders that the test happened to interact with. I do think we want to identify the participant affecting a query or check whenever we reasonably can, but I don't think that's possible in all cases. When it's not currently feasible to link a query or check with a test-designer-specified participant (or, perhaps, a "special" participant like the otherwise-unnamed test designer) I think we'd just not specify participant IDs for those actions.

[Shastick]

Ok, in that case I'll defer to the query's participant_id field.

This moves the responsibility for finding out which participant is being queried to mutate/rid.py (here).

Should I look into how to best do this, and fix any other place that uses the qualifier URL as the participant ID?

[BenjaminPelletier]

Yes, I think we should only have ParticipantIDs by the IDs specified by the test designer, or perhaps in special cases, identify a few special ParticipantIDs (e.g., "uss_qualifier"/"test_runner"/"test_owner"/"test_designer"). When we have ParticipantIDs that aren't in this set, yes, I think we should probably adjust them.

In general, we should definitely prioritize fixing any place where the participant ID is set to the wrong participant (it would be bad to attribute failures or successes to the wrong participant). It would be good to attribute all feasible checks to a particular participant (it's hard to know what to do when something fails but no one's responsible). Attributing queries to particular participants is nice and we should do it when possible, but I think we should only go out of our ways to update existing code when that update might affect the outcome of a check (e.g., a performance requirement). Attributing things to a participant the test designer doesn't recognize can be confusing -- along the same kind of lines as unattributed checks & queries since the test designer will have to do some manual analysis to understand who might be responsible.

[Shastick]

Good, then I'll do a quick pass on the places that were setting a URL as the participant ID without changing anything else: now that some scenarios are creating subscriptions, crashes of the qualifier become much more likely.

[Shastick]

Side question:

the standard does not mention anything related to an area being queried. However, in order to list the subscriptions from the DSS, we need to provide an area in which to search (the Swagger spec from the A4.1 annex mentions that the area is a required parameter).

For reference, DSS0030-f reads:

GET Subscriptions—this interface enables a Net-RID Display Provider to retrieve the details of all existing subscriptions it created.

A valid interpretation thereof is that there should be a way to list all created subscriptions without specifying an area.

As the swagger spec makes the requirement a bit stricter, should we add an entry to /requirements/interuss/f3411/dss_endpoints.md and mention it here?

[Shastick]

Given that I had other reasons to update dss_endpoints.md I went ahead and added a SearchSubscriptionsentry there.

[BenjaminPelletier]

When the preface is included, I think the requirement should be fairly clear:

(3) A DSS implementation shall (DSS0030) minimally
include the following interfaces for use by Net-RID Service
Providers and Display Providers, in accordance with the DSS
portion of the OpenAPI specification presented in Annex A4:
[...]
(f) GET Subscriptions—this interface enables a Net-RID
Display Provider to retrieve the details of all existing subscrip-
tions it created.

The (f) text doesn't specify the need to provide an area, but the preface says each of the subparts (including f) is in accordance with Annex A4, and Annex A4 requires an area. So, I think DSS0030,f should be fine and we don't need an additional requirement. We do need an additional requirement to, e.g., retrieve one specific ISA (GET by ID) since only PUT and DELETE are listed as requirements in the standard (I think this is a simple oversight, but regardless, it doesn't appear to be a requirement in F3411-22a).

[Shastick]

Ok, thanks. Looks like I took "the requirement needs to be listed in the standard" a bit too literally :) I'll update the relevant docs.

[Shastick]

Several of the subscription scenarios have a need to clean up: I made a note to go over them once they are merged to see if this logic can be regrouped a bit.