diff --git a/build/dev/docker-compose.yaml b/build/dev/docker-compose.yaml index 113a5af513..8ee88e50bd 100644 --- a/build/dev/docker-compose.yaml +++ b/build/dev/docker-compose.yaml @@ -20,7 +20,7 @@ services: - dss_internal_network rid_bootstrapper: - image: interuss/dss:v0.13.0 + image: interuss/dss:v0.14.0 volumes: - dss_component_coordination:/var/dss_component_coordination - $PWD/startup:/startup:ro @@ -31,7 +31,7 @@ services: - dss_internal_network scd_bootstrapper: - image: interuss/dss:v0.13.0 + image: interuss/dss:v0.14.0 volumes: - dss_component_coordination:/var/dss_component_coordination - $PWD/startup:/startup:ro @@ -43,7 +43,7 @@ services: dss: hostname: dss.uss1.localutm - image: interuss/dss:v0.13.0 + image: interuss/dss:v0.14.0 volumes: - $PWD/../test-certs:/var/test-certs:ro - dss_component_coordination:/var/dss_component_coordination diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_correct.md similarity index 61% rename from monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search.md rename to monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_correct.md index a3cf999c14..f15740c7ec 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_correct.md @@ -2,9 +2,9 @@ This test step fragment validates that subscriptions can be searched for. -## 🛑 Successful subscription search query check +## [Search query succeeds](./search_query.md) -If the DSS fails to let us search in the area for which the subscription was created, it is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**. +Check query succeeds. ## 🛑 Created Subscription is in search results check diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_query.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_query.md new file mode 100644 index 0000000000..28f6356596 --- /dev/null +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_query.md @@ -0,0 +1,7 @@ +# Search subscription query test step fragment + +This test step fragment validates that a query to search for subscriptions succeeds. + +## 🛑 Successful subscription search query check + +If the DSS fails to let us search in the area for which the subscription was created, it is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**. diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_correct.md similarity index 79% rename from monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update.md rename to monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_correct.md index 0221cf46d8..d23900229d 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_correct.md @@ -2,9 +2,9 @@ This test step fragment validates that subscriptions can be updated. -## 🛑 Subscription can be mutated check +## [Update query succeeds](./update_query.md) -If a subscription cannot be modified with a valid set of parameters, the DSS is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**. +Check query succeeds. ## 🛑 Mutate subscription response format conforms to spec check diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_query.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_query.md new file mode 100644 index 0000000000..a7c237588f --- /dev/null +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_query.md @@ -0,0 +1,7 @@ +# Update subscription query test step fragment + +This test step fragment validates that a query to update a subscription succeeds. + +## 🛑 Subscription can be mutated check + +If a subscription cannot be updated with a valid set of parameters, the DSS is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**. diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.md index ffc799cae2..9d9fbf7128 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.md @@ -126,4 +126,33 @@ The subscription created on a DSS instance must be retrievable from all other DS If the subscription does not exist on one of the other DSS instances, one of the instances fails to comply with **[astm.f3548.v21.DSS0210,A2-7-2,4a](../../../../requirements/astm/f3548/v21.md)**. + +## Expiration of subscriptions removes them test case + +This test case validates that expired subscriptions (created explicitly) are removed from all DSS instances. +To validate this requirement, the subscriptions that were previously created at each DSS instance are modified so that they expire shortly after the modification. +Then it is checked that all the associated subscriptions were removed from all the DSS instances by searching for them in their planning area. +Do note that they are not queried directly, as it is deemed acceptable for expired subscription that were not explicitly deleted to still be retrievable. + +### Expire explicit subscriptions at every DSS in sequence test step + +This test step will modify explicit subscriptions that were previously created at each DSS instance so that they expire shortly after the modification. + +Note that this step is run once for each involved DSS (that is, once for the primary DSS and once for every secondary DSS) + +#### [Modify subscription on a DSS instance so that it expires soon](./fragments/sub/crud/update_query.md) + +Check that the subscription modifications succeed and wait for them to expire. + +#### [Search for subscriptions from all other DSS instances succeeds](./fragments/sub/crud/search_query.md) + +Check that query succeeds. + +#### 🛑 Subscription does not exist on all other DSS instances check + +The explicit subscription expired on a DSS instance must have been removed from all other DSS instances. + +If the subscription still exists on one of the other DSS instances, one of the instances fails to comply with **[astm.f3548.v21.DSS0210,A2-7-2,4d](../../../../requirements/astm/f3548/v21.md)**. + + ## [Cleanup](./clean_workspace.md) diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.py b/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.py index 709fea0512..bad6512c35 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.py +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.py @@ -12,8 +12,10 @@ from uas_standards.astm.f3548.v21.constants import Scope from monitoring.monitorlib import fetch +from monitoring.monitorlib.delay import sleep from monitoring.monitorlib.fetch import QueryError from monitoring.monitorlib.geotemporal import Volume4D +from monitoring.monitorlib.temporal import Time from monitoring.prober.infrastructure import register_resource_type from monitoring.uss_qualifier.configurations.configuration import ParticipantID from monitoring.uss_qualifier.resources.astm.f3548.v21 import PlanningAreaResource @@ -55,7 +57,7 @@ class SubscriptionInteractions(TestScenario): # Reference times for the subscriptions and operational intents _time_start: datetime - _time_send: datetime + _time_end: datetime _manager: str @@ -120,6 +122,9 @@ def run(self, context: ExecutionContext): self._steps_create_subs_at_each_dss() self.end_test_case() + self.begin_test_case("Expiration of subscriptions removes them") + self._steps_expire_subs_at_each_dss() + self.end_test_case() self.end_test_scenario() def _step_create_background_sub(self): @@ -129,7 +134,7 @@ def _step_create_background_sub(self): sub_now_params = self._planning_area.get_new_subscription_params( subscription_id=self._background_sub_id, start_time=self._time_start, - duration=self._time_send - self._time_start, + duration=self._time_end - self._time_start, # This is a planning area without constraint processing notify_for_op_intents=True, notify_for_constraints=False, @@ -188,7 +193,7 @@ def _implicit_subs_check( state=OperationalIntentState.Accepted, uss_base_url="https://example.interuss.org/oir_base_url", time_start=datetime.utcnow(), - time_end=self._time_send + timedelta(minutes=10), + time_end=self._time_end + timedelta(minutes=10), subscription_id=None, implicit_sub_base_url="https://example.interuss.org/sub_base_url", ) @@ -235,7 +240,7 @@ def _implicit_subs_check( state=OperationalIntentState.Accepted, uss_base_url="https://example.interuss.org/oir_base_url_bis", # dummy modification of the OIR time_start=datetime.utcnow(), - time_end=self._time_send + timedelta(minutes=10), + time_end=self._time_end + timedelta(minutes=10), subscription_id=self._current_oirs[oir_id].subscription_id, ) @@ -283,7 +288,7 @@ def _steps_create_subs_at_each_dss(self): common_params = self._planning_area.get_new_subscription_params( subscription_id="", start_time=self._time_start, - duration=self._time_send - self._time_start, + duration=self._time_end - self._time_start, notify_for_op_intents=True, notify_for_constraints=False, ) @@ -342,6 +347,76 @@ def _steps_create_subs_at_each_dss(self): self.end_test_step() + def _steps_expire_subs_at_each_dss(self): + self.begin_test_step("Expire explicit subscriptions at every DSS in sequence") + for i, dss in enumerate([self._dss] + self._secondary_instances): + sub_id = self._sub_ids[i] + sub_params = self._planning_area.get_new_subscription_params( + subscription_id=sub_id, + start_time=datetime.utcnow(), + duration=timedelta(seconds=SUBSCRIPTION_EXPIRY_DELAY_SEC), + notify_for_op_intents=True, + notify_for_constraints=False, + ) + + with self.check( + "Subscription can be mutated", + [dss.participant_id], + ) as check: + sub = self._dss.upsert_subscription( + **sub_params, + version=self._current_subs[sub_id].version, + ) + self.record_query(sub) + if not sub.success: + check.record_failed( + summary="Update subscription query failed", + details=f"Failed to update a subscription on DSS instance with code {sub.status_code}: {sub.error_message}", + query_timestamps=[sub.request.timestamp], + ) + self._current_subs.pop(sub_id) + + sleep( + timedelta(seconds=WAIT_FOR_EXPIRY_SEC), + "waiting for subscriptions to expire", + ) + + for i, dss in enumerate([self._dss] + self._secondary_instances): + sub_id = self._sub_ids[i] + for other_dss in {self._dss, *self._secondary_instances} - {dss}: + other_dss_subs = other_dss.query_subscriptions( + Volume4D( + volume=self._planning_area.volume, + time_start=Time(self._time_start), + time_end=Time(self._time_end), + ).to_f3548v21() + ) + self.record_query(other_dss_subs) + + with self.check( + "Successful subscription search query", + dss.participant_id, + ) as check: + if not other_dss_subs.success: + check.record_failed( + summary="Search subscriptions query failed", + details=f"Failed to search for subscriptions from DSS with code {other_dss_subs.status_code}: {other_dss_subs.error_message}", + query_timestamps=[other_dss_subs.request.timestamp], + ) + + with self.check( + "Subscription does not exist on all other DSS instances", + [dss.participant_id, other_dss.participant_id], + ) as check: + if sub_id in other_dss_subs.subscriptions: + check.record_failed( + summary="Subscription that expired on a DSS instance was found on another instance", + details=f"Subscription {sub_id} expired on DSS instance {dss.participant_id} was found on DSS instance {other_dss.participant_id}.", + query_timestamps=[other_dss_subs.request.timestamp], + ) + + self.end_test_step() + def _create_sub_with_params( self, params: SubscriptionParams ) -> Tuple[Subscription, List[OperationalIntentReference], fetch.Query]: @@ -362,7 +437,7 @@ def _setup_case(self): # Subscription from now to 20 minutes in the future self._time_start = datetime.utcnow() - self._time_send = self._time_start + timedelta(minutes=20) + self._time_end = self._time_start + timedelta(minutes=20) # Multiple runs of the scenario seem to rely on the same instance: # thus we need to reset the state of the scenario before running it. diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md index bd82e15247..c51719386e 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md @@ -57,6 +57,12 @@ Retrieve and validate synchronization of the created operational intent at every Check that read query succeeds. +#### Newly created OIR can be consistently retrieved from all DSS instances check + +If the operational intent retrieved from a secondary DSS instance is not consistent with the newly created one on the +primary DSS instance, this check will fail per **[astm.f3548.v21.DSS0210,A2-7-2,1a](../../../../../requirements/astm/f3548/v21.md)** +and **[astm.f3548.v21.DSS0210,A2-7-2,1d](../../../../../requirements/astm/f3548/v21.md)**. + #### [OIR is synchronized](../fragments/oir/sync.md) Confirm that each DSS provides direct access to the created operational intent reference. @@ -70,6 +76,12 @@ Search for and validate synchronization of the created operational intent at eve Check that search query succeeds. +#### Newly created OIR can be consistently searched for from all DSS instances check + +If the operational intent searched from a secondary DSS instance is not consistent with the newly created one on the +primary DSS instance, this check will fail per **[astm.f3548.v21.DSS0210,A2-7-2,1a](../../../../../requirements/astm/f3548/v21.md)** +and **[astm.f3548.v21.DSS0210,A2-7-2,1c](../../../../../requirements/astm/f3548/v21.md)**. + #### [OIR is synchronized](../fragments/oir/sync.md) Confirm that each DSS returns the operational intent in relevant search results. @@ -100,6 +112,12 @@ Retrieve and validate synchronization of the updated operational intent at every Check that read query succeeds. +#### Updated OIR can be consistently retrieved from all DSS instances check + +If the operational intent retrieved from a secondary DSS instance is not consistent with the updated one on the +primary DSS instance, this check will fail per **[astm.f3548.v21.DSS0210,A2-7-2,1b](../../../../../requirements/astm/f3548/v21.md)** +and **[astm.f3548.v21.DSS0210,A2-7-2,1d](../../../../../requirements/astm/f3548/v21.md)**. + #### [OIR is synchronized](../fragments/oir/sync.md) Confirm that each DSS provides direct access to the updated operational intent reference. @@ -113,6 +131,12 @@ Search for and validate synchronization of the updated operational intent at eve Check that search query succeeds. +#### Updated OIR can be consistently searched for from all DSS instances check + +If the operational intent searched from a secondary DSS instance is not consistent with the updated one on the +primary DSS instance, this check will fail per **[astm.f3548.v21.DSS0210,A2-7-2,1b](../../../../../requirements/astm/f3548/v21.md)** +and **[astm.f3548.v21.DSS0210,A2-7-2,1c](../../../../../requirements/astm/f3548/v21.md)**. + #### [OIR is synchronized](../fragments/oir/sync.md) Confirm that each DSS returns the operational intent in relevant search results. @@ -140,9 +164,24 @@ Verify that the operational intent reference's version fields are as expected. Attempt to query and search for the deleted operational intent reference in various ways -#### 🛑 Secondary DSS should not return the deleted operational intent reference check +#### [Get OIR query](../fragments/oir/crud/read_query.md) + +Check that read query succeeds. + +#### 🛑 Deleted OIR cannot be retrieved from all DSS instances check + +If a DSS returns an operational intent reference that was previously successfully deleted from the primary DSS, +either one of the primary DSS or the DSS that returned the operational intent reference is in violation of **[astm.f3548.v21.DSS0210,2a](../../../../../requirements/astm/f3548/v21.md)** +and **[astm.f3548.v21.DSS0210,A2-7-2,3b](../../../../../requirements/astm/f3548/v21.md)**. + +#### [Search OIR](../fragments/oir/crud/search_query.md) + +Check that search query succeeds. + +#### 🛑 Deleted OIR cannot be searched for from all DSS instances check If a DSS returns an operational intent reference that was previously successfully deleted from the primary DSS, -either one of the primary DSS or the DSS that returned the operational intent reference is in violation of **[astm.f3548.v21.DSS0210,2a](../../../../../requirements/astm/f3548/v21.md)**. +either one of the primary DSS or the DSS that returned the operational intent reference is in violation of **[astm.f3548.v21.DSS0210,2a](../../../../../requirements/astm/f3548/v21.md)** +and **[astm.f3548.v21.DSS0210,A2-7-2,3a](../../../../../requirements/astm/f3548/v21.md)**. ## [Cleanup](../clean_workspace.md) diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.py b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.py index 7b4c3b9399..b1b333ee8a 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.py +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.py @@ -133,11 +133,17 @@ def run(self, context: ExecutionContext): self.end_test_step() self.begin_test_step("Retrieve newly created OIR") - self._query_secondaries_and_compare(self._oir_params) + self._query_secondaries_and_compare( + "Newly created OIR can be consistently retrieved from all DSS instances", + self._oir_params, + ) self.end_test_step() self.begin_test_step("Search for newly created OIR") - self._search_secondaries_and_compare(self._oir_params) + self._search_secondaries_and_compare( + "Newly created OIR can be consistently searched for from all DSS instances", + self._oir_params, + ) self.end_test_step() self.begin_test_step("Mutate OIR") @@ -145,11 +151,17 @@ def run(self, context: ExecutionContext): self.end_test_step() self.begin_test_step("Retrieve updated OIR") - self._query_secondaries_and_compare(self._oir_params) + self._query_secondaries_and_compare( + "Updated OIR can be consistently retrieved from all DSS instances", + self._oir_params, + ) self.end_test_step() self.begin_test_step("Search for updated OIR") - self._search_secondaries_and_compare(self._oir_params) + self._search_secondaries_and_compare( + "Updated OIR can be consistently searched for from all DSS instances", + self._oir_params, + ) self.end_test_step() self.begin_test_step("Delete OIR") @@ -231,7 +243,9 @@ def _create_oir_with_params( self._current_oir = oir def _query_secondaries_and_compare( - self, expected_oir_params: PutOperationalIntentReferenceParameters + self, + main_check_name: str, + expected_oir_params: PutOperationalIntentReferenceParameters, ): for secondary_dss in self._dss_read_instances: with self.check( @@ -241,12 +255,12 @@ def _query_secondaries_and_compare( try: oir, q = secondary_dss.get_op_intent_reference(self._oir_id) self.record_query(q) - except QueryError as e: - self.record_queries(e.queries) + except QueryError as qe: + self.record_queries(qe.queries) check.record_failed( summary="GET for operational intent reference failed", - details=f"Query for operational intent reference failed: {e.msg}", - query_timestamps=e.query_timestamps, + details=f"Query for operational intent reference failed: {qe.msg}", + query_timestamps=qe.query_timestamps, ) involved_participants = list( @@ -260,20 +274,27 @@ def _query_secondaries_and_compare( if q.status_code != 200: check.record_failed( summary="Requested operational intent was not found at secondary DSS.", - details=f"Query for operational intent reference {self._oir_id} failed: {e.msg}", - query_timestamps=e.query_timestamps, + details=f"Query for operational intent reference {self._oir_id} failed: {q.msg}", + query_timestamps=[q.request.timestamp], ) - self._validate_oir_from_secondary( - oir=oir, - q=q, - expected_oir_params=expected_oir_params, - involved_participants=involved_participants, - ) + self._validate_oir_from_secondary( + oir=oir, + q=q, + expected_oir_params=expected_oir_params, + main_check_name=main_check_name, + involved_participants=involved_participants, + ) def _search_secondaries_and_compare( - self, expected_oir_params: PutOperationalIntentReferenceParameters + self, + main_check_name: str, + expected_oir_params: PutOperationalIntentReferenceParameters, ): + """ + Returns: + True if all checks passed, False otherwise; the participant IDs if the checks did not pass. + """ for secondary_dss in self._dss_read_instances: with self.check( "Successful operational intent reference search query", @@ -311,12 +332,13 @@ def _search_secondaries_and_compare( query_timestamps=[q.request.timestamp], ) - self._validate_oir_from_secondary( - oir=oir, - q=q, - expected_oir_params=expected_oir_params, - involved_participants=involved_participants, - ) + self._validate_oir_from_secondary( + oir=oir, + q=q, + expected_oir_params=expected_oir_params, + main_check_name=main_check_name, + involved_participants=involved_participants, + ) # TODO: craft a search with an area of interest that does not intersect with the planning area, # but whose convex hull intersects with the planning area @@ -326,74 +348,88 @@ def _validate_oir_from_secondary( oir: OperationalIntentReference, q: Query, expected_oir_params: PutOperationalIntentReferenceParameters, + main_check_name: str, involved_participants: List[str], ): - with self.check( - "Propagated operational intent reference contains the correct manager", - involved_participants, - ) as check: - if oir.manager != self._expected_manager: - check.record_failed( - summary="Propagated OIR has an incorrect manager", - details=f"Expected: {self._expected_manager}, Received: {oir.manager}", - query_timestamps=[q.request.timestamp], - ) + # TODO: this main check mechanism may be removed if we are able to specify requirements to be validated in test step fragments - with self.check( - "Propagated operational intent reference contains the correct USS base URL", - involved_participants, - ) as check: - if oir.uss_base_url != expected_oir_params.uss_base_url: - check.record_failed( - "Propagated OIR has an incorrect USS base URL", - details=f"Expected: {expected_oir_params.base_url}, Received: {oir.uss_base_url}", - query_timestamps=[q.request.timestamp], - ) + with self.check(main_check_name, involved_participants) as main_check: + with self.check( + "Propagated operational intent reference contains the correct manager", + involved_participants, + ) as check: + if oir.manager != self._expected_manager: + check_args = dict( + summary="Propagated OIR has an incorrect manager", + details=f"Expected: {self._expected_manager}, Received: {oir.manager}", + query_timestamps=[q.request.timestamp], + ) + check.record_failed(**check_args) + main_check.record_failed(**check_args) - with self.check( - "Propagated operational intent reference contains the correct state", - involved_participants, - ) as check: - if oir.state != expected_oir_params.state: - check.record_failed( - summary="Propagated OIR has an incorrect state", - details=f"Expected: {expected_oir_params.state}, Received: {oir.state}", - query_timestamps=[q.request.timestamp], - ) + with self.check( + "Propagated operational intent reference contains the correct USS base URL", + involved_participants, + ) as check: + if oir.uss_base_url != expected_oir_params.uss_base_url: + check_args = dict( + summary="Propagated OIR has an incorrect USS base URL", + details=f"Expected: {expected_oir_params.base_url}, Received: {oir.uss_base_url}", + query_timestamps=[q.request.timestamp], + ) + check.record_failed(**check_args) + main_check.record_failed(**check_args) - expected_volume_collection = Volume4DCollection.from_interuss_scd_api( - expected_oir_params.extents - ) - expected_end = expected_volume_collection.time_end.datetime - expected_start = expected_volume_collection.time_start.datetime - with self.check( - "Propagated operational intent reference contains the correct start time", - involved_participants, - ) as check: - if ( - abs(oir.time_start.value.datetime - expected_start).total_seconds() - > TIME_TOLERANCE_SEC - ): - check.record_failed( - "Propagated OIR has an incorrect start time", - details=f"Expected: {expected_start}, Received: {oir.time_start}", - query_timestamps=[q.request.timestamp], - ) + with self.check( + "Propagated operational intent reference contains the correct state", + involved_participants, + ) as check: + if oir.state != expected_oir_params.state: + check_args = dict( + summary="Propagated OIR has an incorrect state", + details=f"Expected: {expected_oir_params.state}, Received: {oir.state}", + query_timestamps=[q.request.timestamp], + ) + check.record_failed(**check_args) + main_check.record_failed(**check_args) - with self.check( - "Propagated operational intent reference contains the correct end time", - involved_participants, - ) as check: - if ( - abs(oir.time_end.value.datetime - expected_end).total_seconds() - > TIME_TOLERANCE_SEC - ): - check.record_failed( - "Propagated OIR has an incorrect end time", - details=f"Expected: {expected_end}, Received: {oir.time_end}", - query_timestamps=[q.request.timestamp], - ) + expected_volume_collection = Volume4DCollection.from_interuss_scd_api( + expected_oir_params.extents + ) + expected_end = expected_volume_collection.time_end.datetime + expected_start = expected_volume_collection.time_start.datetime + with self.check( + "Propagated operational intent reference contains the correct start time", + involved_participants, + ) as check: + if ( + abs(oir.time_start.value.datetime - expected_start).total_seconds() + > TIME_TOLERANCE_SEC + ): + check_args = dict( + summary="Propagated OIR has an incorrect start time", + details=f"Expected: {expected_start}, Received: {oir.time_start}", + query_timestamps=[q.request.timestamp], + ) + check.record_failed(**check_args) + main_check.record_failed(**check_args) + + with self.check( + "Propagated operational intent reference contains the correct end time", + involved_participants, + ) as check: + if ( + abs(oir.time_end.value.datetime - expected_end).total_seconds() + > TIME_TOLERANCE_SEC + ): + check_args = dict( + summary="Propagated OIR has an incorrect end time", + details=f"Expected: {expected_end}, Received: {oir.time_end}", + query_timestamps=[q.request.timestamp], + ) + check.record_failed(**check_args) + main_check.record_failed(**check_args) def _test_mutate_oir_shift_time(self): """Mutate the OIR by adding 10 seconds to its start and end times. @@ -497,22 +533,57 @@ def _test_get_deleted_oir(self): def _confirm_secondary_has_no_oir(self, secondary_dss: DSSInstance): with self.check( - "Secondary DSS should not return the deleted operational intent reference", - [secondary_dss.participant_id], + "Get operational intent reference by ID", + secondary_dss.participant_id, ) as check: try: oir, q = secondary_dss.get_op_intent_reference(self._oir_id) self.record_query(q) - status = q.status_code - q_ts = [q.request.timestamp] except QueryError as qe: - status = qe.cause_status_code - q_ts = qe.query_timestamps[-1] - if status != 404: + q = qe.cause + self.record_query(q) + if q.status_code != 404: + check.record_failed( + summary="GET for operational intent reference failed", + details=f"Query for operational intent reference failed: {qe.msg}", + query_timestamps=qe.query_timestamps, + ) + + with self.check( + "Deleted OIR cannot be retrieved from all DSS instances", + [self._primary_pid, secondary_dss.participant_id], + ) as check: + if q.status_code != 404: + check.record_failed( + summary="Secondary DSS still has the deleted operational intent reference", + details=f"Expected 404, received {q.status_code}", + query_timestamps=[q.request.timestamp], + ) + + with self.check( + "Successful operational intent reference search query", + [secondary_dss.participant_id], + ) as check: + try: + oirs, q = secondary_dss.find_op_intent(self._planning_area_volume4d) + self.record_query(q) + except QueryError as qe: + self.record_queries(qe.queries) + check.record_failed( + summary="Failed to search for operational intent references", + details=f"Failed to query operational intent references: got response code {qe.cause_status_code}: {qe.msg}", + query_timestamps=qe.query_timestamps, + ) + + with self.check( + "Deleted OIR cannot be searched for from all DSS instances", + [self._primary_pid, secondary_dss.participant_id], + ) as check: + if self._oir_id in oirs: check.record_failed( - "Secondary DSS still has the deleted operational intent reference", - details=f"Expected 404, received {status}", - query_timestamps=q_ts, + summary="Secondary DSS still has the deleted operational intent reference", + details=f"OIR {self._oir_id} was found in the secondary DSS when searched for its expected geo-temporal extent", + query_timestamps=[q.request.timestamp], ) def cleanup(self): diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/subscription_synchronization.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/subscription_synchronization.md index 19c3874d0c..db8feb1229 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/subscription_synchronization.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/subscription_synchronization.md @@ -91,7 +91,7 @@ Confirm that the subscription that was just created is properly synchronized acr Confirms that each DSS provides access to the created subscription, -#### [Search subscription](../fragments/sub/crud/search.md) +#### [Search subscription](../fragments/sub/crud/search_correct.md) Confirms that each DSS returns the created subscription when searched for. @@ -108,7 +108,7 @@ Verify that the version of the subscription returned by every DSS is as expected This test step mutates the previously created subscription, by accessing the primary DSS, to verify that the update is propagated to all other DSSes. Notably, it checks that the subscription version is updated, including for changes that are not directly visible, such as changing the subscription's footprint. -#### [Update subscription](../fragments/sub/crud/update.md) +#### [Update subscription](../fragments/sub/crud/update_correct.md) Confirm that the subscription can be mutated. @@ -132,7 +132,7 @@ Confirm that the subscription that was just mutated is properly synchronized acr Confirms that the subscription that was just mutated can be retrieved from any DSS. -#### [Search subscription](../fragments/sub/crud/search.md) +#### [Search subscription](../fragments/sub/crud/search_correct.md) Confirms that the subscription that was just mutated can be searched for from any DSS. @@ -163,7 +163,7 @@ When queried for a subscription that was created via another DSS, a DSS instance If it does not, it might be in violation of **[astm.f3548.v21.DSS0005,5](../../../../../requirements/astm/f3548/v21.md)**. -#### [Update subscription](../fragments/sub/crud/update.md) +#### [Update subscription](../fragments/sub/crud/update_correct.md) Confirm that the secondary DSS handles the update properly. @@ -189,7 +189,7 @@ Confirm that the subscription that was just mutated is properly synchronized acr Confirms that the subscription that was just mutated can be retrieved from any DSS, and that it has the expected content. -#### [Search subscription](../fragments/sub/crud/search.md) +#### [Search subscription](../fragments/sub/crud/search_correct.md) Confirms that the subscription that was just mutated can be searched for from any DSS, and that it has the expected content. diff --git a/monitoring/uss_qualifier/suites/astm/utm/dss_probing.md b/monitoring/uss_qualifier/suites/astm/utm/dss_probing.md index 74db9b71e0..f864e35efb 100644 --- a/monitoring/uss_qualifier/suites/astm/utm/dss_probing.md +++ b/monitoring/uss_qualifier/suites/astm/utm/dss_probing.md @@ -27,7 +27,7 @@ Checked in - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction @@ -142,6 +142,26 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,A2-7-2,1a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1c + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1d + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,2a Implemented @@ -152,6 +172,16 @@ Implemented ASTM SCD DSS: Operational Intent Reference Key Validation + + DSS0210,A2-7-2,3a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,3b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,4a Implemented @@ -167,6 +197,11 @@ Implemented ASTM SCD DSS: Subscription and entity interaction + + DSS0210,A2-7-2,4d + Implemented + ASTM SCD DSS: Subscription and entity interaction + DSS0210,A2-7-2,5a Implemented diff --git a/monitoring/uss_qualifier/suites/astm/utm/f3548_21.md b/monitoring/uss_qualifier/suites/astm/utm/f3548_21.md index b2a757f791..f64d290178 100644 --- a/monitoring/uss_qualifier/suites/astm/utm/f3548_21.md +++ b/monitoring/uss_qualifier/suites/astm/utm/f3548_21.md @@ -35,7 +35,7 @@ Checked in - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548 flight planners preparation
ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction
Nominal planning: not permitted conflict with equal priority
Off-Nominal planning: down USS
Off-Nominal planning: down USS with equal priority conflicts not permitted @@ -155,6 +155,26 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,A2-7-2,1a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1c + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1d + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,2a Implemented @@ -165,6 +185,16 @@ Implemented ASTM SCD DSS: Operational Intent Reference Key Validation + + DSS0210,A2-7-2,3a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,3b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,4a Implemented @@ -180,6 +210,11 @@ Implemented ASTM SCD DSS: Subscription and entity interaction + + DSS0210,A2-7-2,4d + Implemented + ASTM SCD DSS: Subscription and entity interaction + DSS0210,A2-7-2,5a Implemented diff --git a/monitoring/uss_qualifier/suites/faa/uft/message_signing.md b/monitoring/uss_qualifier/suites/faa/uft/message_signing.md index b3cb9eb381..bd567cf24b 100644 --- a/monitoring/uss_qualifier/suites/faa/uft/message_signing.md +++ b/monitoring/uss_qualifier/suites/faa/uft/message_signing.md @@ -18,7 +18,7 @@ Checked in - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548 flight planners preparation
ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction
Nominal planning: not permitted conflict with equal priority
Off-Nominal planning: down USS
Off-Nominal planning: down USS with equal priority conflicts not permitted @@ -138,6 +138,26 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,A2-7-2,1a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1c + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1d + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,2a Implemented @@ -148,6 +168,16 @@ Implemented ASTM SCD DSS: Operational Intent Reference Key Validation + + DSS0210,A2-7-2,3a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,3b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,4a Implemented @@ -163,6 +193,11 @@ Implemented ASTM SCD DSS: Subscription and entity interaction + + DSS0210,A2-7-2,4d + Implemented + ASTM SCD DSS: Subscription and entity interaction + DSS0210,A2-7-2,5a Implemented diff --git a/monitoring/uss_qualifier/suites/interuss/dss/all_tests.md b/monitoring/uss_qualifier/suites/interuss/dss/all_tests.md index 4c4def593a..734d001692 100644 --- a/monitoring/uss_qualifier/suites/interuss/dss/all_tests.md +++ b/monitoring/uss_qualifier/suites/interuss/dss/all_tests.md @@ -408,7 +408,7 @@ ASTM NetRID DSS: Concurrent Requests
ASTM NetRID DSS: ISA Expiry
ASTM NetRID DSS: ISA Subscription Interactions
ASTM NetRID DSS: Simple ISA
ASTM NetRID DSS: Submitted ISA Validations
ASTM NetRID DSS: Subscription Simple
ASTM NetRID DSS: Subscription Validation
ASTM NetRID DSS: Token Validation - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction @@ -523,6 +523,26 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,A2-7-2,1a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1c + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1d + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,2a Implemented @@ -533,6 +553,16 @@ Implemented ASTM SCD DSS: Operational Intent Reference Key Validation + + DSS0210,A2-7-2,3a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,3b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,4a Implemented @@ -548,6 +578,11 @@ Implemented ASTM SCD DSS: Subscription and entity interaction + + DSS0210,A2-7-2,4d + Implemented + ASTM SCD DSS: Subscription and entity interaction + DSS0210,A2-7-2,5a Implemented diff --git a/monitoring/uss_qualifier/suites/uspace/flight_auth.md b/monitoring/uss_qualifier/suites/uspace/flight_auth.md index 5ee68ba652..ed8e3d2fd2 100644 --- a/monitoring/uss_qualifier/suites/uspace/flight_auth.md +++ b/monitoring/uss_qualifier/suites/uspace/flight_auth.md @@ -19,7 +19,7 @@ Checked in - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548 flight planners preparation
ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction
Nominal planning: not permitted conflict with equal priority
Off-Nominal planning: down USS
Off-Nominal planning: down USS with equal priority conflicts not permitted @@ -139,6 +139,26 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,A2-7-2,1a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1c + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1d + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,2a Implemented @@ -149,6 +169,16 @@ Implemented ASTM SCD DSS: Operational Intent Reference Key Validation + + DSS0210,A2-7-2,3a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,3b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,4a Implemented @@ -164,6 +194,11 @@ Implemented ASTM SCD DSS: Subscription and entity interaction + + DSS0210,A2-7-2,4d + Implemented + ASTM SCD DSS: Subscription and entity interaction + DSS0210,A2-7-2,5a Implemented diff --git a/monitoring/uss_qualifier/suites/uspace/required_services.md b/monitoring/uss_qualifier/suites/uspace/required_services.md index 0dccb79125..9e60c75a53 100644 --- a/monitoring/uss_qualifier/suites/uspace/required_services.md +++ b/monitoring/uss_qualifier/suites/uspace/required_services.md @@ -454,7 +454,7 @@ ASTM NetRID DSS: Concurrent Requests
ASTM NetRID DSS: ISA Expiry
ASTM NetRID DSS: ISA Subscription Interactions
ASTM NetRID DSS: Simple ISA
ASTM NetRID DSS: Submitted ISA Validations
ASTM NetRID DSS: Subscription Simple
ASTM NetRID DSS: Subscription Validation
ASTM NetRID DSS: Token Validation - astm
.f3548
.v21
+ astm
.f3548
.v21
DSS0005,1 Implemented ASTM F3548 flight planners preparation
ASTM F3548-21 UTM DSS Operational Intent Reference Access Control
ASTM SCD DSS: Interfaces authentication
ASTM SCD DSS: Operational Intent Reference Key Validation
ASTM SCD DSS: Operational Intent Reference Synchronization
ASTM SCD DSS: Subscription Simple
ASTM SCD DSS: Subscription Synchronization
ASTM SCD DSS: Subscription Validation
ASTM SCD DSS: Subscription and entity deletion interaction
ASTM SCD DSS: Subscription and entity interaction
Nominal planning: not permitted conflict with equal priority
Off-Nominal planning: down USS
Off-Nominal planning: down USS with equal priority conflicts not permitted @@ -574,6 +574,26 @@ Implemented ASTM SCD DSS: Operational Intent Reference Synchronization + + DSS0210,A2-7-2,1a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1c + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,1d + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,2a Implemented @@ -584,6 +604,16 @@ Implemented ASTM SCD DSS: Operational Intent Reference Key Validation + + DSS0210,A2-7-2,3a + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + + + DSS0210,A2-7-2,3b + Implemented + ASTM SCD DSS: Operational Intent Reference Synchronization + DSS0210,A2-7-2,4a Implemented @@ -599,6 +629,11 @@ Implemented ASTM SCD DSS: Subscription and entity interaction + + DSS0210,A2-7-2,4d + Implemented + ASTM SCD DSS: Subscription and entity interaction + DSS0210,A2-7-2,5a Implemented