diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.md b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.md index f4353bcfaf..17c470b4c7 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.md +++ b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.md @@ -45,16 +45,10 @@ which is of equal priority and came first. #### [Validate operational intent is shared](../validate_shared_operational_intent.md) -### Check for notification to tested_uss due to subscription in flight 2 area test step -In the following test step, we want to assert that tested_uss must have retrieved operational intent details from -mock_uss via a GET request. This assertion is only valid, however, if tested_uss did not obtain the operational -intent details in a different way -- specifically, a notification due to a pre-existing subscription. In this test -step, we determine if tested_uss had a pre-existing subscription by: - -#### [checking if mock_uss sent a notification to tested_uss](../../../interuss/mock_uss/get_mock_uss_interactions.md) - -### [Validate flight2 GET interaction, if no notification test step](test_steps/validate_get_operational_intent.md) -This step is skipped if a notification to tested_uss was found in the previous step since tested_uss obtained the operational intent details of flight 2 without needing to perform a GET interaction. +### [Validate that tested_uss obtained flight2 details test step](test_steps/validate_operational_intent_details_obtained.md) +Validate that tested_uss obtained flight2 details from mock_uss, by means of either +a notification pushed by mock_uss to tested_uss due to the pre-existing subscription, or +direct retrieval by tested_uss from mock_uss. ### [Validate flight1 Notification sent to mock_uss test step](test_steps/validate_notification_operational_intent.md) tested_uss notifies mock_uss of flight 1, due to mock_uss's subscription covering flight 2 (which is necessarily relevant to flight 1 per test design). @@ -93,16 +87,10 @@ The planning attempt should fail because tested_uss will be unable to obtain val Validate flight 1 is not shared with DSS, as plan failed. -### Check for notification to tested_uss due to subscription in flight 2 area test step -In the following test step, we want to assert that tested_uss must have retrieved operational intent details from -mock_uss via a GET request. This assertion is only valid, however, if tested_uss did not obtain the operational -intent details in a different way -- specifically, a notification due to a pre-existing subscription. In this test -step, we determine if tested_uss had a pre-existing subscription by: - -#### [Check if mock_uss sent a notification to tested_uss](../../../interuss/mock_uss/get_mock_uss_interactions.md) - -### [Validate flight2 GET interaction, if no notification test step](test_steps/validate_get_operational_intent.md) -This step is skipped if a notification to tested_uss was found in the previous step. +### [Validate that tested_uss obtained flight2 details test step](test_steps/validate_operational_intent_details_obtained.md) +Validate that tested_uss obtained flight2 details from mock_uss, by means of either +a notification pushed by mock_uss to tested_uss due to the pre-existing subscription, or +direct retrieval by tested_uss from mock_uss. ### [Validate flight 1 Notification not sent to mock_uss test step](test_steps/validate_no_notification_operational_intent.md) diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.py b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.py index 511c2fb70d..c88f5890fb 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.py +++ b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.py @@ -11,7 +11,10 @@ PlanningActivityResult, FlightPlanStatus, ) -from monitoring.monitorlib.clients.mock_uss.interactions import QueryDirection +from monitoring.uss_qualifier.scenarios.astm.utm.data_exchange_validation.test_steps.wait import ( + MaxTimeToWaitForSubscriptionNotificationSeconds as max_wait_time, +) +from monitoring.monitorlib.delay import sleep from monitoring.monitorlib.temporal import TimeDuringTest import arrow from monitoring.monitorlib.temporal import Time @@ -40,16 +43,11 @@ from monitoring.uss_qualifier.scenarios.astm.utm.data_exchange_validation.test_steps.expected_interactions_test_steps import ( expect_no_interuss_post_interactions, expect_mock_uss_receives_op_intent_notification, + expect_uss_obtained_op_intent_details, ) from monitoring.monitorlib.clients.mock_uss.mock_uss_scd_injection_api import ( MockUssFlightBehavior, ) -from monitoring.uss_qualifier.scenarios.interuss.mock_uss.test_steps import ( - get_mock_uss_interactions, - operation_filter, - direction_filter, - notif_op_intent_id_filter, -) from monitoring.uss_qualifier.scenarios.scenario import ( TestScenario, ScenarioCannotContinueError, @@ -61,7 +59,7 @@ submit_flight, ) from monitoring.uss_qualifier.suites.suite import ExecutionContext -from uas_standards.astm.f3548.v21.api import OperationID, EntityID +from uas_standards.astm.f3548.v21.api import EntityID from uas_standards.astm.f3548.v21.constants import Scope @@ -198,47 +196,20 @@ def _plan_successfully_test_case(self, times: Dict[TimeDuringTest, Time]): self.op_intent_ids.add(flight_1_oi_ref.id) self.end_test_step() - self.begin_test_step( - "Check for notification to tested_uss due to subscription in flight 2 area" + self.begin_test_step("Validate that tested_uss obtained flight2 details") + sleep( + max_wait_time, + "we have to wait the longest it may take a USS to send a notification before we can establish another USS has obtained operational intent details", ) - tested_uss_notifications, _ = get_mock_uss_interactions( + expect_uss_obtained_op_intent_details( self, self.mock_uss, flight_2_planning_time, - operation_filter(OperationID.NotifyOperationalIntentDetailsChanged), - direction_filter(QueryDirection.Outgoing), - notif_op_intent_id_filter(flight_2_oi_ref.id), + flight_2_oi_ref.id, + self.tested_uss_client.participant_id, ) self.end_test_step() - self.begin_test_step("Validate flight2 GET interaction, if no notification") - if not tested_uss_notifications: - tested_uss_get_requests, query = get_mock_uss_interactions( - self, - self.mock_uss, - flight_1_planning_time, - operation_filter( - OperationID.GetOperationalIntentDetails, entityid=flight_2_oi_ref.id - ), - direction_filter(QueryDirection.Incoming), - ) - with self.check( - "Expect GET request when no notification", - [self.tested_uss_client.participant_id], - ) as check: - if not tested_uss_get_requests: - check.record_failed( - summary=f"mock_uss did not GET op intent details when planning", - details=f"mock_uss did not receive a request to GET operational intent details for operational intent {flight_2_oi_ref.id}. tested_uss was not sent a notification with the operational intent details, so they should have requested the operational intent details during planning.", - query_timestamps=[query.request.timestamp], - ) - else: - self.record_note( - "No flight 2a GET expected reason", - f"Notifications found to {', '.join(n.query.request.url for n in tested_uss_notifications)}", - ) - self.end_test_step() - self.begin_test_step("Validate flight1 Notification sent to mock_uss") expect_mock_uss_receives_op_intent_notification( self, @@ -321,47 +292,20 @@ def _plan_unsuccessfully_test_case(self, times: Dict[TimeDuringTest, Time]): validator.expect_not_shared() self.end_test_step() - self.begin_test_step( - "Check for notification to tested_uss due to subscription in flight 2 area" + self.begin_test_step("Validate that tested_uss obtained flight2 details") + sleep( + max_wait_time, + "we have to wait the longest it may take a USS to send a notification before we can establish another USS has obtained operational intent details", ) - tested_uss_notifications, _ = get_mock_uss_interactions( + expect_uss_obtained_op_intent_details( self, self.mock_uss, flight_2_planning_time, - operation_filter(OperationID.NotifyOperationalIntentDetailsChanged), - direction_filter(QueryDirection.Outgoing), - notif_op_intent_id_filter(flight_2_oi_ref.id), + flight_2_oi_ref.id, + self.tested_uss_client.participant_id, ) self.end_test_step() - self.begin_test_step("Validate flight2 GET interaction, if no notification") - if not tested_uss_notifications: - tested_uss_get_requests, query = get_mock_uss_interactions( - self, - self.mock_uss, - flight_1_planning_time, - operation_filter( - OperationID.GetOperationalIntentDetails, entityid=flight_2_oi_ref.id - ), - direction_filter(QueryDirection.Incoming), - ) - with self.check( - "Expect GET request when no notification", - [self.tested_uss_client.participant_id], - ) as check: - if not tested_uss_get_requests: - check.record_failed( - summary=f"mock_uss did not GET op intent details when planning", - details=f"mock_uss did not receive a request to GET operational intent details for operational intent {flight_2_oi_ref.id}. tested_uss was not sent a notification with the operational intent details, so they should have requested the operational intent details during planning.", - query_timestamps=[query.request.timestamp], - ) - else: - self.record_note( - "No flight 2b GET expected reason", - f"Notifications found to {', '.join(n.query.request.url for n in tested_uss_notifications)}", - ) - self.end_test_step() - self.begin_test_step("Validate flight 1 Notification not sent to mock_uss") expect_no_interuss_post_interactions( self, diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/expected_interactions_test_steps.py b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/expected_interactions_test_steps.py index dd4c4f4cec..c95e53791f 100644 --- a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/expected_interactions_test_steps.py +++ b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/expected_interactions_test_steps.py @@ -11,16 +11,17 @@ ) from monitoring.monitorlib.clients.mock_uss.interactions import QueryDirection -from monitoring.monitorlib.delay import sleep from monitoring.uss_qualifier.resources.interuss.mock_uss.client import MockUSSClient from monitoring.uss_qualifier.scenarios.astm.utm.data_exchange_validation.test_steps.wait import ( wait_in_intervals, - MaxTimeToWaitForSubscriptionNotificationSeconds as max_wait_time, ) from monitoring.uss_qualifier.scenarios.interuss.mock_uss.test_steps import ( get_mock_uss_interactions, operation_filter, direction_filter, + filter_interactions, + notif_op_intent_id_filter, + status_code_filter, ) from monitoring.uss_qualifier.scenarios.scenario import TestScenarioType @@ -72,10 +73,6 @@ def expect_no_interuss_post_interactions( shared_op_intent_ids: the set of IDs of previously shared operational intents for which it is expected that notifications are present regardless of their timings participant_id: id of the participant responsible to send the notification """ - sleep( - max_wait_time, - "we have to wait the longest it may take a USS to send a notification before we can establish that they didn't send a notification", - ) interactions, query = get_mock_uss_interactions( scenario, mock_uss, @@ -111,3 +108,61 @@ def expect_no_interuss_post_interactions( details=f"Notification for operational intent ID {req.operational_intent_id} triggered by subscriptions {', '.join([sub.subscription_id for sub in req.subscriptions])} with timestamp {interaction.query.request.timestamp}.", query_timestamps=[query.request.timestamp], ) + + +def expect_uss_obtained_op_intent_details( + scenario: TestScenarioType, + mock_uss: MockUSSClient, + st: StringBasedDateTime, + op_intent_id: EntityID, + participant_id: str, +): + """ + This step verifies that a USS obtained operational intent details from a Mock USS by means of either a notification + from the Mock USS (push), or a GET request (operation *getOperationalIntentDetails*) to the Mock USS. + + Implements the test step fragment in `validate_operational_intent_details_obtained.md`. + + Args: + st: the earliest time a notification may have been sent + op_intent_id: the operational intent ID subject of the notification + participant_id: id of the participant responsible to obtain the details + """ + + all_interactions, query = get_mock_uss_interactions( + scenario, + mock_uss, + st, + ) + + notifications = filter_interactions( + all_interactions, + [ + operation_filter(OperationID.NotifyOperationalIntentDetailsChanged), + direction_filter(QueryDirection.Outgoing), + notif_op_intent_id_filter(op_intent_id), + status_code_filter(204), + ], + ) + + get_requests = filter_interactions( + all_interactions, + [ + operation_filter( + OperationID.GetOperationalIntentDetails, entityid=op_intent_id + ), + direction_filter(QueryDirection.Incoming), + status_code_filter(200), + ], + ) + + with scenario.check( + "USS obtained operational intent details by means of either notification or GET request", + [participant_id], + ) as check: + if not notifications and not get_requests: + check.record_failed( + summary=f"USS {participant_id} did not obtained details of operational intent {op_intent_id} from mock_uss", + details=f"operational intent {op_intent_id}: mock_uss did not notify successfully {participant_id} of the details and {participant_id} did not do a successful GET request to retrieve them either since {st}", + query_timestamps=[query.request.timestamp], + ) diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_get_operational_intent.md b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_get_operational_intent.md deleted file mode 100644 index ca33cf2216..0000000000 --- a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_get_operational_intent.md +++ /dev/null @@ -1,9 +0,0 @@ -# Validate GET interaction test step fragment - -This step verifies that a USS makes a GET request to get the intent_details of an existing operation when needed as per ASTM F3548-21 by checking the interuss interactions of mock uss - -## [Get Mock USS interactions logs](../../../../interuss/mock_uss/get_mock_uss_interactions.md) - -## 🛑 Expect GET request when no notification check -**[astm.f3548.v21.SCD0035](../../../../../requirements/astm/f3548/v21.md)** -SCD0035 needs a USS to verify before transitioning to Accepted that it does not conflict with a type of operational intent, and the only way to have verified this is by knowing all operational intent details, and (from previous checks of no notifications) the only way to know the operational intent details of flight is to have requested them via a GET details interaction. diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_operational_intent_details_obtained.md b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_operational_intent_details_obtained.md new file mode 100644 index 0000000000..9112ba110a --- /dev/null +++ b/monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_operational_intent_details_obtained.md @@ -0,0 +1,11 @@ +# Validate operational intent details obtained test step fragment +This step verifies that a USS obtained operational intent details from a Mock USS by means of either a notification from +the Mock USS (push), or a GET request (operation *getOperationalIntentDetails*) to the Mock USS. + +## [Get Mock USS interactions logs](../../../../interuss/mock_uss/get_mock_uss_interactions.md) + +## 🛑 USS obtained operational intent details by means of either notification or GET request check +SCD0035 requires a USS to verify before transitioning to Accepted that it does not conflict with another operational +intent, and the only way to have verified this is by knowing all operational intent details. +As such, if the USS was neither notified of the details by the Mock USS, nor did it retrieve them directly from the Mock +USS, this check will fail per **[astm.f3548.v21.SCD0035](../../../../../requirements/astm/f3548/v21.md)**