[uss_qualifier] Add scenario for testing data validation by SUT for Get operational_intent

monitoring/uss_qualifier/requirements/interuss/f3548/notification_requirements.md

@@ -0,0 +1,9 @@
+# ASTM F3548-21 notifications requirement
+
+While ASTM F3548-21 does not specifically prohibit sending notifications for operational intent changes when those notifications were not specifically prescribed by a notification that an operational intent may be relevant to a subscription (response from the DSS), InterUSS expects that notifications will only be sent at these particular times.
+SCD0085 requirement states only a positive requirement (preconditions satisified => notification must be sent).
+It doesn't establish a negative requirement (preconditions not satisfied => no notifications may be sent).
+As these negative requirements are reasonable  and related to SCD0085, interUSS has added it as a requirement for testing.
+This can happen in following situations -
+* <tt>NoDssEntityNoNotification</tt> If a USS was unable to write an entity reference to the DSS, it should not erroneously notify that operational intent, to another USS subscribed in the area.
+* <tt>NoSubscriptionNoNotification</tt> If a USS wrote an entity reference to the DSS, and was notified of no other USS subscription, then it should not send a notification to any USS. So the USS  shall (OnlyPrescribedNotifications) only send operational intent notifications when prescribed by SCD0085.

monitoring/uss_qualifier/requirements/interuss/mock_uss/interactions.md

@@ -0,0 +1,9 @@
+InterUss mock_uss interactions requirements
+
+## Overview
+Mock_uss records interactions on its uss/v1/operational_intents endpoints. It provides an endpoint for uss_qualifier to get these interactions for testing various scenarios.
+
+## Requirements
+
+### Interactions
+GET /mock_uss/interuss_logging/logs returns a list of interactions that took place between the time specified by query parameter from_time and the current time.

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.md

@@ -0,0 +1,106 @@
+# Data Validation of GET operational intents by USS test scenario
+
+## Description
+This test checks that the USS being tested validates the operational intents received as response to its GET request from another USS.
+Control_uss which is a mock uss plans a nearby V-shaped operation, and provides the data that tested_uss GETs.
+tested_uss validates the GET response from control_uss and accordingly plan its operation.
+Notably the following requirements:
+
+- **[astm.f3548.v21.SCD0035](../../../../requirements/astm/f3548/v21.md)**
+
+## Resources
+### flight_intents
+FlightIntentsResource provides the two V-shaped flight intents.
+The convex hulls of the 2D footprints of the two flights intersect, but the polygons do not intersect.
+There is an overlap in time and altitude of the two flights.
+- flight_1_planned_time_range_A
+- flight_2_planned_time_range_A
+
+### control_uss
+MockUSSResource that will be used for planning flights, controlling data shared for validation testing, and gathering interuss interactions from mock_uss.
+
+### tested_uss
+FlightPlannerResource that will be used for the USS being tested for its data validation of operational intent.
+
+### dss
+DSSInstanceResource that provides access to a DSS instance where flight creation/sharing can be verified.
+
+## Setup test case
+### Check for flight planning readiness test step
+Both USSs are queried for their readiness to ensure this test can proceed.
+
+#### Flight planning USS not ready check
+If either USS does not respond appropriately to the endpoint queried to determine readiness, this check will fail and the USS will have failed to meet **[astm.f3548.v21.GEN0310](../../../../requirements/astm/f3548/v21.md)** as the USS does not support the InterUSS implementation of that requirement.
+
+### Area clearing test step
+Both USSs are requested to remove all flights from the area under test.
+
+#### Area cleared successfully check
+**[interuss.automated_testing.flight_planning.ClearArea](../../../../requirements/interuss/automated_testing/flight_planning.md)**
+
+## Successfully plan flight near an existing flight test case
+### [Control_uss plans flight 2 test step](../../../flight_planning/plan_flight_intent.md)
+Flight 2 on time range A should be successfully planned by the control USS.
+
+### [Validate flight 2 sharing test step](../validate_shared_operational_intent.md)
+Validate that flight 2 is planned
+
+### [Validate no notification pushed for flight 2](../validate_not_shared_operational_intent.md)
+There should be no subscription by tested_uss to trigger notification of flight 2.
+This will ensure that while planning a nearby flight tested_uss will need to make a GET request to control_uss for flight 2 details.
+If this notification was pushed, the GET operational intent data validation test cannot be done.
+
+### [Tested_uss plans flight 1 test step](../../../flight_planning/plan_flight_intent.md)
+The test driver attempts to plan flight 1 on time range A via the tested USS. It checks if any conflicts with flight 2
+which is of equal priority and came first.
+
+### [Validate flight 1 sharing test step](../validate_shared_operational_intent.md)
+Validate flight 1 is planned.
+
+### [Validate flight2 GET interaction test step](test_steps/validate_get_operational_intent.md)
+Tested_uss needs to make GET request for obtaining details of flight 2.
+In a previous step, we checked there was no notification of flight 2 to tested_uss.
+
+### [Validate flight1 Notification sent to Control_uss test step](test_steps/validate_notification_operational_intent.md)
+Tested_uss notifies flight 1 to Control_uss, due to its subscription through flight 2.
+
+### [Delete tested_uss flight test step](../../../flight_planning/delete_flight_intent.md)
+Teardown
+
+### [Delete control_uss flight test step](../../../flight_planning/delete_flight_intent.md)
+Teardown
+
+## Flight planning prevented due to invalid data sharing test case
+### [Control_uss plans flight 2, sharing invalid operational intent data test step](../../../flight_planning/plan_flight_intent.md)
+Flight 2 on time range A should be successfully planned by the control_uss.
+The control_uss, which is mock_uss is instructed to share invalid data with other USS, for negative test.
+
+### [Validate flight 2 shared operational intent with invalid data test step](test_steps/validate_sharing_operational_intent_but_with_invalid_interuss_data.md)
+Validate that flight 2 is shared with invalid data as a modified behavior is injected by uss_qualifier for a negative test.
+
+### [Validate no notification pushed for flight 2](../validate_not_shared_operational_intent.md)
+There should be no subscription by tested_uss to trigger notification of flight 2.
+This will ensure that while planning a nearby flight tested_uss will need to make a GET request to control_uss for flight 2 details.
+If this notification was pushed, the GET operational intent data validation test cannot be carried out.
+
+### [Test_uss attempts to plan flight 1, expect failure test step](test_steps/plan_flight_intent_expect_failed.md)
+The test driver attempts to plan the flight 1 on time range A via the tested_uss. It checks if any conflicts with flight 2
+which is of equal priority and came first.
+
+### [Validate flight 1 not shared by tested_uss test step](../validate_not_shared_operational_intent.md)
+Validate flight 1 is not shared with DSS, as plan failed.
+
+### [Validate flight 2 GET interaction test step](test_steps/validate_get_operational_intent.md)
+Tested_uss needs to make GET request for obtaining details of flight 2 from control_uss.
+In a previous step, we checked there was no notification of flight 2 to tested_uss.
+Hence USS will have to obtain details using GET request.
+
+### [Validate flight 1 Notification not sent to Control_uss test step](test_steps/validate_no_notification_operational_intent.md)
+
+### [Delete Control_uss flight test step](../../../flight_planning/delete_flight_intent.md)
+Teardown
+
+## Cleanup
+### Successful flight deletion check
+This cleanup is for both - after testcase ends and after test scenario ends
+**[interuss.automated_testing.flight_planning.DeleteFlightSuccess](../../../../requirements/interuss/automated_testing/flight_planning.md)**

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/get_op_data_validation.py

@@ -0,0 +1,52 @@
+from typing import Optional
+
+from monitoring.uss_qualifier.resources.astm.f3548.v21 import DSSInstanceResource
+from monitoring.uss_qualifier.resources.astm.f3548.v21.dss import DSSInstance
+from monitoring.uss_qualifier.resources.flight_planning import (
+    FlightIntentsResource,
+)
+from monitoring.uss_qualifier.resources.flight_planning.flight_intent import (
+    FlightIntent,
+)
+from monitoring.uss_qualifier.resources.flight_planning.flight_planner import (
+    FlightPlanner,
+)
+from monitoring.uss_qualifier.resources.flight_planning.flight_planners import (
+    FlightPlannerResource,
+)
+
+from monitoring.uss_qualifier.scenarios.scenario import (
+    TestScenario,
+)
+
+from monitoring.uss_qualifier.resources.interuss.mock_uss.client import (
+    MockUSSClient,
+    MockUSSResource,
+)
+
+
+class GetOpResponseDataValidationByUSS(TestScenario):
+    flight_1_planned_time_range_A: FlightIntent
+
+    flight_2_planned_time_range_A: FlightIntent
+
+    tested_uss: FlightPlanner
+    control_uss: MockUSSClient
+    dss: DSSInstance
+
+    def __init__(
+        self,
+        tested_uss: FlightPlannerResource,
+        control_uss: MockUSSResource,
+        dss: DSSInstanceResource,
+        flight_intents: Optional[FlightIntentsResource] = None,
+    ):
+        super().__init__()
+        self.tested_uss = tested_uss.flight_planner
+        self.control_uss = control_uss.mock_uss
+        self.dss = dss.dss
+
+    def run(self):
+        self.begin_test_scenario()
+        pass
+        self.end_test_scenario()

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/plan_flight_intent_expect_failed.md

@@ -0,0 +1,14 @@
+# Plan flight Expect Failed test step
+
+This page describes the content of a common test case where a valid user flight intent fails in a flight planner, because of invalid data shared for a nearby flight shared by another USS.  See `plan_flight_intent_expect_failed` in invalid_op_test_steps.py.
+
+## Plan should fail check
+
+A USS shouldn't go ahead and plan if it doesn't have accurate information.
+As per SCD0035 a USS needs to verify a particular conflict status.
+If flight intent data shared for a nearby flight shared is invalid, the USS can't successfully perform that verification.
+It couldn't have obtained valid information about the other operational intents it's supposed to verify that conflict status against.
+Therefore, the USS should fail an attempt to plan. If the plan succeeds, we know they've violated SCD0035 because they failed to verify that particular conflict status.
+
+**[astm.f3548.v21.SCD0035](../../../../../requirements/astm/f3548/v21.md)**
+

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_get_operational_intent.md

@@ -0,0 +1,10 @@
+# Validate GET interaction test step
+
+This step verifies that a USS makes a GET request to get the intent_details of an existing operation when needed as per ASTM F3548-21 by checking the interuss interactions of mock uss
+
+## MockUSS interactions request check
+**[interuss.mock_uss.hosted_instance.ExposeInterface](../../../../../requirements/interuss/mock_uss/hosted_instance.md)**.
+
+## Expect GET request check
+**[astm.f3548.v21.SCD0035](../../../../../requirements/astm/f3548/v21.md)**
+SCD0035 needs a USS to verify before transitioning to Accepted that it does not conflict with a type of operational intent, and the only way to have verified this is by knowing all operational intent details, and (from previous checks of no notifications) the only way to know the operational intent details of flight is to have requested them via a GET details interaction.

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_no_notification_operational_intent.md

@@ -0,0 +1,10 @@
+# Validate no notification test step
+
+This step verifies when a flight is not created, it is also not notified by checking the interuss interactions of mock_uss instance.
+
+## MockUSS interactions request check
+**[interuss.mock_uss.hosted_instance.ExposeInterface](../../../../../requirements/interuss/mock_uss/hosted_instance.md)**.
+
+## Expect Notification not sent check
+
+**[interuss.f3548.notification_requirements.NoDssEntityNoNotification](../../../../../requirements/interuss/f3548/notification_requirements.md)**

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_notification_operational_intent.md

@@ -0,0 +1,12 @@
+# Validate notification test step
+
+This step verifies that, when creating or modifying an operational intent, a USS sent the required notification for a relevant subscription owned by a mock_uss instance by checking the interactions of that mock_uss instance.
+
+## MockUSS interactions request check
+**[interuss.mock_uss.hosted_instance.ExposeInterface](../../../../../requirements/interuss/mock_uss/hosted_instance.md)**.
+
+## Expect Notification sent check
+**[astm.f3548.v21.SCD0085](../../../../../requirements/astm/f3548/v21.md)**
+
+## Notification data is valid check
+**[astm.f3548.v21.SCD0085](../../../../../requirements/astm/f3548/v21.md)**

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_sharing_operational_intent_but_with_invalid_interuss_data.md

@@ -0,0 +1,23 @@
+# Validate flight sharing invalid data test step
+
+This step verifies that a created flight is shared properly per ASTM F3548-21 by querying the DSS for flights in the area of the flight intent, and then retrieving the details from the USS if the operational intent reference is found.  See `expect_shared_with_invalid_data` in invalid_op_test_steps.py.
+
+## DSS responses check
+
+**[astm.f3548.v21.DSS0005](../../../../../requirements/astm/f3548/v21.md)**
+
+## Operational intent shared with DSS check
+
+If a reference to the operational intent for the flight is not found in the DSS, this check will fail per **[astm.f3548.v21.USS0005](../../../../../requirements/astm/f3548/v21.md)** and **[astm.f3548.v21.OPIN0025](../../../../../requirements/astm/f3548/v21.md)**.
+
+## Operational intent details retrievable check
+
+If the operational intent details for the flight cannot be retrieved from the USS, this check will fail per **[astm.f3548.v21.USS0105](../../../../../requirements/astm/f3548/v21.md)** and **[astm.f3548.v21.OPIN0025](../../../../../requirements/astm/f3548/v21.md)**.
+
+## Invalid data in Operational intent details shared by Mock USS for negative test check
+
+Mock USS shares operational intent details response for the negative test case as per [the GetOperationalIntentDetailsResponse schema of the OpenAPI specification](https://github.com/astm-utm/Protocol/blob/v1.0.0/utm.yaml#L1120).
+If the operational intent details from mock_uss is valid, this check will fail. It would mean mock_uss is not behaving correctly.
+**[interuss.mock_uss.hosted_instance.ExposeInterface](../../../../../requirements/interuss/mock_uss/hosted_instance.md)**.
+
+

monitoring/uss_qualifier/test_data/usa/kentland/flight_intents/non_conflicting.yaml

@@ -0,0 +1,106 @@
+intents:
+  flight_1_planned_time_range_A:
+    full:
+      basic_information:
+        usage_state: Planned
+        uas_state: Nominal
+        area:
+          - outline_polygon:
+              vertices:
+                - lat: 37.20642344604623
+                  lng: -80.58508994131496
+                - lat: 37.21359527809636
+                  lng: -80.5767365824006
+                - lat: 37.215812746083586
+                  lng: -80.58144645497978
+                - lat: 37.2146332499489
+                  lng: -80.58408279875103
+            altitude_lower:
+              value: 474
+              reference: W84
+              units: M
+            altitude_upper:
+              value: 560
+              reference: W84
+              units: M
+            start_time:
+              offset_from:
+                starting_from:
+                  start_of_test: { }
+                offset: -1s
+            duration: 5m
+
+      astm_f3548_21:
+        priority: 0
+
+      uspace_flight_authorisation:
+        uas_serial_number: 1AF49UL5CC5J6K
+        operation_category: Open
+        operation_mode: Vlos
+        uas_class: C0
+        identification_technologies:
+          - ASTMNetRID
+        connectivity_methods:
+          - cellular
+        endurance_minutes: 30
+        emergency_procedure_url: https://example.interussplatform.org/emergency
+        operator_id: CHEo5kut30e0mt01-qwe
+        uas_id: ''
+        uas_type_certificate: ''
+
+
+  flight_2_planned_time_range_A:
+    full:
+      basic_information:
+        usage_state: Planned
+        uas_state: Nominal
+        area:
+          - outline_polygon:
+              vertices:
+                - lat: 37.214119359044275
+                  lng: -80.58443600701524
+                - lat: 37.212388260776436
+                  lng: -80.58824885811198
+                - lat: 37.20211436858821
+                  lng: -80.5856832012991
+                - lat: 37.21394908884487
+                  lng: -80.57474352572189
+                - lat: 37.206173006943104
+                  lng: -80.5852199577081
+            altitude_lower:
+              value: 483
+              reference: W84
+              units: M
+            altitude_upper:
+              value: 519
+              reference: W84
+              units: M
+            start_time:
+              offset_from:
+                starting_from:
+                  start_of_test: { }
+                offset: -1s
+            duration: 5m
+
+      astm_f3548_21:
+        priority: 0        # TODO: Remove flight_authorisation section when it is optional
+
+      uspace_flight_authorisation:
+        uas_serial_number: 1AF49UL5CC5J6K
+        operation_category: Open
+        operation_mode: Vlos
+        uas_class: C0
+        identification_technologies:
+          - ASTMNetRID
+        connectivity_methods:
+          - cellular
+        endurance_minutes: 30
+        emergency_procedure_url: https://example.interussplatform.org/emergency
+        operator_id: CHEo5kut30e0mt01-qwe
+        uas_id: ''
+        uas_type_certificate: ''
+
+
+
+
+