[uss_qualifier/scenarios/utm/dss/OIRSynchronization] DSS0210,A2-7-2,[…

…1a-b-c-d,3a-b]: Add checks to validate requirements (#613)

[uss_qualifier/scenarios/utm/dss/OIRSynchronization] DSSDSS0210,1a-b-c-d,3a-b: Add checks to validate requirements

build/dev/docker-compose.yaml

@@ -20,7 +20,7 @@ services:
       - dss_internal_network
 
   rid_bootstrapper:
-    image: interuss/dss:v0.13.0
+    image: interuss/dss:v0.14.0
     volumes:
       - dss_component_coordination:/var/dss_component_coordination
       - $PWD/startup:/startup:ro
@@ -31,7 +31,7 @@ services:
       - dss_internal_network
 
   scd_bootstrapper:
-    image: interuss/dss:v0.13.0
+    image: interuss/dss:v0.14.0
     volumes:
       - dss_component_coordination:/var/dss_component_coordination
       - $PWD/startup:/startup:ro
@@ -43,7 +43,7 @@ services:
 
   dss:
     hostname: dss.uss1.localutm
-    image: interuss/dss:v0.13.0
+    image: interuss/dss:v0.14.0
     volumes:
       - $PWD/../test-certs:/var/test-certs:ro
       - dss_component_coordination:/var/dss_component_coordination

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search.md → monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_correct.md

@@ -2,9 +2,9 @@
 
 This test step fragment validates that subscriptions can be searched for.
 
-## 🛑 Successful subscription search query check
+## [Search query succeeds](./search_query.md)
 
-If the DSS fails to let us search in the area for which the subscription was created, it is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**.
+Check query succeeds.
 
 ## 🛑 Created Subscription is in search results check
 

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_query.md

@@ -0,0 +1,7 @@
+# Search subscription query test step fragment
+
+This test step fragment validates that a query to search for subscriptions succeeds.
+
+## 🛑 Successful subscription search query check
+
+If the DSS fails to let us search in the area for which the subscription was created, it is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**.

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update.md → monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_correct.md

@@ -2,9 +2,9 @@
 
 This test step fragment validates that subscriptions can be updated.
 
-## 🛑 Subscription can be mutated check
+## [Update query succeeds](./update_query.md)
 
-If a subscription cannot be modified with a valid set of parameters, the DSS is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**.
+Check query succeeds.
 
 ## 🛑 Mutate subscription response format conforms to spec check
 

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_query.md

@@ -0,0 +1,7 @@
+# Update subscription query test step fragment
+
+This test step fragment validates that a query to update a subscription succeeds.
+
+## 🛑 Subscription can be mutated check
+
+If a subscription cannot be updated with a valid set of parameters, the DSS is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**.

monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.md

@@ -126,4 +126,33 @@ The subscription created on a DSS instance must be retrievable from all other DS
 
 If the subscription does not exist on one of the other DSS instances, one of the instances fails to comply with **[astm.f3548.v21.DSS0210,A2-7-2,4a](../../../../requirements/astm/f3548/v21.md)**.
 
+
+## Expiration of subscriptions removes them test case
+
+This test case validates that expired subscriptions (created explicitly) are removed from all DSS instances.
+To validate this requirement, the subscriptions that were previously created at each DSS instance are modified so that they expire shortly after the modification.
+Then it is checked that all the associated subscriptions were removed from all the DSS instances by searching for them in their planning area.
+Do note that they are not queried directly, as it is deemed acceptable for expired subscription that were not explicitly deleted to still be retrievable.
+
+### Expire explicit subscriptions at every DSS in sequence test step
+
+This test step will modify explicit subscriptions that were previously created at each DSS instance so that they expire shortly after the modification.
+
+Note that this step is run once for each involved DSS (that is, once for the primary DSS and once for every secondary DSS)
+
+#### [Modify subscription on a DSS instance so that it expires soon](./fragments/sub/crud/update_query.md)
+
+Check that the subscription modifications succeed and wait for them to expire.
+
+#### [Search for subscriptions from all other DSS instances succeeds](./fragments/sub/crud/search_query.md)
+
+Check that query succeeds.
+
+#### 🛑 Subscription does not exist on all other DSS instances check
+
+The explicit subscription expired on a DSS instance must have been removed from all other DSS instances.
+
+If the subscription still exists on one of the other DSS instances, one of the instances fails to comply with **[astm.f3548.v21.DSS0210,A2-7-2,4d](../../../../requirements/astm/f3548/v21.md)**.
+
+
 ## [Cleanup](./clean_workspace.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.py

@@ -12,8 +12,10 @@
 from uas_standards.astm.f3548.v21.constants import Scope
 
 from monitoring.monitorlib import fetch
+from monitoring.monitorlib.delay import sleep
 from monitoring.monitorlib.fetch import QueryError
 from monitoring.monitorlib.geotemporal import Volume4D
+from monitoring.monitorlib.temporal import Time
 from monitoring.prober.infrastructure import register_resource_type
 from monitoring.uss_qualifier.configurations.configuration import ParticipantID
 from monitoring.uss_qualifier.resources.astm.f3548.v21 import PlanningAreaResource
@@ -55,7 +57,7 @@ class SubscriptionInteractions(TestScenario):
 
     # Reference times for the subscriptions and operational intents
     _time_start: datetime
-    _time_send: datetime
+    _time_end: datetime
 
     _manager: str
 
@@ -120,6 +122,9 @@ def run(self, context: ExecutionContext):
         self._steps_create_subs_at_each_dss()
         self.end_test_case()
 
+        self.begin_test_case("Expiration of subscriptions removes them")
+        self._steps_expire_subs_at_each_dss()
+        self.end_test_case()
         self.end_test_scenario()
 
     def _step_create_background_sub(self):
@@ -129,7 +134,7 @@ def _step_create_background_sub(self):
         sub_now_params = self._planning_area.get_new_subscription_params(
             subscription_id=self._background_sub_id,
             start_time=self._time_start,
-            duration=self._time_send - self._time_start,
+            duration=self._time_end - self._time_start,
             # This is a planning area without constraint processing
             notify_for_op_intents=True,
             notify_for_constraints=False,
@@ -188,7 +193,7 @@ def _implicit_subs_check(
                 state=OperationalIntentState.Accepted,
                 uss_base_url="https://example.interuss.org/oir_base_url",
                 time_start=datetime.utcnow(),
-                time_end=self._time_send + timedelta(minutes=10),
+                time_end=self._time_end + timedelta(minutes=10),
                 subscription_id=None,
                 implicit_sub_base_url="https://example.interuss.org/sub_base_url",
             )
@@ -235,7 +240,7 @@ def _implicit_subs_check(
                 state=OperationalIntentState.Accepted,
                 uss_base_url="https://example.interuss.org/oir_base_url_bis",  # dummy modification of the OIR
                 time_start=datetime.utcnow(),
-                time_end=self._time_send + timedelta(minutes=10),
+                time_end=self._time_end + timedelta(minutes=10),
                 subscription_id=self._current_oirs[oir_id].subscription_id,
             )
 
@@ -283,7 +288,7 @@ def _steps_create_subs_at_each_dss(self):
         common_params = self._planning_area.get_new_subscription_params(
             subscription_id="",
             start_time=self._time_start,
-            duration=self._time_send - self._time_start,
+            duration=self._time_end - self._time_start,
             notify_for_op_intents=True,
             notify_for_constraints=False,
         )
@@ -342,6 +347,76 @@ def _steps_create_subs_at_each_dss(self):
 
             self.end_test_step()
 
+    def _steps_expire_subs_at_each_dss(self):
+        self.begin_test_step("Expire explicit subscriptions at every DSS in sequence")
+        for i, dss in enumerate([self._dss] + self._secondary_instances):
+            sub_id = self._sub_ids[i]
+            sub_params = self._planning_area.get_new_subscription_params(
+                subscription_id=sub_id,
+                start_time=datetime.utcnow(),
+                duration=timedelta(seconds=SUBSCRIPTION_EXPIRY_DELAY_SEC),
+                notify_for_op_intents=True,
+                notify_for_constraints=False,
+            )
+
+            with self.check(
+                "Subscription can be mutated",
+                [dss.participant_id],
+            ) as check:
+                sub = self._dss.upsert_subscription(
+                    **sub_params,
+                    version=self._current_subs[sub_id].version,
+                )
+                self.record_query(sub)
+                if not sub.success:
+                    check.record_failed(
+                        summary="Update subscription query failed",
+                        details=f"Failed to update a subscription on DSS instance with code {sub.status_code}: {sub.error_message}",
+                        query_timestamps=[sub.request.timestamp],
+                    )
+            self._current_subs.pop(sub_id)
+
+        sleep(
+            timedelta(seconds=WAIT_FOR_EXPIRY_SEC),
+            "waiting for subscriptions to expire",
+        )
+
+        for i, dss in enumerate([self._dss] + self._secondary_instances):
+            sub_id = self._sub_ids[i]
+            for other_dss in {self._dss, *self._secondary_instances} - {dss}:
+                other_dss_subs = other_dss.query_subscriptions(
+                    Volume4D(
+                        volume=self._planning_area.volume,
+                        time_start=Time(self._time_start),
+                        time_end=Time(self._time_end),
+                    ).to_f3548v21()
+                )
+                self.record_query(other_dss_subs)
+
+                with self.check(
+                    "Successful subscription search query",
+                    dss.participant_id,
+                ) as check:
+                    if not other_dss_subs.success:
+                        check.record_failed(
+                            summary="Search subscriptions query failed",
+                            details=f"Failed to search for subscriptions from DSS with code {other_dss_subs.status_code}: {other_dss_subs.error_message}",
+                            query_timestamps=[other_dss_subs.request.timestamp],
+                        )
+
+                with self.check(
+                    "Subscription does not exist on all other DSS instances",
+                    [dss.participant_id, other_dss.participant_id],
+                ) as check:
+                    if sub_id in other_dss_subs.subscriptions:
+                        check.record_failed(
+                            summary="Subscription that expired on a DSS instance was found on another instance",
+                            details=f"Subscription {sub_id} expired on DSS instance {dss.participant_id} was found on DSS instance {other_dss.participant_id}.",
+                            query_timestamps=[other_dss_subs.request.timestamp],
+                        )
+
+        self.end_test_step()
+
     def _create_sub_with_params(
         self, params: SubscriptionParams
     ) -> Tuple[Subscription, List[OperationalIntentReference], fetch.Query]:
@@ -362,7 +437,7 @@ def _setup_case(self):
 
         # Subscription from now to 20 minutes in the future
         self._time_start = datetime.utcnow()
-        self._time_send = self._time_start + timedelta(minutes=20)
+        self._time_end = self._time_start + timedelta(minutes=20)
 
         # Multiple runs of the scenario seem to rely on the same instance:
         # thus we need to reset the state of the scenario before running it.

monitoring/uss_qualifier/scenarios/astm/utm/dss/synchronization/subscription_synchronization.md

@@ -91,7 +91,7 @@ Confirm that the subscription that was just created is properly synchronized acr
 
 Confirms that each DSS provides access to the created subscription,
 
-#### [Search subscription](../fragments/sub/crud/search.md)
+#### [Search subscription](../fragments/sub/crud/search_correct.md)
 
 Confirms that each DSS returns the created subscription when searched for.
 
@@ -108,7 +108,7 @@ Verify that the version of the subscription returned by every DSS is as expected
 This test step mutates the previously created subscription, by accessing the primary DSS, to verify that the update is propagated to all other DSSes.
 Notably, it checks that the subscription version is updated, including for changes that are not directly visible, such as changing the subscription's footprint.
 
-#### [Update subscription](../fragments/sub/crud/update.md)
+#### [Update subscription](../fragments/sub/crud/update_correct.md)
 
 Confirm that the subscription can be mutated.
 
@@ -132,7 +132,7 @@ Confirm that the subscription that was just mutated is properly synchronized acr
 
 Confirms that the subscription that was just mutated can be retrieved from any DSS.
 
-#### [Search subscription](../fragments/sub/crud/search.md)
+#### [Search subscription](../fragments/sub/crud/search_correct.md)
 
 Confirms that the subscription that was just mutated can be searched for from any DSS.
 
@@ -163,7 +163,7 @@ When queried for a subscription that was created via another DSS, a DSS instance
 
 If it does not, it might be in violation of **[astm.f3548.v21.DSS0005,5](../../../../../requirements/astm/f3548/v21.md)**.
 
-#### [Update subscription](../fragments/sub/crud/update.md)
+#### [Update subscription](../fragments/sub/crud/update_correct.md)
 
 Confirm that the secondary DSS handles the update properly.
 
@@ -189,7 +189,7 @@ Confirm that the subscription that was just mutated is properly synchronized acr
 
 Confirms that the subscription that was just mutated can be retrieved from any DSS, and that it has the expected content.
 
-#### [Search subscription](../fragments/sub/crud/search.md)
+#### [Search subscription](../fragments/sub/crud/search_correct.md)
 
 Confirms that the subscription that was just mutated can be searched for from any DSS, and that it has the expected content.
 

monitoring/uss_qualifier/suites/astm/utm/dss_probing.md

@@ -27,7 +27,7 @@
     <th><a href="../../README.md#checked-in">Checked in</a></th>
   </tr>
   <tr>
-    <td rowspan="41" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
+    <td rowspan="42" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
     <td><a href="../../../requirements/astm/f3548/v21.md">DSS0005,1</a></td>
     <td>Implemented</td>
     <td><a href="../../../scenarios/astm/utm/dss/op_intent_ref_access_control.md">ASTM F3548-21 UTM DSS Operational Intent Reference Access Control</a><br><a href="../../../scenarios/astm/utm/dss/authentication/authentication_validation.md">ASTM SCD DSS: Interfaces authentication</a><br><a href="../../../scenarios/astm/utm/dss/op_intent_ref_key_validation.md">ASTM SCD DSS: Operational Intent Reference Key Validation</a><br><a href="../../../scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md">ASTM SCD DSS: Operational Intent Reference Synchronization</a><br><a href="../../../scenarios/astm/utm/dss/subscription_simple.md">ASTM SCD DSS: Subscription Simple</a><br><a href="../../../scenarios/astm/utm/dss/synchronization/subscription_synchronization.md">ASTM SCD DSS: Subscription Synchronization</a><br><a href="../../../scenarios/astm/utm/dss/subscription_validation.md">ASTM SCD DSS: Subscription Validation</a><br><a href="../../../scenarios/astm/utm/dss/subscription_interactions_deletion.md">ASTM SCD DSS: Subscription and entity deletion interaction</a><br><a href="../../../scenarios/astm/utm/dss/subscription_interactions.md">ASTM SCD DSS: Subscription and entity interaction</a></td>
@@ -197,6 +197,11 @@
     <td>Implemented</td>
     <td><a href="../../../scenarios/astm/utm/dss/subscription_interactions.md">ASTM SCD DSS: Subscription and entity interaction</a></td>
   </tr>
+  <tr>
+    <td><a href="../../../requirements/astm/f3548/v21.md">DSS0210,A2-7-2,4d</a></td>
+    <td>Implemented</td>
+    <td><a href="../../../scenarios/astm/utm/dss/subscription_interactions.md">ASTM SCD DSS: Subscription and entity interaction</a></td>
+  </tr>
   <tr>
     <td><a href="../../../requirements/astm/f3548/v21.md">DSS0210,A2-7-2,5a</a></td>
     <td>Implemented</td>

monitoring/uss_qualifier/suites/astm/utm/f3548_21.md

@@ -35,7 +35,7 @@
     <th><a href="../../README.md#checked-in">Checked in</a></th>
   </tr>
   <tr>
-    <td rowspan="69" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
+    <td rowspan="70" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
     <td><a href="../../../requirements/astm/f3548/v21.md">DSS0005,1</a></td>
     <td>Implemented</td>
     <td><a href="../../../scenarios/astm/utm/prep_planners.md">ASTM F3548 flight planners preparation</a><br><a href="../../../scenarios/astm/utm/dss/op_intent_ref_access_control.md">ASTM F3548-21 UTM DSS Operational Intent Reference Access Control</a><br><a href="../../../scenarios/astm/utm/dss/authentication/authentication_validation.md">ASTM SCD DSS: Interfaces authentication</a><br><a href="../../../scenarios/astm/utm/dss/op_intent_ref_key_validation.md">ASTM SCD DSS: Operational Intent Reference Key Validation</a><br><a href="../../../scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md">ASTM SCD DSS: Operational Intent Reference Synchronization</a><br><a href="../../../scenarios/astm/utm/dss/subscription_simple.md">ASTM SCD DSS: Subscription Simple</a><br><a href="../../../scenarios/astm/utm/dss/synchronization/subscription_synchronization.md">ASTM SCD DSS: Subscription Synchronization</a><br><a href="../../../scenarios/astm/utm/dss/subscription_validation.md">ASTM SCD DSS: Subscription Validation</a><br><a href="../../../scenarios/astm/utm/dss/subscription_interactions_deletion.md">ASTM SCD DSS: Subscription and entity deletion interaction</a><br><a href="../../../scenarios/astm/utm/dss/subscription_interactions.md">ASTM SCD DSS: Subscription and entity interaction</a><br><a href="../../../scenarios/astm/utm/nominal_planning/conflict_equal_priority_not_permitted/conflict_equal_priority_not_permitted.md">Nominal planning: not permitted conflict with equal priority</a><br><a href="../../../scenarios/astm/utm/off_nominal_planning/down_uss.md">Off-Nominal planning: down USS</a><br><a href="../../../scenarios/astm/utm/off_nominal_planning/down_uss_equal_priority_not_permitted.md">Off-Nominal planning: down USS with equal priority conflicts not permitted</a></td>
@@ -210,6 +210,11 @@
     <td>Implemented</td>
     <td><a href="../../../scenarios/astm/utm/dss/subscription_interactions.md">ASTM SCD DSS: Subscription and entity interaction</a></td>
   </tr>
+  <tr>
+    <td><a href="../../../requirements/astm/f3548/v21.md">DSS0210,A2-7-2,4d</a></td>
+    <td>Implemented</td>
+    <td><a href="../../../scenarios/astm/utm/dss/subscription_interactions.md">ASTM SCD DSS: Subscription and entity interaction</a></td>
+  </tr>
   <tr>
     <td><a href="../../../requirements/astm/f3548/v21.md">DSS0210,A2-7-2,5a</a></td>
     <td>Implemented</td>