[uss_qualifier/scenarios/utm/dss/OIRSynchronization] DSS0210,A2-7-2,[…

…1a-b-c-d,3a-b]: Add checks to validate requirements (#613) [uss_qualifier/scenarios/utm/dss/OIRSynchronization] DSSDSS0210,1a-b-c-d,3a-b: Add checks to validate requirements
interuss · Mar 28, 2024 · 34dc884 · 34dc884
1 parent 0eee5df
commit 34dc884
Show file tree

Hide file tree

Showing 16 changed files with 559 additions and 121 deletions.
diff --git a/build/dev/docker-compose.yaml b/build/dev/docker-compose.yaml
@@ -20,7 +20,7 @@ services:
       - dss_internal_network
 
   rid_bootstrapper:
-    image: interuss/dss:v0.13.0
+    image: interuss/dss:v0.14.0
     volumes:
       - dss_component_coordination:/var/dss_component_coordination
       - $PWD/startup:/startup:ro
@@ -31,7 +31,7 @@ services:
       - dss_internal_network
 
   scd_bootstrapper:
-    image: interuss/dss:v0.13.0
+    image: interuss/dss:v0.14.0
     volumes:
       - dss_component_coordination:/var/dss_component_coordination
       - $PWD/startup:/startup:ro
@@ -43,7 +43,7 @@ services:
 
   dss:
     hostname: dss.uss1.localutm
-    image: interuss/dss:v0.13.0
+    image: interuss/dss:v0.14.0
     volumes:
       - $PWD/../test-certs:/var/test-certs:ro
       - dss_component_coordination:/var/dss_component_coordination

diff --git a/...astm/utm/dss/fragments/sub/crud/search.md → .../dss/fragments/sub/crud/search_correct.md b/...astm/utm/dss/fragments/sub/crud/search.md → .../dss/fragments/sub/crud/search_correct.md
@@ -2,9 +2,9 @@
 
 This test step fragment validates that subscriptions can be searched for.
 
-## 🛑 Successful subscription search query check
+## [Search query succeeds](./search_query.md)
 
-If the DSS fails to let us search in the area for which the subscription was created, it is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**.
+Check query succeeds.
 
 ## 🛑 Created Subscription is in search results check
 

diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_query.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/search_query.md
@@ -0,0 +1,7 @@
+# Search subscription query test step fragment
+
+This test step fragment validates that a query to search for subscriptions succeeds.
+
+## 🛑 Successful subscription search query check
+
+If the DSS fails to let us search in the area for which the subscription was created, it is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**.
diff --git a/...astm/utm/dss/fragments/sub/crud/update.md → .../dss/fragments/sub/crud/update_correct.md b/...astm/utm/dss/fragments/sub/crud/update.md → .../dss/fragments/sub/crud/update_correct.md
@@ -2,9 +2,9 @@
 
 This test step fragment validates that subscriptions can be updated.
 
-## 🛑 Subscription can be mutated check
+## [Update query succeeds](./update_query.md)
 
-If a subscription cannot be modified with a valid set of parameters, the DSS is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**.
+Check query succeeds.
 
 ## 🛑 Mutate subscription response format conforms to spec check
 

diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_query.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/sub/crud/update_query.md
@@ -0,0 +1,7 @@
+# Update subscription query test step fragment
+
+This test step fragment validates that a query to update a subscription succeeds.
+
+## 🛑 Subscription can be mutated check
+
+If a subscription cannot be updated with a valid set of parameters, the DSS is failing to meet **[astm.f3548.v21.DSS0005,5](../../../../../../../requirements/astm/f3548/v21.md)**.
diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.md b/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.md
@@ -126,4 +126,33 @@ The subscription created on a DSS instance must be retrievable from all other DS
 
 If the subscription does not exist on one of the other DSS instances, one of the instances fails to comply with **[astm.f3548.v21.DSS0210,A2-7-2,4a](../../../../requirements/astm/f3548/v21.md)**.
 
+
+## Expiration of subscriptions removes them test case
+
+This test case validates that expired subscriptions (created explicitly) are removed from all DSS instances.
+To validate this requirement, the subscriptions that were previously created at each DSS instance are modified so that they expire shortly after the modification.
+Then it is checked that all the associated subscriptions were removed from all the DSS instances by searching for them in their planning area.
+Do note that they are not queried directly, as it is deemed acceptable for expired subscription that were not explicitly deleted to still be retrievable.
+
+### Expire explicit subscriptions at every DSS in sequence test step
+
+This test step will modify explicit subscriptions that were previously created at each DSS instance so that they expire shortly after the modification.
+
+Note that this step is run once for each involved DSS (that is, once for the primary DSS and once for every secondary DSS)
+
+#### [Modify subscription on a DSS instance so that it expires soon](./fragments/sub/crud/update_query.md)
+
+Check that the subscription modifications succeed and wait for them to expire.
+
+#### [Search for subscriptions from all other DSS instances succeeds](./fragments/sub/crud/search_query.md)
+
+Check that query succeeds.
+
+#### 🛑 Subscription does not exist on all other DSS instances check
+
+The explicit subscription expired on a DSS instance must have been removed from all other DSS instances.
+
+If the subscription still exists on one of the other DSS instances, one of the instances fails to comply with **[astm.f3548.v21.DSS0210,A2-7-2,4d](../../../../requirements/astm/f3548/v21.md)**.
+
+
 ## [Cleanup](./clean_workspace.md)
diff --git a/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.py b/monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.py
@@ -12,8 +12,10 @@
 from uas_standards.astm.f3548.v21.constants import Scope
 
 from monitoring.monitorlib import fetch
+from monitoring.monitorlib.delay import sleep
 from monitoring.monitorlib.fetch import QueryError
 from monitoring.monitorlib.geotemporal import Volume4D
+from monitoring.monitorlib.temporal import Time
 from monitoring.prober.infrastructure import register_resource_type
 from monitoring.uss_qualifier.configurations.configuration import ParticipantID
 from monitoring.uss_qualifier.resources.astm.f3548.v21 import PlanningAreaResource
@@ -55,7 +57,7 @@ class SubscriptionInteractions(TestScenario):
 
     # Reference times for the subscriptions and operational intents
     _time_start: datetime
-    _time_send: datetime
+    _time_end: datetime
 
     _manager: str
 
@@ -120,6 +122,9 @@ def run(self, context: ExecutionContext):
         self._steps_create_subs_at_each_dss()
         self.end_test_case()
 
+        self.begin_test_case("Expiration of subscriptions removes them")
+        self._steps_expire_subs_at_each_dss()
+        self.end_test_case()
         self.end_test_scenario()
 
     def _step_create_background_sub(self):
@@ -129,7 +134,7 @@ def _step_create_background_sub(self):
         sub_now_params = self._planning_area.get_new_subscription_params(
             subscription_id=self._background_sub_id,
             start_time=self._time_start,
-            duration=self._time_send - self._time_start,
+            duration=self._time_end - self._time_start,
             # This is a planning area without constraint processing
             notify_for_op_intents=True,
             notify_for_constraints=False,
@@ -188,7 +193,7 @@ def _implicit_subs_check(
                 state=OperationalIntentState.Accepted,
                 uss_base_url="https://example.interuss.org/oir_base_url",
                 time_start=datetime.utcnow(),
-                time_end=self._time_send + timedelta(minutes=10),
+                time_end=self._time_end + timedelta(minutes=10),
                 subscription_id=None,
                 implicit_sub_base_url="https://example.interuss.org/sub_base_url",
             )
@@ -235,7 +240,7 @@ def _implicit_subs_check(
                 state=OperationalIntentState.Accepted,
                 uss_base_url="https://example.interuss.org/oir_base_url_bis",  # dummy modification of the OIR
                 time_start=datetime.utcnow(),
-                time_end=self._time_send + timedelta(minutes=10),
+                time_end=self._time_end + timedelta(minutes=10),
                 subscription_id=self._current_oirs[oir_id].subscription_id,
             )
 
@@ -283,7 +288,7 @@ def _steps_create_subs_at_each_dss(self):
         common_params = self._planning_area.get_new_subscription_params(
             subscription_id="",
             start_time=self._time_start,
-            duration=self._time_send - self._time_start,
+            duration=self._time_end - self._time_start,
             notify_for_op_intents=True,
             notify_for_constraints=False,
         )
@@ -342,6 +347,76 @@ def _steps_create_subs_at_each_dss(self):
 
             self.end_test_step()
 
+    def _steps_expire_subs_at_each_dss(self):
+        self.begin_test_step("Expire explicit subscriptions at every DSS in sequence")
+        for i, dss in enumerate([self._dss] + self._secondary_instances):
+            sub_id = self._sub_ids[i]
+            sub_params = self._planning_area.get_new_subscription_params(
+                subscription_id=sub_id,
+                start_time=datetime.utcnow(),
+                duration=timedelta(seconds=SUBSCRIPTION_EXPIRY_DELAY_SEC),
+                notify_for_op_intents=True,
+                notify_for_constraints=False,
+            )
+
+            with self.check(
+                "Subscription can be mutated",
+                [dss.participant_id],
+            ) as check:
+                sub = self._dss.upsert_subscription(
+                    **sub_params,
+                    version=self._current_subs[sub_id].version,
+                )
+                self.record_query(sub)
+                if not sub.success:
+                    check.record_failed(
+                        summary="Update subscription query failed",
+                        details=f"Failed to update a subscription on DSS instance with code {sub.status_code}: {sub.error_message}",
+                        query_timestamps=[sub.request.timestamp],
+                    )
+            self._current_subs.pop(sub_id)
+
+        sleep(
+            timedelta(seconds=WAIT_FOR_EXPIRY_SEC),
+            "waiting for subscriptions to expire",
+        )
+
+        for i, dss in enumerate([self._dss] + self._secondary_instances):
+            sub_id = self._sub_ids[i]
+            for other_dss in {self._dss, *self._secondary_instances} - {dss}:
+                other_dss_subs = other_dss.query_subscriptions(
+                    Volume4D(
+                        volume=self._planning_area.volume,
+                        time_start=Time(self._time_start),
+                        time_end=Time(self._time_end),
+                    ).to_f3548v21()
+                )
+                self.record_query(other_dss_subs)
+
+                with self.check(
+                    "Successful subscription search query",
+                    dss.participant_id,
+                ) as check:
+                    if not other_dss_subs.success:
+                        check.record_failed(
+                            summary="Search subscriptions query failed",
+                            details=f"Failed to search for subscriptions from DSS with code {other_dss_subs.status_code}: {other_dss_subs.error_message}",
+                            query_timestamps=[other_dss_subs.request.timestamp],
+                        )
+
+                with self.check(
+                    "Subscription does not exist on all other DSS instances",
+                    [dss.participant_id, other_dss.participant_id],
+                ) as check:
+                    if sub_id in other_dss_subs.subscriptions:
+                        check.record_failed(
+                            summary="Subscription that expired on a DSS instance was found on another instance",
+                            details=f"Subscription {sub_id} expired on DSS instance {dss.participant_id} was found on DSS instance {other_dss.participant_id}.",
+                            query_timestamps=[other_dss_subs.request.timestamp],
+                        )
+
+        self.end_test_step()
+
     def _create_sub_with_params(
         self, params: SubscriptionParams
     ) -> Tuple[Subscription, List[OperationalIntentReference], fetch.Query]:
@@ -362,7 +437,7 @@ def _setup_case(self):
 
         # Subscription from now to 20 minutes in the future
         self._time_start = datetime.utcnow()
-        self._time_send = self._time_start + timedelta(minutes=20)
+        self._time_end = self._time_start + timedelta(minutes=20)
 
         # Multiple runs of the scenario seem to rely on the same instance:
         # thus we need to reset the state of the scenario before running it.

diff --git a/...alifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md b/...alifier/scenarios/astm/utm/dss/synchronization/op_intent_ref_synchronization.md
@@ -57,6 +57,12 @@ Retrieve and validate synchronization of the created operational intent at every
 
 Check that read query succeeds.
 
+#### Newly created OIR can be consistently retrieved from all DSS instances check
+
+If the operational intent retrieved from a secondary DSS instance is not consistent with the newly created one on the
+primary DSS instance, this check will fail per **[astm.f3548.v21.DSS0210,A2-7-2,1a](../../../../../requirements/astm/f3548/v21.md)**
+and **[astm.f3548.v21.DSS0210,A2-7-2,1d](../../../../../requirements/astm/f3548/v21.md)**.
+
 #### [OIR is synchronized](../fragments/oir/sync.md)
 
 Confirm that each DSS provides direct access to the created operational intent reference.
@@ -70,6 +76,12 @@ Search for and validate synchronization of the created operational intent at eve
 
 Check that search query succeeds.
 
+#### Newly created OIR can be consistently searched for from all DSS instances check
+
+If the operational intent searched from a secondary DSS instance is not consistent with the newly created one on the
+primary DSS instance, this check will fail per **[astm.f3548.v21.DSS0210,A2-7-2,1a](../../../../../requirements/astm/f3548/v21.md)**
+and **[astm.f3548.v21.DSS0210,A2-7-2,1c](../../../../../requirements/astm/f3548/v21.md)**.
+
 #### [OIR is synchronized](../fragments/oir/sync.md)
 
 Confirm that each DSS returns the operational intent in relevant search results.
@@ -100,6 +112,12 @@ Retrieve and validate synchronization of the updated operational intent at every
 
 Check that read query succeeds.
 
+#### Updated OIR can be consistently retrieved from all DSS instances check
+
+If the operational intent retrieved from a secondary DSS instance is not consistent with the updated one on the
+primary DSS instance, this check will fail per **[astm.f3548.v21.DSS0210,A2-7-2,1b](../../../../../requirements/astm/f3548/v21.md)**
+and **[astm.f3548.v21.DSS0210,A2-7-2,1d](../../../../../requirements/astm/f3548/v21.md)**.
+
 #### [OIR is synchronized](../fragments/oir/sync.md)
 
 Confirm that each DSS provides direct access to the updated operational intent reference.
@@ -113,6 +131,12 @@ Search for and validate synchronization of the updated operational intent at eve
 
 Check that search query succeeds.
 
+#### Updated OIR can be consistently searched for from all DSS instances check
+
+If the operational intent searched from a secondary DSS instance is not consistent with the updated one on the
+primary DSS instance, this check will fail per **[astm.f3548.v21.DSS0210,A2-7-2,1b](../../../../../requirements/astm/f3548/v21.md)**
+and **[astm.f3548.v21.DSS0210,A2-7-2,1c](../../../../../requirements/astm/f3548/v21.md)**.
+
 #### [OIR is synchronized](../fragments/oir/sync.md)
 
 Confirm that each DSS returns the operational intent in relevant search results.
@@ -140,9 +164,24 @@ Verify that the operational intent reference's version fields are as expected.
 
 Attempt to query and search for the deleted operational intent reference in various ways
 
-#### 🛑 Secondary DSS should not return the deleted operational intent reference check
+#### [Get OIR query](../fragments/oir/crud/read_query.md)
+
+Check that read query succeeds.
+
+#### 🛑 Deleted OIR cannot be retrieved from all DSS instances check
+
+If a DSS returns an operational intent reference that was previously successfully deleted from the primary DSS,
+either one of the primary DSS or the DSS that returned the operational intent reference is in violation of **[astm.f3548.v21.DSS0210,2a](../../../../../requirements/astm/f3548/v21.md)**
+and **[astm.f3548.v21.DSS0210,A2-7-2,3b](../../../../../requirements/astm/f3548/v21.md)**.
+
+#### [Search OIR](../fragments/oir/crud/search_query.md)
+
+Check that search query succeeds.
+
+#### 🛑 Deleted OIR cannot be searched for from all DSS instances check
 
 If a DSS returns an operational intent reference that was previously successfully deleted from the primary DSS,
-either one of the primary DSS or the DSS that returned the operational intent reference is in violation of **[astm.f3548.v21.DSS0210,2a](../../../../../requirements/astm/f3548/v21.md)**.
+either one of the primary DSS or the DSS that returned the operational intent reference is in violation of **[astm.f3548.v21.DSS0210,2a](../../../../../requirements/astm/f3548/v21.md)**
+and **[astm.f3548.v21.DSS0210,A2-7-2,3a](../../../../../requirements/astm/f3548/v21.md)**.
 
 ## [Cleanup](../clean_workspace.md)