[uss_qualifier/documentation] Split clean_workspace fragment into fragments for different entity types

monitoring/uss_qualifier/reports/globally_expanded/generate.py

@@ -586,21 +586,28 @@ def _update_links(element: marko.element.Element, origin_filename: str) -> None:
 def _add_section_numbers(elements: Sequence[marko.element.Element]) -> None:
     heading_level = 2
     levels = [0]
+    headings = [None]
     prev_heading = None
     for i, element in enumerate(elements):
         if isinstance(element, marko.block.Heading):
             if element.level < heading_level:
                 levels = levels[0 : element.level - heading_level]
+                headings = headings[0 : element.level - heading_level]
                 heading_level = element.level
                 levels[-1] += 1
+                headings[-1] = text_of(element)
             elif element.level == heading_level:
                 levels[-1] += 1
+                headings[-1] = text_of(element)
             elif element.level == heading_level + 1:
                 levels.append(1)
+                headings.append(text_of(element))
                 heading_level += 1
             else:
+                headings.append(text_of(element))
+                heading_trace = " -> ".join(headings)
                 raise ValueError(
-                    f"Encountered a level {element.level} heading ({text_of(element)}) at element {i} following a level {heading_level} heading ({prev_heading}); expected heading levels to increase by 1 level at a time"
+                    f"Encountered a level {element.level} heading ({text_of(element)}) at element {i} following a level {heading_level} heading ({prev_heading}); expected heading levels to increase by 1 level at a time.  Trace: {heading_trace}"
                 )
             prev_heading = text_of(element)
             section_number = ".".join(str(level) for level in levels) + ". "

monitoring/uss_qualifier/scenarios/astm/utm/dss/authentication/authentication_validation.md

@@ -40,7 +40,15 @@ Optional scopes that will allow the scenario to provide additional coverage:
 
 ## Setup test case
 
-### [Ensure clean workspace test step](../clean_workspace.md)
+To perform this scenario, the area must be clear of test entities with the IDs we intend to use.
+
+### Ensure clean workspace test step
+
+#### [Clean any existing OIRs with known test IDs](../clean_workspace_op_intents.md)
+
+#### [Clean any existing subscriptions with known test IDs](../clean_workspace_subs.md)
+
+#### [Clean any existing constraint references with known test IDs](../clean_workspace_constraints.md)
 
 This step ensures that the availability for the test identifier is set to `Unknown`.
 
@@ -540,7 +548,13 @@ it is in violation of **[astm.f3548.v21.DSS0005,4](../../../../../requirements/a
 
 Check response format of a search.
 
-## [Cleanup](../clean_workspace.md)
+## Cleanup
+
+### [Clean any existing OIRs with known test IDs](../clean_workspace_op_intents.md)
+
+### [Clean any existing subscriptions with known test IDs](../clean_workspace_subs.md)
+
+### [Clean any existing constraint references with known test IDs](../clean_workspace_constraints.md)
 
 ### [Availability can be requested](../fragments/availability/read.md)
 

monitoring/uss_qualifier/scenarios/astm/utm/dss/clean_workspace_constraints.md

@@ -0,0 +1,18 @@
+# Ensure clean workspace test step fragment
+
+Ensure a clean workspace for testing interactions with a DSS by removing any constraint references from the DSS that may have been left behind from testing efforts.
+
+## 🛑 Constraint references can be queried by ID check
+
+If an existing constraint reference cannot directly be queried by its ID, or if for a non-existing one the DSS replies with a status code different than 404,
+the DSS implementation is in violation of **[astm.f3548.v21.DSS0005,3](../../../../requirements/astm/f3548/v21.md)**.
+
+## 🛑 Constraint references can be searched for check
+
+A client with valid credentials should be allowed to search for constraint references in a given area.
+Otherwise, the DSS is not in compliance with **[astm.f3548.v21.DSS0005,4](../../../../requirements/astm/f3548/v21.md)**.
+
+## 🛑 Constraint reference removed check
+
+If an existing constraint cannot be deleted by its manager when providing the proper ID and OVN, the DSS implementation is in violation of
+**[astm.f3548.v21.DSS0005,3](../../../../requirements/astm/f3548/v21.md)**.

monitoring/uss_qualifier/scenarios/astm/utm/dss/clean_workspace_op_intents.md

@@ -0,0 +1,18 @@
+# Ensure clean workspace test step fragment
+
+Ensure a clean workspace for testing interactions with a DSS by removing any operational intent references from the DSS that may have been left behind from testing efforts.
+
+## 🛑 Operational intent references can be queried by ID check
+
+If an existing operational intent reference cannot directly be queried by its ID, or if for a non-existing one the DSS replies with a status code different than 404,
+the DSS implementation is in violation of **[astm.f3548.v21.DSS0005,1](../../../../requirements/astm/f3548/v21.md)**.
+
+## 🛑 Operational intent references can be searched for check
+
+A client with valid credentials should be allowed to search for operational intents in a given area.
+Otherwise, the DSS is not in compliance with **[astm.f3548.v21.DSS0005,2](../../../../requirements/astm/f3548/v21.md)**.
+
+## 🛑 Operational intent reference removed check
+
+If an existing operational intent cannot be deleted when providing the proper ID and OVN, the DSS implementation is in violation of
+**[astm.f3548.v21.DSS0005,1](../../../../requirements/astm/f3548/v21.md)**.

monitoring/uss_qualifier/scenarios/astm/utm/dss/clean_workspace_subs.md

@@ -0,0 +1,15 @@
+# Ensure clean workspace test step fragment
+
+Ensure a clean workspace for testing interactions with a DSS by removing any subscriptions from the DSS that may have been left behind from testing efforts.
+
+## 🛑 Successful subscription search query check
+
+**[astm.f3548.v21.DSS0005,5](../../../../requirements/astm/f3548/v21.md)** requires the implementation of the DSS endpoint to allow callers to retrieve the subscriptions they created.
+
+## 🛑 Subscription can be queried by ID check
+
+If the DSS cannot be queried for the existing test ID, the DSS is likely not implementing **[astm.f3548.v21.DSS0005,5](../../../../requirements/astm/f3548/v21.md)** correctly.
+
+## 🛑 Subscription can be deleted check
+
+**[astm.f3548.v21.DSS0005,5](../../../../requirements/astm/f3548/v21.md)** requires the implementation of the DSS endpoint to allow callers to delete subscriptions they created.

monitoring/uss_qualifier/scenarios/astm/utm/dss/constraint_ref_simple.md

@@ -24,7 +24,7 @@ Verifies the behavior of a DSS for simple interactions pertaining to constraint
 
 ## Setup test case
 
-### [Ensure clean workspace test step](./clean_workspace.md)
+### [Ensure clean workspace test step](./clean_workspace_constraints.md)
 
 This step ensures that no entities with the known test IDs exists in the DSS.
 
@@ -76,4 +76,4 @@ This step verifies that an existing CR cannot be mutated with an incorrect OVN.
 If the DSS under test allows the qualifier to mutate an existing CR with a request that provided an incorrect OVN,
 it is in violation of **[astm.f3548.v21.DSS0005,3](../../../../requirements/astm/f3548/v21.md)**
 
-## [Cleanup](./clean_workspace.md)
+## [Cleanup](./clean_workspace_constraints.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/oir_implicit_sub_handling.md

@@ -24,9 +24,11 @@ Checks that implicit subscriptions are properly created, mutated and cleaned up.
 
 ## Setup test case
 
-### [Ensure clean workspace test step](clean_workspace.md)
+### Ensure clean workspace test step
 
-This step ensures that no OIRs with the known test IDs exists in the DSS.
+#### [Clean any existing OIRs with known test IDs](clean_workspace_op_intents.md)
+
+#### [Clean any existing subscriptions with known test IDs](clean_workspace_subs.md)
 
 ## Single OIR implicit subscription is removed upon OIR deletion test case
 
@@ -142,9 +144,13 @@ the DSS is in violation of **[astm.f3548.v21.DSS0005,1](../../../../requirements
 
 This test case verifies that implicit subscriptions are properly removed if they become unnecessary following the mutation of an OIR.
 
-### [Ensure clean workspace test step](clean_workspace.md)
+### Ensure clean workspace test step
+
+<!-- TODO(Shastick): Why do we need to reclean the workspace at this point?  We already ensured it was clean before starting the test; don't we know exactly what happened in the test and therefore know that it's already clean (or something failed)?  If a previous test case created something that we don't need/want in later test cases, the original test case should clean up at the end of the test case. -->
+
+#### [Clean any existing OIRs with known test IDs](clean_workspace_op_intents.md)
 
-Reset the workspace for this test case.
+#### [Clean any existing subscriptions with known test IDs](clean_workspace_subs.md)
 
 ### Create two OIRs with implicit subscription test step
 
@@ -190,9 +196,13 @@ the DSS is failing to properly manage implicit subscriptions for OIRs, and is th
 
 This test case checks that a DSS will properly expand an implicit subscription to cover an OIR that is being attached to it.
 
-### [Ensure clean workspace test step](clean_workspace.md)
+### Ensure clean workspace test step
+
+<!-- TODO(Shastick): Why do we need to reclean the workspace at this point?  We already ensured it was clean before starting the test; don't we know exactly what happened in the test and therefore know that it's already clean (or something failed)?  If a previous test case created something that we don't need/want in later test cases, the original test case should clean up at the end of the test case. -->
+
+#### [Clean any existing OIRs with known test IDs](clean_workspace_op_intents.md)
 
-Reset the workspace for this test case.
+#### [Clean any existing subscriptions with known test IDs](clean_workspace_subs.md)
 
 ### Create an OIR with implicit subscription test step
 
@@ -221,6 +231,8 @@ in which case the DSS is in violation of **[astm.f3548.v21.DSS0005,1](../../../.
 
 Ensure that the attached implicit subscription has been expanded
 
+## Cleanup
 
+### [Remove OIRs created during this test](clean_workspace_op_intents.md)
 
-## [Cleanup](./clean_workspace.md)
+### [Remove subscriptions created during this test](clean_workspace_subs.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/op_intent_ref_access_control.md

@@ -56,7 +56,9 @@ Makes sure that the DSS is in a clean and expected state before running the test
 
 The setup will create two separate operational intent references: one for each set of the available credentials.
 
-### [Ensure clean workspace](clean_workspace.md) test step
+### Ensure clean workspace test step
+
+#### [Clean any existing OIRs](clean_workspace_op_intents.md)
 
 #### ⚠️ Any existing operational intent reference has been removed check
 
@@ -110,4 +112,4 @@ in violation of **[astm.f3548.v21.OPIN0035](../../../../requirements/astm/f3548/
 If an operational intent reference can be deleted by a client which did not create it, the DSS implementation is
 in violation of **[astm.f3548.v21.OPIN0035](../../../../requirements/astm/f3548/v21.md)**.
 
-## [Cleanup](clean_workspace.md)
+## [Cleanup](clean_workspace_op_intents.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/op_intent_ref_key_validation.md

@@ -25,9 +25,9 @@ provide all OVNs for all currently relevant entities.
 
 ## Setup test case
 
-### [Ensure clean workspace test step](./clean_workspace.md)
+### [Ensure clean workspace test step](./clean_workspace_op_intents.md)
 
-This step ensures that no entities with the known test IDs exists in the DSS.
+This step ensures that no operational intent references with the known test IDs exists in the DSS.
 
 ## Key validation on creation test case
 
@@ -141,4 +141,4 @@ The expectation is that the DSS will require the missing OVN.
 
 #### [Non de-conflicted mutation request fails](fragments/oir/crud/update_conflict.md)
 
-## [Cleanup](./clean_workspace.md)
+## [Cleanup](./clean_workspace_op_intents.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/op_intent_ref_key_validation.py

@@ -429,11 +429,6 @@ def _ensure_clean_workspace_step(self):
         for oir_id in self._oir_ids:
             test_step_fragments.cleanup_op_intent(self, self._dss, oir_id)
 
-        # Also drop any subs we might own and that could interfere
-        test_step_fragments.cleanup_active_subs(
-            self, self._dss, self._planning_area_volume4d.to_f3548v21()
-        )
-
     def cleanup(self):
         self.begin_cleanup()
         self._ensure_clean_workspace_step()

monitoring/uss_qualifier/scenarios/astm/utm/dss/op_intent_ref_simple.md

@@ -24,13 +24,13 @@ Verifies the behavior of a DSS for simple interactions pertaining to operational
 
 ## Setup test case
 
-### [Ensure clean workspace test step](./clean_workspace.md)
+### Ensure clean workspace test step
 
-This step ensures that no entities with the known test IDs exists in the DSS.
+#### [Clean any existing OIRs with known test IDs](clean_workspace_op_intents.md)
 
-### [Create an operational intent reference test step](./fragments/oir/crud/create_query.md)
+### Create an operational intent reference test step
 
-Create an operational intent reference to be used in this scenario.
+#### [Create an operational intent reference to be used in this scenario.](./fragments/oir/crud/create_query.md)
 
 ## Deletion requires correct OVN test case
 
@@ -76,4 +76,4 @@ This step verifies that an existing OIR cannot be mutated with an incorrect OVN.
 If the DSS under test allows the qualifier to mutate an existing OIR with a request that provided an incorrect OVN,
 it is in violation of **[astm.f3548.v21.DSS0005,1](../../../../requirements/astm/f3548/v21.md)**
 
-## [Cleanup](./clean_workspace.md)
+## [Cleanup](./clean_workspace_op_intents.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/op_intent_ref_simple.py

@@ -296,11 +296,6 @@ def _ensure_clean_workspace_step(self):
         # Make sure the OIR IDs we are going to use are available
         test_step_fragments.cleanup_op_intent(self, self._dss, self._oir_id)
 
-        # Also drop any subs we might own and that could interfere
-        test_step_fragments.cleanup_active_subs(
-            self, self._dss, self._planning_area_volume4d.to_f3548v21()
-        )
-
     def cleanup(self):
         self.begin_cleanup()
         self._ensure_clean_workspace_step()

monitoring/uss_qualifier/scenarios/astm/utm/dss/op_intent_ref_state_transitions.md

@@ -25,7 +25,9 @@ A `resources.interuss.IDGeneratorResource` that will be used to generate the IDs
 
 Makes sure that the DSS is in a clean and expected state before running the test, and that the passed resources work as required.
 
-### [Ensure clean workspace](clean_workspace.md) test step
+### Ensure clean workspace test step
+
+#### [Clean any existing OIRs with known test IDs](clean_workspace_op_intents.md)
 
 #### [No OIR exists](fragments/oir/cleanup_required.md)
 
@@ -122,4 +124,4 @@ If the DSS allows a client with the `utm.strategic_coordination` scope to transi
 it is in violation of **[astm.f3548.v21.SCD0100](../../../../requirements/astm/f3548/v21.md)** and **[astm.f3548.v21.DSS0005,1](../../../../requirements/astm/f3548/v21.md)**.
 
 
-## [Cleanup](clean_workspace.md)
+## [Cleanup](clean_workspace_op_intents.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions.md

@@ -28,7 +28,11 @@ Create and mutate subscriptions as well as entities, and verify that the DSS han
 
 ## Setup test case
 
-### [Ensure clean workspace test step](clean_workspace.md)
+### Ensure clean workspace test step
+
+#### [Clean any existing OIRs with known test IDs](clean_workspace_op_intents.md)
+
+#### [Clean any existing subscriptions with known test IDs](clean_workspace_subs.md)
 
 This step ensures that no subscriptions and OIRs with the known test IDs exists in the DSS deployment.
 
@@ -155,4 +159,8 @@ The explicit subscription expired on a DSS instance must have been removed from
 If the subscription still exists on one of the other DSS instances, one of the instances fails to comply with **[astm.f3548.v21.DSS0210,A2-7-2,4d](../../../../requirements/astm/f3548/v21.md)**.
 
 
-## [Cleanup](./clean_workspace.md)
+## Cleanup
+
+### [Clean any straggling OIRs with known test IDs](clean_workspace_op_intents.md)
+
+### [Clean any straggling subscriptions with known test IDs](clean_workspace_subs.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_interactions_deletion.md

@@ -28,10 +28,11 @@ Create and mutate subscriptions as well as entities, and verify that the DSS han
 
 ## Setup test case
 
-### [Ensure clean workspace test step](clean_workspace.md)
+### Ensure clean workspace test step
 
-This step ensures that no subscriptions and OIRs with the known test IDs exists in the DSS deployment.
+#### [Clean any existing OIRs with known test IDs](clean_workspace_op_intents.md)
 
+#### [Clean any existing subscriptions with known test IDs](clean_workspace_subs.md)
 
 ## Subscription deletion is reflected on all DSS instances test case
 
@@ -114,4 +115,8 @@ OIR's extents. This does not include subscriptions deleted earlier.
 If the DSS includes a deleted subscription, it fails to implement **[astm.f3548.v21.DSS0210,A2-7-2,5c](../../../../requirements/astm/f3548/v21.md)**.
 
 
-## [Cleanup](./clean_workspace.md)
+## Cleanup
+
+### [Clean any straggling OIRs with known test IDs](clean_workspace_op_intents.md)
+
+### [Clean any straggling subscriptions with known test IDs](clean_workspace_subs.md)

monitoring/uss_qualifier/scenarios/astm/utm/dss/subscription_simple.md

@@ -24,9 +24,9 @@ Perform basic operations on a single DSS instance to create, update and delete s
 
 ## Setup test case
 
-### [Ensure clean workspace test step](clean_workspace.md)
+### Ensure clean workspace test step
 
-This step ensures that no subscription with the known test ID exists in the DSS.
+#### [Clean any existing subscriptions with known test IDs](clean_workspace_subs.md)
 
 ## Subscription Simple test case
 
@@ -193,4 +193,6 @@ If the DSS fails to let us search in the area for which the subscription was jus
 
 If the DSS returns the deleted subscription in a search that covers the area it was originally created for, the DSS is not properly implementing **[astm.f3548.v21.DSS0005,5](../../../../requirements/astm/f3548/v21.md)**.
 
-## [Cleanup](./clean_workspace.md)
+## Cleanup
+
+### [Clean any straggling subscriptions with known test IDs](clean_workspace_subs.md)