Merge branch 'main' into interactions_and_invalid_op_checks

monitoring/uss_qualifier/configurations/dev/library/resources.yaml

@@ -134,7 +134,7 @@ che_conflicting_flights:
     file:
       path: file://./test_data/che/flight_intents/conflicting_flights.yaml
       # Note that this hash_sha512 field can be safely deleted if the content changes
-      hash_sha512: ac109b89c95381ba4685d6db5f7064dac64875a8875e58989aad19d91d2e7b10e568d5091749b5ea2b0892191a84486971d999b63ac8c73fd4249131111a6c58
+      hash_sha512: c35e3536d63b7dd521042cefa094dd1ecd2d3feaf31997ce6a2902361b85c42dec636bec62df853157e46e08d3fc811c00fedfd6dfe4b8bbd0506149cfeb4a17
 
 che_invalid_flight_intents:
   $content_schema: monitoring/uss_qualifier/resources/definitions/ResourceDeclaration.json

monitoring/uss_qualifier/scenarios/README.md

@@ -10,7 +10,7 @@ A test scenario is separated into of a list of test cases, each of which are sep
 
 ### Test cases
 
-1. A test case is a single wholistic operation or action performed as part of a larger test scenario.
+1. A test case is a single holistic operation or action performed as part of a larger test scenario.
     * Test cases are like acts in the "play" of the test scenario they are a part of.
     * Test cases are typically the "gray headers” of the overview sequence diagrams.
 2. A given test case belongs to exactly one test scenario.
@@ -22,14 +22,16 @@ A test scenario is separated into of a list of test cases, each of which are sep
 1. A test step is a single task that must be performed in order to accomplish its associated test case.
     * Test steps are like scenes in the "play/act" of the test scenario/test case they are a part of.
 2. A given test step belongs to exactly one test case.
-3. A test step may have a list of checks associated with it.
+3. A test step should generally have a list of checks associated with it.
 
 ### Checks
 
 1. A check is the lowest-level thing automated testing does – it is a single pass/fail evaluation of a single criterion for a requirement.
 2. A check evaluates information collected during the actions performed for a test step.
 3. A given check belongs to exactly one test step.
 4. Each check defines which requirement(s) are not met if the check fails.
+5. In nearly all cases, the test participant(s) to which a check pertains should be specified when performing the check.
+    * If the test participant is not specified, then either everyone involved in a test or no one is responsible for meeting the requirements of that check, and this is very rarely appropriate.
 
 ## Creation
 
@@ -79,7 +81,23 @@ A scenario must document at least one test case (otherwise the scenario is doing
 
 Each test case in the documentation must document at least one test step (otherwise nothing is happening in the test case).  Each test step must be documented via a subsection of the parent test case named with a " test step" suffix (example: `### Injection test step`).
 
-If the entire test step heading is enclosed in a link, the contents of that linked file will be used to populate the test step (example: `### [Plan flight test step](plan_flight_step.md)`) and any content in this section will be ignored.  The linked file must follow the format requirements for a test step, starting with the first line being a top-level heading ending with " test step" (example: `# Plan flight test step`).
+If the entire test step heading is enclosed in a link, the contents of that linked file will be used to pre-populate the test step (example: `### [Plan flight test step](plan_flight_fragment.md)`) before reading the content in this section.  The linked file must follow the format requirements for a test step, but with the first line being a top-level heading ending with " test step fragment" (example: `# Plan flight test step fragment`).
+
+Multiple test step fragments may be included in a test step by linking to the test step fragment in a heading one level lower than the test step itself, and these lower-level headings may be combined with [checks](#test-checks) specific to the test step; for instance:
+
+```markdown
+### Plan flight test step
+
+This step does (a particular thing).
+
+#### [Actually plan flight](plan_flight_fragment.md)
+
+#### Special check
+
+If the system under test doesn't Foo, then requirement **Bar** will not be met.
+
+#### [Ensure flight baz](check_flight_baz_fragment.md)
+```
 
 ### Test checks
 

monitoring/uss_qualifier/scenarios/astm/netrid/v19/dss/test_steps/clean_workspace.md

@@ -1,4 +1,4 @@
-# Ensure clean workspace test step
+# Ensure clean workspace test step fragment
 
 This page describes the content of a common test step that ensures a clean workspace for testing interactions with a DSS
 

monitoring/uss_qualifier/scenarios/astm/netrid/v19/dss/test_steps/delete_isa.md

@@ -1,4 +1,4 @@
-# Delete ISA test step
+# Delete ISA test step fragment
 
 This page describes the content of a common test step where a deletion of an ISA should be successful.
 See `DSSWrapper.del_isa` in [`dss_wrapper.py`](../../../dss_wrapper.py).

monitoring/uss_qualifier/scenarios/astm/netrid/v19/dss/test_steps/get_isa.md

@@ -1,4 +1,4 @@
-# Get ISA test step
+# Get ISA test step fragment
 
 This page describes the content of a common test step where a request for an ISA by its ID should be successful.
 See `ISAValidator` in [`isa_validator.py`](../../../common/dss/isa_validator.py).

monitoring/uss_qualifier/scenarios/astm/netrid/v19/dss/test_steps/put_isa.md

@@ -1,4 +1,4 @@
-# Create or update ISA test step
+# Create or update ISA test step fragment
 
 This page describes the content of a common test step where a creation or an update of an ISA should be successful.
 See `DSSWrapper.put_isa` in [`dss_wrapper.py`](../../../dss_wrapper.py).

monitoring/uss_qualifier/scenarios/astm/netrid/v19/dss/test_steps/search_isas.md

@@ -1,4 +1,4 @@
-# Search ISAs test step
+# Search ISAs test step fragment
 
 This page describes the content of a common test step where a search for ISAs should be successful.
 See `DSSWrapper.search_isa` in [`dss_wrapper.py`](../../../dss_wrapper.py).

monitoring/uss_qualifier/scenarios/astm/netrid/v22a/dss/test_steps/clean_workspace.md

@@ -1,4 +1,4 @@
-# Ensure clean workspace test step
+# Ensure clean workspace test step fragment
 
 This page describes the content of a common test step that ensures a clean workspace for testing interactions with a DSS
 

monitoring/uss_qualifier/scenarios/astm/netrid/v22a/dss/test_steps/delete_isa.md

@@ -1,4 +1,4 @@
-# Delete ISA test step
+# Delete ISA test step fragment
 
 This page describes the content of a common test step where a deletion of an ISA should be successful.
 See `DSSWrapper.del_isa` in [`dss_wrapper.py`](../../../dss_wrapper.py).

monitoring/uss_qualifier/scenarios/astm/netrid/v22a/dss/test_steps/get_isa.md

@@ -1,4 +1,4 @@
-# Get ISA test step
+# Get ISA test step fragment
 
 This page describes the content of a common test step where a request for an ISA by its ID should be successful.
 See `ISAValidator` in [`isa_validator.py`](../../../common/dss/isa_validator.py).

monitoring/uss_qualifier/scenarios/astm/netrid/v22a/dss/test_steps/put_isa.md

@@ -1,4 +1,4 @@
-# Create or update ISA test step
+# Create or update ISA test step fragment
 
 This page describes the content of a common test step where a creation or an update of an ISA should be successful.
 See `DSSWrapper.put_isa` in [`dss_wrapper.py`](../../../dss_wrapper.py).

monitoring/uss_qualifier/scenarios/astm/netrid/v22a/dss/test_steps/search_isas.md

@@ -1,4 +1,4 @@
-# Search ISAs test step
+# Search ISAs test step fragment
 
 This page describes the content of a common test step where a search for ISAs should be successful.
 See `DSSWrapper.search_isa` in [`dss_wrapper.py`](../../../dss_wrapper.py).

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/plan_flight_intent_expect_failed.md

@@ -1,4 +1,4 @@
-# Plan flight Expect Failed test step
+# Plan flight Expect Failed test step fragment
 
 This page describes the content of a common test case where a valid user flight intent fails in a flight planner, because of invalid data shared for a nearby flight shared by another USS.  See `plan_flight_intent_expect_failed` in invalid_op_test_steps.py.
 

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_get_operational_intent.md

@@ -1,4 +1,4 @@
-# Validate GET interaction test step
+# Validate GET interaction test step fragment
 
 This step verifies that a USS makes a GET request to get the intent_details of an existing operation when needed as per ASTM F3548-21 by checking the interuss interactions of mock uss
 

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_no_notification_operational_intent.md

@@ -1,4 +1,4 @@
-# Validate no notification test step
+# Validate no notification test step fragment
 
 This step verifies when a flight is not created, it is also not notified by checking the interuss interactions of mock_uss instance.
 

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_notification_operational_intent.md

@@ -1,4 +1,4 @@
-# Validate notification test step
+# Validate notification test step fragment
 
 This step verifies that, when creating or modifying an operational intent, a USS sent the required notification for a relevant subscription owned by a mock_uss instance by checking the interactions of that mock_uss instance.
 

monitoring/uss_qualifier/scenarios/astm/utm/data_exchange_validation/test_steps/validate_sharing_operational_intent_but_with_invalid_interuss_data.md

@@ -1,4 +1,4 @@
-# Validate flight sharing invalid data test step
+# Validate flight sharing invalid data test step fragment
 
 This step verifies that a created flight is shared properly per ASTM F3548-21 by querying the DSS for flights in the area of the flight intent, and then retrieving the details from the USS if the operational intent reference is found.  See `expect_shared_with_invalid_data` in invalid_op_test_steps.py.
 

monitoring/uss_qualifier/scenarios/astm/utm/nominal_planning/conflict_equal_priority_not_permitted/conflict_equal_priority_not_permitted.md

@@ -31,48 +31,48 @@ Otherwise, the FlightIntentsResource must provide the following flight intents:
     <th>Must not conflict with</th>
   </tr>
   <tr>
-    <td><code>flight_1_planned_vol_A</code></td>
+    <td><code>flight1_planned</code></td>
     <td rowspan="2">Flight 1</td>
     <td rowspan="8">Any (but all the same)</td>
     <td>Accepted</td>
     <td rowspan="2">Flight 2</td>
     <td rowspan="2">N/A</td>
   </tr>
   <tr>
-    <td><code>flight_1_activated_vol_A</code></td>
+    <td><code>flight1_activated</code></td>
     <td>Activated</td>
   </tr>
   <tr>
-    <td><code>flight_1_activated_vol_A_extended</code></td>
+    <td><code>flight1m_activated</code></td>
     <td>Flight 1m</td>
     <td>Activated</td>
     <td>Flight 2</td>
     <td>N/A</td>
   </tr>
   <tr>
-    <td><code>flight_1_planned_vol_B</code></td>
+    <td><code>flight1c_planned</code></td>
     <td rowspan="2">Flight 1c</td>
     <td>Planned</td>
     <td rowspan="2">N/A</td>
     <td rowspan="2">Flight 2</td>
   </tr>
   <tr>
-    <td><code>flight_1_activated_vol_B</code></td>
+    <td><code>flight1c_activated</code></td>
     <td>Activated</td>
   </tr>
   <tr>
-    <td><code>flight_2_equal_prio_planned_vol_B</code></td>
+    <td><code>equal_prio_flight2_planned</code></td>
     <td rowspan="3">Flight 2</td>
     <td>Planned</td>
     <td rowspan="3">Flight 1, Flight 1m</td>
     <td rowspan="3">Flight 1c</td>
   </tr>
   <tr>
-    <td><code>flight_2_equal_prio_activated_vol_B</code></td>
+    <td><code>equal_prio_flight2_activated</code></td>
     <td>Activated</td>
   </tr>
   <tr>
-    <td><code>flight_2_equal_prio_nonconforming_vol_A</code></td>
+    <td><code>equal_prio_flight2_nonconforming</code></td>
     <td>Nonconforming</td>
   </tr>
 </table>
@@ -95,7 +95,9 @@ DSSInstanceResource that provides access to a DSS instance where flight creation
 ## Attempt to plan flight into conflict test case
 ![Test case summary illustration](assets/attempt_to_plan_flight_into_conflict.svg)
 
-### [Plan flight 2 test step](../../../../flight_planning/plan_flight_intent.md)
+### Plan flight 2 test step
+
+#### [Plan flight 2](../../../../flight_planning/plan_flight_intent.md)
 Flight 2 on time range B should be successfully planned by the control USS.
 
 ### [Validate flight 2 sharing test step](../../validate_shared_operational_intent.md)