comments

monitoring/uss_qualifier/scenarios/astm/utm/dss/authentication/authentication_validation.md

@@ -449,6 +449,10 @@ it is in violation of **[astm.f3548.v21.DSS0210,A2-7-2,7](../../../../../require
 If the DSS does not allow fetching a constraint reference when valid credentials are presented,
 it is in violation of **[astm.f3548.v21.DSS0005,1](../../../../../requirements/astm/f3548/v21.md)**.
 
+#### [Read response format](../fragments/cr/crud/read_format.md)
+
+Check response format of a mutation.
+
 #### 🛑 Mutate constraint reference with missing credentials check
 
 If the DSS under test allows the mutation of a constraint reference without any credentials being presented,
@@ -472,7 +476,7 @@ it is in violation of **[astm.f3548.v21.DSS0210,A2-7-2,7](../../../../../require
 #### 🛑 Mutate constraint reference with valid credentials check
 
 If the DSS does not allow the mutation of a constraint reference when valid credentials are presented,
-it is in violation of **[astm.f3548.v21.DSS0005,1](../../../../../requirements/astm/f3548/v21.md)**.
+it is in violation of **[astm.f3548.v21.DSS0005,3](../../../../../requirements/astm/f3548/v21.md)**.
 
 #### [Mutate response format](../fragments/cr/crud/update_format.md)
 
@@ -503,6 +507,10 @@ it is in violation of **[astm.f3548.v21.DSS0210,A2-7-2,7](../../../../../require
 If the DSS does not allow the deletion of a constraint reference when valid credentials are presented,
 it is in violation of **[astm.f3548.v21.DSS0005,1](../../../../../requirements/astm/f3548/v21.md)**.
 
+#### [Delete response format](../fragments/cr/crud/delete_format.md)
+
+Check response format of a deletion.
+
 #### 🛑 Search constraint references with missing credentials check
 
 If the DSS under test allows searching for constraint references without any credentials being presented,
@@ -526,7 +534,11 @@ it is in violation of **[astm.f3548.v21.DSS0210,A2-7-2,7](../../../../../require
 #### 🛑 Search constraint references with valid credentials check
 
 If the DSS does not allow searching for constraint references when valid credentials are presented,
-it is in violation of **[astm.f3548.v21.DSS0005,1](../../../../../requirements/astm/f3548/v21.md)**.
+it is in violation of **[astm.f3548.v21.DSS0005,4](../../../../../requirements/astm/f3548/v21.md)**.
+
+#### [Search response format](../fragments/cr/crud/search_format.md)
+
+Check response format of a search.
 
 ## [Cleanup](../clean_workspace.md)
 

monitoring/uss_qualifier/scenarios/astm/utm/dss/authentication/cr_api_validator.py

@@ -13,6 +13,7 @@
     PutConstraintReferenceParameters,
     ChangeConstraintReferenceResponse,
     QueryConstraintReferenceParameters,
+    QueryConstraintReferencesResponse,
 )
 
 from monitoring.monitorlib import fetch
@@ -263,6 +264,21 @@ def _verify_cr_get(self):
                     query_timestamps=[query_valid_auth.request.timestamp],
                 )
 
+        with self._scenario.check(
+            "Get constraint reference response format conforms to spec",
+            self._pid,
+        ) as check:
+            try:
+                ImplicitDict.parse(
+                    query_valid_auth.response.json, ChangeConstraintReferenceResponse
+                )
+            except ValueError as e:
+                check.record_failed(
+                    summary="Could not parse the response body",
+                    details=f"Failed to parse the response body as a ChangeConstraintReferenceResponse: {e}",
+                    query_timestamps=[query_valid_auth.request.timestamp],
+                )
+
     def _verify_cr_mutation(self):
         op = OPERATIONS[OperationID.UpdateConstraintReference]
         new_params = PutConstraintReferenceParameters(**self._cr_params)
@@ -429,6 +445,20 @@ def _verify_cr_deletion(self):
                     query_timestamps=[valid_q.request.timestamp],
                 )
 
+        with self._scenario.check(
+            "Delete constraint reference response format conforms to spec", self._pid
+        ) as check:
+            try:
+                ImplicitDict.parse(
+                    valid_q.response.json, ChangeConstraintReferenceResponse
+                )
+            except ValueError as e:
+                check.record_failed(
+                    summary="Could not parse the deletion response",
+                    details=f"Failed to parse the response body as a ChangeConstraintReferenceResponse: {e}",
+                    query_timestamps=[valid_q.request.timestamp],
+                )
+
         self._current_cr = None
 
     def _verify_cr_search(self):
@@ -507,6 +537,21 @@ def _verify_cr_search(self):
                     query_timestamps=[valid_q.request.timestamp],
                 )
 
+        with self._scenario.check(
+            "Search constraint reference response format conforms to spec",
+            self._pid,
+        ) as check:
+            try:
+                ImplicitDict.parse(
+                    valid_q.response.json, QueryConstraintReferencesResponse
+                )
+            except ValueError as e:
+                check.record_failed(
+                    summary="Could not parse the search response",
+                    details=f"Failed to parse the response body as a ChangeConstraintReferenceResponse: {e}",
+                    query_timestamps=[valid_q.request.timestamp],
+                )
+
     def _sanity_check_cr_not_created(
         self, check: PendingCheck, creation_q: fetch.Query
     ):
@@ -532,6 +577,17 @@ def _sanity_check_cr_not_updated(
         try:
             cr, sanity_check = self._dss.get_constraint_ref(self._test_id)
             self._scenario.record_query(sanity_check)
+            # Check if the version changed
+            if cr.version != self._current_cr.version:
+                check.record_failed(
+                    summary="CR version updated by an unauthorized request.",
+                    details=f"The Constraint Reference with id {self._test_id} should not have been updated, as the update attempt was not authenticated.",
+                    query_timestamps=[
+                        creation_q.request.timestamp,
+                        sanity_check.request.timestamp,
+                    ],
+                )
+            # For the unlikely case where the version would not change but the CR would be mutated anyway:
             if (
                 abs(
                     cr.time_end.value.datetime
@@ -540,7 +596,7 @@ def _sanity_check_cr_not_updated(
                 > TIME_TOLERANCE_SEC
             ):
                 check.record_failed(
-                    summary="CR was updated by an unauthorized request.",
+                    summary="CR end time updated by an unauthorized request.",
                     details=f"The Constraint Reference with id {self._test_id} should not have been updated, as the update attempt was not authenticated.",
                     query_timestamps=[
                         creation_q.request.timestamp,

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/cr/crud/delete.md

@@ -6,11 +6,9 @@ This test step fragment validates that constraint references can be deleted
 
 A query to delete a constraint reference, by its owner and when the correct OVN is provided, should succeed, otherwise the DSS is in violation of **[astm.f3548.v21.DSS0005,3](../../../../../../../requirements/astm/f3548/v21.md)**.
 
-## 🛑 Delete constraint reference response format conforms to spec check
+## [Response format](./delete_format.md)
 
-The response to a successful constraint reference deletion query is expected to conform to the format defined by the OpenAPI specification under the `A3.1` Annex of ASTM F3548−21.
-
-If it does not, the DSS is failing to implement **[astm.f3548.v21.DSS0005,3](../../../../../../../requirements/astm/f3548/v21.md)**.
+Check response format
 
 ## 🛑 Delete constraint reference response content is correct check
 

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/cr/crud/delete_format.md

@@ -0,0 +1,9 @@
+# Delete constraint reference response format test step fragment
+
+This test step fragment validates that a constraint references deletion returns a body in the correct format.
+
+## 🛑 Delete constraint reference response format conforms to spec check
+
+The response to a successful constraint reference deletion query is expected to conform to the format defined by the OpenAPI specification under the `A3.1` Annex of ASTM F3548−21.
+
+If it does not, the DSS is failing to implement **[astm.f3548.v21.DSS0005,3](../../../../../../../requirements/astm/f3548/v21.md)**.

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/cr/crud/read_correct.md

@@ -6,11 +6,9 @@ This test step fragment validates that constraint references can be read
 
 Check query succeeds.
 
-## 🛑 Get constraint reference response format conforms to spec check
+## [Read response format](./read_format.md)
 
-The response to a successful get constraint reference query is expected to conform to the format defined by the OpenAPI specification under the `A3.1` Annex of ASTM F3548−21.
-
-If it does not, the DSS is failing to implement **[astm.f3548.v21.DSS0005,3](../../../../../../../requirements/astm/f3548/v21.md)**.
+Check response format
 
 ## 🛑 Get constraint reference response content is correct check
 

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/cr/crud/read_format.md

@@ -0,0 +1,9 @@
+# Read constraint reference response format test step fragment
+
+This test step fragment validates that a request for a constraint reference returns a properly formatted body.
+
+## 🛑 Get constraint reference response format conforms to spec check
+
+The response to a successful get constraint reference query is expected to conform to the format defined by the OpenAPI specification under the `A3.1` Annex of ASTM F3548−21.
+
+If it does not, the DSS is failing to implement **[astm.f3548.v21.DSS0005,3](../../../../../../../requirements/astm/f3548/v21.md)**.

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/cr/crud/search_correct.md

@@ -6,11 +6,9 @@ This test step fragment validates that constraint references can be searched for
 
 Check query succeeds.
 
-## 🛑 Search constraint reference response format conforms to spec check
+## [Response format](./search_format.md)
 
-The response to a successful constraint reference search query is expected to conform to the format defined by the OpenAPI specification under the `A3.1` Annex of ASTM F3548−21.
-
-If it does not, the DSS is failing to implement **[astm.f3548.v21.DSS0005,4](../../../../../../../requirements/astm/f3548/v21.md)**.
+Check response format.
 
 ## 🛑 Expected constraint reference is in search results check
 

monitoring/uss_qualifier/scenarios/astm/utm/dss/fragments/cr/crud/search_format.md

@@ -0,0 +1,9 @@
+# Search constraint reference response format test step fragment
+
+This test step fragment validates that constraint references search responses are properly formatted.
+
+## 🛑 Search constraint reference response format conforms to spec check
+
+The response to a successful constraint reference search query is expected to conform to the format defined by the OpenAPI specification under the `A3.1` Annex of ASTM F3548−21.
+
+If it does not, the DSS is failing to implement **[astm.f3548.v21.DSS0005,4](../../../../../../../requirements/astm/f3548/v21.md)**.