-
Notifications
You must be signed in to change notification settings - Fork 20
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[uss_qualifier] Add test scenario for OVN request API (#816)
- Loading branch information
Showing
21 changed files
with
426 additions
and
35 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
11 changes: 11 additions & 0 deletions
11
monitoring/uss_qualifier/requirements/interuss/f3548/ovn_request.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# OVN Request Optional Extension to ASTM F3548-21 Requirements | ||
This optional extension not part of the original F3548 standard API allows a USS to request a specific OVN when creating | ||
or updating an operational intent. | ||
|
||
## DSS requirements | ||
### <tt>ImplementAPI</tt> | ||
If a DSS has support for the optional extension, it must implement the endpoints `createOperationalIntentReference` and | ||
`updateOperationalIntentReference` with the support for the optional field `requested_ovn_suffix` as defined in the API, | ||
accept requests in the data format prescribed in the API, and respond in the data format prescribed in the API. | ||
If there is a problem using the API such as a connection error, invalid response code, or invalid data, the DSS will | ||
have failed to meet this requirement. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1 change: 1 addition & 0 deletions
1
monitoring/uss_qualifier/scenarios/interuss/ovn_request/__init__.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
from .dss_ovn_request import DSSOVNRequest |
67 changes: 67 additions & 0 deletions
67
monitoring/uss_qualifier/scenarios/interuss/ovn_request/dss_ovn_request.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
# OVN Request Optional Extension to ASTM F3548-21 test scenario | ||
|
||
## Description | ||
This test validates that a DSS correctly implements the [OVN Request Optional Extension to ASTM F3548-21](../../../requirements/interuss/f3548/ovn_request.md). | ||
|
||
## Resources | ||
|
||
### dss | ||
[`DSSInstanceResource`](../../../resources/astm/f3548/v21/dss.py) to be tested in this scenario. | ||
|
||
### id_generator | ||
[`IDGeneratorResource`](../../../resources/interuss/id_generator.py) providing the base entity ID for this scenario. | ||
|
||
### client_identity | ||
[`ClientIdentityResource`](../../../resources/communications/client_identity.py) the client identity that will be used to create and update operational intent references. | ||
|
||
### planning_area | ||
[`PlanningAreaResource`](../../../resources/astm/f3548/v21/planning_area.py) describes the 3D volume in which operational intent references will be created. | ||
|
||
## Setup test case | ||
|
||
### [Ensure clean workspace test step](../../astm/utm/dss/clean_workspace.md) | ||
This step ensures that no entities with the known test IDs exists in the DSS. | ||
|
||
## Request for OIR OVN with valid suffix test case | ||
This case validates the nominal behavior of the OVN request. | ||
|
||
### Create OIR with OVN suffix request test step | ||
|
||
#### [Create OIR with OVN suffix request](../../astm/utm/dss/fragments/oir/crud/create_query.md) | ||
Check that the OIR creation query succeeds. | ||
|
||
#### [DSS has set the expected OVN using the requested OVN suffix](./expected_ovn_set_fragment.md) | ||
Check that the DSS has set the expected OVN correctly. | ||
|
||
### Activate OIR with OVN suffix request test step | ||
|
||
#### [Update OIR with OVN suffix request](../../astm/utm/dss/fragments/oir/crud/update_query.md) | ||
Check that the OIR update query succeeds. | ||
|
||
#### [DSS has set the expected OVN using the requested OVN suffix](./expected_ovn_set_fragment.md) | ||
Check that the DSS has set the expected OVN correctly. | ||
|
||
## Request for OIR OVN with invalid suffix test case | ||
This case validates the off-nominal behaviors of the OVN request. | ||
|
||
### Attempt to create OIR with OVN suffix request not being a UUID test step | ||
#### [Attempt to create OIR with OVN suffix request not being a UUID rejected check](./invalid_ovn_suffix_fragment.md) | ||
Check that the DSS rejects OVN suffix that are not UUIDs. | ||
If the DSS accepts the OVN suffix, or fails with an unexpected error, this check will fail. | ||
|
||
### Attempt to create OIR with OVN suffix request empty test step | ||
#### [Attempt to create OIR with OVN suffix request empty rejected check](./invalid_ovn_suffix_fragment.md) | ||
Check that the DSS rejects OVN suffix that are empty. | ||
If the DSS accepts the OVN suffix, or fails with an unexpected error, this check will fail. | ||
|
||
### Attempt to create OIR with OVN suffix request being a UUID but not v7 test step | ||
#### [Attempt to create OIR with OVN suffix request being a UUID but not v7 rejected check](./invalid_ovn_suffix_fragment.md) | ||
Check that the DSS rejects OVN suffix that are UUIDs but not v7. | ||
If the DSS accepts the OVN suffix, or fails with an unexpected error, this check will fail. | ||
|
||
### Attempt to create OIR with OVN suffix request being an outdated UUIDv7 test step | ||
#### [Attempt to create OIR with OVN suffix request being an outdated UUIDv7 rejected check](./invalid_ovn_suffix_fragment.md) | ||
Check that the DSS rejects OVN suffix that are outdated UUIDv7. | ||
If the DSS accepts the OVN suffix, or fails with an unexpected error, this check will fail. | ||
|
||
## [Cleanup](../../astm/utm/dss/clean_workspace.md) |
234 changes: 234 additions & 0 deletions
234
monitoring/uss_qualifier/scenarios/interuss/ovn_request/dss_ovn_request.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,234 @@ | ||
from typing import List | ||
|
||
from uuid6 import uuid7, uuid6 | ||
from datetime import datetime, timedelta | ||
|
||
from uas_standards.astm.f3548.v21.api import ( | ||
OperationalIntentState, | ||
Volume4D, | ||
OperationalIntentReference, | ||
) | ||
from uas_standards.astm.f3548.v21.constants import Scope | ||
|
||
from monitoring.monitorlib.fetch import QueryError | ||
from monitoring.prober.infrastructure import register_resource_type | ||
from monitoring.uss_qualifier.resources.astm.f3548.v21 import PlanningAreaResource | ||
from monitoring.uss_qualifier.resources.astm.f3548.v21.dss import ( | ||
DSSInstanceResource, | ||
) | ||
from monitoring.uss_qualifier.resources.communications import ClientIdentityResource | ||
from monitoring.uss_qualifier.resources.interuss import IDGeneratorResource | ||
from monitoring.uss_qualifier.resources.resource import MissingResourceError | ||
from monitoring.uss_qualifier.scenarios.astm.utm.dss import test_step_fragments | ||
from monitoring.uss_qualifier.scenarios.scenario import ( | ||
TestScenario, | ||
) | ||
from monitoring.uss_qualifier.suites.suite import ExecutionContext | ||
from monitoring.monitorlib import geotemporal | ||
|
||
OIR_TYPE = register_resource_type( | ||
398, "Operational Intent Reference for OVN suffix request" | ||
) | ||
|
||
|
||
class DSSOVNRequest(TestScenario): | ||
def __init__( | ||
self, | ||
dss: DSSInstanceResource, | ||
id_generator: IDGeneratorResource, | ||
client_identity: ClientIdentityResource, | ||
planning_area: PlanningAreaResource, | ||
): | ||
super().__init__() | ||
if not dss.supports_ovn_request: | ||
raise MissingResourceError( | ||
f"DSS resource with ID {dss.participant_id} does not support OVN requests", | ||
"dss", | ||
) | ||
self._dss = dss.get_instance( | ||
{ | ||
Scope.StrategicCoordination: "create and delete operational intent references" | ||
} | ||
) | ||
|
||
self._oir_id = id_generator.id_factory.make_id(OIR_TYPE) | ||
self._planning_area = planning_area.specification | ||
self._expected_manager = client_identity.subject() | ||
|
||
def run(self, context: ExecutionContext): | ||
self.begin_test_scenario(context) | ||
self._setup_case() | ||
|
||
now = datetime.now() | ||
extents = [ | ||
self._planning_area.get_volume4d( | ||
now - timedelta(seconds=10), | ||
now + timedelta(minutes=45), | ||
).to_f3548v21() | ||
] | ||
|
||
self.begin_test_case("Request for OIR OVN with valid suffix") | ||
|
||
self.begin_test_step("Create OIR with OVN suffix request") | ||
req_ovn_suffix = str(uuid7()) | ||
oir = self._create_oir(extents, req_ovn_suffix) | ||
self._check_expected_ovn(req_ovn_suffix, oir) | ||
self.end_test_step() | ||
|
||
self.begin_test_step("Activate OIR with OVN suffix request") | ||
req_ovn_suffix = str(uuid7()) | ||
self._activate_oir(extents, oir.ovn, req_ovn_suffix) | ||
self._check_expected_ovn(req_ovn_suffix, oir) | ||
self.end_test_step() | ||
|
||
self.end_test_case() | ||
|
||
self.begin_test_case("Request for OIR OVN with invalid suffix") | ||
|
||
self.begin_test_step( | ||
"Attempt to create OIR with OVN suffix request not being a UUID" | ||
) | ||
self._create_invalid_oir_attempt(extents, "abc") | ||
self.end_test_step() | ||
|
||
self.begin_test_step("Attempt to create OIR with OVN suffix request empty") | ||
self._create_invalid_oir_attempt(extents, "") | ||
self.end_test_step() | ||
|
||
self.begin_test_step( | ||
"Attempt to create OIR with OVN suffix request being a UUID but not v7" | ||
) | ||
self._create_invalid_oir_attempt(extents, str(uuid6())) | ||
self.end_test_step() | ||
|
||
self.begin_test_step( | ||
"Attempt to create OIR with OVN suffix request being an outdated UUIDv7" | ||
) | ||
self._create_invalid_oir_attempt( | ||
extents, "0192b9ff-793a-7a18-9b61-552a7ed277b3" | ||
) # Wed, 23 Oct 2024 15:29:40 GMT | ||
self.end_test_step() | ||
|
||
self.end_test_case() | ||
|
||
self.end_test_scenario() | ||
|
||
def _create_oir( | ||
self, extents: List[Volume4D], req_ovn_suffix: str | ||
) -> OperationalIntentReference: | ||
with self.check( | ||
"Create operational intent reference query succeeds", | ||
[self._dss.participant_id], | ||
) as check: | ||
try: | ||
oir, _, q = self._dss.put_op_intent( | ||
extents=extents, | ||
key=[], | ||
state=OperationalIntentState.Accepted, | ||
base_url=self._planning_area.get_base_url(), | ||
oi_id=self._oir_id, | ||
ovn=None, | ||
requested_ovn_suffix=req_ovn_suffix, | ||
) | ||
self.record_query(q) | ||
except QueryError as qe: | ||
self.record_queries(qe.queries) | ||
check.record_failed( | ||
summary="Create operational intent reference failed", | ||
details=qe.msg, | ||
query_timestamps=qe.query_timestamps, | ||
) | ||
|
||
return oir | ||
|
||
def _activate_oir(self, extents: List[Volume4D], ovn: str, req_ovn_suffix: str): | ||
with self.check( | ||
"Mutate operational intent reference query succeeds", | ||
[self._dss.participant_id], | ||
) as check: | ||
try: | ||
oir, _, q = self._dss.put_op_intent( | ||
extents=extents, | ||
key=[], | ||
state=OperationalIntentState.Activated, | ||
base_url=self._planning_area.get_base_url(), | ||
oi_id=self._oir_id, | ||
ovn=ovn, | ||
requested_ovn_suffix=req_ovn_suffix, | ||
) | ||
self.record_query(q) | ||
except QueryError as qe: | ||
self.record_queries(qe.queries) | ||
check.record_failed( | ||
summary="Mutate operational intent reference failed", | ||
details=qe.msg, | ||
query_timestamps=qe.query_timestamps, | ||
) | ||
|
||
def _create_invalid_oir_attempt(self, extents: List[Volume4D], req_ovn_suffix: str): | ||
with self.check( | ||
"Attempt to create OIR with invalid requested OVN suffix query rejected", | ||
[self._dss.participant_id], | ||
) as check: | ||
try: | ||
oir, _, q = self._dss.put_op_intent( | ||
extents=extents, | ||
key=[], | ||
state=OperationalIntentState.Accepted, | ||
base_url=self._planning_area.get_base_url(), | ||
oi_id=self._oir_id, | ||
ovn=None, | ||
requested_ovn_suffix=req_ovn_suffix, | ||
) | ||
self.record_query(q) | ||
check.record_failed( | ||
summary="Creation of an operational intent reference with invalid requested OVN suffix succeeded", | ||
details=f"OIR {oir.id} with OVN {oir.ovn} got incorrectly created with requested OVN suffix {req_ovn_suffix}", | ||
query_timestamps=q.query_timestamps, | ||
) | ||
except QueryError as qe: | ||
self.record_queries(qe.queries) | ||
if qe.cause_status_code != 400: | ||
check.record_failed( | ||
summary="Creation of an operational intent reference with invalid requested OVN suffix failed with incorrect status code", | ||
details=f"OIR {oir.id} with requested OVN suffix {req_ovn_suffix}: expected 400 but got {q.status_code}; {qe.msg}", | ||
query_timestamps=qe.query_timestamps, | ||
) | ||
|
||
def _check_expected_ovn(self, req_ovn_suffix: str, oir: OperationalIntentReference): | ||
with self.check( | ||
"DSS has set the expected OVN using the requested OVN suffix", | ||
[self._dss.participant_id], | ||
) as check: | ||
expected_ovn = f"{self._oir_id}_{req_ovn_suffix}" | ||
if expected_ovn != oir.ovn: | ||
check.record_failed( | ||
summary="DSS returned an invalid OVN after request for OVN suffix", | ||
details=f"Requested OVN suffix {req_ovn_suffix}, expected OVN {expected_ovn} but got {oir.ovn}", | ||
) | ||
|
||
def _setup_case(self): | ||
self.begin_test_case("Setup") | ||
|
||
self.begin_test_step("Ensure clean workspace") | ||
vol = geotemporal.Volume4D( | ||
volume=self._planning_area.volume, | ||
).to_f3548v21() | ||
|
||
test_step_fragments.cleanup_active_oirs( | ||
self, | ||
self._dss, | ||
vol, | ||
self._expected_manager, | ||
) | ||
|
||
test_step_fragments.cleanup_op_intent(self, self._dss, self._oir_id) | ||
test_step_fragments.cleanup_active_subs(self, self._dss, vol) | ||
|
||
self.end_test_step() | ||
self.end_test_case() | ||
|
||
def cleanup(self): | ||
self.begin_cleanup() | ||
test_step_fragments.cleanup_op_intent(self, self._dss, self._oir_id) | ||
self.end_cleanup() |
5 changes: 5 additions & 0 deletions
5
...oring/uss_qualifier/scenarios/interuss/ovn_request/expected_ovn_set_fragment.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# DSS has set the expected OVN using the requested OVN suffix test step fragment | ||
This test step fragment validates that the DSS has set the expected OVN correctly after an USS requested a suffix. | ||
|
||
## 🛑 DSS has set the expected OVN using the requested OVN suffix check | ||
If the DSS has not set the OVN according to the specifications, it will fail this check as per **[interuss.f3548.ovn_request.ImplementAPI](../../../requirements/interuss/f3548/ovn_request.md)**. |
5 changes: 5 additions & 0 deletions
5
...ing/uss_qualifier/scenarios/interuss/ovn_request/invalid_ovn_suffix_fragment.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# Attempt to create OIR with invalid requested OVN suffix query test step fragment | ||
This test step fragment validates that the DSS rejects invalid attempts to request an OVN suffix. | ||
|
||
## 🛑 Attempt to create OIR with invalid requested OVN suffix query rejected check | ||
If the DSS accepts the OVN suffix, or fails with an error other than an HTTP code 400, this check will fail as per **[interuss.f3548.ovn_request.ImplementAPI](../../../requirements/interuss/f3548/ovn_request.md)**. |
Oops, something went wrong.