Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[uss_qualifier] Remove test step management from OpIntentValidator #428

Merged
merged 11 commits into from
Jan 9, 2024
Merged
Original file line number Diff line number Diff line change
Expand Up @@ -29,18 +29,22 @@ DSSInstanceResource that provides access to a DSS instance where flight creation

## Successfully plan flight near an existing flight test case

### [mock_uss plans flight 2 test step](../../../flight_planning/plan_flight_intent.md)
### mock_uss plans flight 2 test step

#### [Plan](../../../flight_planning/plan_flight_intent.md)

Flight 2 should be successfully planned by the control USS.

### [Validate flight 2 sharing test step](../validate_shared_operational_intent.md)
Validate that flight 2 is planned
#### [Validate](../validate_shared_operational_intent.md)

### tested_uss plans flight 1 test step

#### [Plan](../../../flight_planning/plan_flight_intent.md)

### [tested_uss plans flight 1 test step](../../../flight_planning/plan_flight_intent.md)
The test driver attempts to plan flight 1 via the tested USS. It checks if any conflicts with flight 2
which is of equal priority and came first.

### [Validate flight 1 sharing test step](../validate_shared_operational_intent.md)
Validate flight 1 is planned.
#### [Validate](../validate_shared_operational_intent.md)

### Check for notification to tested_uss due to subscription in flight 2 area test step
In the following test step, we want to assert that tested_uss must have retrieved operational intent details from
Expand All @@ -63,18 +67,27 @@ Teardown
Teardown

## Flight planning prevented due to invalid data sharing test case
### [mock_uss plans flight 2, sharing invalid operational intent data test step](../../../flight_planning/plan_flight_intent.md)

### mock_uss plans flight 2, sharing invalid operational intent data test step

#### [Plan](../../../flight_planning/plan_flight_intent.md)

Flight 2 should be successfully planned by the mock_uss.

#### [Validate](test_steps/validate_sharing_operational_intent_but_with_invalid_interuss_data.md)

The mock_uss is instructed to share invalid data with other USS, for negative test.

### [Validate flight 2 shared operational intent with invalid data test step](test_steps/validate_sharing_operational_intent_but_with_invalid_interuss_data.md)
Validate that flight 2 is shared with invalid data as a modified behavior is injected by uss_qualifier for a negative test.
### tested_uss attempts to plan flight 1, expect failure test step

#### [Plan](test_steps/plan_flight_intent_expect_failed.md)

### [tested_uss attempts to plan flight 1, expect failure test step](test_steps/plan_flight_intent_expect_failed.md)
The test driver attempts to plan the flight 1 via the tested_uss. It checks if any conflicts with flight 2
which is of equal priority and came first.
The planning attempt should fail because tested_uss will be unable to obtain valid operational intent details for flight 2.

#### [Validate](../validate_not_shared_operational_intent.md)

### [Validate flight 1 not shared by tested_uss test step](../validate_not_shared_operational_intent.md)
Validate flight 1 is not shared with DSS, as plan failed.

### Check for notification to tested_uss due to subscription in flight 2 area test step
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -167,45 +167,43 @@ def _plan_successfully_test_case(self, times: Dict[TimeDuringTest, Time]):
times[TimeDuringTest.TimeOfEvaluation] = Time(arrow.utcnow().datetime)
flight_2 = self.flight_2.resolve(times)

self.begin_test_step("mock_uss plans flight 2")
with OpIntentValidator(
self,
self.mock_uss_client,
self.dss,
"Validate flight 2 sharing",
self._intents_extent,
) as validator:
self.begin_test_step("mock_uss plans flight 2")
flight_2_planning_time = Time(arrow.utcnow().datetime)
_, self.flight_2_id = plan_flight(
self,
self.mock_uss_client,
flight_2,
)
self.end_test_step()

flight_2_oi_ref = validator.expect_shared(flight_2)
self.end_test_step()

times[TimeDuringTest.TimeOfEvaluation] = Time(arrow.utcnow().datetime)
flight_1 = self.flight_1.resolve(times)

self.begin_test_step("tested_uss plans flight 1")
with OpIntentValidator(
self,
self.tested_uss_client,
self.dss,
"Validate flight 1 sharing",
self._intents_extent,
) as validator:
self.begin_test_step("tested_uss plans flight 1")
flight_1_planning_time = Time(arrow.utcnow().datetime)
plan_res, self.flight_1_id = plan_flight(
self,
self.tested_uss_client,
flight_1,
)
self.end_test_step()
validator.expect_shared(
flight_1,
)
self.end_test_step()

self.begin_test_step(
"Check for notification to tested_uss due to subscription in flight 2 area"
Expand Down Expand Up @@ -281,48 +279,46 @@ def _plan_unsuccessfully_test_case(self, times: Dict[TimeDuringTest, Time]):
)

additional_fields = {"behavior": behavior}
self.begin_test_step(
"mock_uss plans flight 2, sharing invalid operational intent data"
)
with OpIntentValidator(
self,
self.mock_uss_client,
self.dss,
"Validate flight 2 shared operational intent with invalid data",
self._intents_extent,
) as validator:
self.begin_test_step(
"mock_uss plans flight 2, sharing invalid operational intent data"
)
flight_2_planning_time = Time(arrow.utcnow().datetime)
_, self.flight_2_id = plan_flight(
self,
self.mock_uss_client,
flight_info,
additional_fields,
)
self.end_test_step()
flight_2_oi_ref = validator.expect_shared_with_invalid_data(
flight_info,
validation_failure_type=OpIntentValidationFailureType.DataFormat,
invalid_fields=[modify_field1, modify_field2],
)
self.end_test_step()

times[TimeDuringTest.TimeOfEvaluation] = Time(arrow.utcnow().datetime)
flight_1 = self.flight_1.resolve(times)
self.begin_test_step("tested_uss attempts to plan flight 1, expect failure")
with OpIntentValidator(
self,
self.tested_uss_client,
self.dss,
"Validate flight 1 not shared by tested_uss",
self._intents_extent,
) as validator:
self.begin_test_step("tested_uss attempts to plan flight 1, expect failure")
flight_1_planning_time = Time(arrow.utcnow().datetime)
_, self.flight_1_id = plan_flight_intent_expect_failed(
self,
self.tested_uss_client,
flight_1,
)
self.end_test_step()
validator.expect_not_shared()
self.end_test_step()

self.begin_test_step(
"Check for notification to tested_uss due to subscription in flight 2 area"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,19 +44,23 @@ All flight intent data provided was complete and correct. It should have been pr
to reject or accept the flight. If the USS indicates that the injection attempt failed, this check will fail per
**[interuss.automated_testing.flight_planning.ExpectedBehavior](../../../../requirements/interuss/automated_testing/flight_planning.md)**.

### [Validate flight intent too far ahead of time not planned test step](../validate_not_shared_operational_intent.md)
#### [Validate flight intent too far ahead of time not planned](../validate_not_shared_operational_intent.md)

## Validate transition to Ended state after cancellation test case
### [Plan flight intent test step](../../../flight_planning/plan_flight_intent.md)
### Plan flight intent test step

#### [Plan](../../../flight_planning/plan_flight_intent.md)
The valid flight intent should be successfully planned by the flight planner.

### [Validate flight intent shared correctly test step](../validate_shared_operational_intent.md)
#### [Validate flight intent shared correctly](../validate_shared_operational_intent.md)
Validate that the flight intent was shared correctly and is discoverable.

### [Cancel flight intent test step](../../../flight_planning/delete_flight_intent.md)
### Remove flight intent test step

#### [Cancel flight intent](../../../flight_planning/delete_flight_intent.md)
The flight intent should be successfully transition to Ended state by the flight planner.

### Validate flight intent is non-discoverable test step
#### [Validate flight intent is non-discoverable](../validate_removed_operational_intent.md)

#### DSS responses check

Expand Down Expand Up @@ -84,7 +88,7 @@ All flight intent data provided was complete and correct. It should have been pr
to reject or accept the flight. If the USS indicates that the injection attempt failed, this check will fail per
**[interuss.automated_testing.flight_planning.ExpectedBehavior](../../../../requirements/interuss/automated_testing/flight_planning.md)**.

### [Validate conflicting flight not planned test step](../validate_not_shared_operational_intent.md)
#### [Validate conflicting flight not planned](../validate_not_shared_operational_intent.md)

## Cleanup
### Successful flight deletion check
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -165,14 +165,13 @@ def run(self, context: ExecutionContext):
self.end_test_scenario()

def _attempt_invalid(self):
self.begin_test_step("Attempt to plan flight intent too far ahead of time")
with OpIntentValidator(
self,
self.tested_uss,
self.dss,
"Validate flight intent too far ahead of time not planned",
self._intents_extent,
) as validator:
self.begin_test_step("Attempt to plan flight intent too far ahead of time")
submit_flight_intent(
self,
"Incorrectly planned",
Expand All @@ -181,38 +180,36 @@ def _attempt_invalid(self):
self.tested_uss,
self.invalid_too_far_away.request,
)
self.end_test_step()

validator.expect_not_shared()
self.end_test_step()

def _validate_ended_cancellation(self):
self.begin_test_step("Plan flight intent")
with OpIntentValidator(
self,
self.tested_uss,
self.dss,
"Validate flight intent is non-discoverable",
self._intents_extent,
) as cancelled_validator:
with OpIntentValidator(
) as planned_validator:
_, flight_id, _ = plan_flight_intent(
self,
self.tested_uss,
self.dss,
"Validate flight intent shared correctly",
self._intents_extent,
) as planned_validator:
self.begin_test_step("Plan flight intent")
_, flight_id, _ = plan_flight_intent(
self,
self.tested_uss,
self.valid_flight.request,
)
self.end_test_step()
planned_validator.expect_shared(self.valid_flight.request)

_ = delete_flight_intent(
self, "Cancel flight intent", self.tested_uss, flight_id
self.valid_flight.request,
)
cancelled_validator.expect_not_shared()
oi_ref = planned_validator.expect_shared(self.valid_flight.request)
self.end_test_step()

self.begin_test_step("Remove flight intent")
with OpIntentValidator(
self,
self.tested_uss,
self.dss,
self._intents_extent,
) as cancelled_validator:
_ = delete_flight_intent(self, self.tested_uss, flight_id)
cancelled_validator.expect_removed(oi_ref.id)
self.end_test_step()

def _validate_precision_intersection(self):
self.begin_test_step("Plan control flight intent")
Expand All @@ -223,14 +220,13 @@ def _validate_precision_intersection(self):
)
self.end_test_step()

self.begin_test_step("Attempt to plan flight conflicting by a tiny overlap")
with OpIntentValidator(
self,
self.tested_uss,
self.dss,
"Validate conflicting flight not planned",
self._intents_extent,
) as validator:
self.begin_test_step("Attempt to plan flight conflicting by a tiny overlap")
submit_flight_intent(
self,
"Incorrectly planned",
Expand All @@ -242,9 +238,9 @@ def _validate_precision_intersection(self):
self.tested_uss,
self.valid_conflict_tiny_overlap.request,
)
self.end_test_step()

validator.expect_not_shared()
self.end_test_step()

def cleanup(self):
self.begin_cleanup()
Expand Down
Loading
Loading