Skip to content

Commit

Permalink
latest PR comments
Browse files Browse the repository at this point in the history
  • Loading branch information
Shastick committed Mar 19, 2024
1 parent 9361190 commit 849b534
Show file tree
Hide file tree
Showing 9 changed files with 46 additions and 30 deletions.
2 changes: 1 addition & 1 deletion monitoring/prober/infrastructure.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ def wrapper_default_scope(*args, **kwargs):
resource_type_code_descriptions: Dict[ResourceType, str] = {}


# Next code: 381
# Next code: 382
def register_resource_type(code: int, description: str) -> ResourceType:
"""Register that the specified code refers to the described resource.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,18 @@ For the purpose of this scenario, the `second_utm_auth` resource must provide ac

##### Separate subscription

Note that the subscription (or 'sub' claim, not to be confused with an SCD DSS subscription) of the token that will be obtained for this resource
Note that the subject (or 'sub' claim) of the token that will be obtained for this resource
MUST be different from the one of the `dss` resources mentioned above:
this will be verified at runtime, and the depending checks will not be run if this is not the case.

## Setup test case

### [Ensure clean workspace test step](../clean_workspace.md)

This step ensures that no subscription with the known test ID exists in the DSS.
This step ensures that no subscriptions with the known test IDs exists in the DSS.

This includes the main test subscription used in this test, as well as the extra subscription
used for testing the `manager` field sync, if the test is configured to test for it.

## Subscription Synchronization test case

Expand Down Expand Up @@ -147,18 +150,22 @@ Verify that the subscription returned by every DSS is correctly formatted and co

Verify that the version of the subscription returned by every DSS is as expected.

### Verify manager synchronization test step

Checks that the manager of a subscription is properly synchronized across all DSS instances.
### Create subscription with different credentials test step

This is done by means of using a separate set of credentials to create a subscription on the primary DSS,
and then verifying that the main credentials are not able to mutate this subscription via one of the secondary DSS instances
If the second set of credentials is provided, this test step will create a subscription using these credentials,
in order to prepare the next step that checks manager synchronization.

#### [Create subscription](../fragments/sub/crud/create.md)

Verify that a subscription can be created on the primary DSS.
Verify that a subscription can be created on the primary DSS using the separate set of credentials.

### Verify manager synchronization test step

If the second set of credentials is provided, checks that the manager of a subscription is properly synchronized across all DSS instances.

This is done by verifying that the main credentials are not able to delete the subscription via any of the secondary DSS instances.

#### 🛑 Subscription deletion with different non-managing credentials on secondary DSS fails check
#### ⚠️ Subscription deletion with different non-managing credentials on secondary DSS fails check

If the subscription can be deleted by a client which did not create it, via a DSS instance to which the subscription was synced
following its creation on the primary DSS, either one of the primary DSS or the DSS that accepted the deletion failed to properly broadcast, respectively take into account, the manage of the subscription,
Expand Down Expand Up @@ -192,3 +199,8 @@ If a DSS returns a subscription that was previously successfully deleted from th
either one of the primary DSS or the DSS that returned the subscription is in violation of **[astm.f3548.v21.DSS0210,1a](../../../../../requirements/astm/f3548/v21.md)**.

## [Cleanup](../clean_workspace.md)

This step ensures that no subscriptions with the known test IDs exists in the DSS.

This includes the main test subscription used in this test, as well as the extra subscription
used for testing the `manager` field sync, if the test is configured to test for it.
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,9 @@ class SubscriptionSynchronization(TestScenario):
"""

SUB_TYPE = register_resource_type(379, "Subscription")
ACL_SUB_TYPE = register_resource_type(380, "Subscription with different credentials")
ACL_SUB_TYPE = register_resource_type(
381, "Subscription with different credentials"
)

_dss: DSSInstance

Expand Down Expand Up @@ -156,28 +158,28 @@ def run(self, context: ExecutionContext):
return

self.begin_test_scenario(context)
self._setup_case()
self._step_setup_case()
self.begin_test_case("Subscription Synchronization")

self.begin_test_step("Create subscription validation")
self._create_sub_with_params(self._sub_params)
self._step_create_sub_with_params(self._sub_params)
self.end_test_step()

self.begin_test_step("Query newly created subscription")
self._query_secondaries_and_compare(self._sub_params)
self._step_query_secondaries_and_compare(self._sub_params)
self.end_test_step()

self.begin_test_step("Mutate subscription")
self._test_mutate_subscriptions_shift_time()
self.end_test_step()

self.begin_test_step("Query updated subscription")
self._query_secondaries_and_compare(self._sub_params)
self._step_query_secondaries_and_compare(self._sub_params)
self.end_test_step()

if self._dss_separate_creds:
self.begin_test_step("Create subscription with different credentials")

self._step_create_sub_separate_creds()
self.end_test_step()
self.begin_test_step("Verify manager synchronization")
self._step_test_delete_sub_with_separate_creds()
Expand All @@ -189,17 +191,17 @@ def run(self, context: ExecutionContext):
)

self.begin_test_step("Delete subscription")
self._test_delete_sub()
self._step_test_delete_sub()
self.end_test_step()

self.begin_test_step("Query deleted subscription")
self._test_get_deleted_sub()
self._step_test_get_deleted_sub()
self.end_test_step()

self.end_test_case()
self.end_test_scenario()

def _setup_case(self):
def _step_setup_case(self):
self.begin_test_case("Setup")
# Multiple runs of the scenario seem to rely on the same instance of it:
# thus we need to reset the state of the scenario before running it.
Expand Down Expand Up @@ -229,7 +231,7 @@ def _ensure_no_active_subs_exist(self):
self._planning_area_volume4d,
)

def _create_sub_with_params(self, creation_params: SubscriptionParams):
def _step_create_sub_with_params(self, creation_params: SubscriptionParams):

# TODO migrate to the try/except pattern for queries
newly_created = self._dss.upsert_subscription(
Expand Down Expand Up @@ -261,7 +263,9 @@ def _create_sub_with_params(self, creation_params: SubscriptionParams):
# Store the subscription
self._current_subscription = newly_created.subscription

def _query_secondaries_and_compare(self, expected_sub_params: SubscriptionParams):
def _step_query_secondaries_and_compare(
self, expected_sub_params: SubscriptionParams
):
for secondary_dss in self._dss_read_instances:
self._validate_get_sub_from_secondary(
secondary_dss=secondary_dss,
Expand Down Expand Up @@ -631,7 +635,7 @@ def _step_test_delete_sub_with_separate_creds(self):
check.record_failed(
"Subscription deletion with main credentials did not fail",
details=f"Subscription deletion with main credentials did not fail with the expected "
f"status code of 403; instead returned {deleted_sub.status_code}",
f"status code of 403; instead returned {deleted_sub.status_code}",
query_timestamps=[deleted_sub.request.timestamp],
)

Expand All @@ -646,7 +650,7 @@ def _credentials_are_different(self):
!= self._dss.client.auth_adapter.get_sub()
)

def _test_delete_sub(self):
def _step_test_delete_sub(self):
deleted_sub = self._dss.delete_subscription(
sub_id=self._sub_id, sub_version=self._current_subscription.version
)
Expand Down Expand Up @@ -676,7 +680,7 @@ def _test_delete_sub(self):

self._current_subscription = None

def _test_get_deleted_sub(self):
def _step_test_get_deleted_sub(self):
for secondary_dss in self._dss_read_instances:
self._confirm_secondary_has_no_sub(secondary_dss)

Expand Down
2 changes: 1 addition & 1 deletion monitoring/uss_qualifier/suites/astm/utm/dss_probing.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<th><a href="../../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="19" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td rowspan="20" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td><a href="../../../requirements/astm/f3548/v21.md">DSS0005,1</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/utm/op_intent_ref_access_control.md">ASTM F3548-21 UTM DSS Operational Intent Reference Access Control</a><br><a href="../../../scenarios/astm/utm/dss/authentication/authentication_validation.md">ASTM SCD DSS: Interfaces authentication</a><br><a href="../../../scenarios/astm/utm/dss/subscription_simple.md">ASTM SCD DSS: Subscription Simple</a><br><a href="../../../scenarios/astm/utm/dss/synchronization/subscription_synchronization.md">ASTM SCD DSS: Subscription Synchronization</a><br><a href="../../../scenarios/astm/utm/dss/subscription_validation.md">ASTM SCD DSS: Subscription Validation</a></td>
Expand Down
2 changes: 1 addition & 1 deletion monitoring/uss_qualifier/suites/astm/utm/f3548_21.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
<th><a href="../../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="45" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td rowspan="46" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td><a href="../../../requirements/astm/f3548/v21.md">DSS0005,1</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/utm/prep_planners.md">ASTM F3548 flight planners preparation</a><br><a href="../../../scenarios/astm/utm/op_intent_ref_access_control.md">ASTM F3548-21 UTM DSS Operational Intent Reference Access Control</a><br><a href="../../../scenarios/astm/utm/dss/authentication/authentication_validation.md">ASTM SCD DSS: Interfaces authentication</a><br><a href="../../../scenarios/astm/utm/dss/subscription_simple.md">ASTM SCD DSS: Subscription Simple</a><br><a href="../../../scenarios/astm/utm/dss/synchronization/subscription_synchronization.md">ASTM SCD DSS: Subscription Synchronization</a><br><a href="../../../scenarios/astm/utm/dss/subscription_validation.md">ASTM SCD DSS: Subscription Validation</a><br><a href="../../../scenarios/astm/utm/off_nominal_planning/down_uss.md">Off-Nominal planning: down USS</a><br><a href="../../../scenarios/astm/utm/off_nominal_planning/down_uss_equal_priority_not_permitted.md">Off-Nominal planning: down USS with equal priority conflicts not permitted</a></td>
Expand Down
2 changes: 1 addition & 1 deletion monitoring/uss_qualifier/suites/faa/uft/message_signing.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<th><a href="../../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="45" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td rowspan="46" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td><a href="../../../requirements/astm/f3548/v21.md">DSS0005,1</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/utm/prep_planners.md">ASTM F3548 flight planners preparation</a><br><a href="../../../scenarios/astm/utm/op_intent_ref_access_control.md">ASTM F3548-21 UTM DSS Operational Intent Reference Access Control</a><br><a href="../../../scenarios/astm/utm/dss/authentication/authentication_validation.md">ASTM SCD DSS: Interfaces authentication</a><br><a href="../../../scenarios/astm/utm/dss/subscription_simple.md">ASTM SCD DSS: Subscription Simple</a><br><a href="../../../scenarios/astm/utm/dss/synchronization/subscription_synchronization.md">ASTM SCD DSS: Subscription Synchronization</a><br><a href="../../../scenarios/astm/utm/dss/subscription_validation.md">ASTM SCD DSS: Subscription Validation</a><br><a href="../../../scenarios/astm/utm/off_nominal_planning/down_uss.md">Off-Nominal planning: down USS</a><br><a href="../../../scenarios/astm/utm/off_nominal_planning/down_uss_equal_priority_not_permitted.md">Off-Nominal planning: down USS with equal priority conflicts not permitted</a></td>
Expand Down
2 changes: 1 addition & 1 deletion monitoring/uss_qualifier/suites/interuss/dss/all_tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -408,7 +408,7 @@
<td><a href="../../../scenarios/astm/netrid/v22a/dss/heavy_traffic_concurrent.md">ASTM NetRID DSS: Concurrent Requests</a><br><a href="../../../scenarios/astm/netrid/v22a/dss/isa_expiry.md">ASTM NetRID DSS: ISA Expiry</a><br><a href="../../../scenarios/astm/netrid/v22a/dss/isa_subscription_interactions.md">ASTM NetRID DSS: ISA Subscription Interactions</a><br><a href="../../../scenarios/astm/netrid/v22a/dss/isa_simple.md">ASTM NetRID DSS: Simple ISA</a><br><a href="../../../scenarios/astm/netrid/v22a/dss/isa_validation.md">ASTM NetRID DSS: Submitted ISA Validations</a><br><a href="../../../scenarios/astm/netrid/v22a/dss/subscription_simple.md">ASTM NetRID DSS: Subscription Simple</a><br><a href="../../../scenarios/astm/netrid/v22a/dss/subscription_validation.md">ASTM NetRID DSS: Subscription Validation</a><br><a href="../../../scenarios/astm/netrid/v22a/dss/token_validation.md">ASTM NetRID DSS: Token Validation</a></td>
</tr>
<tr>
<td rowspan="19" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td rowspan="20" style="vertical-align:top;"><a href="../../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td><a href="../../../requirements/astm/f3548/v21.md">DSS0005,1</a></td>
<td>Implemented</td>
<td><a href="../../../scenarios/astm/utm/op_intent_ref_access_control.md">ASTM F3548-21 UTM DSS Operational Intent Reference Access Control</a><br><a href="../../../scenarios/astm/utm/dss/authentication/authentication_validation.md">ASTM SCD DSS: Interfaces authentication</a><br><a href="../../../scenarios/astm/utm/dss/subscription_simple.md">ASTM SCD DSS: Subscription Simple</a><br><a href="../../../scenarios/astm/utm/dss/synchronization/subscription_synchronization.md">ASTM SCD DSS: Subscription Synchronization</a><br><a href="../../../scenarios/astm/utm/dss/subscription_validation.md">ASTM SCD DSS: Subscription Validation</a></td>
Expand Down
2 changes: 1 addition & 1 deletion monitoring/uss_qualifier/suites/uspace/flight_auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<th><a href="../README.md#checked-in">Checked in</a></th>
</tr>
<tr>
<td rowspan="45" style="vertical-align:top;"><a href="../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td rowspan="46" style="vertical-align:top;"><a href="../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td><a href="../../requirements/astm/f3548/v21.md">DSS0005,1</a></td>
<td>Implemented</td>
<td><a href="../../scenarios/astm/utm/prep_planners.md">ASTM F3548 flight planners preparation</a><br><a href="../../scenarios/astm/utm/op_intent_ref_access_control.md">ASTM F3548-21 UTM DSS Operational Intent Reference Access Control</a><br><a href="../../scenarios/astm/utm/dss/authentication/authentication_validation.md">ASTM SCD DSS: Interfaces authentication</a><br><a href="../../scenarios/astm/utm/dss/subscription_simple.md">ASTM SCD DSS: Subscription Simple</a><br><a href="../../scenarios/astm/utm/dss/synchronization/subscription_synchronization.md">ASTM SCD DSS: Subscription Synchronization</a><br><a href="../../scenarios/astm/utm/dss/subscription_validation.md">ASTM SCD DSS: Subscription Validation</a><br><a href="../../scenarios/astm/utm/off_nominal_planning/down_uss.md">Off-Nominal planning: down USS</a><br><a href="../../scenarios/astm/utm/off_nominal_planning/down_uss_equal_priority_not_permitted.md">Off-Nominal planning: down USS with equal priority conflicts not permitted</a></td>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@
<td><a href="../../scenarios/astm/netrid/v22a/dss/heavy_traffic_concurrent.md">ASTM NetRID DSS: Concurrent Requests</a><br><a href="../../scenarios/astm/netrid/v22a/dss/isa_expiry.md">ASTM NetRID DSS: ISA Expiry</a><br><a href="../../scenarios/astm/netrid/v22a/dss/isa_subscription_interactions.md">ASTM NetRID DSS: ISA Subscription Interactions</a><br><a href="../../scenarios/astm/netrid/v22a/dss/isa_simple.md">ASTM NetRID DSS: Simple ISA</a><br><a href="../../scenarios/astm/netrid/v22a/dss/isa_validation.md">ASTM NetRID DSS: Submitted ISA Validations</a><br><a href="../../scenarios/astm/netrid/v22a/dss/subscription_simple.md">ASTM NetRID DSS: Subscription Simple</a><br><a href="../../scenarios/astm/netrid/v22a/dss/subscription_validation.md">ASTM NetRID DSS: Subscription Validation</a><br><a href="../../scenarios/astm/netrid/v22a/dss/token_validation.md">ASTM NetRID DSS: Token Validation</a></td>
</tr>
<tr>
<td rowspan="45" style="vertical-align:top;"><a href="../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td rowspan="46" style="vertical-align:top;"><a href="../../requirements/astm/f3548/v21.md">astm<br>.f3548<br>.v21</a></td>
<td><a href="../../requirements/astm/f3548/v21.md">DSS0005,1</a></td>
<td>Implemented</td>
<td><a href="../../scenarios/astm/utm/prep_planners.md">ASTM F3548 flight planners preparation</a><br><a href="../../scenarios/astm/utm/op_intent_ref_access_control.md">ASTM F3548-21 UTM DSS Operational Intent Reference Access Control</a><br><a href="../../scenarios/astm/utm/dss/authentication/authentication_validation.md">ASTM SCD DSS: Interfaces authentication</a><br><a href="../../scenarios/astm/utm/dss/subscription_simple.md">ASTM SCD DSS: Subscription Simple</a><br><a href="../../scenarios/astm/utm/dss/synchronization/subscription_synchronization.md">ASTM SCD DSS: Subscription Synchronization</a><br><a href="../../scenarios/astm/utm/dss/subscription_validation.md">ASTM SCD DSS: Subscription Validation</a><br><a href="../../scenarios/astm/utm/off_nominal_planning/down_uss.md">Off-Nominal planning: down USS</a><br><a href="../../scenarios/astm/utm/off_nominal_planning/down_uss_equal_priority_not_permitted.md">Off-Nominal planning: down USS with equal priority conflicts not permitted</a></td>
Expand Down

0 comments on commit 849b534

Please sign in to comment.