Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[uss_qualifier] Add scenario for testing data validation by SUT for Get operational_intent #293

Merged
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
9bbecfb
Scenario for testing data validation by SUT for Get operational_intent
punamverma Oct 26, 2023
1fd83d5
Removing an import
punamverma Oct 26, 2023
5d618a2
Fix format
punamverma Oct 26, 2023
56e0e8b
Add interactions requirement file
punamverma Oct 26, 2023
427cdf7
Adding more description to the scenario
punamverma Oct 26, 2023
2dddc76
Scenario documentation fix
punamverma Oct 27, 2023
db471a0
Improved requirements description
punamverma Oct 27, 2023
fdc3c7f
Per PR comments, fixed module path, file name, checks and description…
punamverma Oct 31, 2023
572bfeb
Per PR comments removing py files, and improving description
punamverma Oct 31, 2023
246cd3f
Per PR review, created a new requirement for no notifications
punamverma Oct 31, 2023
d322d0a
Adding empty sceanrio and fixing documentation links
punamverma Oct 31, 2023
493247b
Fix format
punamverma Oct 31, 2023
9f46935
Fix the requirement reference in documentation
punamverma Nov 1, 2023
be300fa
Documentation change per PR
punamverma Nov 1, 2023
52f1308
Removed requirement reference from test step, as per PR review
punamverma Nov 1, 2023
42f337a
Added more description to test steps
punamverma Nov 1, 2023
bfda3e3
Fix per PR review
punamverma Nov 1, 2023
a298f4d
Merge branch 'main' into first_data_validation_scenario
punamverma Nov 1, 2023
43c2b3d
Removed time_range suffix for flight intents, as per PR comment
punamverma Nov 1, 2023
4a3df76
Remover time_range_A suffix from flight intent
punamverma Nov 1, 2023
ae54279
Revert "Remover time_range_A suffix from flight intent"
punamverma Nov 1, 2023
9dc0912
Removed time_range_A suffix from flight intents
punamverma Nov 1, 2023
e58a0fe
Removing unreferenced file and description per PR review
punamverma Nov 2, 2023
2b68ef5
fixing variable name per PR reveiw
punamverma Nov 2, 2023
dbcb318
Removed link for no notification pushed
punamverma Nov 2, 2023
4ae2814
Fixed removing link
punamverma Nov 2, 2023
5dc2375
Fix link
punamverma Nov 2, 2023
23141b2
Removing check from a precondition step as per PR review
punamverma Nov 3, 2023
7e3ffcb
Merge branch 'main' into first_data_validation_scenario
punamverma Nov 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
InterUss mock_uss interactions requirements
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

## Overview
InterUSS provides endpoint to get InterUUS interactions recorded by mock_uss


## Requirements

### Interactions
GET interactions from a time instance to now
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# SCD Data Validation of GET operational intents by SUT test scenario
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

## Description
This test checks that the SUT validates the operational intents received as response to its GET request from another USS.
Control uss which is a mock uss plans a nearby V-shaped operation, and provides the data that SUT GETs.
SUT validates the GET response to successfully plan its operation
Notably the following requirements:
- **[astm.f3548.v21.OPIN0015](../../../../../requirements/astm/f3548/v21.md)**
- **[astm.f3548.v21.OPIN0020](../../../../../requirements/astm/f3548/v21.md)**
- **[astm.f3548.v21.OPIN0030](../../../../../requirements/astm/f3548/v21.md)**
- **[astm.f3548.v21.OPIN0040](../../../../../requirements/astm/f3548/v21.md)**
- **[astm.f3548.v21.GEN0500](../../../../../requirements/astm/f3548/v21.md)**
- **[astm.f3548.v21.SCD0035](../../../../../requirements/astm/f3548/v21.md)**
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

## Resources
### flight_intents
FlightIntentsResource that provides the following V-shaped flight intents:
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
- `non-conflicting` : flight1 and flight2
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

### control_uss
FlightPlannerResource that will control the data shared for validation testing of sut.
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

### sut
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
FlightPlannerResource that will be tested for its validation of operational intents.

### dss
DSSInstanceResource that provides access to a DSS instance where flight creation/sharing can be verified.

### mock_uss
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
MockUSSResource

## Setup test case
### Check for flight planning readiness test step
Both USSs are queried for their readiness to ensure this test can proceed.

#### Flight planning USS not ready check
If either USS does not respond appropriately to the endpoint queried to determine readiness, this check will fail and the USS will have failed to meet **[astm.f3548.v21.GEN0310](../../../../../requirements/astm/f3548/v21.md)** as the USS does not support the InterUSS implementation of that requirement.

### Area clearing test step
Both USSs are requested to remove all flights from the area under test.

#### Area cleared successfully check
**[interuss.automated_testing.flight_planning.ClearArea](../../../../../requirements/interuss/automated_testing/flight_planning.md)**

## Attempt to plan no conflict flight - near existing flight test case
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
### [Control USS plans flight 2 test step](../../../../flight_planning/plan_flight_intent.md)
Flight 2 on time range B should be successfully planned by the control USS.
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

### [Validate flight 2 sharing test step](../../validate_shared_operational_intent.md)
Validate that flight 2 is planned

### [SUT plans flight 1 test step](../../../../flight_planning/plan_flight_intent.md)
The test driver attempts to plan the flight 1 on time range B via the tested USS. It checks if any conflicts with flight 2
which is of equal priority and came first.
per **[astm.f3548.v21.SCD0035](../../../../../requirements/astm/f3548/v21.md)**.

### [Validate flight 1 sharing test step](../../validate_shared_operational_intent.md)
Validate Flight 1 is planned.

### [Validate flight2 GET interaction test step](test_steps/validate_get_operational_intent.md)
Flight1 when being planned, SUT needs to GET information of flight 2
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

### [Validate flight1 Notification sent to Control USS test step](test_steps/validate_notification_operational_intent.md)
Flight 2 not notified. As per **[astm.f3548.v21.USS0105](../../../../../requirements/astm/f3548/v21.md)** and **[astm.f3548.v21.SCD0085](../../../../../requirements/astm/f3548/v21.md)**
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

### [Delete sut flight test step](../../../../flight_planning/delete_flight_intent.md)
Delete sut flight

### [Delete control flight test step](../../../../flight_planning/delete_flight_intent.md)
Delete sut flight
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

## Attempt to plan deconflicted flight - near a flight sharing invalid data test case
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
### [Control USS plans flight 2, sharing invalid interuss data test step](../../../../flight_planning/plan_flight_intent.md)
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved
Flight 2 on time range B should be successfully planned by the control USS.
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

### [Validate flight 2 shared intent with invalid interuss data test step](test_steps/validate_sharing_operational_intent_but_with_invalid_interuss_data.md)
Validate that flight 2 is planned
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

### [SUT attempts to plan flight 1, expect failure test step](test_steps/plan_flight_intent_expect_failed.md)
The test driver attempts to plan the flight 1 on time range B via the tested USS. It checks if any conflicts with flight 2
which is of equal priority and came first.
per **[astm.f3548.v21.SCD0035](../../../../../requirements/astm/f3548/v21.md)**.

### [Validate flight 1 not shared by SUT test step](../../validate_not_shared_operational_intent.md)
Validate Flight 1 is not shared.

### [Validate flight2 GET interaction test step](test_steps/validate_get_operational_intent.md)
When Flight1 is being planned, SUT needs to GET information of flight 2
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

### [Validate flight1 Notification not sent to Control USS test step](test_steps/validate_no_notification_operational_intent.md)

### [Delete Control USS flight test step](../../../../flight_planning/delete_flight_intent.md)
Delete control USS flight 2

### [Delete SUT flight test step](../../../../flight_planning/delete_flight_intent.md)
Delete control USS flight 2
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

## Cleanup
### Successful flight deletion check
**[interuss.automated_testing.flight_planning.DeleteFlightSuccess](../../../../../requirements/interuss/automated_testing/flight_planning.md)**
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
from __future__ import annotations
BenjaminPelletier marked this conversation as resolved.
Show resolved Hide resolved

from typing import List, Optional
import time
import jwt

from monitoring.monitorlib.fetch import QueryError
from monitoring.uss_qualifier.common_data_definitions import Severity
from monitoring.uss_qualifier.scenarios.scenario import TestScenarioType
from monitoring.uss_qualifier.resources.interuss.mock_uss.client import MockUSSClient
from implicitdict import StringBasedDateTime
from loguru import logger
from monitoring.monitorlib.clients.mock_uss.interactions import Interaction


def expect_interuss_post_interactions(
scenario: TestScenarioType,
mock_uss: MockUSSClient,
st: StringBasedDateTime,
posted_to_url: str,
test_step: str,
):
found = False
if mock_uss is not None:
interactions = _get_interuss_interactions(scenario, mock_uss, st, test_step)
logger.debug(f"Checking for Post to {posted_to_url}")
with scenario.check("Expect Notification sent") as check:
found = False
for interaction in interactions:
method = interaction.query.request.method
url = interaction.query.request.url
if method == "POST" and posted_to_url in url:
found = True
if found == False:
check.record_failed(
summary=f"Notification to {posted_to_url} not received",
severity=Severity.Medium,
details=f"Notification to {posted_to_url} not received",
requirements="SCDxxxx",
)
scenario.end_test_step()

return found


def expect_no_interuss_post_interactions(
scenario: TestScenarioType,
mock_uss: MockUSSClient,
st: StringBasedDateTime,
posted_to_url: str,
test_step: str,
):
found = False
if mock_uss is not None:
interactions = _get_interuss_interactions(scenario, mock_uss, st, test_step)
logger.debug(f"Checking for POST request to {posted_to_url}")
with scenario.check("Expect Notification not sent") as check:
found = False
for interaction in interactions:
method = interaction.query.request.method
url = interaction.query.request.url
if method == "POST" and posted_to_url in url:
found = True
if found == True:
check.record_failed(
summary=f"Notification to {posted_to_url} wrongly sent",
severity=Severity.Medium,
details=f"Notification to {posted_to_url} wrongly sent",
requirements="SCDxxxx",
)
scenario.end_test_step()

return found


def expect_interuss_get_interactions(
scenario: TestScenarioType,
mock_uss: MockUSSClient,
st: StringBasedDateTime,
get_from_url: str,
id: str,
test_step: str,
):
found = False
if mock_uss is not None:
interactions = _get_interuss_interactions(scenario, mock_uss, st, test_step)
logger.debug(f"Checking for GET request to {get_from_url} for id {id}")
with scenario.check("Expect GET request") as check:
found = False
for interaction in interactions:
method = interaction.query.request.method
url = interaction.query.request.url
if method == "GET" and get_from_url in url and id in url:
found = True
if found == False:
check.record_failed(
summary=f"No GET request received at {get_from_url} for {id} ",
severity=Severity.Medium,
details=f"No GET request received at {get_from_url} for {id}",
requirements="SCDxxxx",
)

scenario.end_test_step()
return found


def _get_interuss_interactions(
scenario: TestScenarioType,
mock_uss: MockUSSClient,
st: StringBasedDateTime,
test_step: str,
) -> List[Interaction]:
scenario.begin_test_step(test_step)
time.sleep(5)
with scenario.check("MockUSS interactions request") as check:
try:
all_interactions, query = mock_uss.get_interactions(st)
scenario.record_query(query)
except QueryError as e:
for q in e.queries:
scenario.record_query(q)
check.record_failed(
summary=f"Error from mock_uss when attempting to get interactions from_time {st}",
severity=Severity.High,
details=f"{str(e)}\n\nStack trace:\n{e.stacktrace}",
query_timestamps=[q.request.timestamp for q in e.queries],
)

exclude_sub = mock_uss.session.auth_adapter.get_sub()

def is_uss_interaction(interaction: Interaction, excl_sub: str) -> bool:
headers = interaction.query.request.headers
if "Authorization" in headers:
token = headers.get("Authorization").split(" ")[1]
payload = jwt.decode(
token, algorithms="RS256", options={"verify_signature": False}
)
sub = payload["sub"]
logger.debug(f"sub of interuss_interaction token: {sub}")
if sub == excl_sub:
logger.debug(f"Excluding interaction with sub: {sub} ")
return False
else:
return True
else:
logger.error(f"Interaction received without Authorization : {interaction}")
return False

interuss_interactions = []
for interaction in all_interactions:
if is_uss_interaction(interaction, exclude_sub):
interuss_interactions.append(interaction)
logger.debug(
f"Interuss interaction reported : {interaction.query.request.method} {interaction.query.request.url} "
f"with response {interaction.query.response.status_code}"
)

return interuss_interactions
Loading
Loading