Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZachG update parametric documentation and add contributing doc #3325

Open
wants to merge 45 commits into
base: main
Choose a base branch
from

Conversation

ZStriker19
Copy link
Contributor

@ZStriker19 ZStriker19 commented Oct 25, 2024

Motivation

A guide on how to contribute to system-tests will be very helpful for first time contributors and those who haven't contributed for some time. This should greatly reduce the startup time on getting an engineer able to write the tests they need.

Changes

Contributing guide as well as updates to the current parametric docs.

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

docs/edit/features.md Outdated Show resolved Hide resolved
@ZStriker19 ZStriker19 marked this pull request as ready for review November 14, 2024 18:16
@ZStriker19 ZStriker19 requested a review from a team as a code owner November 14, 2024 18:16
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the hundreds of comments, my creative writing background gets the best of me!

docs/edit/features.md Outdated Show resolved Hide resolved
docs/scenarios/parametric.md Outdated Show resolved Hide resolved
docs/scenarios/parametric.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Show resolved Hide resolved
docs/edit/features.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Show resolved Hide resolved
docs/edit/README.md Outdated Show resolved Hide resolved
docs/edit/add-new-test.md Outdated Show resolved Hide resolved
docs/edit/features.md Show resolved Hide resolved
docs/edit/features.md Show resolved Hide resolved
docs/scenarios/parametric.md Outdated Show resolved Hide resolved
docs/scenarios/parametric.md Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Show resolved Hide resolved
docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
Copy link
Contributor

@mabdinur mabdinur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one last nit. Overall looks great to me!

docs/scenarios/parametric_contributing.md Outdated Show resolved Hide resolved
Copy link
Contributor

@mtoffl01 mtoffl01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small nits but otherwise, good to go.
But also, since you said your docs don't contain contributing information for weblog, can you just make the change in your PR to remove the empty [TODO]: LINK TO CONTRIBUTING DOC link in this doc? https://github.com/DataDog/system-tests/blob/main/docs/edit/enable-test.md


Several key points:

* Each new feature should be defined in [_features.py](/utils/_features.py). This consists of adding a feature in [Feature Parity Dashbaord](https://feature-parity.us1.prod.dog/), get the feature id and copying one of the already added features, changing the name and the feature id in the url, and the feature number. In this case we'd add
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Each new feature should be defined in [_features.py](/utils/_features.py). This consists of adding a feature in [Feature Parity Dashbaord](https://feature-parity.us1.prod.dog/), get the feature id and copying one of the already added features, changing the name and the feature id in the url, and the feature number. In this case we'd add
* Each new feature should be defined in [_features.py](/utils/_features.py). This consists of adding a feature in [Feature Parity Dashboard](https://feature-parity.us1.prod.dog/), get the feature id and copying one of the already added features, changing the name and the feature id in the url, and the feature number. In this case we'd add

return test_object
```

* One class test one feature
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* One class test one feature
* One class tests one feature


1. `./utils/scripts/parametric/run_reference_http.sh`
2. Navigate to http://localhost:8000/docs in your web browser to access the documentation.
3. You can download the OpenAPI schema from http://localhost:8000/openapi.json. This schema can be imported into tools like [Postman](https://learning.postman.com/docs/integrations/available-integrations/working-with-openAPI/) or other API clients to facilitate development and testing.

Not all endpoint implementations per language are up to spec with regards to their parameters and return values. To see these please reference the [feature parity board](https://feature-parity.us1.prod.dog/#/?runDateFilter=7d&feature=339)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Not all endpoint implementations per language are up to spec with regards to their parameters and return values. To see these please reference the [feature parity board](https://feature-parity.us1.prod.dog/#/?runDateFilter=7d&feature=339)
Not all endpoint implementations per language are up to spec with regards to their parameters and return values. To view endpoints that are not up to spec, see the [feature parity board](https://feature-parity.us1.prod.dog/#/?runDateFilter=7d&feature=339)

@@ -8,6 +8,10 @@ System tests allow developers define scenarios and ensure datadog libraries prod

To make changes, you must be able to run tests locally. Instructions for running **end-to-end** tests can be found [here](https://github.com/DataDog/system-tests/blob/main/docs/execute/README.md#run-tests) and for **parametric**, [here](https://github.com/DataDog/system-tests/blob/main/docs/scenarios/parametric.md#running-the-tests).

**Note**

For information on contributing to specifically **parametric** tests, see [here](/docs/scenarios/parametric_contributing.md).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming 😃

Suggested change
For information on contributing to specifically **parametric** tests, see [here](/docs/scenarios/parametric_contributing.md).
For information on contributing to specifically **parametric** scenario, see [here](/docs/scenarios/parametric_contributing.md).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants