[uss_qualifier] Clean up artifacts #282
Conversation
| path = os.path.join(config.artifacts.output_path, "report.json") | ||
| logger.info(f"Writing raw report to {path}") | ||
| raw_report = config.artifacts.raw_report | ||
| report_to_write = redacted_report if _should_redact(raw_report) else report |
There was a problem hiding this comment.
Looks like the configuration object should be passed in _should_redact instead of the report itself.
There was a problem hiding this comment.
_should_redact is looking for an object that may have a redact_access_tokens attribute which should match config.artifacts.raw_report (what raw_report is assigned the line above). I'll take a look at s8 for uspace.
There was a problem hiding this comment.
Oh. sorry just realized that we redact tokens by removing the signature. Everything is ok. Somehow, I missed the end of the token which is correctly redacted.
There was a problem hiding this comment.
Hmm, could you elaborate on where you're seeing unredacted tokens? I downloaded the CI run report artifacts and the first few tokens appear to be redacted (redacted means that the third/last part of the token is replaced with REDACTED, not that the token is removed entirely) in uspace/sequence/s8.html. The raw uspace report.json also seems to have REDACTED tokens (per spot check).
* Clean up artifacts * Fix dss_probing artifacts * Fix shell lint 726592d
This PR attempts to clean up artifact generation for uss_qualifier.
All of the artifacts for a particular test run are confined to a single folder by placing a single
output_pathfield on the ArtifactsConfiguration. This cleans up theoutputfolder substantially. Each individual artifact type is modified slightly to just specify a name within the output path rather than specifying the full output path (when applicable); artifacts that only have one way to be generated are assigned static names (e.g., report.json, report.html, sequence).Two artifact types are removed: graph visualization and tested roles. These artifacts did not seem to be providing much value and they can be retrieved from the commit history if they ever need to be revived. In the mean time, removing them reduces maintenance burden.
Report redaction is now specified on a per-artifact basis, and the redacted report is provided by default to artifact generators that shouldn't need full details (tested requirements, templated reports).
The GitHub Pages index is updated to provide links to more of the outputs.
External report*.json verification is removed from run_locally.sh as the report validation in #257 should have removed it (probably was un-removed as part of a suboptimal merge).
A few smaller adjustments are made to various artifacts -- "report" is clarified to "raw_report", multiple tested requirements configurations are now possible.
Note that these changes are not backwards-compatible with the current configuration format, but I think this is acceptable as we are still in pre-v1.0.0 state and it is better to reach an optimal v1.0.0 format sooner than ease the configuration upgrade burden of alpha adopters.