OSCAL CLI and External Constraints Improvements #603
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
david-waltermire
force-pushed
the
feature/external-constraints
branch
from
f45bd7c to
ad970d5
Compare
david-waltermire
force-pushed
the
feature/external-constraints
branch
from
63c2923 to
0318c8f
Compare
|
i think this is ready to be converted to a real PR and review, so we can start making PR's into develop instead of this feature branch |
aj-stein-gsa
force-pushed
the
develop
branch
from
fb5e758 to
35b66c9
Compare
…ts for use with OSCAL-cli.
* Introduce cucumber testing of yaml unit tests * introduce content generation and validation via CLI * use junit * eslint format
* Draft allowed values metaschema and YAML unit test. * automate content generation and validation via CLI (#614) * Introduce cucumber testing of yaml unit tests * introduce content generation and validation via CLI * better test summary reporting * introduce constraint coverage checking + improve coverage * throw an error if we do not find the matching rule * store output in .sarif folder * update constraints & test strategy, allow for mixed results as failure test * pretty print sarif & correct file name output * add time to sarif output file * introduce validation-cache for performance Co-authored-by: David Waltermire <[email protected]> Co-authored-by: Rene Tshiteya <[email protected]> Co-authored-by: DimitriZhurkin <[email protected]>
* Improve constraint coverage tests * Update features/steps/fedramp_extensions_steps.ts --------- Co-authored-by: David Waltermire <[email protected]>
* Add README.md to OSCAL CLI instructions * Implemented reviewers' comments
* make test runner aware of informational constraint results * Update features/steps/fedramp_extensions_steps.ts Co-authored-by: A.J. Stein <[email protected]> * improve test runner to handle warn and informational tests better * Update fedramp_extensions_steps.ts --------- Co-authored-by: A.J. Stein <[email protected]>
* add make update command * add make constraint * improve first run on fresh constraint
* Constraint-specific CONTRIBUTING to its own dir Add the diagram of the constraint and testing components specific to this area of code base here and outline other sections to follow. * Reference prerequisites in README for install * Reorder CONTRIBUTING sections, add Metaschema one * Add references to relevant Metaschema docs * Add docs for new constraint tests * Sigh, whitespace from code blocks break numbering * Add detailed docs on modifying existing constraint * Shorten and clean up explanatory copy * Explain purpose of oscal-external constraints * Clarify oscal file for generic constraints only * Add guidance for using which FR constraints file * [skip ci] Docs for deleting tests, links to PR docs * Add @Rene2mt's feedback about testing one constraint by ID Clearly this guy constraints! Co-authored-by: Rene Tshiteya <[email protected]> * [skip ci] Clean up typos, grammar, and missing info per @Rene2mt's PR feedback Co-authored-by: Rene Tshiteya <[email protected]> --------- Co-authored-by: Rene Tshiteya <[email protected]>
* Added constraints and tests for resource-has-(title/rlink) * metapath cleanup * Add comment Co-authored-by: A.J. Stein <[email protected]> * Add comment Co-authored-by: A.J. Stein <[email protected]> * Added or base64 condition * Cleanup * Edit constraint name --------- Co-authored-by: A.J. Stein <[email protected]>
) * [skip ci] More appropriate README title for #659 * [skip ci] Context for constraints, tools for #659 Be sure to talk about the constraints and their relationship to the tool, do not just talk about the `oscal-cli` without context. Re-order some of the info, talk about target audience before install instructions. * [skip ci] Remove dupe copy of 'who for?' for #659 * [skip ci] Subject is FR not only FR devs in #659 * [skip ci] No more header numbers, add headers #659 * [skip ci] Better intro and simple diagram for #659 * [skip ci] Docker install prerequisites for #659 * [skip ci] More install and command docs for #659 * [skip ci] Clearer wording on OCI tool for #659 * [skip ci] Feedback for #659, re manual clone step * [skip ci] Remove dangling this for #659 Thanks to @david-waltermire for catching that. * [skip ci] Align arguments docs, examples for #659 Based on some more detailed feedback from @Rene2mt that matched other comments from Dave in the PR. * Good catch, @gabelis, fix numbering for #659
* introduce data center constraints * complete data center constraints * Update src/validations/constraints/content/ssp-all-INVALID.xml Co-authored-by: A.J. Stein <[email protected]> * Update src/validations/constraints/content/ssp-all-VALID.xml Co-authored-by: A.J. Stein <[email protected]> * Update src/validations/constraints/content/ssp-location-INVALID.xml Co-authored-by: A.J. Stein <[email protected]> * remove allowed-type data center country code * Late review feedback: align country code example with constraint * Fix the correction that broke negative testing, sorry Paul --------- Co-authored-by: A.J. Stein <[email protected]>
…write perms dropping (#665) * Add initial OCI spec for container for #655 * Now add FR constraint files for #655 * Woops, fix typo in clone path for fd_data_dl scratch container * Constraints in /opt/fedramp sudir, make it WORKDIR * Switch to Alpine Maven scratch image not Debian * Switch to Node for final image, install oscaljs * Add checkout data to final image * Fix missed parameterization of git image * Add non-default OCI image build target for make * Verify GPG signaure of oscal-cli build * Add clean target for OCI image builds * Allow for TLS bypass and proxy in Makefile Disable cert-checking for the local version that is built on laptops for GSA staff who make use of a VPN/proxy solution that intercept all TLS communication for security monitoring. This includes not just Docker, but also the containers as they build an image. Since production images will be made in GitHub Actions without the Makefile, these directives will be ignored. * Do not do slow git clone, use local COPY instead For speed, ease of access, and leave commit metadata from the container ID linked to the commit hash itself, just copy from the outside context of the image build. * Add publish target to Makefile with useful tags Also try docker push to GHCR to start before moving on the "in pipeline" build with GitHub Actions. * Fix repeat docker commands for correct tag-n-push * Correct the org.opencontainers.image.source label * Actions: perms for writing packages (ghcr.io) * Actions: follow GH tutorial, more perms added * Actions: build, sign, push, attest and OCI image This workflow change is the first attempt at building, pushing, and signing the validation-tools image to push to the ghcr.io registry. * Actions: ref_name for image tags problematic For both PRs and non-PR branches, that seems to cause problems for tags that we ought to avoid for now. * Actions: use action correctly, no manual labels * Actions: remove metadata from Dockerfile, use GHA * Actions: woops, forgot explicit checkout path Our GHA CI/CD checks out to `./git-content`, `.` by default so the action directive looking for context did not find the Dockerfile. * Actions: check if least privilege perms block push See more details in this reply and the larger context from others who cannot push a built container to ghcr.io. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: scratch that, `write-all` blocked by org The github.com/GSA organization still blocks the write to an org-level package in very permissive move. Tips from the discussions posts did not help here. https://github.com/orgs/community/discussions/57724#discussioncomment-7779731 * Actions: add metadata action SHA options We need to force SHA1 long (not seven-digit short version to avoid collisions), remove both `sha-` prefix and remove suffix explicitly. * Actions, sigh, really remove `sha256` prefix again It seems that didn't stick the last time, so I will try this config again and follow the official custom hash label strategy from the action example from the official README. * Support MVP platforms, arm64 and amd64 If not we will only support modern Apple computers with modern M1 chips, not Intel environments for PC and older Macs. We need broad support for these top platforms. * Explicit platform option for buildx too for #656 It seems this may be needed because I still get similar but different warnings on multi-platform docker builds when using on macOS on an Apple laptop with a M1 processor and amd64 processor for personal computers with Windows and Linux operating systems respectively. > WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v3) and no specific platform was requested * Pin metadata action and update configs for #656 - Had a slightly wrong version of docker/metadata-action that could not use annotations properly, hence no annotations on image. - Use annotations instead of custom override labels with that action. - Update docker/build-push-registry action to retrieve those labels as well. - Change subject name for attestation to end with `-attestation` suffix to make the GHCR registry entries less confusing. * Woops, attestation subject === image name for #656 I re-read the dogs. Attestations will be uploaded to Sigstore but I will not busy up the registry with them every moment as it will make it even more confusing for novice users and advanced developers what data they are looking for by content-addressable git commit hash ID. * Explanatory comments on Dockerfile lint for #656 For future analysis or assessment, I am leaving information in the Dockerfile as comments to address warning output in docker build and push flagging a potential finding re secrets based on variable names. ```sh 4 warnings found (use docker --debug to expand): - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "OSCAL_CLI_GPG_KEY") (line 20) - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "TEMURIN_APK_KEY_URL") (line 45) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 17) - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 43) ``` They are IDs to secrets, not actually secrets, now I have documented it. * Attestations need explicit reg push off for #656 Just removing it may not have done the trick.
* Added back-matter 'has' constraints * Set levels to 'ERROR'
* Actions: tighten when docker build runs and how - We do not want to fails build when staff and community make fork PRs. - We want to make sure the latest feature branch is tagged and deployed for now, stop push PR container builds before merge. * Actions: more explicit branch targeting I am not sure that syntax is air-tight with var == 'value1' || 'value2', so make it more explicit and have var on left side and right side of the boolean OR check. * Actions: even more explicit use startsWith syntax * Actions: one last attempt to force annotations
I had incorrectly put it on workflow_dispatch which will not help as needed.
* Add system-owner role * Adjust wording to be more friendly to users with less OSCAL knowledge. --------- Co-authored-by: A.J. Stein <[email protected]>
* introduce missing-response-components constraint * Add review feedback from AJ before merge --------- Co-authored-by: A.J. Stein <[email protected]>
* Update SSP metadata role constraints * Adjust message text to be more approachable, per PR feedback --------- Co-authored-by: A.J. Stein <[email protected]>
* Added system-characteristics 'categorization-has constraints and tests * Adjust path style and message wording per my PR feedback --------- Co-authored-by: A.J. Stein <[email protected]>
* Add system-characteristics 'has-assurance-level' constraints & tests * Make uniform wording for informational findings per PR review --------- Co-authored-by: A.J. Stein <[email protected]>
* introduce metaquery script * Update CONTRIBUTING.md * Tweak header for mq explanation in CONTRIBUTING.md --------- Co-authored-by: A.J. Stein <[email protected]>
* Added constraints & tests * Corrected message
…702) * Added system-characteristics 'has-authorization-boundary' constraints & tests * Style corrections * Rephrase * Fix IAL-FAL acronym typo and wrap up review. --------- Co-authored-by: A.J. Stein <[email protected]>
* Fixed targets, created separate invalid files where necessary, redirected yaml fail test files where necessary, removed 2 constraints that are handled by the schema * Fixed targets, created separate invalid files where necessary, redirected yaml fail test files where necessary, removed 1 constraint that is handled by the schema * cleanup
* Introduce test data optimization ADR * Correction: only scaffold invalid test data file * Provide more detail in ADR * Date correction * Add solution & consequences * Add reference to #710. --------- Co-authored-by: A.J. Stein <[email protected]>
* add option to specify test content be made from template or use all invalid * improve scaffolding for negative test cases * introduce re-run script * update contributing docs for rerun info * improve scaffolding * adjust constraint script to match feedback * Update dev-constraint.js
* Created separate invalid test data file and edited the yaml fail case file to reference the new invalid test data file for each constraint in * Created separate invalid test data file and edited the yaml fail case file to reference the new invalid test data file for each constraint in * Removed deprecated invalid test data files * Rename invalid test data files * Delete ssp-all-INVALID.xml (obviously)
aj-stein-gsa
force-pushed
the
feature/external-constraints
branch
from
c59e72c to
f6d2009
Compare
* Add allowed-values (virtual, public, allows-authenticated-scan) * Add YAML and ssp-virtual-INVALID..xml * Add YAML and INVALID for public and authorized-scan allowed values. * Rebase the branch and add "inventory-item" to constraints * Fix feature inventory of tests per feedback --------- Co-authored-by: A.J. Stein <[email protected]>
* hotfix post scaffold run * Update dev-constraint.js * remove conflicting legacy files
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Committer Notes
This feature branch provides external constraints for use with the OSCAL-cli and CI/CD improvements.
Two files are provided:
All Submissions:
If applicable, have all FedRAMP Documents Related to OSCAL Adoption affected by the changes in this issue have been updated?If applicable, does this PR reference the issue it addresses and explain how it addresses the issue?By submitting a pull request, you are agreeing to provide this contribution under the CC0 1.0 Universal public domain dedication.