Skip to content

Commit

Permalink
Test GHCR integration with "in upstream" branch instead of fork with …
Browse files Browse the repository at this point in the history
…write perms dropping (#665)

* Add initial OCI spec for container for #655

* Now add FR constraint files for #655

* Woops, fix typo in clone path for fd_data_dl scratch container

* Constraints in /opt/fedramp sudir, make it WORKDIR

* Switch to Alpine Maven scratch image not Debian

* Switch to Node for final image, install oscaljs

* Add checkout data to final image

* Fix missed parameterization of git image

* Add non-default OCI image build target for make

* Verify GPG signaure of oscal-cli build

* Add clean target for OCI image builds

* Allow for TLS bypass and proxy in Makefile

Disable cert-checking for the local version that is built on laptops for
GSA staff who make use of a VPN/proxy solution that intercept all TLS
communication for security monitoring. This includes not just Docker,
but also the containers as they build an image. Since production images
will be made in GitHub Actions without the Makefile, these directives
will be ignored.

* Do not do slow git clone, use local COPY instead

For speed, ease of access, and leave commit metadata from the container
ID linked to the commit hash itself, just copy from the outside context
of the image build.

* Add publish target to Makefile with useful tags

Also try docker push to GHCR to start before moving on the "in pipeline"
build with GitHub Actions.

* Fix repeat docker commands for correct tag-n-push

* Correct the org.opencontainers.image.source label

* Actions: perms for writing packages (ghcr.io)

* Actions: follow GH tutorial, more perms added

* Actions: build, sign, push, attest and OCI image

This workflow change is the first attempt at building, pushing, and
signing the validation-tools image to push to the ghcr.io registry.

* Actions: ref_name for image tags problematic

For both PRs and non-PR branches, that seems to cause problems for tags
that we ought to avoid for now.

* Actions: use action correctly, no manual labels

* Actions: remove metadata from Dockerfile, use GHA

* Actions: woops, forgot explicit checkout path

Our GHA CI/CD checks out to `./git-content`, `.` by default so the action
directive looking for context did not find the Dockerfile.

* Actions: check if least privilege perms block push

See more details in this reply and the larger context from others who
cannot push a built container to ghcr.io.

https://github.com/orgs/community/discussions/57724#discussioncomment-7779731

* Actions: scratch that, `write-all` blocked by org

The github.com/GSA organization still blocks the write to an org-level
package in very permissive move. Tips from the discussions posts did not
help here.

https://github.com/orgs/community/discussions/57724#discussioncomment-7779731

* Actions: add metadata action SHA options

We need to force SHA1 long (not seven-digit short version to avoid
collisions), remove both `sha-` prefix and remove suffix explicitly.

* Actions, sigh, really remove `sha256` prefix again

It seems that didn't stick the last time, so I will try this config
again and follow the official custom hash label strategy from the action
example from the official README.

* Support MVP platforms, arm64 and amd64

If not we will only support modern Apple computers with modern M1 chips,
not Intel environments for PC and older Macs. We need broad support for
these top platforms.

* Explicit platform option for buildx too for #656

It seems this may be needed because I still get similar but different
warnings on multi-platform docker builds when using on macOS on an Apple
laptop with a M1 processor and amd64 processor for personal computers
with Windows and Linux operating systems respectively.

> WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v3) and no specific platform was requested

* Pin metadata action and update configs for #656

- Had a slightly wrong version of docker/metadata-action that could not
use annotations properly, hence no annotations on image.
- Use annotations instead of custom override labels with that action.
- Update docker/build-push-registry action to retrieve those labels as
well.
- Change subject name for attestation to end with `-attestation` suffix
to make the GHCR registry entries less confusing.

* Woops, attestation subject === image name for #656

I re-read the dogs. Attestations will be uploaded to Sigstore but I will
not busy up the registry with them every moment as it will make it even
more confusing for novice users and advanced developers what data they
are looking for by content-addressable git commit hash ID.

* Explanatory comments on Dockerfile lint for #656

For future analysis or assessment, I am leaving information in the
Dockerfile as comments to address warning output in docker build and
push flagging a potential finding re secrets based on variable names.

```sh
 4 warnings found (use docker --debug to expand):
 - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "OSCAL_CLI_GPG_KEY") (line 20)
 - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ARG "TEMURIN_APK_KEY_URL") (line 45)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 17)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 43)
```

 They are IDs to secrets, not actually secrets, now I have documented it.

* Attestations need explicit reg push off for #656

Just removing it may not have done the trick.
  • Loading branch information
aj-stein-gsa committed Sep 24, 2024
1 parent f2979a9 commit 2fa3d36
Show file tree
Hide file tree
Showing 4 changed files with 166 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Ignore everything
*
!/src/validations/constraints
65 changes: 65 additions & 0 deletions .github/workflows/content-artifacts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,19 @@ on:
name: Process Content
env:
HOME_REPO: GSA/fedramp-automation
IMAGE_NAME: GSA/fedramp-automation/validation-tools
REGISTRY: ghcr.io
# Docs: github.com/docker/metadata-action/?tab=readme-ov-file#typesha
DOCKER_METADATA_PR_HEAD_SHA: true
jobs:
validate-and-publish-content:
name: Content Validation Checking, Conversion and Validation
runs-on: ubuntu-20.04
permissions:
contents: read
packages: write
attestations: write
id-token: write
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332
with:
Expand Down Expand Up @@ -53,3 +62,59 @@ jobs:
commit_user_name: OSCAL GitHub Actions Bot
commit_user_email: [email protected]
commit_author: OSCAL GitHub Actions Bot <[email protected]>
- name: Container image QEMU setup for cross-arch builds
id: image_setup_qemu
uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf
- name: Container image buildx setup for cross-arch builds
id: image_setup_buildx
with:
platforms: linux/amd64,linux/arm64
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db
- name: Container image login
id: image_login
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Container image metadata and tag generation
id: image_metadata
uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
with:
images:
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=sha,prefix=,suffix=,format=long
type=ref,event=branch
type=ref,event=tag
type=ref,event=pr
# For now, do not auto-tag latest, maintainers will decided what is
# release-worthy.
flavor: |
latest=false
annotations:
maintainers="FedRAMP Automation Team <[email protected]>"
org.opencontainers.image.authors="FedRAMP Automation Team <[email protected]>"
org.opencontainers.image.documentation="https://automate.fedramp.gov"
org.opencontainers.image.source="https://github.com/GSA/fedramp-automation"
org.opencontainers.image.vendor="GSA Technology Transformation Services"
org.opencontainers.image.title="FedRAMP Validation Tools"
org.opencontainers.image.description="FedRAMP's tools for validating OSCAL data"
org.opencontainers.image.licenses="CC0-1.0"
- name: Container image registry push
id: image_registry_push
uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
with:
context: git-content
push: true
tags: ${{ steps.image_metadata.outputs.tags }}
labels: ${{ steps.image_metadata.outputs.annotations }}
platforms: linux/amd64,linux/arm64
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Container image push attestations
uses: actions/attest-build-provenance@1c608d11d69870c2092266b3f9a6f3abbf17002c
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-digest: ${{ steps.image_registry_push.outputs.digest }}
push-to-registry: false
67 changes: 67 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
ARG MAVEN_IMAGE=maven:3.9.9-eclipse-temurin-22-alpine
ARG NODE_IMAGE=node:22-alpine3.20
ARG APK_EXTRA_ARGS
ARG WGET_EXTRA_ARGS
# Static analysis from docker build and push warns this is a secret, it is not.
# Per official developer instructions, it is necessary to verify the APK packages
# for Alpine or properly signed. This information is inherently public.
# https://adoptium.net/installation/linux/#_alpine_linux_instructions
ARG TEMURIN_APK_KEY_URL=https://packages.adoptium.net/artifactory/api/security/keypair/public/repositories/apk
ARG TEMURIN_APK_REPO_URL=https://packages.adoptium.net/artifactory/apk/alpine/main
ARG TEMURIN_APK_VERSION=temurin-22-jdk
ARG MAVEN_DEP_PLUGIN_VERSION=3.8.0
ARG OSCAL_CLI_VERSION=2.0.2
# Current public key ID for [email protected] releases of oscal-cli
# Static analysis from docker build and push warns this is a secret, it is not
# and is necessary to cross-ref the Maven GPG key for checking build signatures.
# https://keyserver.ubuntu.com/pks/lookup?search=0127D75951997E00&fingerprint=on&op=index
ARG OSCAL_CLI_GPG_KEY=0127D75951997E00
ARG OSCAL_JS_VERSION=1.4.4
ARG FEDRAMP_AUTO_GIT_URL=https://github.com/GSA/fedramp-automation.git
ARG FEDRAMP_AUTO_GIT_REF=feature/external-constraints
ARG FEDRAMP_AUTO_GIT_COMMIT

FROM ${MAVEN_IMAGE} as oscal_cli_downloader
ARG MAVEN_DEP_PLUGIN_VERSION
ARG OSCAL_CLI_VERSION
ARG OSCAL_CLI_GPG_KEY
ARG APK_EXTRA_ARGS
ARG WGET_EXTRA_ARGS
RUN apk add --no-cache gpg gpg-agent unzip && \
mkdir -p /opt/oscal-cli && \
mvn \
org.apache.maven.plugins:maven-dependency-plugin:${MAVEN_DEP_PLUGIN_VERSION}:copy \
-DoutputDirectory=/opt/oscal-cli \
-DremoteRepositories=https://repo1.maven.org/maven2 \
-Dartifact=dev.metaschema.oscal:oscal-cli-enhanced:${OSCAL_CLI_VERSION}:zip:oscal-cli && \
mvn \
org.apache.maven.plugins:maven-dependency-plugin:${MAVEN_DEP_PLUGIN_VERSION}:copy \
-DoutputDirectory=/opt/oscal-cli \
-DremoteRepositories=https://repo1.maven.org/maven2 \
-Dartifact=dev.metaschema.oscal:oscal-cli-enhanced:${OSCAL_CLI_VERSION}:zip.asc:oscal-cli && \
gpg --recv-keys ${OSCAL_CLI_GPG_KEY} && \
gpg -k ${OSCAL_CLI_GPG_KEY} && \
cd /opt/oscal-cli && \
gpg --verify *.zip.asc && \
unzip *.zip && \
rm -f *.zip && \
rm -f *.zip.asc

FROM ${NODE_IMAGE} as final
ARG OSCAL_JS_VERSION
ARG TEMURIN_APK_KEY_URL
ARG TEMURIN_APK_REPO_URL
ARG TEMURIN_APK_VERSION
ARG APK_EXTRA_ARGS
ARG WGET_EXTRA_ARGS
COPY --from=oscal_cli_downloader /opt/oscal-cli /opt/oscal-cli
RUN wget ${WGET_EXTRA_ARGS} -O /etc/apk/keys/adoptium.rsa.pub "${TEMURIN_APK_KEY_URL}" && \
echo "${TEMURIN_APK_REPO_URL}" >> /etc/apk/repositories && \
apk add ${APK_EXTRA_ARGS} --no-cache ${TEMURIN_APK_VERSION} && \
mkdir -p /opt/fedramp/oscaljs && \
mkdir -p /opt/fedramp/constraints && \
(cd /opt/fedramp/oscaljs && npm install oscal@${OSCAL_JS_VERSION})
COPY ./src/validations/constraints/*.xml /opt/fedramp/constraints/
ENV PATH="$PATH:/opt/oscal-cli/bin:/opt/fedramp/oscaljs/node_modules/.bin"
WORKDIR /opt/fedramp/constraints
ENTRYPOINT [ "/opt/oscal-cli/bin/oscal-cli" ]
31 changes: 31 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
export BASE_DIR=$(shell pwd)
OCI_REV_TAG=$(shell git rev-parse HEAD)

help:
@grep -h -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
Expand Down Expand Up @@ -26,6 +27,12 @@ clean-dist: ## Clean non-RCS-tracked dist files
@echo "Cleaning dist..."
git clean -xfd dist

clean-oci-image:
docker rmi -f \
validation-tools:$(OCI_REV_TAG) \
ghcr.io/gsa/fedramp-automation/validation-tools:$(OCI_REV_TAG) \
gsatts/validation-tools:$(OCI_REV_TAG) \

test: build-validations ## Test all

build: build-validations build-web dist ## Build all artifacts and copy into dist directory
Expand All @@ -39,3 +46,27 @@ build: build-validations build-web dist ## Build all artifacts and copy into di

@echo '#/bin/bash\necho "Serving FedRAMP ASAP documentation at http://localhost:8000/..."\npython3 -m http.server 8000 --directory web/' > ./dist/serve-documentation
chmod +x ./dist/serve-documentation

build-oci-image:
docker build \
--build-arg APK_EXTRA_ARGS="--no-check-certificate" \
--build-arg WGET_EXTRA_ARGS="--no-check-certificate" \
-t validation-tools:$(OCI_REV_TAG) \
-t ghcr.io/gsa/fedramp-automation/validation-tools:$(OCI_REV_TAG) \
-t gsatts/validation-tools:$(OCI_REV_TAG) \
.

publish-oci-image:
docker tag \
validation-tools:$(OCI_REV_TAG) validation-tools:latest

docker tag \
ghcr.io/gsa/fedramp-automation/validation-tools:$(OCI_REV_TAG) \
ghcr.io/gsa/fedramp-automation/validation-tools:latest

docker tag \
gsatts/validation-tools:$(OCI_REV_TAG) \
gsatts/validation-tools:latest

docker push ghcr.io/gsa/fedramp-automation/validation-tools:$(OCI_REV_TAG)
docker push ghcr.io/gsa/fedramp-automation/validation-tools:latest

0 comments on commit 2fa3d36

Please sign in to comment.