Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for docker build caching, disabled by default #313

Merged
merged 11 commits into from
Aug 14, 2024
57 changes: 42 additions & 15 deletions .github/actions/docker/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,9 @@ inputs:
default: 'false'
description: 'Generate tag only.'
required: false
Comment on lines 63 to 65
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This input should be removed as well if we're eliminating TAG_ONLY entirely.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i can keep it if we think it is or should be used? would need to understand the use case that led to it being added

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can see, the only use I've seen so far is in the cd-retag workflow, which tags an existing image with another tag and pushes it. Let's keep things as-is for now.

CACHE_FROM:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are not used by any consuming repos and not exposed in the ci workflows so removing them

default: 'type=gha'
description: 'Cache from.'
required: false
CACHE_TO:
default: 'type=gha,mode=max'
description: 'Cache to.'
ENABLE_CACHE:
default: 'false'
description: 'Whether to use cache when building the image'
required: false
outputs:
IMAGE_TAG:
Expand Down Expand Up @@ -126,15 +122,31 @@ runs:
run: echo BUILD_ARGS=${{inputs.BUILD_ARGS}} >> $GITHUB_ENV
shell: bash

- name: Build and export to Docker
- name: Build and export to Docker without cache
uses: docker/build-push-action@v6
if: inputs.TAG_ONLY == 'false'
Copy link
Collaborator Author

@ryanwi ryanwi Aug 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TAG_ONLY is never used as far as I can tell, in any consuming repos, so removing to simplify the flag logic

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TAG_ONLY looks to be used in workflows/cd-retag.yml. We should remove references to TAG_ONLY there as well.

if: inputs.ENABLE_CACHE == 'false'
with:
load: true
tags: |
${{ steps.meta.outputs.tags }}
no-cache: true
file: ${{ inputs.FILE }}
context: ${{ inputs.CONTEXT }}
# cannot use multiple platforms with `load`, build a single arch image for validation purposes in CI
platforms: linux/amd64
build-args: ${{ env.BUILD_ARGS }}
secrets: ${{ env.DOCKER_SECRETS }}

- name: Build and export to Docker with cache
uses: docker/build-push-action@v6
if: inputs.ENABLE_CACHE == 'true'
with:
load: true
tags: |
${{ steps.meta.outputs.tags }}
# cache-from: type=gha
# cache-to: type=gha,mode=max
cache-from: type=gha
cache-to: type=gha,mode=max
no-cache: false
file: ${{ inputs.FILE }}
context: ${{ inputs.CONTEXT }}
# cannot use multiple platforms with `load`, build a single arch image for validation purposes in CI
Expand All @@ -155,16 +167,31 @@ runs:
docker logs test
${{ inputs.CONTAINER_TEST_COMMAND }}

- name: Build and push
if: inputs.PUSH == 'true' # && inputs.TAG_ONY == 'false'
- name: Build and push with cache
if: inputs.PUSH == 'true' && inputs.ENABLE_CACHE == 'true'
uses: docker/build-push-action@v6
with:
push: true
tags: |
${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
no-cache: false
file: ${{ inputs.FILE }}
context: ${{ inputs.CONTEXT }}
platforms: ${{ inputs.PLATFORMS }}
build-args: ${{ env.BUILD_ARGS }}
secrets: ${{ env.DOCKER_SECRETS }}

- name: Build and push without cache
if: inputs.PUSH == 'true' && inputs.ENABLE_CACHE == 'false'
uses: docker/build-push-action@v6
with:
push: true
tags: |
${{ steps.meta.outputs.tags }}
cache-from: ${{ inputs.CACHE_FROM }}
cache-to: ${{ inputs.CACHE_TO }}}
file: ${{ inputs.FILE }}
no-cache: true
context: ${{ inputs.CONTEXT }}
platforms: ${{ inputs.PLATFORMS }}
build-args: ${{ env.BUILD_ARGS }}
Expand Down
18 changes: 6 additions & 12 deletions .github/workflows/ci-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -106,15 +106,10 @@ on:
type: string
description: ref to pull and build default to github.ref
required: false
CACHE_FROM:
type: string
default: 'type=gha'
description: 'Cache from.'
required: false
CACHE_TO:
type: string
default: 'type=gha,mode=max'
description: 'Cache to.'
ENABLE_DOCKER_BUILD_CACHE:
type: boolean
default: false
description: 'Whether to use cache when building the image'
required: false
## Vault Secrets
VAULT_SECRETS:
Expand Down Expand Up @@ -229,7 +224,7 @@ jobs:
uses: actions/checkout@v4
with:
repository: signalwire/actions-template
ref: main
ref: ryanwi/docker-cache
path: actions

- uses: ./actions/.github/actions/gpg
Expand Down Expand Up @@ -280,8 +275,7 @@ jobs:
CONTAINER_TEST_RUN_OPTIONS: ${{ inputs.CONTAINER_TEST_RUN_OPTIONS }}
CONTAINER_TEST_COMMAND: ${{ inputs.CONTAINER_TEST_COMMAND }}
OUTPUT_TAG_INDEX: ${{ inputs.OUTPUT_TAG_INDEX }}
CACHE_FROM: ${{ inputs.CACHE_FROM }}
CACHE_TO: ${{ inputs.CACHE_TO }}
ENABLE_CACHE: ${{ inputs.ENABLE_DOCKER_BUILD_CACHE }}
env:
GITHUB_TOKEN: ${{ github.token }}
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
Expand Down
10 changes: 8 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,11 @@ on:
default: .
description: Whenever the WF requires a different path than . for the revision file
required: false
ENABLE_DOCKER_BUILD_CACHE:
type: boolean
default: false
description: 'Whether to use cache when building the image'
required: false
## Vault Secrets
VAULT_SECRETS:
type: string
Expand Down Expand Up @@ -207,7 +212,7 @@ jobs:

CHECK:
name: CI
uses: signalwire/actions-template/.github/workflows/ci-check.yml@main
uses: signalwire/actions-template/.github/workflows/ci-check.yml@ryanwi/docker-cache
with:
ENVIRONMENT: ${{ inputs.ENVIRONMENT }}
PROJECT_NAME: ${{ inputs.PROJECT_NAME }}
Expand Down Expand Up @@ -245,7 +250,7 @@ jobs:
BUILD:
name: BUILD
needs: CHECK
uses: signalwire/actions-template/.github/workflows/ci-build.yml@main
uses: signalwire/actions-template/.github/workflows/ci-build.yml@ryanwi/docker-cache
with:
ENVIRONMENT: ${{ inputs.ENVIRONMENT }}
PROJECT_NAME: ${{ inputs.PROJECT_NAME }}
Expand All @@ -269,6 +274,7 @@ jobs:
VAULT_SECRETS: ${{ inputs.VAULT_SECRETS }}
TELEPORT_APP: ${{ inputs.TELEPORT_APP }}
TELEPORT_PROXY_URL: ${{ inputs.TELEPORT_PROXY_URL }}
ENABLE_DOCKER_BUILD_CACHE: ${{ inputs.ENABLE_DOCKER_BUILD_CACHE }}
secrets:
GH_BOT_DEPLOY_KEY: ${{ secrets.GH_BOT_DEPLOY_KEY }}
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
Expand Down