Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for docker build caching, disabled by default #313

Merged
merged 11 commits into from
Aug 14, 2024
Merged

Add support for docker build caching, disabled by default #313

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e4847a6
Add support for caching, disabled by default
ryanwi Aug 9, 2024
845eb9f
add branch
ryanwi Aug 9, 2024
b5ba9dc
add branch
ryanwi Aug 9, 2024
34cf7e9
Add support for caching, disabled by default
ryanwi Aug 9, 2024
9e26b7e
Add support for caching, disabled by default
ryanwi Aug 9, 2024
b7a9a3c
Add support for caching, disabled by default
ryanwi Aug 9, 2024
49d2f33
Add support for caching, disabled by default
ryanwi Aug 9, 2024
f572f09
Add support for caching, disabled by default
ryanwi Aug 9, 2024
c76acc4
test with no cache
ryanwi Aug 9, 2024
7521945
restore
ryanwi Aug 9, 2024
92e73cd
restore
ryanwi Aug 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 42 additions & 15 deletions .github/actions/docker/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,9 @@ inputs:
default: 'false'
description: 'Generate tag only.'
required: false
Comment on lines 63 to 65
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This input should be removed as well if we're eliminating TAG_ONLY entirely.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i can keep it if we think it is or should be used? would need to understand the use case that led to it being added

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can see, the only use I've seen so far is in the cd-retag workflow, which tags an existing image with another tag and pushes it. Let's keep things as-is for now.

CACHE_FROM:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are not used by any consuming repos and not exposed in the ci workflows so removing them

default: 'type=gha'
description: 'Cache from.'
required: false
CACHE_TO:
default: 'type=gha,mode=max'
description: 'Cache to.'
ENABLE_CACHE:
default: 'false'
description: 'Whether to use cache when building the image'
required: false
outputs:
IMAGE_TAG:
Expand Down Expand Up @@ -126,15 +122,31 @@ runs:
run: echo BUILD_ARGS=${{inputs.BUILD_ARGS}} >> $GITHUB_ENV
shell: bash

- name: Build and export to Docker
- name: Build and export to Docker without cache
uses: docker/build-push-action@v6
if: inputs.TAG_ONLY == 'false'
Copy link
Collaborator Author

@ryanwi ryanwi Aug 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TAG_ONLY is never used as far as I can tell, in any consuming repos, so removing to simplify the flag logic

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TAG_ONLY looks to be used in workflows/cd-retag.yml. We should remove references to TAG_ONLY there as well.

if: inputs.ENABLE_CACHE == 'false'
with:
load: true
tags: |
${{ steps.meta.outputs.tags }}
no-cache: true
file: ${{ inputs.FILE }}
context: ${{ inputs.CONTEXT }}
# cannot use multiple platforms with `load`, build a single arch image for validation purposes in CI
platforms: linux/amd64
build-args: ${{ env.BUILD_ARGS }}
secrets: ${{ env.DOCKER_SECRETS }}

- name: Build and export to Docker with cache
uses: docker/build-push-action@v6
if: inputs.ENABLE_CACHE == 'true'
with:
load: true
tags: |
${{ steps.meta.outputs.tags }}
# cache-from: type=gha
# cache-to: type=gha,mode=max
cache-from: type=gha
cache-to: type=gha,mode=max
no-cache: false
file: ${{ inputs.FILE }}
context: ${{ inputs.CONTEXT }}
# cannot use multiple platforms with `load`, build a single arch image for validation purposes in CI
Expand All @@ -155,16 +167,31 @@ runs:
docker logs test
${{ inputs.CONTAINER_TEST_COMMAND }}

- name: Build and push
if: inputs.PUSH == 'true' # && inputs.TAG_ONY == 'false'
- name: Build and push with cache
if: inputs.PUSH == 'true' && inputs.ENABLE_CACHE == 'true'
uses: docker/build-push-action@v6
with:
push: true
tags: |
${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
no-cache: false
file: ${{ inputs.FILE }}
context: ${{ inputs.CONTEXT }}
platforms: ${{ inputs.PLATFORMS }}
build-args: ${{ env.BUILD_ARGS }}
secrets: ${{ env.DOCKER_SECRETS }}

- name: Build and push without cache
if: inputs.PUSH == 'true' && inputs.ENABLE_CACHE == 'false'
uses: docker/build-push-action@v6
with:
push: true
tags: |
${{ steps.meta.outputs.tags }}
cache-from: ${{ inputs.CACHE_FROM }}
cache-to: ${{ inputs.CACHE_TO }}}
file: ${{ inputs.FILE }}
no-cache: true
context: ${{ inputs.CONTEXT }}
platforms: ${{ inputs.PLATFORMS }}
build-args: ${{ env.BUILD_ARGS }}
Expand Down
16 changes: 5 additions & 11 deletions .github/workflows/ci-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,15 +106,10 @@ on:
type: string
description: ref to pull and build default to github.ref
required: false
CACHE_FROM:
type: string
default: 'type=gha'
description: 'Cache from.'
required: false
CACHE_TO:
type: string
default: 'type=gha,mode=max'
description: 'Cache to.'
ENABLE_DOCKER_BUILD_CACHE:
type: boolean
default: false
description: 'Whether to use cache when building the image'
required: false
## Vault Secrets
VAULT_SECRETS:
Expand Down Expand Up @@ -280,8 +275,7 @@ jobs:
CONTAINER_TEST_RUN_OPTIONS: ${{ inputs.CONTAINER_TEST_RUN_OPTIONS }}
CONTAINER_TEST_COMMAND: ${{ inputs.CONTAINER_TEST_COMMAND }}
OUTPUT_TAG_INDEX: ${{ inputs.OUTPUT_TAG_INDEX }}
CACHE_FROM: ${{ inputs.CACHE_FROM }}
CACHE_TO: ${{ inputs.CACHE_TO }}
ENABLE_CACHE: ${{ inputs.ENABLE_DOCKER_BUILD_CACHE }}
env:
GITHUB_TOKEN: ${{ github.token }}
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,11 @@ on:
default: .
description: Whenever the WF requires a different path than . for the revision file
required: false
ENABLE_DOCKER_BUILD_CACHE:
type: boolean
default: false
description: 'Whether to use cache when building the image'
required: false
## Vault Secrets
VAULT_SECRETS:
type: string
Expand Down Expand Up @@ -269,6 +274,7 @@ jobs:
VAULT_SECRETS: ${{ inputs.VAULT_SECRETS }}
TELEPORT_APP: ${{ inputs.TELEPORT_APP }}
TELEPORT_PROXY_URL: ${{ inputs.TELEPORT_PROXY_URL }}
ENABLE_DOCKER_BUILD_CACHE: ${{ inputs.ENABLE_DOCKER_BUILD_CACHE }}
secrets:
GH_BOT_DEPLOY_KEY: ${{ secrets.GH_BOT_DEPLOY_KEY }}
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
Expand Down