Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker container #1

Merged
merged 33 commits into from
Nov 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
477b82d
add devcontainer with gpu support
eberrigan Nov 21, 2024
5228dc5
add dockerfile
eberrigan Nov 21, 2024
6f5b049
add dockerignore
eberrigan Nov 21, 2024
daa9943
add test data
eberrigan Nov 21, 2024
9cdc69c
add initial readme
eberrigan Nov 21, 2024
9eb1deb
add test workflow for pushes to branches other than main
eberrigan Nov 21, 2024
7a942df
add production workflow
eberrigan Nov 21, 2024
b140d27
set-output is deprecated
eberrigan Nov 21, 2024
c70630d
use env instead of output
eberrigan Nov 21, 2024
d750287
remove colon and change back to output
eberrigan Nov 21, 2024
a4ab742
macos runner does not have docker!
eberrigan Nov 21, 2024
0b5eb2a
use QEMU to emulate different architectures
eberrigan Nov 21, 2024
67ed5f2
sanitize docker tags
eberrigan Nov 21, 2024
58e7704
fix sanitization for github action
eberrigan Nov 21, 2024
3e425e2
remove pruning because it causes error
eberrigan Nov 21, 2024
3337aba
forgot sanitized_platform as an output
eberrigan Nov 21, 2024
1b46eb6
we don't have compatible dependencies for linux/arm64
eberrigan Nov 21, 2024
98ff0d3
needs to be a list
eberrigan Nov 21, 2024
5482a36
only using amd64 for now
eberrigan Nov 21, 2024
34ccb17
skip combine-manifest for now
eberrigan Nov 21, 2024
f47f679
simplify workflow to only run with amd64 but retain ease to extend to…
eberrigan Nov 22, 2024
44855cb
make manifest to group tags
eberrigan Nov 22, 2024
e08037e
update manifest name to be more specific
eberrigan Nov 22, 2024
abdeba1
removing manifest since it only makes sense for multi-arch images
eberrigan Nov 22, 2024
70a9ef3
simplify production workflow for amd64
eberrigan Nov 22, 2024
2042e79
update readme
eberrigan Nov 22, 2024
16803b8
fix readme
eberrigan Nov 22, 2024
4dddd6c
update readme
eberrigan Nov 22, 2024
61aeddf
add VNC connect
eberrigan Nov 22, 2024
a02908c
first docker container has vnc support
eberrigan Nov 24, 2024
b175ba3
make directory for base container
eberrigan Nov 25, 2024
f88c930
fix typo
eberrigan Nov 25, 2024
fd56c1d
update readmes
eberrigan Nov 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/docker-existing-dockerfile
{
"name": "Existing Dockerfile",
"build": {
// Sets the run context to the subfolder.
"context": "../sleap_cuda",
// Update the 'dockerFile' property if you aren't using the standard 'Dockerfile' filename.
"dockerfile": "../sleap_cuda/Dockerfile"
},
"features": {
},
"runArgs": [
"--gpus", "all"
]

// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Uncomment the next line to run commands after the container is created.
// "postCreateCommand": "cat /etc/os-release",

// Configure tool-specific properties.
// "customizations": {},

// Uncomment to connect as an existing user other than the container default. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "devcontainer"
}
68 changes: 68 additions & 0 deletions .github/workflows/sleap_cuda_production.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Build and Push sleap-cuda (Production Workflow)

# Run on push to main branch after testing is complete
on:
push:
branches:
- main
paths:
- sleap_cuda/** # Only run on changes to sleap_cuda

jobs:
build:
runs-on: ubuntu-latest # Only build on Ubuntu for now since Docker is not available on macOS runners
strategy:
matrix:
platform: [linux/amd64] # Only build amd64 for now
max-parallel: 2 # Build both architectures in parallel (if more than one)
outputs:
git_sha: ${{ steps.get_sha.outputs.sha }}
sanitized_platform: ${{ steps.sanitize_platform.outputs.sanitized_platform }}
steps:
- name: Checkout code
# https://github.com/actions/checkout
uses: actions/checkout@v4

- name: Get Git SHA
id: get_sha
run: echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT

- name: Debug Git SHA
run: echo "Git SHA ${{ steps.get_sha.outputs.sha }}"

# Generate a sanitized platform string with slashes replaced by dashes
- name: Sanitize platform name
id: sanitize_platform
run: |
sanitized_platform="${{ matrix.platform }}" # Copy platform value
sanitized_platform="${sanitized_platform/\//-}" # Replace / with -
echo "sanitized_platform=$sanitized_platform" >> $GITHUB_OUTPUT

- name: Set up Docker Buildx
# https://github.com/docker/setup-buildx-action
uses: docker/setup-buildx-action@v3
with:
driver: docker-container # Use a container driver for Buildx (default)

- name: Log in to Docker Hub
# https://github.com/docker/login-action
uses: docker/login-action@v3
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build and push Docker image
# https://github.com/docker/build-push-action
uses: docker/build-push-action@v6
with:
context: ./docker # Build context
file: ./docker/Dockerfile # Path to Dockerfile
platforms: ${{ matrix.platform }}
push: true # Push the image to Docker Hub
# Tags for the production images, including the "latest" tag
tags: |
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:latest
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:${{ steps.sanitize_platform.outputs.sanitized_platform }}
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:${{ steps.sanitize_platform.outputs.sanitized_platform }}-nvidia-cuda-11.3.1-cudnn8-runtime-ubuntu20.04
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:${{ steps.sanitize_platform.outputs.sanitized_platform }}-sleap-1.3.4
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:${{ steps.sanitize_platform.outputs.sanitized_platform }}-${{ steps.get_sha.outputs.sha }}
67 changes: 67 additions & 0 deletions .github/workflows/sleap_cuda_test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: Build and Push sleap-cuda (Test Workflow)

# Run on push to branches other than main for sleap_cuda
on:
push:
branches-ignore:
- main
paths:
- sleap_cuda/** # Only run on changes to sleap_cuda

jobs:
build:
runs-on: ubuntu-latest # Only build on Ubuntu for now since Docker is not available on macOS runners
strategy:
matrix:
platform: [linux/amd64] # Only build amd64 for now
max-parallel: 2 # Build both architectures in parallel (if more than one)
outputs:
git_sha: ${{ steps.get_sha.outputs.sha }}
sanitized_platform: ${{ steps.sanitize_platform.outputs.sanitized_platform }}
steps:
- name: Checkout code
# https://github.com/actions/checkout
uses: actions/checkout@v4

- name: Get Git SHA
id: get_sha
run: echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT

- name: Debug Git SHA
run: echo "Git SHA ${{ steps.get_sha.outputs.sha }}"

# Generate a sanitized platform string with slashes replaced by dashes
- name: Sanitize platform name
id: sanitize_platform
run: |
sanitized_platform="${{ matrix.platform }}" # Copy platform value
sanitized_platform="${sanitized_platform/\//-}" # Replace / with -
echo "sanitized_platform=$sanitized_platform" >> $GITHUB_OUTPUT

- name: Set up Docker Buildx
# https://github.com/docker/setup-buildx-action
uses: docker/setup-buildx-action@v3
with:
driver: docker-container # Use a container driver for Buildx (default)

- name: Log in to Docker Hub
# https://github.com/docker/login-action
uses: docker/login-action@v3
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build and push Docker image
# https://github.com/docker/build-push-action
uses: docker/build-push-action@v6
with:
context: ./sleap_cuda # Build context wrt the root of the repository
file: ./sleap_cuda/Dockerfile # Path to Dockerfile wrt the root of the repository
platforms: ${{ matrix.platform }}
push: true # Push the image to Docker Hub
# Tags all include "-test" to differentiate from production images
tags: |
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:${{ steps.sanitize_platform.outputs.sanitized_platform }}-test
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:${{ steps.sanitize_platform.outputs.sanitized_platform }}-nvidia-cuda-11.3.1-cudnn8-runtime-ubuntu20.04-test
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:${{ steps.sanitize_platform.outputs.sanitized_platform }}-sleap-1.3.4-test
${{ vars.DOCKERHUB_USERNAME }}/sleap-cuda:${{ steps.sanitize_platform.outputs.sanitized_platform }}-${{ steps.get_sha.outputs.sha }}-test
68 changes: 68 additions & 0 deletions .github/workflows/sleap_vnc_connect_production.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Build and Push sleap-vnc-connect (Production Workflow)

# Run on push to main branch after testing is complete
on:
push:
branches:
- main
paths:
- sleap_vnc_connect/** # Only run on changes to sleap_vnc_connect

jobs:
build:
runs-on: ubuntu-latest # Only build on Ubuntu for now since Docker is not available on macOS runners
strategy:
matrix:
platform: [linux/amd64] # Only build amd64 for now
max-parallel: 2 # Build both architectures in parallel (if more than one)
outputs:
git_sha: ${{ steps.get_sha.outputs.sha }}
sanitized_platform: ${{ steps.sanitize_platform.outputs.sanitized_platform }}
steps:
- name: Checkout code
# https://github.com/actions/checkout
uses: actions/checkout@v4

- name: Get Git SHA
id: get_sha
run: echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT

- name: Debug Git SHA
run: echo "Git SHA ${{ steps.get_sha.outputs.sha }}"

# Generate a sanitized platform string with slashes replaced by dashes
- name: Sanitize platform name
id: sanitize_platform
run: |
sanitized_platform="${{ matrix.platform }}" # Copy platform value
sanitized_platform="${sanitized_platform/\//-}" # Replace / with -
echo "sanitized_platform=$sanitized_platform" >> $GITHUB_OUTPUT

- name: Set up Docker Buildx
# https://github.com/docker/setup-buildx-action
uses: docker/setup-buildx-action@v3
with:
driver: docker-container # Use a container driver for Buildx (default)

- name: Log in to Docker Hub
# https://github.com/docker/login-action
uses: docker/login-action@v3
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build and push Docker image
# https://github.com/docker/build-push-action
uses: docker/build-push-action@v6
with:
context: ./docker # Build context
file: ./docker/Dockerfile # Path to Dockerfile
platforms: ${{ matrix.platform }}
push: true # Push the image to Docker Hub
# Tags for the production images, including the "latest" tag
tags: |
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:latest
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:${{ steps.sanitize_platform.outputs.sanitized_platform }}
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:${{ steps.sanitize_platform.outputs.sanitized_platform }}-nvidia-cuda-11.3.1-cudnn8-runtime-ubuntu20.04
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:${{ steps.sanitize_platform.outputs.sanitized_platform }}-sleap-1.3.4
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:${{ steps.sanitize_platform.outputs.sanitized_platform }}-${{ steps.get_sha.outputs.sha }}
67 changes: 67 additions & 0 deletions .github/workflows/sleap_vnc_connect_test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: Build and Push sleap-vnc-connect (Test Workflow)

# Run on push to branches other than main for sleap_vnc_connect
on:
push:
branches-ignore:
- main
paths:
- sleap_vnc_connect/** # Only run on changes to sleap_vnc_connect

jobs:
build:
runs-on: ubuntu-latest # Only build on Ubuntu for now since Docker is not available on macOS runners
strategy:
matrix:
platform: [linux/amd64] # Only build amd64 for now
max-parallel: 2 # Build both architectures in parallel (if more than one)
outputs:
git_sha: ${{ steps.get_sha.outputs.sha }}
sanitized_platform: ${{ steps.sanitize_platform.outputs.sanitized_platform }}
steps:
- name: Checkout code
# https://github.com/actions/checkout
uses: actions/checkout@v4

- name: Get Git SHA
id: get_sha
run: echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT

- name: Debug Git SHA
run: echo "Git SHA ${{ steps.get_sha.outputs.sha }}"

# Generate a sanitized platform string with slashes replaced by dashes
- name: Sanitize platform name
id: sanitize_platform
run: |
sanitized_platform="${{ matrix.platform }}" # Copy platform value
sanitized_platform="${sanitized_platform/\//-}" # Replace / with -
echo "sanitized_platform=$sanitized_platform" >> $GITHUB_OUTPUT

- name: Set up Docker Buildx
# https://github.com/docker/setup-buildx-action
uses: docker/setup-buildx-action@v3
with:
driver: docker-container # Use a container driver for Buildx (default)

- name: Log in to Docker Hub
# https://github.com/docker/login-action
uses: docker/login-action@v3
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build and push Docker image
# https://github.com/docker/build-push-action
uses: docker/build-push-action@v6
with:
context: ./sleap_vnc_connect # Build context wrt the root of the repository
file: ./sleap_vnc_connect/Dockerfile # Path to Dockerfile wrt the root of the repository
platforms: ${{ matrix.platform }}
push: true # Push the image to Docker Hub
# Tags all include "-test" to differentiate from production images
tags: |
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:${{ steps.sanitize_platform.outputs.sanitized_platform }}-test
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:${{ steps.sanitize_platform.outputs.sanitized_platform }}-nvidia-cuda-11.3.1-cudnn8-runtime-ubuntu20.04-test
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:${{ steps.sanitize_platform.outputs.sanitized_platform }}-sleap-1.3.4-test
${{ vars.DOCKERHUB_USERNAME }}/sleap-vnc-connect:${{ steps.sanitize_platform.outputs.sanitized_platform }}-${{ steps.get_sha.outputs.sha }}-test
Loading