Skip to content

Commit

Permalink
Allow to use public registries (#24)
Browse files Browse the repository at this point in the history 
## 📖 Description

This PR allows to use public images by removing the need for credentials
and adding the option to resize the VM's disk once it's been pulled.

## 📝 Notes

The setup now includes the installation of `sshpass` as this is the
recommended method from tart's documentation. The ssh server is not
exposed outside of the host machine anyway and this simplifies the
overall configuration.

## 🦀 Dispatch

- `#dispatch/devops`
  • Loading branch information
@randy-sab-roy
randy-sab-roy authored Oct 4, 2024
1 parent 39bc197 commit 4b64856
Show file tree
Hide file tree
Showing 4 changed files with 85 additions and 50 deletions.
4 changes: 2 additions & 2 deletions guest/runner.pkr.hcl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,8 @@ build {
provisioner "shell" {
inline = [
"source ~/.zprofile",
"brew install --cask homebrew/cask-versions/temurin8",
"brew install android-sdk android-ndk",
"brew install --cask temurin",
"brew install android-commandlinetools android-ndk",
"echo \"export ANDROID_HOME=/opt/homebrew/share/android-sdk\" >> ~/.zprofile",
"echo \"export ANDROID_SDK_ROOT=/opt/homebrew/share/android-sdk\" >> ~/.zprofile",
"echo \"export ANDROID_NDK_HOME=/opt/homebrew/share/android-ndk\" >> ~/.zprofile",
Expand Down
7 changes: 4 additions & 3 deletions host/.env.example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@ GITHUB_API_TOKEN=<TOKEN>
GITHUB_REGISTRATION_ENDPOINT=https://api.github.com/orgs/<ORG>/actions/runners/registration-token
GITHUB_REGISTRATION_ENDPOINT=https://api.github.com/repos/<ORG>/<REPO>/actions/runners/registration-token

VM_USERNAME=runner
VM_USERNAME=admin
VM_PASSWORD=admin

RUNNER_LABELS=self-hosted,arm64
RUNNER_URL=https://github.com/<ORG>
RUNNER_URL=https://github.com/<ORG>/<REPO>
RUNNER_NAME=runner

REGISTRY_URL=<REGISTRY_URL>
REGISTRY_USERNAME=user
REGISTRY_PASSWORD=password
REGISTRY_IMAGE_NAME=runner:latest

TRUNCATE_SIZE=200g
20 changes: 3 additions & 17 deletions host/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,35 +39,21 @@ $ ssh-copy-id -i ~/.ssh/id_rsa.pub admin@<HOST_IP>

### Copy Files

Still from your local machine, copy a few files from this repository. The `domain.crt` file is the certificate required to access the VM registry. The `.env` contains secrets for the runner (see `.env.example` for a template). An ssh key is also installed on the host to facilitate the connection to the VM.
Still from your local machine, copy a few files from this repository. The `.env` contains configurations for the runner (see `.env.example` for a template).

```
$ scp domain.crt admin@<HOST_IP>
$ scp launch.sh admin@<HOST_IP>:vm
$ scp com.mirego.ekiden.plist admin@<HOST_IP>:vm
$ scp .env admin@<HOST_IP>:vm
$ scp id_rsa admin@<HOST_IP>:.ssh
$ scp id_rsa.pub admin@<HOST_IP>:.ssh
```

### Install Tools

On the remote machine, install [Homebrew](https://brew.sh), `wget` and [tart](https://github.com/cirruslabs/tart/)
On the remote machine, install [Homebrew](https://brew.sh), `wget`, `sshpass` and [tart](https://github.com/cirruslabs/tart/)

```
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ brew install wget cirruslabs/cli/tart
```

### Install Certificate

Open the certificate in the keychain and make sure it is marked as "Always Trust".
Alternatively, this can also be done from command line:

```
$ sudo security authorizationdb write com.apple.trust-settings.admin allow
$ security unlock-keychain ~/Library/Keychains/login.keychain-db
$ security add-trusted-cert -d -k ~/Library/Keychains/login.keychain domain.crt
$ brew install wget cirruslabs/cli/tart cirruslabs/cli/sshpass
```

### Start the Runner
Expand Down
104 changes: 76 additions & 28 deletions host/launch.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,24 @@
GITHUB_API_TOKEN=
GITHUB_REGISTRATION_ENDPOINT=

VM_USERNAME=runner
VM_USERNAME="admin"
VM_PASSWORD="admin"

RUNNER_LABELS=self-hosted,M1
RUNNER_LABELS="self-hosted,M1"
RUNNER_URL=
RUNNER_NAME=Runner
RUNNER_NAME="Runner"

REGISTRY_URL=
REGISTRY_USERNAME=
REGISTRY_PASSWORD=
REGISTRY_IMAGE_NAME=runner

LOGFILE=runner.log
REGISTRY_IMAGE_NAME="runner"

LOGFILE="runner.log"
SCHEDULE_SHUTDOWN=false

function log_output {
echo "$(date "+%Y/%m/%d %H:%M:%S") $1"
echo "$(date "+%Y/%m/%d %H:%M:%S") [${RUN_ID:-PREPARING}] $1" >>$LOGFILE
if [ -z "$2" ] || [ "$2" = "true" ]; then
echo "$(date "+%Y/%m/%d %H:%M:%S") $1"
echo "$(date "+%Y/%m/%d %H:%M:%S") [${RUN_ID:-PREPARING}] $1" >>$LOGFILE
fi
}

function stream_output {
Expand All @@ -41,9 +41,71 @@ function cleanup {
exit 0
}

function ssh_command() {
local command=$1
local show_output=$2

if [ -z "${show_output}" ] || [ "${show_output}" = "true" ]; then
SSHPASS=$VM_PASSWORD sshpass -e ssh -q -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" "$command" 2>&1 | sed -nru 's/^(.+)$/[GUEST] 📀 \1/p' | stream_output
else
SSHPASS=$VM_PASSWORD sshpass -e ssh -q -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" "$command" >/dev/null
fi
}

function boot_vm {
BASE_IMAGE=$1
INSTANCE_NAME=$2
ENABLE_LOGGING=$3

TART_NO_AUTO_PRUNE="" tart clone "$BASE_IMAGE" "$INSTANCE_NAME"
trap 'log_output "[HOST] 🪓 Killing the VM"; tart delete $INSTANCE_NAME; cleanup' SIGINT SIGTERM

tart run --no-graphics "$INSTANCE_NAME" >/dev/null 2>&1 &

log_output "[HOST] 💤 Waiting for VM to boot" "$ENABLE_LOGGING"
IP_ADDRESS=$(tart ip "$INSTANCE_NAME")
until [[ "$IP_ADDRESS" =~ ^([0-9]+\.){3}[0-9]+$ ]]; do
IP_ADDRESS=$(tart ip "$INSTANCE_NAME")
sleep 1
done

log_output "[HOST] 💤 Waiting for SSH to be available on VM" "$ENABLE_LOGGING"
until [ "$(SSHPASS=$VM_PASSWORD sshpass -e ssh -q -o ConnectTimeout=1 -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" pwd)" ]; do
sleep 1
done
}

function pull_image {
log_output "[HOST] ⬇️ Downloading from remote registry"
TART_REGISTRY_USERNAME=$REGISTRY_USERNAME TART_REGISTRY_PASSWORD=$REGISTRY_PASSWORD tart pull "$REGISTRY_PATH" --concurrency 1
if [ -z "$REGISTRY_USERNAME" ]; then
tart pull "$REGISTRY_PATH" --concurrency 1
else
TART_REGISTRY_USERNAME=$REGISTRY_USERNAME TART_REGISTRY_PASSWORD=$REGISTRY_PASSWORD tart pull "$REGISTRY_PATH" --concurrency 1
fi

# The images from cirruslabs are too small for some builds
# This step allows to resize the disk by truncating the disk file and booting the VM to resize the partition
if [ -n "${TRUNCATE_SIZE}" ]; then
log_output "[HOST] 📊 Resizing the disk to $TRUNCATE_SIZE"
REGISTRY_DISK_PATH="${REGISTRY_PATH//://}"
truncate -s "$TRUNCATE_SIZE" ~/.tart/cache/OCIs/"$REGISTRY_DISK_PATH"/disk.img

log_output "[HOST] 📊 Booting instance"
local INSTANCE_NAME="truncate_instance"
boot_vm "$REGISTRY_PATH" "$INSTANCE_NAME" false

log_output "[HOST] 📊 Repairing disk"
ssh_command "echo y | diskutil repairDisk disk0"

log_output "[HOST] 📊 Resizing partition"
ssh_command "diskutil apfs resizeContainer disk0s2 0"

log_output "[HOST] 📊 Stoping instance"
tart stop $INSTANCE_NAME
rm ~/.tart/cache/OCIs/"$REGISTRY_DISK_PATH"/disk.img
cp -c ~/.tart/vms/"$INSTANCE_NAME"/disk.img ~/.tart/cache/OCIs/"$REGISTRY_DISK_PATH"/
tart delete $INSTANCE_NAME
fi
}

function run_loop {
Expand All @@ -54,27 +116,13 @@ function run_loop {

log_output "[HOST] 💻 Launching macOS VM"
INSTANCE_NAME=runner_"$RUNNER_NAME"_"$RUN_ID"
TART_NO_AUTO_PRUNE="" tart clone "$REGISTRY_PATH" "$INSTANCE_NAME"
trap 'log_output "[HOST] 🪓 Killing the VM"; tart delete $INSTANCE_NAME; cleanup' SIGINT SIGTERM
tart run --no-graphics $INSTANCE_NAME >/dev/null 2>&1 &

log_output "[HOST] 💤 Waiting for VM to boot"
IP_ADDRESS=$(tart ip $INSTANCE_NAME)
until [[ "$IP_ADDRESS" =~ ^([0-9]+\.){3}[0-9]+$ ]]; do
IP_ADDRESS=$(tart ip $INSTANCE_NAME)
sleep 1
done

log_output "[HOST] 💤 Waiting for SSH to be available on VM"
until [ "$(ssh -q -o ConnectTimeout=1 -o StrictHostKeyChecking=no -oBatchMode=yes "$VM_USERNAME@$IP_ADDRESS" pwd)" ]; do
sleep 1
done
boot_vm "$REGISTRY_PATH" "$INSTANCE_NAME"

log_output "[HOST] 🔨 Configuring runner on VM"
ssh -q -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" "./actions-runner/config.sh --url $RUNNER_URL --token $REGISTRATION_TOKEN --ephemeral --name $RUNNER_NAME --labels $RUNNER_LABELS --unattended --replace" >/dev/null
ssh_command "./actions-runner/config.sh --url $RUNNER_URL --token $REGISTRATION_TOKEN --ephemeral --name $RUNNER_NAME --labels $RUNNER_LABELS --unattended --replace" false

log_output "[HOST] 🏃 Starting runner on VM"
ssh -q -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" "source ~/.zprofile && ./actions-runner/run.sh" 2>&1 | sed -nru 's/^(.+)$/[GUEST] 📀 \1/p' | stream_output
ssh_command "source ~/.zprofile && ./actions-runner/run.sh"

log_output "[HOST] ✋ Stop the VM"
tart stop "$INSTANCE_NAME"
Expand Down

0 comments on commit 4b64856

Please sign in to comment.