Skip to content

Commit

Permalink
Optimizing GH actions, to use less minutes. (WIP)
Browse files Browse the repository at this point in the history
We can use `.lagoon.yml` to set up a GH deployment,
that we can use to listen for in GH actions.
This is an alternative to us spending a lot of GH minutes
just waiting for the site to become available.

As the `@todo`'s say, this needs some work, as I'm unsure
how to use secrets and get around `set -e` as part of
the deployment.
**I'm looking for input for this :)**

Ontop of that, also setting up `concurrency` rules to
`ci-tests`, so if a second push is made, we cancel the
old and unrelated workflow.
  • Loading branch information
rasben committed Oct 1, 2024
1 parent 91252b1 commit ec84997
Show file tree
Hide file tree
Showing 5 changed files with 76 additions and 82 deletions.
6 changes: 6 additions & 0 deletions .github/workflows/ci-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ env:
PHP_VERSION: 8.1
COMPOSER_VERSION: v2

# Detect if this action is already running, and cancel it.
# This most likely happened because a second push has been made to a branch.
concurrency:
group: ${{ github.repository_id }}-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
ValidateComposer:
name: Validate Composer
Expand Down
16 changes: 16 additions & 0 deletions .github/workflows/lagoon-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
name: "Tests of deployment"

on:
deployment_status

# @todo - this action does nothing useful right now, but it is to show that
# the site URL is available as part of deployment_status.
jobs:
tests:
if: github.event.deployment_status.state == 'success'
runs-on: ubuntu-latest
steps:
- run: echo "$URL"
env:
URL: ${{ github.event.deployment_status.target_url }}
82 changes: 6 additions & 76 deletions .github/workflows/lagoon.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,5 @@
on:
pull_request:
# We have two groups of jobs in this workflow that reacts on actions:
#
# 1. We update the status of a Github Deployment on:
# - opened
# - synchronize
# - reopened
# - closed
#
# 2. We forward all events to lagoon via InformLagoon
types: [ opened, synchronize, reopened, closed, edited ]
name: Lagoon integration

Expand All @@ -31,62 +22,6 @@ jobs:
/^.{1,100}$/
errorMessage: 'Branch name too long. This cannot be deployed to Lagoon.'

CheckEnvironment:
name: Check environment
runs-on: ubuntu-latest
if: ${{ github.event.action == 'opened' || github.event.action == 'reopened' || github.event.action == 'synchronize' }}
needs: [BranchNameLength]
permissions:
# Give the default GITHUB_TOKEN permission to create and update deployments
deployments: write
steps:
- name: Generate environment data
id: environment
run: |
echo ::set-output name=id::pr-${{github.event.number}}
echo ::set-output name=url::'https://varnish.pr-${{github.event.number}}.${{ env.LAGOON_PROJECT }}.${{ env.LAGOON_HOST }}/'
echo ::set-output name=logs::'https://ui.lagoon.${{ env.LAGOON_HOST }}/projects/${{ env.LAGOON_PROJECT }}/${{ env.LAGOON_PROJECT }}-pr-${{github.event.number}}/deployments'
- name: Start deployment
uses: bobheadxi/[email protected]
id: deployment
with:
step: start
token: ${{ secrets.GITHUB_TOKEN }}
env: ${{ steps.environment.outputs.id }}
ref: ${{ github.head_ref }}
logs: ${{ steps.environment.outputs.logs }}
debug: ${{ runner.debug && 'true' || 'false' }}
- name: Generate wait-on config
# Retrieval of Let's Encrypt certificate sometimes fail in Lagoon.
# In this case a self-signed certificate will be used. Allow this.
run: |
echo "{\"strictSSL\": false}" > $RUNNER_TEMP/wait-on.config.json
- name: Wait for environment to become available
uses: iFaxity/[email protected]
with:
resource: ${{ steps.environment.outputs.url }}
# Time in ms. Wait for 20 mins for deployment to complete. We have
# seen deployments taking up to 17 mins.
timeout: 1200000
# Poll every 10 seconds. For whatever reason Lagoon environments may
# return 200 during the deployment process even though the deployment
# is not complete. Reduce polling interval to the risk of this
# happening.
interval: 10000
config: ${{ runner.temp }}/wait-on.config.json
- name: Finish deployment
if: always()
uses: bobheadxi/[email protected]
with:
step: finish
token: ${{ secrets.GITHUB_TOKEN }}
status: ${{ job.status }}
deployment_id: ${{ steps.deployment.outputs.deployment_id }}
env: ${{ steps.deployment.outputs.env }}
env_url: ${{ steps.environment.outputs.url }}
logs: ${{ steps.environment.outputs.logs }}
debug: ${{ runner.debug && 'true' || 'false' }}

CloseEnvironment:
name: Close environment
runs-on: ubuntu-latest
Expand All @@ -112,19 +47,14 @@ jobs:
# legitimate contributions.
# The integration is controlled by creating synthetic events related to select
# pull-request events, and send them to Lagoon.
#
# The job expects the following secrets:
# LAGOON_WEBHOOK_URL: The url events are to be delivered to
# LAGOON_WEBHOOK_SECRET: Shared lagoon webhook secret
#
InformLagoon:
name: Send synthetic event to Lagoon
runs-on: ubuntu-latest
needs: [BranchNameLength]
steps:
- name: Send pull request event
uses: distributhor/workflow-webhook@v3
env:
webhook_url: ${{ secrets.LAGOON_WEBHOOK_URL }}
webhook_secret: ${{ secrets.LAGOON_WEBHOOK_SECRET }}
webhook_type: 'json-extended'
- name: Send pull request event
uses: distributhor/workflow-webhook@v3
env:
webhook_url: ${{ secrets.LAGOON_WEBHOOK_URL }}
webhook_secret: ${{ secrets.LAGOON_WEBHOOK_SECRET }}
webhook_type: 'json-extended'
40 changes: 34 additions & 6 deletions .lagoon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,26 @@ project: dpl-cms-core

ssh: 20.238.147.183:22
api: https://api.lagoon.dplplat01.dpl.reload.dk/graphql
environment_variables:
git_sha: 'true'

tasks:
post-rollout:
- run:
name: Create new GH deployment
command: |
export GH_DEPLOYMENT_ID = $(curl -L \
-X POST \
-H "Authorization: Bearer $GH_DEPLOYMENT_TOKEN" \
https://api.github.com/repos/danskernesdigitalebibliotek/dpl-cms/deployments \
-d '{"ref":"$LAGOON_GIT_SHA","description":"Triggered by Lagoon"}');
service: cli
shell: bash
- run:
name: If drupal is not installed
command: |
set -e
source dev-scripts/lagoon-error-handling.sh
if tables=$(drush sqlq "show tables like 'node';") && [ -z "$tables" ]; then
# Install and set the admin password to a Lagoon variable if it exists.
if [[ -n $PR_DRUPAL_PWD ]]; then
Expand All @@ -29,7 +42,8 @@ tasks:
- run:
name: drush deploy
command: |
set -e
source dev-scripts/lagoon-error-handling.sh
if [[ -f config/sync/system.site.yml ]]; then
echo "Config detected, doing a drush deploy"
drush deploy
Expand All @@ -49,7 +63,8 @@ tasks:
# it will be gone.
name: Create module upload directory in public files
command: |
set -e
source dev-scripts/lagoon-error-handling.sh
if [[ ! -d "web/sites/default/files/modules_local" ]]; then
echo "Creating directory for module uploads"
mkdir web/sites/default/files/modules_local
Expand All @@ -58,14 +73,16 @@ tasks:
- run:
name: Import translations
command: |
set -e;
source dev-scripts/lagoon-error-handling.sh
drush locale-check
drush locale-update
service: cli
- run:
name: Create test users
command: |
set -e
source dev-scripts/lagoon-error-handling.sh
# Only create test users if they do not exist already.
if editor_user=$(drush sqlq 'select * from users_field_data where name = "editor"') && [ -z "$editor_user" ]; then
drush user:create editor --password="$PR_DRUPAL_PWD"
Expand All @@ -89,10 +106,21 @@ tasks:
- run:
name: Enable example content
command: |
set -e
source dev-scripts/lagoon-error-handling.sh
drush en -y dpl_example_content
service: cli
- run:
name: Setting Deployment status success
command: |
DEPLOYMENT_STATUS="success";
DRUPAL_URL=$(drush browse);
export GH_DEPLOYMENT_ID = $(curl -L \
-X POST \
-H "Authorization: Bearer $GH_DEPLOYMENT_TOKEN" \
https://api.github.com/repos/danskernesdigitalebibliotek/dpl-cms/deployments/$GH_DEPLOYMENT_ID/statuses \
-d '{"environment":"$LAGOON_GIT_SHA","state":"$DEPLOYMENT_STATUS", "target_url":"$DRUPAL_URL"}');
environments:
main:
cronjobs:
Expand Down
14 changes: 14 additions & 0 deletions dev-scripts/lagoon-error-handling.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/bash

error_handler() {
curl -L \
-X POST \
-H "Authorization: Bearer $GH_DEPLOYMENT_TOKEN" \
https://api.github.com/repos/danskernesdigitalebibliotek/dpl-cms/deployments/$GH_DEPLOYMENT_ID/statuses \
-d "{\"environment\":\"$LAGOON_GIT_SHA\",\"state\":\"failure\"}"

exit "$1"
}

# Set up trap for ERR signal
trap 'error_handler $?' ERR

0 comments on commit ec84997

Please sign in to comment.