Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration test #80

Closed
wants to merge 30 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
8490ff4
Debug docker
slawosz Sep 20, 2023
20fe9cd
Add generated terraform and Makefile config
slawosz Sep 21, 2023
ab06c7f
Temporairly disable statuscake
slawosz Sep 21, 2023
e68c928
Add simple healthcheck
slawosz Sep 21, 2023
c04810c
Fix docker repo domain
peteryates Sep 22, 2023
e302c99
Change the way to create an extension
slawosz Sep 25, 2023
4f27453
Allow to use any image as database docker image in kubernetes deployment
slawosz Sep 26, 2023
c947fb4
Create staging environment
slawosz Sep 27, 2023
18ffa7a
Add devops related files to gitignore
slawosz Sep 28, 2023
29420cd
Create proper service tag
slawosz Sep 28, 2023
f38b184
Update terraform/domains/infrastructure/config/zones.tfvars.json
slawosz Sep 28, 2023
edf2e7d
Update terraform/domains/infrastructure/config/zones.tfvars.json
slawosz Sep 28, 2023
9b013db
Update terraform/domains/infrastructure/config/zones.tfvars.json
slawosz Sep 28, 2023
5445769
Change domain resource group name format to match one from Makefile
slawosz Sep 29, 2023
48025aa
Change name of the service
slawosz Oct 2, 2023
1fb19e6
Add console for production environment
slawosz Oct 2, 2023
4a38e77
Change terraform version
slawosz Oct 4, 2023
cfc5b23
Small config change
slawosz Oct 5, 2023
8a42db1
Add review config
slawosz Oct 6, 2023
c0feac6
Add review config
slawosz Oct 9, 2023
7a9aeb6
Improve Makefile
slawosz Oct 9, 2023
53fa4e9
Make review apps working
slawosz Oct 12, 2023
c7f7c11
Review app deployment actions
slawosz Oct 13, 2023
6f263fe
Disable brakeman for a moment
slawosz Oct 13, 2023
6af503d
Use official terraform modules
slawosz Oct 13, 2023
1ad1e74
Cleanup docker image lookup
slawosz Oct 13, 2023
5cc9819
Fix path
slawosz Oct 13, 2023
081e0eb
Add environments
slawosz Oct 16, 2023
c832c17
Fix seed step
slawosz Oct 16, 2023
7654595
Testing
slawosz Oct 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/actions/deploy-environment-to-aks/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Deploy environment to AKS
description: Deploys an application environment to AKS

inputs:
environment:
description: The name of the environment
required: true
docker-image:
description: The Docker image to deploy
required: true
azure-credentials:
description: JSON object containing a key for the service principal authorised on the Azure subscription
required: true
pull-request-number:
description: The pull request number which triggered this deploy. If set, this will automatically seed the database.
required: false
current-commit-sha:
description: The commit sha for the current commit
required: true

outputs:
url:
description: The base URL for the deployed environment
value: ${{ steps.apply-terraform.outputs.url }}

runs:
using: composite

steps:
- uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.5.4
terraform_wrapper: false

- uses: DFE-Digital/github-actions/set-arm-environment-variables@master
with:
azure-credentials: ${{ inputs.azure-credentials }}

- name: Apply Terraform
id: apply-terraform
shell: bash
run: |
make ci ${{ inputs.environment }} terraform-apply
cd terraform/application && echo "url=$(terraform output -raw url)" >> $GITHUB_OUTPUT
env:
TF_VAR_azure_sp_credentials_json: ${{ inputs.azure-credentials }}
TF_VAR_statuscake_api_token: ${{ inputs.statuscake-api-token }}
DOCKER_IMAGE: ${{ inputs.docker-image }}
PULL_REQUEST_NUMBER: ${{ inputs.pull-request-number }}

- uses: Azure/login@v1
with:
creds: ${{ inputs.azure-credentials }}

- name: Seed database
if: ${{ inputs.pull-request-number != '' }}
shell: bash
run: |
az aks get-credentials --resource-group s189t01-tsc-ts-rg --name s189t01-tsc-test-aks
kubectl exec -n cpd-development deployment/cpd-tsh-review-${{ inputs.pull-request-number }} -- sh -c "cd /app && /usr/local/bin/bundle exec rails db:seed"

78 changes: 78 additions & 0 deletions .github/workflows/aks_deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: "Deploy"

concurrency:
group: ${{ github.ref }}
cancel-in-progress: true

on:
workflow_dispatch:
inputs:
environment:
description: "Deploy environment"
required: true
default: review_aks
type: environment
options:
- review_aks
push:
branches:
- main

pull_request:
branches:
- main
types:
- labeled
- synchronize
- reopened
- opened

jobs:
docker:
name: Build and push Docker image
runs-on: ubuntu-latest
if: github.actor != 'dependabot[bot]'
outputs:
docker-image: ${{ steps.build-docker-image.outputs.image }}
steps:
- uses: actions/checkout@v3

- uses: DFE-Digital/github-actions/build-docker-image@master
id: build-docker-image
with:
docker-repository: ghcr.io/dfe-digital/teaching-school-hub-finder
github-token: ${{ secrets.GITHUB_TOKEN }}

# brakeman:
# name: Run Brakeman vulnerability scanner
# uses: ./.github/workflows/brakeman.yml
# with:
# ruby-version: "3.2.2"

deploy_review:
name: Deploy review
concurrency: deploy_review_${{ github.event.pull_request.number }}
if: github.actor != 'dependabot[bot]' && github.event_name == 'pull_request'
# needs: [docker, brakeman] TODO: redo brekeman
needs: [docker]
runs-on: ubuntu-latest
environment:
name: review
steps:
- uses: actions/checkout@v3

- uses: ./.github/actions/deploy-environment-to-aks
id: deploy
with:
environment: review
docker-image: ${{ needs.docker.outputs.docker-image }}
azure-credentials: ${{ secrets.AZURE_CREDENTIALS }}
pull-request-number: ${{ github.event.pull_request.number }}
current-commit-sha: ${{ github.event.pull_request.head.sha }}

- name: Post sticky pull request comment
if: github.event_name == 'pull_request'
uses: marocchino/sticky-pull-request-comment@v2
with:
message: |
Review app deployed to ${{ steps.deploy.outputs.url }}
61 changes: 61 additions & 0 deletions .github/workflows/aks_destroy_review.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Delete review

on:
pull_request:
types: [closed]
branches: [main]

jobs:
delete-review-app:
name: Delete Review App ${{ github.event.pull_request.number }}
concurrency: deploy_review_${{ github.event.pull_request.number }}
if: ${{ !contains(github.event.pull_request.labels.*.name, 'dependencies') }}
runs-on: ubuntu-latest
environment: review
steps:
- uses: actions/checkout@v3

- uses: Azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}

- uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.5.4
terraform_wrapper: false

- name: Set environment variables
run: |
state_file_name=terraform-${{ github.event.pull_request.number }}.tfstate

echo "TF_STATE_FILE=$state_file_name" >> $GITHUB_ENV

state_file_status=$(az storage blob list -c cpdecf-tfstate \
--account-name "s189t01cpdecftfstatervsa" \
--prefix $state_file_name --query "[].name" -o tsv)

if [ -n "$state_file_status" ]; then
echo "TF_STATE_EXISTS=true" >> $GITHUB_ENV
fi

- uses: DFE-Digital/github-actions/set-arm-environment-variables@master
if: env.TF_STATE_EXISTS == 'true'
with:
azure-credentials: ${{ secrets.AZURE_CREDENTIALS }}

- name: Destroy Terraform
if: env.TF_STATE_EXISTS == 'true'
id: destroy-terraform
shell: bash
run: make ci review terraform-destroy
env:
TF_VAR_azure_sp_credentials_json: ${{ secrets.AZURE_CREDENTIALS }}
TF_VAR_statuscake_api_token: ${{ secrets.STATUSCAKE_API_TOKEN }}
DOCKER_IMAGE: "ghcr.io/dfe-digital/early-careers-framework:no-tag"
PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }}

- name: Delete Terraform state file
if: env.TF_STATE_EXISTS == 'true'
run: |
az storage blob delete -c cpdecf-tfstate --name ${{ env.TF_STATE_FILE }} \
--account-name "s189t01cpdecftfstatervsa"
73 changes: 37 additions & 36 deletions .github/workflows/specs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -120,40 +120,41 @@ jobs:

- name: Run rubocop
run: bundle exec rubocop --format clang --parallel
brakeman:
name: Run Brakeman

runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: 3.1.2

- name: Set up Node
uses: actions/setup-node@v3
with:
node-version: '16.20.0'

- name: Set up ruby gem cache
uses: actions/cache@v3
with:
path: vendor/bundle
key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
restore-keys: |
${{ runner.os }}-gems-

- name: Install dependencies
run: sudo apt-get install -y libproj-dev proj-bin

- name: Install gems
run: |
bundle config path vendor/bundle
bundle install --jobs 4 --retry 3

- name: Run brakeman
run: bundle exec brakeman
# brakeman:
# name: Run Brakeman
#
# runs-on: ubuntu-latest
#
# steps:
# - name: Checkout code
# uses: actions/checkout@v3
#
# - name: Set up Ruby
# uses: ruby/setup-ruby@v1
# with:
# ruby-version: 3.1.2
#
# - name: Set up Node
# uses: actions/setup-node@v3
# with:
# node-version: '16.20.0'
#
# - name: Set up ruby gem cache
# uses: actions/cache@v3
# with:
# path: vendor/bundle
# key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
# restore-keys: |
# ${{ runner.os }}-gems-
#
# - name: Install dependencies
# run: sudo apt-get install -y libproj-dev proj-bin
#
# - name: Install gems
# run: |
# bundle config path vendor/bundle
# bundle install --jobs 4 --retry 3
#
# - name: Run brakeman
# run: bundle exec brakeman
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,9 @@ tmp/pids/*
!tmp/pids/.keep

/config/master.key

.terraform
terraform/application/vendor
terraform/domains/environment_domains/vendor
terraform.tfstate*
bin/terrafile
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -72,5 +72,5 @@ RUN apk add --no-cache proj-dev
COPY --from=builder /app /app
COPY --from=builder /usr/local/bundle/ /usr/local/bundle/

CMD RAILS_ENV=production bundle exec rails db:migrate && \
RAILS_ENV=production bundle exec rails server -b 0.0.0.0
CMD bundle exec rails db:migrate && \
bundle exec rails server -b 0.0.0.0
Loading