Skip to content

Commit

Permalink
JASPER-173: Develop the mTLS API Gateway components (#68)
Browse files Browse the repository at this point in the history
* - Initial GHA for deploying Lambda functions to ECR
- Test publish-lambdas GHA
- Added lambdas
- Added Dockerfile for creating lambda images
- Added "initial" stack to separate resources that needs to be provisioned first
- Refactor TF code to make it maintainable and reusable
- Created Github Action to deploy lambda functions
- Implemented first set of lambda functions
- Added httpService class to isolate axios calls and certificate processing
- Added mtls_cert in secrets
- Updated openshift job to pull the new secret
- Added eslint support for aws
- Added deploying of initial TF stack
- Cleanup in publish-lambdas
- Used IAM in apigw method authorization for now until authorizer has been setup

---------

Co-authored-by: Ronaldo Macapobre <[email protected]>
  • Loading branch information
ronaldo-macapobre and Ronaldo Macapobre authored Nov 13, 2024
1 parent 1dd9e2e commit 7e8c235
Show file tree
Hide file tree
Showing 88 changed files with 5,905 additions and 768 deletions.
36 changes: 36 additions & 0 deletions .github/workflows/actions/build-lambdas/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Build Lambda
description: Builds all Lambda functions

inputs:
working_directory:
description: The working directory where the code will be built.
required: true
node_version:
description: The node version that will be used.
required: true

runs:
using: composite

steps:
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.node_version }}

- run: npm ci
shell: bash
working-directory: ${{ inputs.working_directory }}

- run: npm run lint
shell: bash
working-directory: ${{ inputs.working_directory }}
continue-on-error: false

- run: npm run build
shell: bash
working-directory: ${{ inputs.working_directory }}

- run: npm run test --if-present
shell: bash
working-directory: ${{ inputs.working_directory }}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Deploy to AWS
description: Deploy image to AWS Instance
name: Deploy App
description: Deploy web or API image to AWS ECS

inputs:
environment:
Expand Down Expand Up @@ -41,11 +41,11 @@ runs:
id: vars
shell: bash
run: |
echo "task_definition_name=${{ inputs.app_name }}-${{ inputs.tier_name }}-task-definition-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "task_definition_name=${{ inputs.app_name }}-${{ inputs.tier_name }}-td-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "container_name=${{ inputs.app_name }}-${{ inputs.tier_name }}-container-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "ecs_cluster_name=${{ inputs.app_name }}-ecs-cluster-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "ecs_service_name=${{ inputs.app_name }}-ecs-${{ inputs.tier_name }}-service-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "full_ecr_repo_url=${{ inputs.aws_account }}.dkr.ecr.${{ inputs.region }}.amazonaws.com/${{ inputs.app_name }}-ecr-repo-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "ecs_cluster_name=${{ inputs.app_name }}-app-cluster-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "ecs_service_name=${{ inputs.app_name }}-${{ inputs.tier_name }}-ecs-service-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "full_ecr_repo_url=${{ inputs.aws_account }}.dkr.ecr.${{ inputs.region }}.amazonaws.com/${{ inputs.app_name }}-app-repo-${{ inputs.environment }}" >> $GITHUB_OUTPUT
- name: Log in to the GHCR
uses: docker/login-action@v2
Expand All @@ -71,7 +71,7 @@ runs:
shell: bash
run: |
IMAGE_TAG=${{ inputs.image_name }}-${{ inputs.short_sha}}
REPOSITORY_NAME=${{ inputs.app_name }}-ecr-repo-${{ inputs.environment }}
REPOSITORY_NAME=${{ inputs.app_name }}-app-repo-${{ inputs.environment }}
IMAGE_EXISTS=$(aws ecr describe-images --repository-name $REPOSITORY_NAME --query "imageDetails[?contains(imageTags, '$IMAGE_TAG')]" --output text)
Expand Down
100 changes: 100 additions & 0 deletions .github/workflows/actions/deploy-lambda/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
name: Deploy Lambda
description: Deploy image to a Lambda function to AWS

inputs:
environment:
description: The environment to which the image will be deployed.
required: true
aws_account:
description: The AWS Account ID.
required: true
region:
description: The AWS Region of the AWS Account.
required: true
app_name:
description: The application name.
required: true
resource:
description: The resource path of the lambda function.
required: true
lambda_name:
description: The lambda function name name.
required: true
aws_role_arn:
description: The AWS Role ARN to assume.
required: true
ghcr_token:
description: The token to use to login to the GHCR.
required: true
github_image_repo:
description: The GCHR repo where images are stored.
required: true
image_name:
description: The name of the image to be deployed.
required: true
short_sha:
description: The short SHA used to tag image in GCHR.
required: true

runs:
using: composite

steps:
- name: Set reusable variables
id: vars
shell: bash
run: |
echo "full_ecr_repo_url=${{ inputs.aws_account }}.dkr.ecr.${{ inputs.region }}.amazonaws.com/${{ inputs.app_name }}-lambda-repo-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "container_name=${{ inputs.app_name }}-${{ inputs.tier_name }}-container-${{ inputs.environment }}" >> $GITHUB_OUTPUT
- name: Log in to the GHCR
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ inputs.ghcr_token }}

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-skip-session-tagging: true
aws-region: ${{ inputs.region }}
role-to-assume: ${{ inputs.aws_role_arn }}
role-duration-seconds: 1800
role-session-name: ci-deployment

- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@v2

- name: Check ECR Image exists
id: ecr-check
shell: bash
run: |
IMAGE_TAG=${{ inputs.resource }}${{ inputs.lambda_name }}-${{ inputs.short_sha }}
REPOSITORY_NAME=${{ inputs.app_name }}-lambda-repo-${{ inputs.environment }}
IMAGE_EXISTS=$(aws ecr describe-images --repository-name $REPOSITORY_NAME --query "imageDetails[?contains(imageTags, '$IMAGE_TAG')]" --output text)
if [ -z "$IMAGE_EXISTS" ]; then
echo "Image with tag $IMAGE_TAG does not exist."
echo "exists=false" >> $GITHUB_OUTPUT
else
echo "Image with tag $IMAGE_TAG already exists."
echo "exists=true" >> $GITHUB_OUTPUT
fi
- name: Push if Docker image does not exist
if: steps.ecr-check.outputs.exists == 'false'
shell: bash
run: |
docker pull ${{ inputs.github_image_repo }}/${{ inputs.image_name }}:${{ inputs.short_sha}}
docker tag ${{ inputs.github_image_repo }}/${{ inputs.image_name }}:${{ inputs.short_sha}} ${{ steps.vars.outputs.full_ecr_repo_url }}:${{ inputs.image_name }}-${{ inputs.short_sha }}
docker push ${{ steps.vars.outputs.full_ecr_repo_url }}:${{ inputs.image_name }}-${{ inputs.short_sha }}
- name: Update Lambda Function
shell: bash
run: |
aws lambda update-function-code \
--function-name ${{ inputs.app_name }}-${{ inputs.lambda_name }}-lambda-${{ inputs.environment }} \
--image-uri ${{ env.full_ecr_repo_url }}:${{ inputs.resource }}.${{ inputs.lambda_name }}-${{ inputs.short_sha }}
5 changes: 0 additions & 5 deletions .github/workflows/aws-template-terraform.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ on:
CHANGE_FOLDER_NAME:
required: true
type: string
TEST_BUCKET_NAME:
required: true
type: string
APPLY_TF_CODE:
required: true
default: false
Expand Down Expand Up @@ -117,7 +114,6 @@ jobs:
id: plan
env:
CONTEXT_FOLDER: ${{ inputs.CONTEXT_FOLDER }}
#TF_VAR_test_s3_bucket_name: ${{ inputs.TEST_BUCKET_NAME }}
run: |
terraform plan -no-color -input=false -var-file=${{ inputs.ENVIRONMENT_NAME }}.tfvars
continue-on-error: true
Expand All @@ -129,7 +125,6 @@ jobs:
if: inputs.APPLY_TF_CODE == true
env:
CONTEXT_FOLDER: ${{ inputs.CONTEXT_FOLDER }}
#TF_VAR_test_s3_bucket_name: ${{ inputs.TEST_BUCKET_NAME }}
run: |
terraform apply --auto-approve -input=false -var-file=${{ inputs.ENVIRONMENT_NAME }}.tfvars
working-directory: ${{ inputs.CONTEXT_FOLDER }}
28 changes: 28 additions & 0 deletions .github/workflows/build-and-test-lambdas.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Build and Test Lambdas

on:
pull_request:
branches:
- master
paths:
- "aws/**"

workflow_dispatch:

env:
WORKING_DIRECTORY: ./aws
NODE_VERSION: 20

jobs:
build-and-test:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Building Lambdas codebase
uses: ./.github/workflows/actions/build-lambdas
with:
working_directory: ${{ env.WORKING_DIRECTORY }}
node_version: ${{ env.NODE_VERSION }}
12 changes: 11 additions & 1 deletion .github/workflows/build-infra.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,22 @@ on:
- prod

jobs:
build-initial:
uses: ./.github/workflows/aws-template-terraform.yml
with:
CONTEXT_FOLDER: "./infrastructure/cloud/environments/initial"
CHANGE_FOLDER_NAME: environments/${{ inputs.environment || 'dev' }}
ENVIRONMENT_NAME: ${{ inputs.environment || 'dev' }}
APPLY_TF_CODE: false
secrets: inherit

build:
needs: [build-initial]
uses: ./.github/workflows/aws-template-terraform.yml
with:
CONTEXT_FOLDER: "./infrastructure/cloud/environments/${{ inputs.environment || 'dev' }}"
CHANGE_FOLDER_NAME: environments/${{ inputs.environment || 'dev' }}
ENVIRONMENT_NAME: ${{ inputs.environment || 'dev' }}
TEST_BUCKET_NAME: jasper-test-bucket
APPLY_TF_CODE: false
secrets: inherit

6 changes: 3 additions & 3 deletions .github/workflows/publish-api.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ jobs:
uses: actions/checkout@v4

- name: Deploy to ${{ env.ENVIRONMENT }}
uses: ./.github/workflows/actions/deploy-to-aws
uses: ./.github/workflows/actions/deploy-app
with:
environment: ${{ env.ENVIRONMENT }}
aws_account: ${{ vars.AWS_ACCOUNT }}
Expand Down Expand Up @@ -120,7 +120,7 @@ jobs:

# Uncomment when infra in AWS in TEST environment has been configured
# - name: Deploy to ${{ env.ENVIRONMENT }}
# uses: ./.github/workflows/actions/deploy-to-aws
# uses: ./.github/workflows/actions/deploy-app
# with:
# environment: ${{ env.ENVIRONMENT }}
# aws_account: ${{ vars.AWS_ACCOUNT }}
Expand Down Expand Up @@ -150,7 +150,7 @@ jobs:

# Uncomment when infra in AWS in PROD environment has been configured
# - name: Deploy to ${{ env.ENVIRONMENT }}
# uses: ./.github/workflows/actions/deploy-to-aws
# uses: ./.github/workflows/actions/deploy-app
# with:
# environment: ${{ env.ENVIRONMENT }}
# aws_account: ${{ vars.AWS_ACCOUNT }}
Expand Down
94 changes: 93 additions & 1 deletion .github/workflows/publish-infra.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,106 @@ on:
- dev
- test
- prod
run_initial:
description: "Run initial Terraform setup?"
required: false
default: "No"
type: choice
options:
- "No"
- "Yes"

jobs:
initial:
if: ${{ inputs.run_initial == 'Yes' }}
uses: ./.github/workflows/aws-template-terraform.yml
with:
CONTEXT_FOLDER: "./infrastructure/cloud/environments/initial"
CHANGE_FOLDER_NAME: environments/${{ inputs.environment }}
ENVIRONMENT_NAME: ${{ inputs.environment || 'dev' }}
APPLY_TF_CODE: true
secrets: inherit

post-initial:
runs-on: ubuntu-latest
needs: initial
if: ${{ inputs.run_initial == 'Yes' }}
environment: ${{ inputs.environment }}
permissions:
id-token: write
packages: write
env:
DUMMY_IMAGE_NAME: dummy-image
GITHUB_IMAGE_REPO: ghcr.io/bcgov/jasper
APP_ECR_REPO_URL: ${{ vars.AWS_ACCOUNT }}.dkr.ecr.${{ vars.AWS_REGION }}.amazonaws.com/${{ vars.APP_NAME }}-app-repo-${{ inputs.environment }}
LAMBDA_ECR_REPO_URL: ${{ vars.AWS_ACCOUNT }}.dkr.ecr.${{ vars.AWS_REGION }}.amazonaws.com/${{ vars.APP_NAME }}-lambda-repo-${{ inputs.environment }}
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Log in to the GHCR
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Setup Image Metadata
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.GITHUB_IMAGE_REPO }}/${{ env.DUMMY_IMAGE_NAME }}
tags: |
type=raw,value=latest
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver: docker

- name: Build ${{ env.DUMMY_IMAGE_NAME }} image
uses: docker/build-push-action@v6
with:
push: true
file: ./docker/dummy-image/Dockerfile.release
context: .
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-skip-session-tagging: true
aws-region: ${{ vars.AWS_REGION }}
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
role-duration-seconds: 1800
role-session-name: ci-deployment

- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@v2

- name: Push dummy images to App ECR
shell: bash
run: |
echo 'Deploying ${{ env.DUMMY_IMAGE_NAME }} to App ECR'
docker tag ${{ env.GITHUB_IMAGE_REPO }}/${{ env.DUMMY_IMAGE_NAME }}:latest ${{ env.APP_ECR_REPO_URL }}:${{ env.DUMMY_IMAGE_NAME }}
docker push ${{ env.APP_ECR_REPO_URL }}:${{ env.DUMMY_IMAGE_NAME }}
- name: Push dummy images to Lambda ECR
shell: bash
run: |
echo 'Deploying ${{ env.env.DUMMY_IMAGE_NAME }} to Lambda ECR'
docker tag ${{ env.GITHUB_IMAGE_REPO }}/${{ env.DUMMY_IMAGE_NAME }}:latest ${{ env.LAMBDA_ECR_REPO_URL }}:${{ env.DUMMY_IMAGE_NAME }}
docker push ${{ env.LAMBDA_ECR_REPO_URL }}:${{ env.DUMMY_IMAGE_NAME }}
deploy:
needs: [initial, post-initial]
if: always()
uses: ./.github/workflows/aws-template-terraform.yml
with:
CONTEXT_FOLDER: "./infrastructure/cloud/environments/${{ inputs.environment || 'dev' }}"
CHANGE_FOLDER_NAME: environments/${{ inputs.environment || 'dev' }}
ENVIRONMENT_NAME: ${{ inputs.environment || 'dev' }}
TEST_BUCKET_NAME: jasper-test-bucket
APPLY_TF_CODE: true
secrets: inherit
Loading

0 comments on commit 7e8c235

Please sign in to comment.