5. Continuous Integration and Deployment (CI CD) Workflow

Overview

This GitHub Actions workflow automates the build, test, and deployment processes of the Java project whenever code is pushed or a pull request is created on the dev, staging, or main branches. This setup ensures that changes are tested in development and staging environments before being promoted to production, enhancing the reliability and stability of the application.

It handles four workflows:

1. CI (Continuous Integration): Builds, tests, and archives artifacts for your Java project.
2. Dev-CD: Deploys the project to a development environment when the CI workflow succeeds on the dev branch.
3. Staging-CD: Deploys the project to a staging environment when the CI workflow succeeds on the staging branch.
4. Prod-CD: Deploys the project to a production environment when the CI workflow succeeds on the main

Set Up CI/CD with GitHub Actions

• In the root of your project folder, create a .github directory and a workflows subdirectory. The folder structure should be .github/workflows/.
• Inside the workflows folder, create the following YAML files for different branches and environments: CI.yml, dev-cd.yml, staging-cd.yml, and prod-cd.yml.

CI Workflow

Workflow Overview

This workflow automates continuous integration (CI) by running a job whenever there is a push or pull request to the dev, staging, or main branches.

Paste the below in your CI.yml file:

name: CI

on:
  push:
    branches: [dev, staging, main]
  pull_request:
    branches: [dev, staging, main]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    services:
        postgres:  
          image: postgres:latest
          env:
            POSTGRES_USER: ${{ secrets.POSTGRES_USER }}
            POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
            POSTGRES_DB: ${{ secrets.POSTGRES_DB }}
          ports:
            - 5432:5432
          options: >-
            --health-cmd pg_isready
            --health-interval 10s
            --health-timeout 5s
            --health-retries 5
            
    env:
      connection_string: postgresql://localhost:${{ secrets.POSTGRES_PORT }}/${{ secrets.POSTGRES_DB }}
      database_username: ${{ secrets.POSTGRES_USER }}
      database_password: ${{ secrets.POSTGRES_PASSWORD }}

    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Set up JDK 17
        uses: actions/setup-java@v4
        with:
          java-version: 17
          distribution: temurin
          cache: maven

      - name: Download Dependencies
        run: mvn dependency:resolve

      - name: Build the application
        run: mvn clean install

      - name: List target directory contents
        run: ls -l target

      - name: Start Application
        run: nohup java -jar target/hng-java-boilerplate-0.0.1-SNAPSHOT.jar &

      - name: Wait for Application to be Ready
        run: sleep 30

      - name: Unit and Integration Testing
        run: mvn -B test

      - name: Archive Test Result
        uses: actions/upload-artifact@v3
        if: always()
        with: 
          name: test-results
          path: target/*.jar

Breakdown

1. Trigger Conditions:

• Push: Triggers when code is pushed to the dev, staging, or main branches.
• Pull Request: Triggers when a pull request is created targeting these branches.

2. Job: build-and-test

• Environment: Runs on the latest Ubuntu environment provided by GitHub Actions.
• Services:
  • PostgreSQL: Sets up a PostgreSQL service with the postgres:latest Docker image.
  • Environment Variables: Configures PostgreSQL with the credentials stored as secrets in GitHub
  • Health Checks: Ensures PostgreSQL is ready before running further steps.
• Steps:
  • Checkout Repository: Uses the actions/checkout@v3 action to pull the code from the repository.
  • Set Up JDK 17: Uses the actions/setup-java@v4 action to install Java Development Kit 17 and cache Maven dependencies.
  • Download Dependencies: Resolves Maven project dependencies.
  • Build Application: Compiles and packages the application using Maven.
  • List target directory contents: Verifies the presence of the built JAR file.
  • Start the application: The application is started in the background.
  • Wait for the application to be ready: Ensures the application has time to start up.
  • Unit and Integration Testing: Executes unit and integration tests in the src/test path to validate the code.
  • Archive Test Results: Uploads test results as artifacts for later inspection, regardless of test outcomes.

Dev-CD Workflow

Workflow Overview

The Dev-Deployment workflow automates the deployment of the application to the development environment after a successful CI run on the dev branch.

Paste the below in your dev-cd.yml file:

name: Dev-Deployment

on:
  workflow_run:
    workflows: [CI]
    types:
      - completed

jobs:
  on-success:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion=='success' && github.event.workflow_run.head_branch == 'dev' }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Add SSH Key to Known Hosts
        run: |
          mkdir -p ~/.ssh
          ssh-keyscan $SSH_HOST >> ~/.ssh/known_hosts
        env:
          SSH_HOST: ${{ secrets.SSH_HOST }}

      - name: SSH into Server and Execute Command
        run: |
          sshpass -p $SSH_PASSWORD ssh -o StrictHostKeyChecking=no $SSH_USERNAME@$SSH_HOST "\
          cd ~/hng_boilerplate_java_web && git checkout dev && git pull && chmod +x deploy.sh && ./deploy.sh && \
          exit"
        env:
          SSH_HOST: ${{ secrets.SSH_HOST }}
          SSH_USERNAME: ${{ secrets.DEV_SSH_USERNAME }}
          SSH_PASSWORD: ${{ secrets.DEV_SSH_PASSWORD }}
  
  on-failure:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion=='failure' }}
    steps:
      - run: echo "Triggering the Workflow Failed"

Breakdown:

1. Trigger Conditions:

• Workflow Run: Triggers upon the completion of the CI workflow.

2. Jobs:

• on-success:

  • Environment: Runs on the latest Ubuntu environment.
  • Conditions: Executes if the CI workflow completes successfully and the branch is dev.
  • Steps:
    • Checkout Code: Uses actions/checkout@v3 to retrieve the code.
    • Add SSH Key to Known Hosts: Adds the SSH server’s key to the known hosts file for secure SSH connections.
    • SSH into Server and Execute Command: Connects to the dev environment via SSH and runs deployment commands (checkout branch, pull updates, make deploy script executable, and execute it).

• on-failure:

  • Environment: Runs on Ubuntu if the CI workflow fails.
  • Steps: Logs a failure message.

Note: Ensure your SSH_HOST, DEV_SSH_USERNAME, and DEV_SSH_PASSWORD secrets are configured in GitHub.

Bash Scripts

The script that will be executed in the SSH into Server and Execute Command step is the deploy.sh script. It will be created in the server to copy the application.properties file and flyway.conf file into the project folder. It also resolves and installs the dependencies for the application.

Create a deploy.sh file in the root of your server and paste the below content:

#!/bin/bash
cd ~
./stop-app.sh
cp application.properties ~/hng_boilerplate_java_web/src/main/resources/
cp flyway.conf ~/hng_boilerplate_java_web/
cd ~/hng_boilerplate_java_web/
mvn dependency:resolve
./mvnw clean install

The deploy.sh script calls a stop-app.sh script. The stop-app.sh script is stops any running Java application process so that the updated application folder can be deployed and started in the server. This helps to resolve any port conflicts.

Create the stop-app.sh file in the root of the server:

#!/bin/bash

# Port to check
PORT=8080

# Find PID using lsof
PID=$(lsof -t -i:$PORT)

# Check if the PID is found
if [ -z "$PID" ]; then
  echo "No process found on port $PORT."
else
  echo "Killing Java application with PID: $PID"
  kill -9 $PID
  echo "Java application stopped."
fi

Staging-CD Workflow

Workflow Overview

The Staging-Deployment workflow is triggered when the CI workflow completes successfully on the staging branch. It deploys the latest successful build to the staging environment.

Paste the below in your staging-cd.yml file:

name: Staging-Deployment

on:
  workflow_run:
    workflows: [CI]
    types:
      - completed

jobs:
  on-success:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion=='success' && github.event.workflow_run.head_branch == 'staging' }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Add SSH Key to Known Hosts
        run: |
          mkdir -p ~/.ssh
          ssh-keyscan $SSH_HOST >> ~/.ssh/known_hosts
        env:
          SSH_HOST: ${{ secrets.SSH_HOST }}

      - name: SSH into Server and Execute Command
        run: |
          sshpass -p $SSH_PASSWORD ssh -o StrictHostKeyChecking=no $SSH_USERNAME@$SSH_HOST "\
          cd ~/hng_boilerplate_java_web && git fetch origin staging && git stash && git checkout staging && git pull && chmod +x deploy.sh && ./deploy.sh && \
          exit"
        env:
          SSH_HOST: ${{ secrets.SSH_HOST }}
          SSH_USERNAME: ${{ secrets.STAGING_SSH_USERNAME }}
          SSH_PASSWORD: ${{ secrets.STAGING_SSH_PASSWORD }}

  on-failure:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion=='failure' }}
    steps:
      - run: echo "Triggering the Workflow Failed"

Breakdown:

1. Trigger Conditions:

• Workflow Run: Triggers upon the completion of the CI workflow.

2. Jobs:

• on-success:

  • Environment: Runs on the latest Ubuntu environment.
  • Conditions: Executes if the CI workflow completes successfully and the branch is staging.
  • Steps:
    • Checkout Code: Uses actions/checkout@v3 to retrieve the code.
    • Add SSH Key to Known Hosts: Adds the SSH server’s key to the known hosts file for secure SSH connections.
    • SSH into Server and Execute Command: Connects to the staging environment via SSH and runs deployment commands (checkout branch, pull updates, make deploy script executable, and execute it).

• on-failure:

  • Environment: Runs on Ubuntu if the CI workflow fails.
  • Steps: Logs a failure message.

Note: Ensure your SSH_HOST, STAGING_SSH_USERNAME, and STAGING_SSH_PASSWORD secrets are configured in GitHub.

Prod-CD Workflow

Workflow Overview

The Prod-Deployment workflow handles the deployment to the production environment after a successful CI run on the main branch.

Paste the below in your prod-cd.yml file:

name: Prod-Deployment

on:
  workflow_run:
    workflows: [CI]
    types:
      - completed

jobs:
  on-success:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion=='success' && github.event.workflow_run.head_branch == 'main' }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Add SSH Key to Known Hosts
        run: |
          mkdir -p ~/.ssh
          ssh-keyscan $SSH_HOST >> ~/.ssh/known_hosts
        env:
          SSH_HOST: ${{ secrets.SSH_HOST }}

      - name: SSH into Server and Execute Command
        run: |
          sshpass -p $SSH_PASSWORD ssh -o StrictHostKeyChecking=no $SSH_USERNAME@$SSH_HOST "\
          cd ~/hng_boilerplate_java_web && git fetch origin main && git stash && git checkout main && git pull && chmod +x deploy.sh && ./deploy.sh && \
          exit"
        env:
          SSH_HOST: ${{ secrets.SSH_HOST }}
          SSH_USERNAME: ${{ secrets.PROD_SSH_USERNAME }}
          SSH_PASSWORD: ${{ secrets.PROD_SSH_PASSWORD }}

  on-failure:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion=='failure' }}
    steps:
      - run: echo "Triggering the Workflow Failed"

Breakdown:

1. Trigger Conditions:

• Workflow Run: Triggers upon the completion of the CI workflow.

2. Jobs:

• on-success:

  • Environment: Runs on the latest Ubuntu environment.
  • Conditions: Executes if the CI workflow completes successfully and the branch is main.
  • Steps:
    • Checkout Code: Uses actions/checkout@v3 to retrieve the code.
    • Add SSH Key to Known Hosts: Adds the SSH server’s key to the known hosts file for secure SSH connections.
    • SSH into Server and Execute Command: Connects to the production environment via SSH and runs deployment commands (checkout branch, pull updates, make deploy script executable, and execute it).

• on-failure:

  • Environment: Runs on Ubuntu if the CI workflow fails.
  • Steps: Logs a failure message.

Note: Ensure your SSH_HOST, PROD_SSH_USERNAME, and PROD_SSH_PASSWORD secrets are uploaded to GitHub.

Running the Pipeline

The CI pipeline is set to run automatically once a push is made to the repository. After the pushed changes have been merged to the respective branch (e.g dev), the workflow for that branch will automatically be triggered.