Skip to content

Latest commit

 

History

History
178 lines (120 loc) · 9.55 KB

README.md

File metadata and controls

178 lines (120 loc) · 9.55 KB
Raw

WP10 Image Factory

The WP10 Image Factory is a self-service system designed to facilitate the creation of VM and container images. This repository serves as the primary resource for deploying the underlying infrastructure through GitHub Actions workflows that automate the deployment and configuration processes.

Additionally, it provides reusable GitHub Actions workflows that users can invoke from their own repositories for image-building purposes.

Table of Contents

Repository Structure

  • ansible/: Contains Ansible configuration and playbooks for setting up the VM.
  • docs/: Documentation files related to the project.
  • images/: Holds the Docker and Packer image configurations.
    • docker/: Custom Docker image for GitHub runners.
    • packer/: Packer template for building VM images.
  • terraform/: Terraform scripts for deploying various Azure components (resource group, vnet, subnet, runner host vm etc.)
    • modules/: Reusable Terraform modules for Azure resources.
    • scripts/: Helper scripts for automation.
  • terraform-init/: Terraform scripts for setting up an Azure Storage Account used as Terraform state storage

Prerequisites

To use this repository, the following prerequisites are needed:

  • An Azure subscription
  • An Azure service principal with credentials (Client ID, Client Secret, and Tenant ID)
  • GitHub Actions secrets and variables configured in the repo

Repository Secrets and variables

The following GitHub secrets need to be configured in the repository to run GitHub Actions workflows:

  • ACR_PASSWORD: Password for Azure Container Registry.
  • ACR_USERNAME: Username for Azure Container Registry.
  • AZURE_CLIENT_ID: Client ID of the Azure service principal.
  • AZURE_CLIENT_SECRET: Client Secret of the Azure service principal.
  • AZURE_SUBSCRIPTION_ID: Subscription ID for the Azure account.
  • AZURE_TENANT_ID: Tenant ID of the Azure service principal.
  • AZURE_VM_SSH_KEY: SSH key for Azure VM access.
  • AZURE_IF_RUNNER_IP: IP address of the image factory runner.
  • JUMPHOST_IP: IP address of the Azure jump host.
  • RUNNER_HOST_IP: IP address for the VM hosting the Image Factory runners in the air-gapped environment.
  • RUNNER_HOST_IP_DEVEL: IP address for the VM hosting the Image Factory runners in the development environment (public subnet).
  • SSH_KEY: SSH key for accessing the runner host VM in the air-gapped environment.
  • SSH_KEY_DEVEL: SSH key for accessing the runner host VM in the development environment.
  • GH_PAT_TOKEN: Personal Access Token for GitHub used for Image Factory runner registration.
  • REGISTRY: Azure Container Registry registry URL.

The following GitHub repository variables need to be configured in the repository to run GitHub Actions workflows:

  • ACR_RUNNER_IMAGE_NAME: Name of the runner image in Azure Container Registry.
  • AZURE_ACG: Azure Compute Gallery name.
  • AZURE_LOCATION: Azure region where resources will be deployed.
  • AZURE_RESOURCE_GROUP: Resource group in Azure for the deployment.
  • GH_OWNER: GitHub owner (user or organization) where Image Factory runners will be connected to.
  • GH_REPO: GitHub repository name where Image Factory runners will be connected to.
  • VM_ENV: Environment setting for the runner host VM. Valid values are "development" and "air-gapped".

Building Platform Images

The following images are used in the main platform deployment:

  • Runner Host Image (virtual machine)
  • Runner (container)

Platform Images are built using GitHub Actions workflows.

Building Runner Host Image

The packer-build-if-runner.yml workflow is used to build the runner host VM image with Packer and push it to the Azure Compute Gallery (ACG).
Packer image and code used in this GitHub action can be found in the images/packer folder.

Building Runner Container Image

The docker-build-if-runner.yml workflow is used to build the runner container image with Docker and push it to the Azure Container Registry (ACR).
Docker image and all resources needed in the Docker build action can be found in the images/docker folder.


Platform Deployment

The platform deployment is managed through a .github/workflows/terraform-deploy.yml GitHub Actions workflow that leverages Terraform to provision and configure the necessary infrastructure on Azure.

Detailed information about the Terraform configurations and scripts can be found in the following locations:

Diagam

Executing the Workflow

To deploy the platform, ensure the necessary GitHub secrets and variables are configured as described in the Prerequisites section. The workflow requires manual execution and user inputs, which can be provided in the GitHub Actions tab.


Platform Configuration

Further platform configuration is done in .github/workflows/ansible-configure-vm.yml GitHub Actions workflow. This workflow uses Ansible to configure the deployed runner host VM.

For more detailed information about the Ansible playbooks and configurations, refer to the following locations:

  • Ansible playbooks and configurations: ansible/

Executing the Workflow

To configure the platform, ensure the necessary GitHub secrets and variables are configured as described in the Prerequisites section. The workflow can be found in the GitHub Actions tab and can be executed manually once infractructure is deployed with terraform.


Image Factory Reusable Workflows

This repository provides two reusable GitHub Actions workflows that customers can call from their repositories to build Docker and Packer images.

Reusable Docker Image Workflow

The reusable-workflow-docker-image.yml workflow is used to build Docker images. Customers can integrate this workflow into their repositories to automate the Docker image-building process.

Reusable Packer Image Workflow

The reusable-workflow-packer-image.yml workflow is used to build Packer images. Customers can use this workflow in their repositories to automate the creation of VM images with Packer.

Important Note:
Packer is currently using the same Azure service principal with permissions across all platform resources. As a result, it provisions all necessary resources for building the VM image autonomously. In the final implementation, we will isolate the build process within a dedicated environment for enhanced security and control.

Example User Repository

An example user repository that demonstrates how to call these reusable workflows can be found here.


GitHub Action Runners

Multi-target architecture

Our infrastructure now supports the production of both arm64 and amd64 container images. This enhancement has been achieved by enabling multi-architecture support on our Image Factory GitHub Actions runners.

The following commands were executed to facilitate this capability:

sudo update-binfmts --enable qemu-aarch64
sudo update-binfmts --enable qemu-arm

By enabling these QEMU interpreters, our CI/CD pipeline can seamlessly handle and build container images for multiple architectures, empowering customers with greater flexibility and compatibility for their deployment needs.

Quality gate and SBOM generation

Quality Gate Checking with Trivy helps us enforce quality gates automatically. This integration reduces the risk of deploying insecure or non-compliant images to production, maintaining high security.

SBOM Generation with Trivy provides detailed visibility into the components, libraries, and dependencies within our Docker images. This transparency allows us to manage software components effectively, quickly identify and mitigate vulnerabilities, and ensure compliance with licensing and regulatory requirements.

SBOM is generated in following formats:

  • CycloneDX
  • SPDX

Each invocation of our reusable workflow generates and uploads the results to the GitHub Actions run. The output is consolidated into a file named image-scan-results.zip.