Skip to content

Latest commit

 

History

History
206 lines (162 loc) · 11.7 KB

README.md

File metadata and controls

206 lines (162 loc) · 11.7 KB
Raw

CI/CD bootstrap

The primary purpose of this stage is to set up your CI/CD project structure automatically, with most of the necessary configuration to run the pipelines out of the box.

How to run this stage

This stage is meant to be executed after the bootstrap stage has run, as it leverages the automation service account and bucket created there. The entire stage is optional, you may also choose to create your repositories manually.

Providers configuration

The default way of making sure you have the right permissions, is to use the identity of the service account pre-created for this stage during bootstrap, and that you are a member of the group that can impersonate it via provider-level configuration (gcp-devops or organization-admins).

To simplify setup, the previous stage pre-configures a valid providers file in its output, and optionally writes it to a local file if the outputs_location variable is set to a valid path.

If you have set a valid value for outputs_location in the bootstrap stage (see the bootstrap stage README for more details), simply link the relevant providers.tf file from this stage's folder in the path you specified:

# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/providers/00-cicd-providers.tf .

If you have not configured outputs_location in bootstrap, you can derive the providers file from that stage's outputs:

cd ../00-bootstrap
terraform output -json providers | jq -r '.["00-cicd"]' \
  > ../00-cicd/providers.tf

If you want to continue to rely on outputs_location logic, create a terraform.tfvars file and configure it as described here.

Variable configuration

There are two broad sets of variables you will need to fill in:

  • variables shared by other stages (org id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.)
  • variables specific to resources managed by this stage

To avoid the tedious job of filling in the first group of variable with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured .tfvars files.

If you configured a valid path for outputs_location in the bootstrap stage, simply link the relevant terraform-*.auto.tfvars.json files from the outputs folder. For this stage, you need the .tfvars file compiled manually for the bootstrap stage, and the one generated by it:

# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/tfvars/00-bootstrap.auto.tfvars.json .
# also copy the tfvars file used for the bootstrap stage
cp ../00-bootstrap/terraform.tfvars .

A second set of variables is specific to this stage, they are all optional so if you need to customize them, create an extra terraform.tfvars file or add them to the file copied from bootstrap.

Refer to the Variables table at the bottom of this document, for a full list of variables, their origin (e.g. a stage or specific to this one), and descriptions explaining their meaning. The sections below also describe some of the possible customizations.

CI/CD systems

Gitlab

To configure Gitlab, add the following variable:

gitlab = {
  url                    = "https://gitlab.com" # Or self-hosted URL
  project_visibility     = "private"
  shared_runners_enabled = true
}

Also set GITLAB_TOKEN to a token that has appropriate permissions.

GitHub

To configure GitHub, add the following variable:

github = {
  url        = null # Or GitHub Enterprise base URL
  visibility = "private"
}

Also set GITHUB_TOKEN to a token that has appropriate permissions.

CI/CD repositories

While the other stages create the necessary supporting structure for their CI/CD pipelines, like service accounts and such, the 00-cicd stage creates all the repositories in your CI/CD system through automation. Its configuration is essentially a combination of all the cicd_repositories variables of the other stages plus additional CI/CD system specific configuration information.

This is an example of configuring the repositories in this stage.

cicd_repositories = {
  bootstrap = {
    branch            = null
    identity_provider = "github-sample"
    name              = "my-gh-org/fast-bootstrap"
    description       = "Google Cloud bootstrapping"
    type              = "github"
    create            = true
    create_group      = true
  }
  cicd = {
    branch            = null
    identity_provider = "github-sample"
    name              = "my-gh-org/fast-cicd"
    description       = "Fabric FAST CI/CD setup"
    type              = "github"
    create            = true
    create_group      = true
  }
  resman = {
    branch            = "main"
    identity_provider = "github-sample"
    name              = "my-gh-org/fast-resman"
    description       = "Google Cloud resource management"
    type              = "github"
    create            = true
    create_group      = true
  }
  networking = {
    branch            = "main"
    identity_provider = "github-sample"
    name              = "my-gh-org/fast-networking"
    description       = "Google Cloud networking setup"
    type              = "github"
    create            = true
    create_group      = true
  }
  security = {
    branch            = "main"
    identity_provider = "github-sample"
    description       = "Google Cloud security settings"
    name              = "my-gh-org/fast-security"
    type              = "github"
    create            = true
    create_group      = true
  }
  data-platform = {
    branch            = "main"
    identity_provider = "github-sample"
    name              = "my-gh-org/fast-data-platform"
    description       = "Google Cloud data platform"
    type              = "github"
    create            = true
    create_group      = true
  }
  project-factory = {
    branch            = "main"
    identity_provider = "github-sample"
    name              = "my-gh-org/fast-project-factory"
    description       = "Google Cloud project factory"
    type              = "github"
    create            = true
    create_group      = true
  }
}

The type attribute can be set to one of the supported repository types: github or gitlab.

Once the stage is applied the generated output files will contain pre-configured workflow files for each repository, that will use Workload Identity Federation via a dedicated service account for each repository to impersonate the automation service account for the stage.

Once done, you can run this stage:

terraform init
terraform apply

Files

name description resources
cicd.tf None tls_private_key
github.tf None github_actions_secret · github_repository
gitlab.tf None gitlab_group · gitlab_project · gitlab_project_variable
main.tf Module-level locals and resources.
outputs-files.tf Output files persistence to local filesystem. local_file
outputs-gcs.tf Output files persistence to automation GCS bucket. google_storage_bucket_object
outputs.tf Module outputs.
variables.tf Module variables.
versions.tf Version pins.

Variables

name description type required default producer
automation Automation resources created by the bootstrap stage. object({…}) ✓ 00-bootstrap
cicd_repositories CI/CD repository configuration. Identity providers reference keys in the federated_identity_providers variable. Set to null to disable, or set individual repositories to null if not needed. object({…}) null
custom_roles Custom roles defined at the org level, in key => id format. object({…}) null 00-bootstrap
github GitHub settings object({…}) {…}
gitlab Gitlab settings object({…}) {…}
outputs_location Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable string null

Outputs

name description sensitive consumers
tfvars Terraform variable files for the following stages. ✓