Pipeline Builder

The Pipeline Builder is a collection of tools related to GitHub Actions and other GitHub-related concerns for Paketo-style buildpacks. This collection includes GitHub actions, a CLI that configures pipelines for repositories, and a collection of descriptors for existing repositories.

Pipeline Builder

Octo Pipelines

Octo Pipelines is a CLI that generates a collection of GitHub Workflows and other Github-related artifacts in a repository. The pipeline generator takes a simplified descriptor as input and transforms it into more complicated workflows and artifacts.

Getting Started

The command line can either be built or run directly using Go.

$: go run github.com/paketo-buildpacks/pipeline-builder/cmd/octo --descriptor <DESCRIPTOR>

The input is a YAML-based descriptor, examples of which can be found in .github/pipeline-descriptor.yml in other repositories.

Common Workflows

Regardless of the contents of the descriptor the pipeline builder creates a minimal set of workflows and artifacts.

Dependendabot

Example dependabot.yml

A Dependendabot configuration file is added to all repositories. It will always add an update for github-actions, and if the repository contains a go.mod file, it will also add an update for gomod.

Labels

In order to facilitate the automated creation of draft releases and notes, a set of semver-scope labels (semver:major, semver:minor, and semver:patch) and type labels (type:bug, type:dependency-upgrade, type:documentation, type:enhancement, type:question, type:task) are added to the repository. In addition, a workflow that enforces exactly one of each label is attached to every PR.

Pipeline Updater

Example update-pipeline.yml

A pipeline-updating workflow is added to the repository to ensure that workflows are kept up to date as the pipeline-builder evolves.

Release Drafter

Draft release notes are created on every commit to main. These notes take into account every PR since the previous release in order to create a division of the types of changes that were made and the semver scope of the change to work out the next release number.

Descriptor

The descriptor is a YAML document with a number of top-level keys that correspond to new workflows, modified workflows, or artifacts. All top-level keys except github are optional.

`github` (REQUIRED)

github:
  username: ${{ secrets.JAVA_GITHUB_USERNAME }}
  token:    ${{ secrets.JAVA_GITHUB_TOKEN }}

github is an object the describes the GitHub identity that the pipeline should use in the many places that GitHub API access is required. The token must be granted the repo, write:packages, and workflow scopes.

`codeowners`

codeowners:
- path:  "*"
  owner: "@paketo-buildpacks/java-buildpacks"

Example CODEOWNERS

codeowners is a list of objects test describe a path and owner. Each of these objects corresponds to a line in CODEOWNERS file.

`package`

package:
  repository:           gcr.io/paketo-buildpacks/adopt-openjdk
  include_dependencies: false
  register:             true
  registry_token:       ${{ secrets.JAVA_REGISTRY_TOKEN }}

package is an object that describes the repository a buildpackage should be published to as well as whether to include the buildpackage's dependencies when creating it (false by default). If defined, a create-package workflow is created that creates and publishes a new package when a release is published as well as adds a create-package job to the tests workflow that is run on each PR and each commit. It will also add additional content to the draft release notes about the contents of the build package and will update the digest of the buildpackage in the published release notes. If register is true, after the package is created, it is registered with the Buildpack Registry Index.

`builder`

builder:
  repository: gcr.io/projectriff/builder

builder is an object that describes the repository a builder should be published to. If defined, a create-builder workflow is created that's creates and publishes a new builder when a release is published as well as adds a create-builder job to the tests workflow that is run on each PR and each commit. It will also add additional content to the draft release notes about the contents of the build package and will update the digest of the builder in the published release notes. Finally it will add update workflows for the lifecycle and builder image elements of the builder.toml file.

`docker_credentials`

docker_credentials:
- registry: gcr.io
  username: _json_key
  password: ${{ secrets.JAVA_GCLOUD_SERVICE_ACCOUNT_KEY }}

docker_credentials is a list of objects that describes a registry, username, and password for authenticating against a Docker registry. In every job that registry access is required, a Docker Login Action is created and conditionalized so that it will only run on pushes to main and on PRs that come from the same repository (to protect the secrets).

`http_credentials`

http_credentials:
- host:     download.aquasec.com
  username: ${{ secrets.AQUA_SECURITY_USERNAME }}
  password: ${{ secrets.AQUA_SECURITY_PASSWORD }}

http_credentials is a list of objects that describe a host, username, and password for authenticating requests against an HTTP endpoint. In every job that HTTP access is required a .netrc file is written and conditionalized so that it will only be written on pushes to main and on PRs that come from the same repository (to protect the secrets).

`dependencies`

dependencies:
- name:            JRE 11
  id:              jre
  version_pattern: "11\\.[\\d]+\\.[\\d]+"
  uses:            docker://ghcr.io/paketo-buildpacks/actions/adopt-openjdk-dependency:main
  with:
    implementation: hotspot
    type:           jre
    version:        "[11,12)"

Example update-*.yml

dependencies is a list of objects that define how dependencies are detected and updated by describing an optional name (defaults to id), id that matches a buildpack.toml defined dependency id, an optional version_pattern that defines which dependency with a given id to update, a uses to define which GitHub Action to use to find the next version, and a with used to configure the GitHub action to use to find the next version. If defined, each object will create an update workflow that is responsible for detecting a new version, updating buildpack.toml and opening a PR to include the change in the repository, if appropriate.

`test`

test:
  steps:
  - name: Install richgo
    run: |
      #!/usr/bin/env bash

      set -euo pipefail

      GO111MODULE=on go get -u -ldflags="-s -w" github.com/kyoh86/richgo
  - name: Run Tests
    run: |
      #!/usr/bin/env bash

      set -euo pipefail

      richgo test ./...
    env:
      RICHGO_FORCE_COLOR: "1"

test is an object that allows you to override the default test configuration by specifying a list of steps as defined by the GitHub Actions reference documentation. The default value is described in the example above.

`path`

path: ..

path is a pointer to a Git repository, on your local filesystem, where files will be created. If unspecified, the default is ...

`offline_packages`

offline_packages:
- source: paketo-buildpacks/adopt-openjdk
  target: gcr.io/tanzu-buildpacks/adopt-openjdk

offline_packages is a list of objects that describe a source GitHub repository and a target Docker registry location. If defined, each object will create a create-package workflow that is responsible for detecting a new online buildpackage release and creating a matching offline buildpackage release and publishing it.

`actions`

actions:
- source: adopt-openjdk-dependency
  target: ghcr.io/paketo-buildpacks/actions/adopt-openjdk-dependency

Example create-action-*.yml

actions is a list of objects that describe a source directory within this repository's actions/ directory, and a target Docker registry location. If defined, each object will create a create-action workflow that is responsible for building and publishing new versions of the action.

Actions

This repository contains the source code and build pipelines for a collection of GitHub actions that are used by buildpacks. Most of these actions are used to find the latest version of given dependencies.

All the of the dependency actions also accept a pre_release configuration option (true by default) of whether to return pre-release versions.

AdoptOpenJDK Dependency

The AdoptOpenJDK Dependency queries the AdoptOpenJDK API for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/adopt-openjdk-dependency:main
with:
  implementation: hotspot
  type:           jre
  version:        "[11,12)"

Amazon Corretto Dependency

The Amazon Corretto Dependency watches Amazon Corretto repositories for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/amazon-corretto-dependency:main
with:
  glob:       amazon-corretto-11\.[\d.-]+-linux-x64.tar.gz
  repository: corretto-11

AppDynamics Dependency

The AppDynamics Dependency queries the AppDynamics API for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/appdynamics-dependency:main
with:
  type: sun-jvm

Aqua Security Dependency

The Aqua Security Dependency queries the Aqua Security API for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/aqua-security-dependency:main
with:
  username: ${{ secrets.AQUA_SECURITY_USERNAME }}
  password: ${{ secrets.AQUA_SECURITY_PASSWORD }}

Azul Zulu Dependency

The Azul Zulu Dependency queries the Azul Zulu API for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/azul-zulu-dependency:main
with:
  type:    headfull
  version: "11"

Bellsoft Liberica Dependency

The Bellsoft Liberica Dependency queries the Bellsoft Liberica API for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/bellsoft-liberica-dependency:main
with:
  type:    jre
  version: "11"

CA APM Dependency

The CA APM Dependency watches the CA APM Download Page for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/ca-apm-dependency:main
with:
  type: java

GCS Dependency

The GCS Dependency watches GCS Buckets for new versions. The first capture in glob is used as the version.

uses: docker://ghcr.io/paketo-buildpacks/actions/gcs-dependency:main
with:
  bucket: projectriff
  glob:   command-function-invoker/command-function-invoker-linux-amd64-([\d\.]+).tgz

GitHub Release Dependency

The GitHub Release Dependency watches GitHub Releases for new versions. The first capture in glob is used as the version.

uses: docker://ghcr.io/paketo-buildpacks/actions/github-release-dependency:main
with:
  glob:       sapmachine-jre-.+_linux-x64_bin.tar.gz
  owner:      SAP
  repository: SapMachine
  tag_filter: sapmachine-(11.*)
  token:      ${{ secrets.GITHUB_TOKEN }}

Google Stackdriver Profiler Dependency

The Google Stackdriver Profiler Dependency watches the Google Cloud Profiler bucket for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/google-stackdriver-profiler-dependency:main

GraalVM Dependency

The GraalVM Dependency watches the GraalVM Repository for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/graalvm-dependency:main
with:
  glob:    graalvm-ce-java11-linux-amd64-.+.tar.gz
  token:   ${{ secrets.GITHUB_TOKEN }}
  version: 11

Gradle Dependency

The Gradle Dependency queries the Gradle API for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/gradle-dependency:main

JProfiler Dependency

The JProfiler Dependency watches the JProfiler Changelog for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/jprofiler-dependency:main

JRebel Dependency

The JRebel Dependency watches the JRebel Download Page for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/jrebel-dependency:main

Leiningen Dependency

The Leiningen Dependency watches the Leiningen Repository for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/leiningen-dependency:main
with:
  token: ${{ secrets.GITHUB_TOKEN }}

Maven Dependency

The Maven Dependency queries a Maven Repository for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/maven-dependency:main
with:
  uri:         https://repo1.maven.org/maven2
  group_id:    org.apache.maven
  artifact_id: apache-maven
  classifier:  bin
  packaging:   tar.gz

New Relic Dependency

The New Relic Dependency watches the New Relic Download Page for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/new-relic-dependency:main
with:
  type: php

NPM Dependency

The NPM Dependency queries the NPM API for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/npm-dependency:main
with:
  package: "@google-cloud/debug-agent"

OverOps Dependency

The OverOps Dependency watches the OverOps Download Page for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/overops-dependency:main

Riverbed Dependency

The Riverbed Dependency watches the Riverbed Bucket for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/riverbed-dependency:main

Skywalking Dependency

The Skywalking Dependency watches the Apache Skywalking Download Page for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/skywalking-dependency:main

Spring Generations

The Spring Generations queries the Spring Generations API for the generation lifecycle of Spring projects.

uses: docker://ghcr.io/paketo-buildpacks/actions/spring-generations:main
with:
  uri: https://spring.io/api

Tomcat Dependency

The Tomcat Dependency watches the Apache Tomcat Download Page for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/tomcat-dependency:main
with:
  uri: https://archive.apache.org/dist/tomcat/tomcat-9

YourKit Dependency

The YourKit Dependency watches the YourKit Download Page for new versions.

uses: docker://ghcr.io/paketo-buildpacks/actions/yourkit-dependency:main

License

This library is released under version 2.0 of the Apache License.

Name		Name	Last commit message	Last commit date
Latest commit History 313 Commits
.github		.github
actions		actions
cmd/octo		cmd/octo
octo		octo
.gitignore		.gitignore
LICENSE		LICENSE
NOTICE		NOTICE
README.md		README.md
go.mod		go.mod
go.sum		go.sum
token-scopes.png		token-scopes.png

License

micahyoung/pipeline-builder

Folders and files

Latest commit

History

Repository files navigation

Pipeline Builder

Octo Pipelines

Getting Started

Common Workflows

Dependendabot

Labels

Pipeline Updater

Release Drafter

Descriptor

github (REQUIRED)

codeowners

package

builder

docker_credentials

http_credentials

dependencies

test

path

offline_packages

actions

Actions

AdoptOpenJDK Dependency

Amazon Corretto Dependency

AppDynamics Dependency

Aqua Security Dependency

Azul Zulu Dependency

Bellsoft Liberica Dependency

CA APM Dependency

GCS Dependency

GitHub Release Dependency

Google Stackdriver Profiler Dependency

GraalVM Dependency

Gradle Dependency

JProfiler Dependency

JRebel Dependency

Leiningen Dependency

Maven Dependency

New Relic Dependency

NPM Dependency

OverOps Dependency

Riverbed Dependency

Skywalking Dependency

Spring Generations

Tomcat Dependency

YourKit Dependency

License

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

`github` (REQUIRED)

`codeowners`

`package`

`builder`

`docker_credentials`

`http_credentials`

`dependencies`

`test`

`path`

`offline_packages`

`actions`

Packages