Skip to content

Latest commit

 

History

History
316 lines (213 loc) · 15.3 KB

releasing.adoc

File metadata and controls

316 lines (213 loc) · 15.3 KB
Raw

Releasing

This document describes how releases of the packages (aka modules) in this repository are versioned and published.

Quickstart

To publish a release, first make sure the npm token is still valid. Next, click Run Pipeline on the Pipelines page, select the branch from which you’re releasing, set the RELEASE_VERSION variable to a version number or an increment keyword, then click Run Pipeline. The CI job takes over from there. See Release Trigger to find more details and step-by-step commands.

The remainder of this document describes how the release process works, including configuration, credentials, and process.

Overview

Here’s what happens automatically when a release is made:

  • The version property in package.json and each package.json file under packages/ is set to the new version.

  • The Unreleased section title is in CHANGELOG.adoc is replaced with the release version and current date.

  • The version in docs/antora.yml is replaced with the release version.

  • The copyright year is updated to the current year in the project README.adoc and each package README.adoc.

  • The changes are committed to the git branch from which the release is made (only changed files are committed).

  • An unsigned, annotated tag with the name of the version prefixed with “v” (i.e., v1.0.0) is added to the git repository.

  • Each package under packages/ is packed and published to the @antora scope of the npm registry.

    • Publishing to the npm registry also makes the packages available in the Yarn registry.

    • The packages cannot be unpublished from the npm registry once published.

  • The tag and corresponding commit on the branch from which the release is made are pushed to “origin” remote (i.e., the repository on GitLab).

If the CI pipeline succeeds, we can assume the release was successful. The logs of the release activity are available through the GitLab CI interface. The release logic will prevent the same CI job from making another release.

Keep in mind that the release does not update the entries in the CHANGELOG. It only replaces the Unreleased section title with the release version and current date. The entries must be added manually, prior to the release.

Versioning

Version numbers are chosen for releases according to semantic versioning rules. All packages in this repository are released together and thus share a common version. Changes to the repository as a whole should be considered when selecting the version number for a release.

The current published version is tracked in the file package.json at the root of this repository. This value is used to generate the next version in the sequence (unless an explicit version is provided).

The version number (e.g., 1.1.0) or increment keyword (e.g., patch) for a release is set using the RELEASE_VERSION variable on the “Run Pipeline” page. This process is described in detail later.

Release Process

Releases are published from the CI pipeline by the release job. The release job is defined in the file .gitlab-ci.yml. The release job only runs on a protected branch when the pipeline is run from the Pipeline page and the RELEASE_VERSION variable is set. The job runs in a phase named “deploy” at the end of the CI pipeline.

The release job delegates to the npm/release.sh script to perform the release.

Credentials

In order to perform a release, npm must be able to:

  • publish packages to npmjs.com

  • push commits to this git repository

The auth token for publishing to npmjs.com is stored in the RELEASE_NPM_TOKEN secret variable defined in the CI/CD configuration. Make sure this token is still valid before releasing! (You can check if the token is registered by running npm token list). npm uses this token to authenticate as the release manager (currently mojavelinux). This is done by writing the token to an .npmrc file located at the root of the project (i.e., .npmrc).

After the package is published, the tag and the corresponding commit to the main branch must be pushed from the CI job to the git repository. A dedicated SSH key (with no passphrase) has been generated for this purpose using ssh-keygen -m PEM -t rsa -b 4096 -C antora-releases -f antora-releases. The private SSH key is stored in the RELEASE_SSH_PRIV_KEY secret variable defined in the CI/CD configuration. The corresponding public SSH key is registered as a deploy key in the repository configuration. During the release job, an SSH agent session is started, initialized, and killed by npm/release.sh. The private key is read from the RELEASE_SSH_PRIV_KEY environment variable and added to the SSH agent session. The git client is then configured to push the commits over SSH.

The RELEASE_GIT_NAME and RELEASE_GIT_EMAIL secret variables, also defined in the CI/CD configuration, are used to set the git user (name and email) for the release commits. The values used are those of the release manager.

The secret variables described in this section are scoped to the releases environment. The release job associates itself with the releases environment, allowing it to receive the secret variables.

Since we’re publishing packages to a scope (i.e., @antora), we have to set access=public in .npmrc in order to successfully publish new packages. Otherwise, scoped packages are not public by default.

README Dance

The npmjs.com registry does not currently support READMEs written in AsciiDoc. To complicate matters, the npm client doesn’t allow the README to be specified explicitly when packaging, and it has an affinity for README.adoc. Therefore, we have to do some fancy footwork to stuff a Markdown version of the README into the package that’s being shipped off to the registry. Thanks to npm lifecycle hooks, this is possible.

Each package defines two package-scoped npm lifecycle scripts, prepublishOnly and postpublish. These scripts link to the following two shared scripts:

In the prepublishOnly script, the README.adoc file is moved out of the way, its contents are transformed to Markdown and saved as README.md. The package.json is also updated to include a readmeFilename key that points to this file. The packaging script then picks up the generated README.md file and bundles it with the package (ignoring the hidden README.adoc).

The postpublish script restores the original state by removing the README.md file and moving README.adoc back into place. The package.json is also updated to remove the readmeFilename key. As a result, the npm registry displays the Markdown version of the README, but we get to maintain the content in AsciiDoc format.

In addition to these two package-scope scripts, the version.js script runs once at the start of the npm publish lifecycle to apply the following changes before the files are committed and tagged for the release:

  • Replace the Unreleased section title in CHANGELOG.adoc with the release version and date.

  • Update the docs version in docs/antora.yml.

To verify the correct package metadata is sent over the wire, write the stringified metadata passed to the afterChange callback in verdaccio (build/api/endpoint/api/publish.js) to a file and inspect it.

Version Selection

By default, npm is configured to select the next prerelease version in the sequence when selecting the version number. This default can be overridden by setting the RELEASE_VERSION variable. The RELEASE_VERSION variable accepts a version number or increment keyword. For example:

RELEASE_VERSION=patch

If the value assigned to the RELEASE_VERSION variable matches one of the increment keywords (major, minor, patch, premajor, preminor, prepatch, prerelease), that value is passed to npm’s version bump argument. Consult the npm documentation for details about how these values determine the next version number. Otherwise, the value is assumed to be an explicit version number, which is also passed to npm’s version bump argument (as npm figures out which is which).

If the RELEASE_NPM_TAG variable is set, its value is passed to npm’s --dist-tag option. This setting can be used to publish releases that do not get installed by default (e.g., prereleases). If the RELEASE_NPM_TAG variable is not set, and the RELEASE_VERSION beings with pre or contains a hyphen, the RELEASE_NPM_TAG variable defaults to testing. Otherwise, it defaults to latest. To override these defaults, set the RELEASE_NPM_TAG environment variable when running the pipeline. For example:

RELEASE_VERSION=2.0.0-alpha.1
RELEASE_NPM_TAG=preview

The release is always run from the branch which was selected from the “Run Pipeline” page.

Release Trigger

A release is made by running a pipeline from the Pipelines page. The steps to perform a release are as follows:

  • Navigate to the Pipelines page in the project on GitLab.

  • Click the Run Pipeline button.

  • (optional) If you aren’t releasing from the default branch, select a different (protected) branch.

  • Type RELEASE_VERSION in the "Input variable key" field

  • Type a version number (e.g. 1.0.0-alpha.1) or increment token (e.g., prerelease) in the "Input variable value" field

  • Click the Run Pipeline button

The release script handles the rest.

The job is configured to only run on a protected branch in the upstream project. The release job immediately clones the release branch and performs the release from there.

The CI job will publish the packages to npmjs.com at the end of the job execution.

Post Release Trigger

Once the release completes, the job to build the Docker image is automatically triggered. The version of the Docker image matches the latest version of Antora. The Docker image will be tagged testing if the release number contains a hyphen. Otherwise, it will be tagged latest (the default version).

Testing a Release

It’s possible to test the release locally using a private npm registry. This section briefly describes how.

Set Up a Private Npm Registry

You can run a private npm registry on your own machine using Verdaccio.

Verdaccio is available as an npm package. To install Verdaccio, run the following command:

$ npm i -g verdaccio

You’ll then need to create a user for publishing packages. First, start Verdaccio using the verdaccio command:

$ verdaccio

Verdaccio runs at http://localhost:4873 by default. You can visit this URL to see what packages have been installed. (Of course, there’s nothing there yet!)

Next, create a user using the npm adduser command (assuming the default port):

$ npm adduser --registry http://localhost:4873

You’ll never need to login as this user. This just adds an auth token for localhost:4873 to the ~/.npmrc file that the npm client will use for publishing to this registry.

Next, we’ll need to disable the proxy for the @antora scope so that we can test locally without interfering with or getting interference from the released packages.

First, stop Verdaccio. Then, open the file ~/.config/verdaccio/config.yaml, add the entry under the packages: key, and save it.

  '@antora/*':
    access: $all
    publish: $authenticated

Run the Private Npm Registry

You must start Verdaccio in order to use it for publishing:

$ verdaccio

Prepare the Clones

Now we need to prepare two clones so that they are configured as the release repository (test-release-from) and the upstream repository (test-release-to). This ensures that you can perform a full publish without affecting the real repository.

Start by cloning the repository to use as the upstream target:

$ git clone [email protected]:antora/antora.git test-release-to
  cd test-release-to

Next, switch to another branch so the main branch can receive commits and we have something to come back to:

$ git checkout -b current
  cd ..

Next, clone the repository from which you will perform the release:

$ git clone [email protected]:antora/antora.git test-release-from
  cd test-release-from

Next, change the remote origin to point to our local (fake) target:

$ git remote set-url origin ../test-release-to

Confirm this is set up correctly by doing an empty push:

$ git push origin main

Perform a Local Release

Now you are ready to perform a local release. Switch to the test-release-from repository and run the following command, adjusting the npm_config_registry and version bump value as needed:

$ npm_config_registry=http://localhost:4873 ./npm/local-release.sh prerelease

If you’ve set up everything correctly, this will not affect the official repository and won’t publish anything to npmjs.com.

Verify that the packages have been published to the private npm registry by visiting http://localhost:4873.

Reverting a Release

The beauty of testing locally is that you can revert a release. The quickest way to do it is to run the provided script from the test-release-from repository:

$ npm_config_registry=http://localhost:4873 ./npm/revert-local-release.sh ../test-release-to

Or you can do it manually. Start by purging the packages from the private npm repository:

$ for package in `find packages -mindepth 1 -maxdepth 1 -printf "%f\n"`; do
    npm --registry http://localhost:4873 unpublish --force @antora/$package
  done

Next, move to the test-release-to repository and clean stuff up:

$ git tag -d `git tag`
  git checkout main
  git reset --hard `git rev-parse current`
  git checkout current

Now, go back to the test-release-from repository and sync with the local origin:

$ git tag -d `git tag`
  git fetch origin
  git reset --hard origin/main

Now you should be all set to try the release again!

Future Ideas

In the future, we may consider other ways to trigger a release aside from using a dedicated branch.

Pipeline Triggers

One possible approach is to use a pipeline trigger against the main branch. The reason we decided to defer using this strategy is because there’s no clear way to disable a job for a certain trigger or distinguish one trigger from another. If we want to use pipeline triggers for other purposes, there’s a risk we could inadvertently trigger a release. Even with protections in place, such as checking for a special environment variable, the release job would still run on any trigger.

Commit Tags

Another way to trigger a release is to use a commit tag. When pushing a commit to the main branch, either directly or by merging an MR, we could use a special tag in the commit message to indicate that a release could follow. For example, the commit message might look like:

add a cool new feature [ci release]

The increment keyword could be specified as an optional qualifier:

totally change the API [ci release major]

The upside to this approach is that it’s very easy to control when a release is performed and to track what triggered it. The downside is that it adds noise to commit messages.