To provide an automated, pull request-based development workflow to validate and build future releases of Webspark 2 (WS2) using continuous integration (CI).
Background • Getting Started • Local Development • Sprints • Deployment • Resources • JIRA
This project aims to provide an automated, pull request-based development workflow to validate and build future releases of Webspark 2 (WS2) using continuous integration (CI).
This project leverages the Pantheon Build Tools workflow to streamline the development, testing, and deployment of the Webspark Drupal codebase.
This approach enables us to use GitHub as our preferred Git provider, CircleCI for Continuous Integration, and Pantheon Multidev environments for testing builds.
When creating a website on Pantheon that will use the Build Tools workflow, it is important to follow the instructions outlined in the Build Tools documentation. A summary of those instructions is provided below, along with the example code we used to set up the Webspark CI project. In our examples, we will assume the use of GitHub and CircleCI.
Review the following Build Tools Prerequisites Guide to
- Verify Composer is installed
- Verify Terminus is installed
- Verify the Terminus Build Tools Plugin is installed
If you like, you can manually generate the GitHub and CircleCI access tokens that will be used to create the project in Pantheon. This step is optional, as the Build Tools Plugin will generate them as needed during the project creation process.
Review the following Build Tools Token Generation Guide to
- Generate a GitHub access token
- Generate a CircleCI access token
Review the following Build Tools Project Creation Guide to
- Create a new site on Pantheon using the Build Tools workflow
It is important to take note of all the available options you have when creating your project. Review the Build Tools Plugin for full options.
The examples below assume the name
webspark-ci
Create a new project using the default Pantheon Drupal distribution:
terminus build:project:create --team='Arizona State University' --org='ASUWebPlatforms' d9 webspark-ciCreate a new project using the existing Webspark upstream:
terminus build:project:create --team='Arizona State University' --org='ASUWebPlatforms' asuwebplatforms/webspark-release-testing webspark-ciRotate the access tokens used to create the project:
terminus build:project:repair webspark-ciThis workflow differs from a usual Pantheon workflow in that when developing locally, there is no need for the Pantheon site itself to be cloned. The only use for the Pantheon site locally is to pull down the database and files.
Although you may use whatever tool you choose, it is always a good idea to use the tool with the best documentation!
We recommend the use of DDEV or Lando for local Drupal development. Both will require Docker to be installed.
After your tool of choice is installed, you then need to set up that tool specifically for Pantheon. This gives you the benefit of having Pantheon and Drupal-specific tools pre-installed, as well as being able to match the Pantheon server configuration as closely as possible.
A note on DDEV: At this point, we only want to ensure the Pantheon machine token is added. The rest of the steps outlined in the article above do not apply specifically to the webspark-ci project.
Once your local development environment is set up for Pantheon, you are ready to begin developing locally for the webspark-ci project. Remember, we do not need to use the Pantheon site for anything other than its database and files, so we will not need to pull the Pantheon site locally.
Clone the webspark-ci repo:
git clone [email protected]:ASUWebPlatforms/webspark-ci.gitNavigate to the repo, and install its dependencies via Composer:
cd webspark-ci
composer installInitiate the repo as a Drupal project:
ddev config
ddev startSince we already installed the dependencies, DDEV will automatically configure itself as a Drupal 9 project for us. When using the ddev config command, you simply need to continue until you are prompted to run ddev start.
We now have a container for our webspark-ci project, so next it is time to pull down the database and files from the Pantheon site.
First, we need to update the default PHP version in the DDEV config to 8.1, since Webspark expects us to use that:
# .ddev/config.yaml
php_version: "8.1"Next, we need to tell DDEV which site in Pantheon to pull the database and files from. In .ddev/providers, you will find a file named pantheon.yaml.example. Copy the contents of this file into a new file called pantheon.yaml. Next, we will add the Webspark CI Pantheon site as our target site:
# .ddev/providers/pantheon.yaml
environment_variables:
project: webspark-ci.devFinally, we can pull the database and files from the Pantheon site:
# Restart the container first since we changed the PHP version
ddev restart
# Establish a connection using our machine token
ddev auth ssh
# Pull the database and files
ddev pull pantheonComing soon!
Arizona State University follows Agile software development principles. As such, teams work in sprints to accomplish tasks in a timely, but orderly fashion. Webspark sprints are usually two weeks in length, and each sprint has a set number of tasks assigned to it.
Each sprint has a name, often, it is simply the sprint number (ex: Sprint 39). To keep development organized, the Webspark CI GitHub repository will have a branch created for each specific sprint. So, if we are currently in Sprint 39, there will also be a ws2-sprint-39 branch. This sprint branch will be used by all developers on the team as the base branch. All new branches should be created from this base branch, and all Pull Request(s) (PR) should be merged into this base branch. When the sprint is complete, the base branch will then be merged into the master branch via its own Pull Request.
For this example, let's assume we are working on Sprint 39 and your task is WS2-1596
As a developer working on the Webspark CI project, here is a typical workflow:
Sprint 39is started in JIRA, and tasks are assigned to you.- You begin your first task,
WS2-1596. - You fetch the
webspark-cirepo from GitHub, gaining access to theorigin/ws2-sprint-39branch. - You create a new branch from
origin/ws2-sprint-39, and name it after your task using all lowercase letters:ws2-1596. - You complete your work, and push your branch to the repo, creating the
origin/ws2-1596branch. - You create a new PR, looking to merge
origin/ws2-1596intoorigin/ws2-sprint-39, and assign thewebspark-maintainersgroup as Reviewers. - The build process deploys a new Pantheon multidev named after your PR, ready for other team members to review.
The benefit of the Pantheon Build Tools workflow is its automated deployment process between the GitHub repository and the Pantheon website. Previously, developers had to create and manage their own Pantheon sites to test their work. The Pantheon Build Tools workflow allows for a centralized site for all team members to use and contribute to. Below, you will find a simplified breakdown of the Build Tools Pull Request Workflow:
- Pull Request Issued: A developer makes changes to a task branch in their local environment and pushes these changes to the remote
webspark-cirepository. They then create a pull request (PR) to propose merging these changes into the sprint branch. - Continuous Integration (CI) Kicks In: Upon the creation of the PR, the continuous integration (CI) process starts via CircleCI. CircleCI is configured to automatically pick up the PR event and initiate a series of tasks defined in a configuration file within the project, which Pantheon has already bootstrapped for us.
- Automated Build Process: CircleCI first checks out the proposed changes from the task branch, then performs tasks such as installing dependencies, compiling code, running unit tests, etc. The base configuration simply runs Composer, however, we have the flexibility to expand upon this process as we see fit.
- Creation of the Pantheon Multidev: If the build process is successful, CircleCI then uses Terminus to create a new Multidev environment in the
webspark-ciPantheon site, using the database and files from thedevenvironment. The Multidev will be named after the PR. - Automated Testing: Depending on the project's configuration, CircleCI may then perform additional automated tests in the new Multidev environment. Pantheon provides Behat testing and visual regression testing by default, but we can expand these to include performance tests, accessibility tests, and more.
- PR Review: Once the Multidev environment is ready, CircleCI updates the original PR with the URL of the new Multidev. This allows the team to review the proposed changes in a fully functional environment. Additional commits made to the task branch for the PR will trigger the build process, ensuring the Multidev is always using the most up-to-date code.
- Deletion of the Pantheon Multidev: When the PR is merged into the sprint branch and closed, CircleCI will automatically destroy the Multidev environment for that PR.