Buildkite Rails Parallel Example with Knapsack Pro

This repository is an example on how to easily run 20 parallel testing agents for a Rails application using Buildkite, rbenv and Knapsack Pro.

This repository doesn't actually include a Rails sample application, but has the same CI scripts you would use for a production application. For a Docker based setup, see the Rails Docker Parallel Example with Knapsack Pro.

Files to note:

• .buildkite/hooks/environment - loads rbenv
• .buildkite/pipeline.yml - the pipeline config showing how to have parallel steps
• scripts/ci/setup.sh - sets up the CI environment for each step
• scripts/ci/parallel_specs.sh - runs the specs in parallel using knapsack_pro gem with Queue Mode. Tests are dynamically distributed across parallel jobs. Please read this article to learn more about environment variables configuration for Buildkite and Knapsack Pro.

How does it work?

In the pipeline configuration file the parallelism property for the test step is set to 20. When a build, the step will appear 20 times in the pipeline, each with different environment variables exposed so you can divvy up your test suite accordingly. You can then run 20 agents (on the same machine, or distributed) to work on the 20 jobs in parallel.

steps:
  - name: ":rspec:"
    command: "scripts/ci/setup.sh && scripts/ci/parallel_specs.sh"
    parallelism: 20

See the parallelizing builds guide for more information to create parallelized and distributed builds with Buildkite.

Consult the Knapsack Pro documentation for configuring your database and dependent services to support running parallel steps on the one machine.

You can obtain API key for Knapsack Pro here.

License

See Licence.md (MIT)