Continuous Integration Build Comparison

Build comparison

In a continuous integration context, your performance/load tests should be integrated in your build pipeline. Well we provide you with a way to compare two test runs (or more) in the visualization tool of choice (Grafana, Kibana).

How to define the builds

Defining your build number is as simple as it gets. All you need to do is use the environment variables of your CI solution and provide the build number to your jmeter test execution. Here are examples for three different CI solutions.

Jenkins

sh "jmeter -t someTest.jmx -JBuildNumber=${env.BUILD_NUMBER}"

CircleCI

- run:
    shell: /bin/sh
    command: |
      jmeter -t someTest.jmx -JBuildNumber=CIRCLE_BUILD_NUM

TravisCI

script: jmeter -t someTest.jmx -JBuildNumber=TRAVIS_BUILD_NUMBER

Adding graphs to your visualizer

Whether you use Grafana or Kibana, you can then compare the builds by splitting the series with the BuildNumber field.