-
Notifications
You must be signed in to change notification settings - Fork 2
Running Tests
This guide will walk you through the steps needed to run one execution cycle and then use this information to execute a complete test cycle with a generated report.
Make sure your project's config/environment.rb defines jmeter properties and there is at least 1 cluster defined.
A configuration file with an additional target monitoring -
# Hailstorm configuration
Hailstorm.application.config do |config|
# This is the JMeter configuration
config.jmeter do |jmeter|
# default properties for all app/**/*.jmx files
jmeter.properties do |map|
map['ThreadGroup.loops_count'] = 1
map['ThreadGroup.num_threads'] = 10
map['ThreadGroup.ramp_time'] = map['ThreadGroup.num_threads']
end
end
# EC2 configuration
config.clusters(:amazon_cloud) do |cluster|
cluster.access_key = 'BRICKREDTO3PGNOIDAFX'
cluster.secret_key = 'B+{Kr;<^Cxt:(Z*4>Bc4#z%r7Kdd9&J(?cwa}{7@'
cluster.region = 'us-east-1'
cluster.max_threads_per_agent = 25
end
# Target configuration
config.monitors(:nmon) do |monitor|
monitor.executable_path = '/usr/bin/nmon'
monitor.ssh_identity = '3pg_identity'
monitor.user_name = '3pg'
monitor.sampling_interval = 5
monitor.groups('Application Server') do |group|
group.hosts('p.3pillarglobal.com') # don't forget to put the host name in quotes!
end
end
config.samples_breakup_interval = "1,3,5,10"
endStart the CLI application with script/hailstorm within the project directory. At the prompt, invoke setup command and you should see INFO messages come up. Once the setup completes you can see the different components that have been setup.
Once setup completes, you will see output similar to this:
hs > setup
3774: 17:41:46 [INFO] Setting up the project...
3774: 17:41:46 [INFO] Reading and validating JMeter plans...
3774: 17:41:47 [INFO] Setting up clusters...
4028: 17:41:48 [INFO] Searching available AMI...
4028: 17:42:01 [INFO] Using AMI ami-d02ff0b9 for agents...
4028: 17:42:02 [INFO] Starting new agent on us-east-1...
4028: 17:42:29 [INFO] agent#i-2f0cc249 is running, ensuring SSH access...
4028: 17:44:38 [INFO] Uploading script jmx/TestDroidLogin...
3774: 17:44:50 [INFO] Setting up targets...
# more output truncated and explained in next sections
💡 If you encounter errors in setup you can retry the setup command safely. Components which are setup correctly will remain unaffected. Hailstorm will offer diagnostic messages on errors with recommendations on how to resolve the issue.
The environment details displayed above consists of the following components:
You can view the test plans that have been included in the application. Remember, multiple test plans are run in parallel.
Invoke show jmeter in the console and you'll see output similar to this:
hs > show jmeter
Active JMeter Plans
===============================================================================
1] TestDroidLogin
+-------------------------+-------------------------+-----------------------+
| Properties |
+-------------------------+-------------------------+-----------------------+
| ThreadGroup.loops_count | ThreadGroup.num_threads | ThreadGroup.ramp_time |
+-------------------------+-------------------------+-----------------------+
| 1 | 10 | 10 |
+-------------------------+-------------------------+-----------------------+
You can view the active clusters and the load agents (once they are created).
Invoke show cluster and you should see output similar to this.
Active Clusters
===============================================================================
1] Amazon Cloud, region: us-east-1
+----------------+--------+----------------+------------+
| JMeter Plan | Type | IP Address | JMeter PID |
+----------------+--------+----------------+------------+
| TestDroidLogin | Master | 184.169.195.43 | |
+----------------+--------+----------------+------------+
You can view the target hosts which have been successfully added to the application.
Invoke show monitor and you should see output similar to this.
hs > show monitor
Active Monitors
===============================================================================
+--------------------+-----------------+---------+-----+
| Role | Host | Monitor | PID |
+--------------------+-----------------+---------+-----+
| Application Server | qa.brickred.com | nmon | |
+--------------------+-----------------+---------+-----+
Once you have setup the application to your satisfaction, it is time to start the load generation. Invoke the start command and you should see output similar to this:
hs > start
4534: 15:25:28 [INFO] Starting load generation and monitoring on targets...
Active Clusters
===============================================================================
1] Amazon Cloud, region: us-west-1
+----------------+--------+----------------+------------+
| JMeter Plan | Type | IP Address | JMeter PID |
+----------------+--------+----------------+------------+
| TestDroidLogin | Master | 184.169.195.43 | 1111 |
+----------------+--------+----------------+------------+
Active Monitors
===============================================================================
+--------------------+-----------------+---------+------+
| Role | Host | Monitor | PID |
+--------------------+-----------------+---------+------+
| Application Server | qa.brickred.com | nmon | 7321 |
+--------------------+-----------------+---------+------+
Note the JMeter PID column and the Active Monitors PID column now have values - indicating the load generation and monitoring was started successfully.
You can stop the load generation and target monitoring after they have been started. You use the stop command for this. The stop command will not prematurely stop the load generation. When you issue a stop command the command checks your JMeter plans to see if you have defined a loop count or a time period; if so it refuses to stop the tests. In case you have set your tests to loop forever, the application will stop JMeter.
The stop command can be used with a wait option, this will wait till the JMeter instances have executed your tests completely. This is safe even if you are using multiple JMeter plans, since the target monitoring will not be stopped till all JMeter plans have stopped executing.
Invoke stop wait and you will see output similar to this:
hs > stop wait
3774: 18:08:48 [INFO] Stopping load generation and monitoring on targets...
4586: 18:08:55 [INFO] JMeter is still running, waiting as asked...
4586: 18:09:56 [INFO] JMeter has exited, proceeding...
4918: 15:27:48 [INFO] Fetching logs from Amazon Cloud, region: us-east-1...
6504: 15:28:12 [INFO] Monitoring stopped at qa.brickred.com
6504: 15:28:12 [INFO] Collecting usage data from qa.brickred.com...
Active Clusters
===============================================================================
1] Amazon Cloud, region: us-east-1
+----------------+--------+----------------+------------+
| JMeter Plan | Type | IP Address | JMeter PID |
+----------------+--------+----------------+------------+
| TestDroidLogin | Master | 184.169.195.43 | |
+----------------+--------+----------------+------------+
Active Monitors
===============================================================================
+--------------------+-----------------+---------+-----+
| Role | Host | Monitor | PID |
+--------------------+-----------------+---------+-----+
| Application Server | qa.brickred.com | nmon | |
+--------------------+-----------------+---------+-----+
Notice that the JMeter PID and Active Monitors PID columns are now empty, indicating that remote processes were terminated successfully.
💡 If you are using Amazon EC2, you can temporarily suspend your agents while stopping the tests. This can be useful if you wish to resume testing at a later point but do not want to go through a long setup stage. Invoke stop wait suspend in this case.
In a typical test cycle, you will tend to alter some bits of the load generation and/or the target environments. In such cases, you can make changes to the config/environment.rb. The changes to this file are loaded automatically by the application.
For example, you can increase the number of threads for the next test - map['ThreadGroup.num_threads'] = 20.
Now rerun the start and stop wait commands. Run the same sequence of steps for 30, 40, 50 users.
💡 If you exceed the maximum number of threads per instance, a new instance will automatically be started and the virtual users (/threads), will be distributed equally.
3774: 18:34:20 [INFO] Setting up clusters...
9126: 18:34:21 [INFO] Starting new agent on us-east-1...
9126: 18:34:55 [INFO] agent#i-2f5d9349 is running, ensuring SSH access...
⭐ The Hailstorm console supports in session history without any extra configuration. Simply use the up and down arrow keys to access previous commands. This history is automatically persisted across sessions (history will be maintained between Hailstorm restarts).
You can generate reports by issuing the results report command. By default the report will include data for all stopped tests; if you abort a test, the data for the test will be ignored.
You can view the tests which will be included in the report prior to generating the report by issuing results command or results show command:
hs > results
+----------+---------------------+---------------------+---------+
| TEST | Started | Stopped | Threads |
+----------+---------------------+---------------------+---------+
| 1 | 2012-05-07 18:08:37 | 2012-05-07 18:10:10 | 10 |
| 2 | 2012-05-07 18:30:51 | 2012-05-07 18:33:24 | 20 |
| 3 | 2012-05-07 18:34:20 | 2012-05-07 18:39:50 | 30 |
| 4 | 2012-05-07 18:40:08 | 2012-05-07 18:42:50 | 40 |
| 5 | 2012-05-07 18:43:24 | 2012-05-07 18:46:07 | 50 |
+----------+---------------------+---------------------+---------+
You can exclude/include specific tests or you can choose tests as well. Type results help to see all options.
Go ahead and just invoke the results report command and you should see output similar to this:
hs > results report
3774: 18:50:31 [INFO] Creating report for stopped tests...
3774: 18:50:41 [INFO] Report generated to: /home/ubuntu/Workspace/jmeter/testdroid_page_visits/reports/testdroid_page_loads-1-5.docx
Reports are always generated to the application/project reports directory. Check out an auto-generated report.
⭐ The generated report is in Word (docx) format and is editable, the sample is a PDF to share easily.
Once you are through with your testing, you can issue the terminate command. This will terminate your Amazon EC2 instances.
hs > terminate
3774: 18:57:55 [INFO] Terminating test cycle...
15982: 18:57:55 [INFO] Terminating agent#i-2f5d9349...
15980: 18:57:55 [INFO] Terminating agent#i-2f0cc249...
# more output truncated (will be similar to show command)
You can check the state of the load generation and the application will tell you if it's still running on any load agent or if all tests are finished.
hs > status
4534: 21:24:43 [INFO] No tests have been started
⭐ If you want to purge your cluster of all Hailstorm related artifacts on your cluster, you can use the purge cluster option. The exact semantics of this command depend on the cluster type. If you are using Amazon AWS, this will terminate running instances, remove Hailstorm AMI and security groups from all regions configured in your environment. If you are sharing an Amazon account, you should not invoke this command.