Skip to content

Commit

Permalink
Update a bunch of docs and examples
Browse files Browse the repository at this point in the history 
Move nl.js stuff under lib/nl
Move reporting stuff under lib/reporting
  • Loading branch information
@jonjlee
jonjlee committed Dec 1, 2010
1 parent 8ee881b commit a5e3f73
Show file tree
Hide file tree
Showing 30 changed files with 1,165 additions and 869 deletions.
8 changes: 4 additions & 4 deletions Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
.PHONY: clean templates compile
PROCESS_TPL = scripts/process_tpl.js
SOURCES = lib/header.js lib/*.tpl.js lib/template.js lib/config.js lib/util.js lib/stats.js lib/loop/loop.js lib/loop/multiloop.js lib/monitoring/collectors.js lib/monitoring/statslogger.js lib/monitoring/monitor.js lib/monitoring/monitorgroup.js lib/http.js lib/reporting.js lib/loadtesting.js lib/remote/endpoint.js lib/remote/endpointclient.js lib/remote/slave.js lib/remote/slaves.js lib/remote/slavenode.js lib/remote/cluster.js lib/remote/httphandler.js lib/remote/remotetesting.js
SOURCES = lib/header.js lib/config.js lib/util.js lib/stats.js lib/loop/loop.js lib/loop/multiloop.js lib/monitoring/collectors.js lib/monitoring/statslogger.js lib/monitoring/monitor.js lib/monitoring/monitorgroup.js lib/http.js lib/reporting/*.tpl.js lib/reporting/template.js lib/reporting/index.js lib/loadtesting.js lib/remote/endpoint.js lib/remote/endpointclient.js lib/remote/slave.js lib/remote/slaves.js lib/remote/slavenode.js lib/remote/cluster.js lib/remote/httphandler.js lib/remote/remotetesting.js

all: compile

clean:
rm -rf ./lib-cov
rm -f ./nodeload.js ./lib/*.tpl.js
rm -f ./nodeload.js ./lib/reporting/*.tpl.js
rm -f results-*-err.log results-*-stats.log results-*-summary.html

templates:
$(PROCESS_TPL) REPORT_SUMMARY_TEMPLATE lib/summary.tpl > lib/summary.tpl.js
$(PROCESS_TPL) DYGRAPH_SOURCE lib/dygraph.tpl > lib/dygraph.tpl.js
$(PROCESS_TPL) REPORT_SUMMARY_TEMPLATE lib/reporting/summary.tpl > lib/reporting/summary.tpl.js
$(PROCESS_TPL) DYGRAPH_SOURCE lib/reporting/dygraph.tpl > lib/reporting/dygraph.tpl.js

compile: templates
echo "#!/usr/bin/env node" > ./nodeload.js
Expand Down
569 changes: 0 additions & 569 deletions NODELOADLIB.md
View file

This file was deleted.

214 changes: 146 additions & 68 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,87 +1,165 @@
NODELOAD
INSTALLING
================

`nodeload` is both a **standalone tool** and a **`node.js` library** for load testing HTTP services.
Using [npm](http://npmjs.org/):

curl http://npmjs.org/install.sh | sh # install npm if not already installed
npm install nodeload

See [NODELOADLIB.md](http://github.com/benschmaus/nodeload/blob/master/NODELOADLIB.md) for using `nodeload` as a `node.js` library.
From source:

See [NODELOAD.md](http://github.com/benschmaus/nodeload/blob/master/NODELOAD.md) for instructions on using the standalone load test tool.
git clone git://github.com/benschmaus/nodeload.git
cd nodeload
npm link # optional. enables require('nodeload/<module>') instead of require('./lib/<module>').

Or as a single file (this does not install the `nl.js` tool):

wget https://github.com/benschmaus/nodeload/raw/master/nodeload.js

NODELOAD QUICKSTART
NODELOAD
================

1. Install node.js.
2. Clone nodeload.
3. cd into nodeload working copy.
4. git submodule update --init
5. Start testing!
`nodeload` is collection of [node.js](http://nodejs.org/) modules for load testing HTTP services.

nodeload contains a toy server that you can use for a quick demo.
Try the following:
As a developer, you should be able write load tests and get informative reports without having to learn another framework. You should be able to build tests by example and selectively use the parts of the tool that fit your task. Being a library means that you can use as much or as little of `nodeload` as makes sense, and you can create load tests with the power of a full programming language. For example, if you need to execute some function at a given rate, just use the [`'nodeload/loop'`](https://github.com/benschmaus/nodeload/tree/master/doc/loop.md) module, and write the rest yourself.

[~/code/nodeload] node examples/test-server.js &
[1] 2756
[~/code/nodeload] Server running at http://127.0.0.1:8080/
[~/code/nodeload] ./dist/nodeload.js -f -c 10 -n 10000 -i 1 -r ../examples/test-generator.js localhost:8080
In addition, `nodeload` is built for operability. It can always be deployed by simply copying the single file, `nodeload.js`.

You should now see some test output in your console. The generated webpage contains a graphical chart of test results.
Here are examples of each module, which can be used separately. Look for more examples in the [`examples/`](https://github.com/benschmaus/nodeload/tree/master/examples) directory and in test cases prefixed with "example" in [`test/`](https://github.com/benschmaus/nodeload/tree/master/test):

### [nl](https://github.com/benschmaus/nodeload/tree/master/doc/nl.md)

`nl` is an [Apache Bench (ab)](http://httpd.apache.org/docs/2.0/programs/ab.html) like command line tool for running tests quickly. See the [nl documentation](https://github.com/benschmaus/nodeload/tree/master/doc/nl.md) for details.

NODELOADLIB QUICKSTART
================
$ examples/test-server.js & # starts a simple server on port 9000 to load test
$ ./nl.js -c 10 -n 10000 -i 2 localhost:9000

will send 10,000 queries to http://localhost:9000 using 10 connections. Statistics are printed to the console and graphs can be seen at <http://localhost:8000/>.

### [nodeload](https://github.com/benschmaus/nodeload/tree/master/doc/nodeload.md)

The `nodeload` module is the primary interface for creating load tests. It includes all of the other modules described below, so if you `require('nodeload')`, you don't need to `require()` any of the other ones. Look at the examples in [`examples/loadtesting.ex.js`](https://github.com/benschmaus/nodeload/tree/master/examples/loadtesting.ex.js) and [`examples/riaktest.ex.js`]((https://github.com/benschmaus/nodeload/tree/master/examples/riaktest.ex.js) or read the [nodeload module documentation](https://github.com/benschmaus/nodeload/tree/master/doc/nodeload.md).

var nl = require('nodeload');
var loadtest = nl.run({
host: 'localhost',
port: 9000,
timeLimit: 60,
targetRps: 500,
requestGenerator: function(client) {
var request = client.request('GET', "/" + Math.floor(Math.random()*10000));
request.end();
return request;
}
});
loadtest.on('end', function() { console.log('Load test done.'); });

### [remote](https://github.com/benschmaus/nodeload/tree/master/doc/remote.md)

* **Write a load test:**
The `remote` module provides a mechanism for running a distributed load test. See the [remote module documentation](https://github.com/benschmaus/nodeload/tree/master/doc/remote.md).

$ vi example.js ## Add the following text to example.js
Start slave instances:

$ HTTP_PORT=10001 ./nodeload.js # start a local slave instance on :10001
$ HTTP_PORT=10002 ./nodeload.js # start a 2nd slave instance on :10002

// This test will hit localhost:8080 with 20 concurrent connections for 10 minutes.
var http = require('http'),
nl = require('./lib/nodeloadlib');

http.createServer(function (req, res) { res.writeHead(200); res.end(); }).listen(8080);
console.log("Server to load test listening on 8080.")

nl.runTest({
host: 'localhost',
port: 8080,
numClients: 20,
timeLimit: 600,
successCodes: [200],
targetRps: 200,
requestLoop: function(loopFun, client) {
var url = '/data/object-' + Math.floor(Math.random()*10000),
req = nl.traceableRequest(client, 'GET', url, { 'host': 'localhost' });
req.on('response', function(res) {
loopFun({req: req, res: res});
});
req.end();
}
});

$ node example.js ## while running, browse to http://localhost:8000
Listening on 8080.
Opening log files.
Started HTTP server on port 8000.
......done.
Finishing...
Shutdown HTTP server.

Browse to http://localhost:8000 during the test for graphs. Non-200 responses are logged to `results-{timestamp}-err.log`, `results-{timestamp}-stats.log` contains statistics, and the summary web page is written to `results-{timestamp}-summary.html`. Check out [examples/nodeloadlib-ex.js](http://github.com/benschmaus/nodeload/blob/master/examples/nodeloadlib-ex.js) for a example of a full read+write test.

* **Run a function at given rate:**

// Print 0..19 over 10 seconds
var nl = require('./lib/nodeloadlib').disableServer();
var i = 0;

new nl.Job({
rps: 2, // run 2 times/sec
duration: 10, // run for 10 seconds
fun: function(loopFun) {
console.log(i++);
loopFun();
}
Create the distributed load test:

var nl = require('nodeload/remote');
var cluster = new nl.LoadTestCluster('localhost:8000', ['localhost:8002', 'localhost:8001']);
cluster.run({
host: 'localhost',
port: 9000,
timeLimit: 60,
targetRps: 500,
requestGenerator: function(client) {
var request = client.request('GET', "/" + Math.floor(Math.random()*10000));
request.end();
return request;
}
});
cluster.on('end', function() { console.log('Load test done.'); });

### [stats](https://github.com/benschmaus/nodeload/tree/master/doc/stats.md)

The `stats` module provides implementations of various statistics objects, like Histograms and Accumulators, and functions, like randomString(), and nextGaussian(). See the [stats module documentation](https://github.com/benschmaus/nodeload/tree/master/doc/stats.md).

var stats = require('nodeload/stats');
var histogram = new stats.Histogram();
for (var i = 0; i < 1000; i++)
histogram.put(Math.abs(Math.floor(stats.nextGaussian())));
console.log('Mean: ' + histogram.mean() + ', 99%: ' + histogram.percentile(0.99));

will output "`Mean: 0.852, 99%: 3`".

### [monitoring](https://github.com/benschmaus/nodeload/tree/master/doc/monitoring.md)

The `monitoring` module provides a way to track runtime statistics for code that is run concurrently. See the [monitoring module documentation](https://github.com/benschmaus/nodeload/tree/master/doc/monitoring.md).

var monitoring = require('nodeload/monitoring');
var monitor = new monitoring.Monitor('runtime');
function asyncFunction() {
var m = monitor.start();
setTimeout(function() { m.end(); }, Math.floor(Math.random()*1000));
}
for (var i = 0; i < 1000; i++) { asyncFunction(); }
process.on('exit', function() {
console.log('Median runtime (ms): ' + monitor.stats['runtime'].percentile(0.5));
});

will output "`Median runtime (ms): 497`".

### [reporting](https://github.com/benschmaus/nodeload/tree/master/doc/reporting.md)

The `reporting` module provides a way to produce HTML graphs from code. See the [reporting module documentation](https://github.com/benschmaus/nodeload/tree/master/doc/reporting.md).

var reporting = require('nodeload/reporting'), stats = require('nodeload/stats'),
report = reporting.REPORT_MANAGER.addReport('Random Numbers'),
chart = report.getChart('Gaussian / Pareto vs. Time (minutes)');
for (var i = 0; i < 10; i++) {
setTimeout(function() {
chart.put({'Pareto': stats.nextPareto(0, 100), 'Gaussian': stats.nextGaussian()});
}, i * 500);
}

will display a graph on http://localhost:8000/ and save it to an HTML file in the local directory.

### [loop](https://github.com/benschmaus/nodeload/tree/master/doc/loop.md)

The `loop` module provides a way to execute a function at a set rate and concurrency. See [`test/loop.test.js`](https://github.com/benschmaus/nodeload/tree/master/test/loop.test.js) for examples and read the [loop module documentation](https://github.com/benschmaus/nodeload/tree/master/doc/loop.md) for details.

var http = require('http'),
loop = require('nodeload/loop'),
requests = 0,
client = http.createClient(80, 'www.google.com'),
l = new loop.MultiLoop({
fun: function(finished) {
client.request('GET', '/').end();
requests++;
finished();
},
rps: 10,
duration: 3,
concurrency: 5
}).start();
l.on('end', function() { console.log('Total requests: ' + requests) });

will output "`Total requests: 30`".

### [http](https://github.com/benschmaus/nodeload/tree/master/doc/http.md)

The `http` module provides a generic HTTP server that serves static files and that can be configured with new routes. See the [http module documentation](https://github.com/benschmaus/nodeload/tree/master/doc/http.md).

var http = require('nodeload/http');
var server = new http.HttpServer().start(10000);
server.addRoute('^/hello$', function(url, req, res) {
res.writeHead(200);
res.end("Hello");
});

<http://localhost:8000/package.json> will output the contents of `./package.json`, and <http://localhost:10000/resource> will display "Hello".


CONTRIBUTING
================
Contributions are always welcome. File bugs on [github](https://github.com/benschmaus/nodeload/issues), email any of the authors, and fork away! [developers.md](https://github.com/benschmaus/nodeload/tree/master/doc/developers.md) has brief instructions on getting tests up and running, and will hold more design details in the future.
19 changes: 19 additions & 0 deletions doc/developers.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Setting up

First, it's recommended that [`npm`](http://npmjs.org/) is installed. Just run:

[~/]> curl http://npmjs.org/install.sh | sh

The clone nodeload and run `npm link`

[~/]> git clone git://github.com/benschmaus/nodeload.git
[~/]> cd nodeload
[~/nodeload]> npm link

which will installs the unit testing framework [expresso](http://visionmedia.github.com/expresso) and puts a symlink to `nodeload` in the node library path.

Use expresso to run the tests under test/:

[~/nodeload]> expresso

100% 20 tests
Loading

0 comments on commit a5e3f73

Please sign in to comment.