Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Completed Lua/Luvit version of the plugin. #3

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Completed Lua/Luvit version of the plugin. #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
eac573a
Completed Lua/Luvit version of LiteSpeed plugin.
Feb 21, 2015
2552fca
Made requirements tables less confusing.
Feb 21, 2015
34001e5
Added metrics.json.
Feb 21, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
369 changes: 190 additions & 179 deletions LICENSE
View file
Open in desktop

Large diffs are not rendered by default.

87 changes: 52 additions & 35 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,52 +1,69 @@
Boundary Litespeed Plugin
-------------------------
# Boundary Litespeed Plugin

Collects metrics from Litespeed HTTP Server instance.
Collects metrics from LiteSpeed HTTP Web-Server.

### Prerequisites
## Prerequisites

| OS | Linux | Windows | SmartOS | OS X |
|:----------|:-----:|:-------:|:-------:|:----:|
| Supported | v | - | v | v |
| Supported | yes | no | yes | yes |

- **OS**: Tested to work on **Debian-based Linux distributions** (although any Linux-based OS should work).
- Requires access to "/tmp/lshttpd" (LiteSpeed real-time reporting folder).

| Runtime | node.js | Python | Java |
### **Meter V4.0 or greater**

To get the new meter:

curl -fsS \
-d "{\"token\":\"<your API token here>\"}" \
-H "Content-Type: application/json" \
"https://meter.boundary.com/setup_meter" > setup_meter.sh
chmod +x setup_meter.sh
./setup_meter.sh

| Runtime | node.js | Python | Java |
|:---------|:-------:|:------:|:----:|
| Required | + | | |
| Required | no | no | no |

- [How to install node.js?](https://help.boundary.com/hc/articles/202360701)
### Meter less than V4.0

### Plugin Setup
None
| Runtime | node.js | Python | Java |
|:---------|:-------:|:------:|:----:|
| Required | yes | | |

- [How to install node.js?](https://help.boundary.com/hc/articles/202360701)

#### Plugin Configuration Fields
## Plugin Setup
None required.

### Plugin Configuration Fields
Litespeed writes server statistics to multiple report files in the /tmp/lshttpd folder by default, check your configuration to ensure that the path is correct. The number of reports are based on the number of CPUs on your server and the number of CPUs that you have licences for.

|Field Name |Description |
|:------------|:---------------------------------------|
|Report path |The path to the lshttp .rtreport files |
|Virtual Hosts|Include individual VHosts in your graphs|
|Field Name |Identifier |Type |Description |
|:--------------|:------------|--------------|:---------------------------------------------------------------------------------|
|Report Path |reportPath |string |The path to the LiteSpeed '.rtreport' file(s) to watch (default: '/tmp/lshttpd'). |
|Virtual Hosts |virtualHosts |array[string] |Include individual VHosts in your graphs. |
|Poll Interval |pollInterval |integer |How often (in milliseconds) to poll the metrics file(s) (default: 5000). |

### Metrics Collected
Tracks the following metrics for the [LiteSpeed](http://www.litespeedtech.com/) HTTP Web-Server.

Tracks the following metrics for the [litespeed](http://www.litespeedtech.com/) webserver.

|Metric Name |Description |
|:-------------------------------|:-------------------------------------------|
|Litespeed HTTP Connection limit |The ratio of connections to the Max Limit |
|Litespeed Http Conns |The number of current HTTP connections |
|Litespeed Idle Http Conns |The number of current idle HTTP connections |
|Litespeed HTTPS Connection limit|The ratio of connections to the Max Limit |
|Litespeed Https Conns |The number of current HTTPS connections |
|Litespeed Idle Https Conns |The number of current idle HTTPS connections|
|Litespeed Http Bytes In |The amount of inbound data over HTTP |
|Litespeed Http Bytes Out |The amount of outbound data over HTTP |
|Litespeed Https Bytes In |The amount of inbound data over HTTPS |
|Litespeed Https Bytes Out |The amount of outbound data over HTTPS |
|Litespeed Total Bytes In |The total amount of inbound data |
|Litespeed Total Bytes Out |The total amount of outbound data |
|Litespeed Cache Hits |The number of cache hits |
|Litespeed Cache Hit Ratio |The ratio of cache hits to overall requests |
|Litespeed Requests in Process |The number of requests in process |
|Litespeed Requests |The number of requests |
|Metric Name |Description |
|:--------------------------------|:---------------------------------------------|
|Litespeed HTTP Connection limit |The ratio of connections to the Max Limit. |
|Litespeed HTTP Connections |The number of current HTTP connections. |
|Litespeed Idle HTTP Connections |The number of current idle HTTP connections. |
|Litespeed HTTPS Connection limit |The ratio of connections to the Max Limit. |
|Litespeed HTTPS Connections |The number of current HTTPS connections. |
|Litespeed Idle HTTPS Connections |The number of current idle HTTPS connections. |
|Litespeed HTTP Bytes In |The amount of inbound data over HTTP. |
|Litespeed HTTP Bytes Out |The amount of outbound data over HTTP. |
|Litespeed HTTPS Bytes In |The amount of inbound data over HTTPS. |
|Litespeed HTTPS Bytes Out |The amount of outbound data over HTTPS. |
|Litespeed Total Bytes In |The total amount of inbound data. |
|Litespeed Total Bytes Out |The total amount of outbound data. |
|Litespeed Cache Hits |The number of cache hits. |
|Litespeed Cache Hit Ratio |The ratio of cache hits to overall requests. |
|Litespeed Requests in Process |The number of requests in process. |
|Litespeed Requests |The number of requests. |
4 changes: 2 additions & 2 deletions index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,8 +300,8 @@ function poll(cb)
console.log('LITESPEED_HTTP_BYTES_OUT %d %s', cur.BPS_OUT, _source);
console.log('LITESPEED_HTTPS_BYTES_IN %d %s', cur.SSL_BPS_IN, _source);
console.log('LITESPEED_HTTPS_BYTES_OUT %d %s', cur.SSL_BPS_OUT, _source);
console.log('LITESPEED_TOTAL_BYTES_IN %d %s', sum(cur.BPS_IN + cur.SSL_BPS_IN), _source);
console.log('LITESPEED_TOTAL_BYTES_OUT %d %s', sum(cur.BPS_OUT + cur.SSL_BPS_OUT), _source);
console.log('LITESPEED_TOTAL_BYTES_IN %d %s', sum(cur.BPS_IN, cur.SSL_BPS_IN), _source);
console.log('LITESPEED_TOTAL_BYTES_OUT %d %s', sum(cur.BPS_OUT, cur.SSL_BPS_OUT), _source);

// PER VHOST STATS
if (cur.vhosts && Object.keys(cur.vhosts).length > 0)
Expand Down
273 changes: 273 additions & 0 deletions init.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,273 @@
--
-- [boundary.com] Couchbase Lua Plugin
-- [author] Valeriu Paloş <[email protected]>
--

--
-- Imports.
--
local fs = require('fs')
local json = require('json')
local http = require('http')
local os = require('os')
local path = require('path')
local timer = require('timer')
local tools = require('tools')
local url = require('url')

--
-- Initialize.
--
local _buckets = {}
local _parameters = json.parse(fs.readFileSync('param.json')) or {}

local _reportPath = _parameters.reportPath or '/tmp/lshttpd'
local _virtualHosts = _parameters.virtualHosts or {}
local _pollInterval = tools.fence(tonumber(_parameters.pollInterval) or 5000, 100, 1000 * 60 * 60 * 24)

local _ignoredLines = { '^VERSION', '^UPTIME', '^BLOCKED_IP', '^EOF' }

--
-- Metrics source.
--
local _source =
(type(_parameters.source) == 'string' and _parameters.source:gsub('%s+', '') ~= '' and _parameters.source) or
os.hostname()

--
-- Scan paths.
--
local reports = {}
local file = path.join(_reportPath, '.rtreport')
if fs.existsSync(file) then
table.insert(reports, file)
end
for i = 0, #os.cpus() do
local file = path.join(_reportPath, '.rtreport' .. i)
if fs.existsSync(file) then
table.insert(reports, file)
end
end

--
-- Scan virtual hosts.
--
local virtualHosts = {}
if #_parameters.virtualHosts == 0 then
table.insert(_parameters.virtualHosts, '')
end
for _, item in ipairs(_parameters.virtualHosts) do
if item then
local vhost = item:match('^([^|]*)')
local alias = item:match('|(.*)$')

if virtualHosts[vhost] then
error(("The virtual host '%s' is defined twice and hosts must be distinct!"):format(vhost))
end

virtualHosts[vhost] = _source .. '-' .. tools.trim(alias or vhost)
end
end

--
-- Calculus utility functions.
--
function sum(a, b)
return (a and b and math.max(a + b, 0)) or a or b
end
function delta(a, b)
local na, nb = tonumber(a), tonumber(b)
return (na and nb and math.max(na - nb, 0)) or 0
end
function parseFloat(x)
return tonumber(x) or 0
end
function parseKeyValues(data)
local result = {}
for key, value in data:gmatch("([%w_]+):%s*(%d+)") do
result[key:upper()] = parseFloat(value)
end
return result
end

--
-- Schedule poll.
--
function schedule()
timer.setTimeout(_pollInterval, poll)
end

--
-- Print a metric.
--
function metric(stamp, id, value, source)
print(string.format('%s %s %s %d', id, value, source or _source, stamp))
end

--
-- Parse a report file.
--
function parseReport(file, callback)

-- Sample Report:
--
-- VERSION: LiteSpeed Web Server/Standard/4.2.21
-- UPTIME: 02:38:00
-- BPS_IN: 0, BPS_OUT: 0, SSL_BPS_IN: 0, SSL_BPS_OUT: 0
-- MAXCONN: 150, MAXSSL_CONN: 150, PLAINCONN: 0, AVAILCONN: 150, IDLECONN: 0, SSLCONN: 0, AVAILSSL: 150
-- REQ_RATE []: REQ_PROCESSING: 0, REQ_PER_SEC: 0.0, TOT_REQS: 0, CACHE_HITS_PER_SEC: 0.0, TOTAL_CACHE_HITS: 0
-- REQ_RATE [Example]: REQ_PROCESSING: 0, REQ_PER_SEC: 0.0, TOT_REQS: 0, CACHE_HITS_PER_SEC: 0.0, TOTAL_CACHE_HITS: 0
-- REQ_RATE [_AdminVHost]: REQ_PROCESSING: 0, REQ_PER_SEC: 0.0, TOT_REQS: 0, CACHE_HITS_PER_SEC: 0.0, TOTAL_CACHE_HITS: 0
-- BLOCKED_IP:
-- EOF
--
fs.readFile(file, function(failure, data)

-- Safety check.
if failure or not data then
print("ERROR: " .. failure)
return callback({})
end

-- Walk through lines.
local results = { ['virtualHosts'] = {} }
for line in tools.lines(data) do

-- Catch ignored lines.
local valid = true
for _, ignored in ipairs(_ignoredLines) do
if line:find(ignored) then
valid = false
break
end
end

-- Parse valid lines.
if valid then
local sink = results
local data = line

-- General metrics or virtual-host specifc?
local vhost, metrics = line:match('^REQ_RATE %[(.-)%]:(.*)')
if vhost then
data = ''

if virtualHosts[vhost] then
results.virtualHosts[vhost] = results.virtualHosts[vhost] or {}
sink = results.virtualHosts[vhost]
data = metrics
end
end

-- Parse metrics.
for key, value in pairs(parseKeyValues(data)) do
sink[key] = (sink[key] or 0) + value
end
end

end

callback(results)
end)
end

--
-- Collect and aggregate reports.
--
function collectReports(callback)
local sets = {}
local remaining = #reports

-- Add results to the stack.
function aggregate(values)
table.insert(sets, values)
if remaining == 0 then

-- Aggregate if all reports are parsed.
local store = { ['virtualHosts'] = {} }
for _, metrics in ipairs(sets) do

-- Walk generic values.
for name, value in pairs(metrics) do
if name ~= 'virtualHosts' then
store[name] = (store[name] or 0) + value
end
end

-- Walk virtual-host-specific values.
for vhost, vhMetrics in pairs(metrics.virtualHosts) do
store.virtualHosts[vhost] = store.virtualHosts[vhost] or {}
for name, value in pairs(vhMetrics) do
store.virtualHosts[vhost][name] = (store.virtualHosts[vhost][name] or 0) + value
end
end

end

callback(store)
end
end

-- Trigger asynchronous parsing of report files.
for _, report in ipairs(reports) do
parseReport(report, function(values)
remaining = remaining - 1
aggregate(values)
end)
end

end

--
-- Produce metrics.
--
function poll()
local stamp = os.time()

-- Trigger one collection.
collectReports(function(metrics)
local httpConnLimit = (metrics.MAXCONN == 0 and 0) or (metrics.PLAINCONN / metrics.MAXCONN)
local httpsConnLimit = (metrics.MAXSSL_CONN == 0 and 0) or (metrics.SSLCONN / metrics.MAXSSL_CONN)

metrics.BPS_IN = metrics.BPS_IN * 1024;
metrics.BPS_OUT = metrics.BPS_OUT * 1024;
metrics.SSL_BPS_IN = metrics.SSL_BPS_IN * 1024;
metrics.SSL_BPS_OUT = metrics.SSL_BPS_OUT * 1024;

-- Generic metrics.
metric(stamp, 'LITESPEED_HTTP_CONNECTION_LIMIT', httpConnLimit)
metric(stamp, 'LITESPEED_HTTP_CONNECTIONS', metrics.PLAINCONN)
metric(stamp, 'LITESPEED_HTTP_IDLE_CONNECTIONS', metrics.IDLECONN)
metric(stamp, 'LITESPEED_HTTPS_CONNECTION_LIMIT', httpsConnLimit)
metric(stamp, 'LITESPEED_HTTPS_CONNECTIONS', metrics.SSLCONN)
metric(stamp, 'LITESPEED_HTTPS_IDLE_CONNECTIONS', delta(metrics.AVAILSSL, metrics.SSLCONN))
metric(stamp, 'LITESPEED_HTTP_BYTES_IN', metrics.BPS_IN)
metric(stamp, 'LITESPEED_HTTP_BYTES_OUT', metrics.BPS_OUT)
metric(stamp, 'LITESPEED_HTTPS_BYTES_IN', metrics.SSL_BPS_IN)
metric(stamp, 'LITESPEED_HTTPS_BYTES_OUT', metrics.SSL_BPS_OUT)
metric(stamp, 'LITESPEED_TOTAL_BYTES_IN', sum(metrics.BPS_IN, metrics.SSL_BPS_IN))
metric(stamp, 'LITESPEED_TOTAL_BYTES_OUT', sum(metrics.BPS_OUT, metrics.SSL_BPS_OUT))

-- Virtual-host specific metrics.
for vhost, vhMetrics in pairs(metrics.virtualHosts) do
local virtualHost = metrics.virtualHosts[vhost];
local virtualName = virtualHosts[vhost];

local cacheHits = virtualHost.CACHE_HITS_PER_SEC or 0;
local requests = virtualHost.REQ_PER_SEC;
local requestsInProcess = virtualHost.REQ_PROCESSING or 0;
local cacheRatio = (requests == 0 and 0) or (cacheHits / requests);

metric(stamp, 'LITESPEED_CACHE_HITS', cacheHits, virtualName)
metric(stamp, 'LITESPEED_CACHE_RATIO', cacheRatio, virtualName)
metric(stamp, 'LITESPEED_REQUESTS', requests, virtualName)
metric(stamp, 'LITESPEED_REQUESTS_IN_PROCESS', requestsInProcess, virtualName)
end

-- Reschedule.
schedule()
end)
end

-- Trigger polling.
poll()
Loading