DESCRIPTION
testcookie-nginx-module is a simple robot mitigation module using cookie based challenge/response.
Challenge cookies can be set using different methods:
* "Set-Cookie" + 302/307 HTTP Location redirect
* "Set-Cookie" + HTML meta refresh redirect
* Custom template, JavaScript can be used here.
To prevent automatic parsing, challenge cookie value can
be encrypted with AES-128 in CBC mode using custom/random key and iv,
and then decrypted at client side with JavaScript.
If use Redis feature, only IPs which are stored in Redis need to resolve challenge, remaining is allowed (allow all but block some)
We can use any Redis client to dynamically add or new IP to blacklist without reloading Nginx. Example:
## select DB which store blacklist
127.0.0.1:6379> SELECT 3
##add new key-value to block permanent
127.0.0.1:6379> SET malicious_ip malicious_ip
or block in some times:
127.0.0.1:6379>SETEX malicious_ip blocktime_in_seconds malicious_ip
### delete IP:
127.0.0.1:6379>DEL not_malicious_ip_anymore
DIRECTIVES
testcookie
on - enable module
off - disable module
var - don't intercept requests, only set cookie vars
testcookie_name
cookie name, default is TCK
testcookie_domain
cookie domain, default is none, set by browser
testcookie_expires
cookie expiration value, default 31 Dec 2037 23:55:55 GMT
testcookie_path
cookie path, useful if you plan to use different keys for locations. default is /
testcookie_secret
secret string, used in challenge cookie computation, should be 32 bytes or more,
better to be long but static to prevent cookie reset for legitimate users every server restart.
if set to "random" - new secret will be generated every server restart, not recomended(all cookies with previous key will be invalid),
testcookie_session
sets the challenge generation function input,
$remote_addr - clients IP address will be used as an user unique identifier
$remote_addr$http_user_agent - clients IP + User-Agent
* required configuration directive
testcookie_arg
GET parameter name, used for cookie setting attempts computation
if not set - server will try to set cookie infinitely(actually, browser will show the error page after 5 attempts).
testcookie_max_attempts
maximum number of redirects before user will be sent to fallback URL, according to RFC1945 can't be more than 5
if set to 0 or testcookie_arg not set - server will try to set cookie infinitely.
testcookie_p3p
P3P policy, default is none.
testcookie_fallback
sets the fallback URL, user will be redirected to after maximum number of attempts, specified by directive
testcookie_max_attempts exceded. nginx scripting variables can be used here.
if not set - client will get 403 after max attempts reached.
testcookie_whitelist
sets the networks for which the testing will not be used, add search engine networks here
currently IPv4 CIDR only.
testcookie_pass
variable name, if variable set to 1 cookie check will not be performed,
can be used for more complex whitelisting.
testcookie_redirect_via_refresh
set cookie and redirect using HTTP meta refresh, required if testcookie_refresh_template used (on|off)
default is off.
testcookie_refresh_template
custom html instead of simple HTTP meta refresh, you need to set cookie manually from the template
available all the nginx variables and
$testcookie_nexturl - URL the client should be redirected to
$testcookie_got - cookie value received from client, empty if no cookie or it does not match format
$testcookie_set - correct cookie value we're expecting from client
$testcookie_ok - user passed test (1/0). Note: changed from "yes"/"no" in v1.10
also, if testcookie_refresh_encrypt_cookie enabled there are three more variables
$testcookie_enc_key - encryption key (32 hex digits)
$testcookie_enc_iv - encryption iv (32 hex digits)
$testcookie_enc_sec - encrypted cookie value (32 hex digits)
testcookie_refresh_status
custom HTTP response status. 200 by default.
testcookie_deny_keepalive
close connection just after setting the cookie, no reason to keep connections with bots (on|off)
default is off.
testcookie_get_only
process only GET requests, POST requests will be bypassed (on|off)
default is off.
testcookie_https_location
redirect to https protocol after setting the cookie, also affects $testcookie_nexturl
useful with 3dparty SSL offload (on|off)
default is off.
testcookie_refresh_encrypt_cookie
encrypt cookie variable, used with testcookie_refresh_template to force client-side decryption
AES-128 CBC mode used (on|off)
default is off.
testcookie_refresh_encrypt_cookie_key
encryption key
possible values:
random - new key generated every nginx restart
32 hex digits - static key, useful if you plan to obfuscate it deep in client-side javascript
* required directive if encryption enabled
testcookie_refresh_encrypt_iv
encryption iv
possible values:
random - new iv generated for every client request
random2 - new iv generated for every nginx restart
32 hex digits - static iv, useful if you plan to obfuscate it deep in client-side javascript
default is random
testcookie_internal
enable testcookie check for internal redirects (on|off)
useful for this type of configs:
rewrite ^/(.*)$ /index.php?$1 last;
default is off.
testcookie_httponly_flag
adds HttpOnly flag for cookie (on|off)
default is off.
testcookie_secure_flag
adds Secure flag for cookie (on|off|$variable)
default is off.
any variable value except "off" interpreted as True.
testcookie_port_in_redirect
Keep server port in redirect (on|off)
default is off.
testcookie_redis
if enable, only IPs in logical database number which configured in testcookie_redisdb is required to verify, all are allow
on - enable module, using Redis to store blacklist IPs
off - disable module with Redis intergration
var - don't intercept requests, only set cookie vars
testcookie_redisip
Redis server's IP
testcookie_redisport
Redis port
testcookie_redisdb
Redis logical DB used for storing blacklist IPs
INSTALLATION
Install hiredis library, grab the nginx source code from nginx.org (<http://nginx.org/>), for
example, the version 1.1.15 (see nginx compatibility), and then build
the source with this module:
wget 'http://nginx.org/download/nginx-1.1.15.tar.gz'
tar -xzvf nginx-1.1.15.tar.gz
cd nginx-1.1.15/
./configure --add-module=/path/to/testcookie-nginx-module
make
make install
If you use nginx >= 1.9.11 you can compile Dynamic module.
wget 'http://nginx.org/download/nginx-1.9.11.tar.gz'
tar -xzvf nginx-1.9.11.tar.gz
cd nginx-1.9.11/
./configure --add-dynamic-module=/path/to/testcookie-nginx-module
make
make install
Then load "ngx_http_testcookie_access_module.so" using "load_module" directive.
For using client-side cookie decryption,
you need to manually grab SlowAES (<http://code.google.com/p/slowaes/>)
JavaScript AES implementation, patch it(utils/aes.patch) and put it to document root.
COMPATIBILITY
Module was tested with nginx 1.1+, but should work with 1.0+.
EXAMPLE CONFIGURATION
http {
#default config, module disabled
testcookie off;
#config for redis
testcookie_reis off;
testcookie_redisip 127.0.0.1;
testcookie_redisport 6379;
testcookie_redisdb 3;
#setting cookie name
testcookie_name BPC;
#setting secret
testcookie_secret keepmesecret;
#setting session key
testcookie_session $remote_addr;
#setting argument name
testcookie_arg ckattempt;
#setting maximum number of cookie setting attempts
testcookie_max_attempts 3;
#setting p3p policy
testcookie_p3p 'CP="CUR ADM OUR NOR STA NID", policyref="/w3c/p3p.xml"';
#setting fallback url
testcookie_fallback http://google.com/cookies.html?backurl=http://$host$request_uri;
#configuring whitelist
testcookie_whitelist {
8.8.8.8/32;
}
#setting redirect via html code
testcookie_redirect_via_refresh on;
#enable encryption
testcookie_refresh_encrypt_cookie on;
#setting encryption key
testcookie_refresh_encrypt_cookie_key deadbeefdeadbeefdeadbeefdeadbeef;
#setting encryption iv
testcookie_refresh_encrypt_cookie_iv deadbeefdeadbeefdeadbeefdeadbeef;
#setting response template
testcookie_refresh_template '<html><body>setting cookie...<script type=\"text/javascript\" src=\"/aes.min.js\" ></script><script>function toNumbers(d){var e=[];d.replace(/(..)/g,function(d){e.push(parseInt(d,16))});return e}function toHex(){for(var d=[],d=1==arguments.length&&arguments[0].constructor==Array?arguments[0]:arguments,e="",f=0;f<d.length;f++)e+=(16>d[f]?"0":"")+d[f].toString(16);return e.toLowerCase()}var a=toNumbers("$testcookie_enc_key"),b=toNumbers("$testcookie_enc_iv"),c=toNumbers("$testcookie_enc_set");document.cookie="BPC="+toHex(slowAES.decrypt(c,2,a,b))+"; expires=Thu, 31-Dec-37 23:55:55 GMT; path=/";location.href="$testcookie_nexturl";</script></body></html>';
server {
listen 80;
server_name test.com;
location = /aes.min.js {
gzip on;
gzip_min_length 1000;
gzip_types text/plain;
root /var/www/public_html;
}
location = /w3c/p3p.xml {
root /var/www/public_html;
}
location / {
#enable module for specific location
testcookie on;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:80;
}
location /dynamic_block {
#enable testcookie with redis support for this specific location, then we can dynamically add IP to block without reloading Nginx
testcookie_redis on;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:80;
}
}
}
TESTS SUITE
This module comes with a Perl-driven test suite.
Thanks to the Test::Nginx (<http://search.cpan.org/perldoc?Test::Nginx>) module in the Perl world.
SOURCES
Available on github at kyprizel/testcookie-nginx-module
(<http://github.com/kyprizel/testcookie-nginx-module>).
TODO
* Code review
* More encryption algos (-)
* Statistics (-)
BUGS
Feel free to report bugs and send patches to [email protected]
or use github's issue tracker(<http://github.com/kyprizel/testcookie-nginx-module/issues>).
SUPPORT THE PROJECT
Send your donations to 1FHmPTP6aDBAzVtM7Pe7Y69zqhjPRx847s
COPYRIGHT & LICENSE
Copyright (C) 2011-2017 Eldar Zaitov ([email protected]).
All rights reserved.
This module is licenced under the terms of BSD license.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the authors nor the names of its contributors
may be used to endorse or promote products derived from this
software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.