Last updated: June 9th, 2014
The tool is just quick proof of concept but it might be useful when one doesn't have the time and resources to add integrated bruteforce detection to a possibly very complex web application stack. It's easy to filter for, say, very CPU-intensive URL's and set specific tresholds for those URL's. Two examples of usage of the tool are provided below; one in which a webserver is protected and another example in which TCP SYN scans are detected.
Since the tool does not keep track of which IP's were banned nor when, it cannot unban those IP's either. To get this functionality one can simply use fail2ban. After installing fail2ban open up /etc/fail2ban/jail.conf and add the following lines:
[brutedet] enabled = true filter = brutedet logpath = /var/log/brutedet.log action = iptables-allports maxretry = 1
Then create /etc/fail2ban/filter.d/brutedet.conf with the following content:
[Definition] failregex = <HOST>
Create the file /var/log/brutedet.log and make sure it's readable by the credentials fail2ban runs under.
Reload the fail2ban rules by running fail2ban-client reload and make sure there are no errors being displayed.
Now we are ready to start parsing HTTP log files and feed the data into the tool. First we disable any buffering and we run
a tail -f on the logfile of our webserver such as this:
$ stdbuf -i0 -o0 -e0 tail -f /var/log/fnord/current @40000000538e48a02e25609c x.x.x.x 200 3453 santarago.org:80 Mozilla/5.0_(Windows_NT_6.1;_WOW64;_rv:29.0)_Gecko/20100101_Firefox/29.0 none /brutedet.html @40000000538e48a13b676b9c x.x.x.x 304 0 santarago.org:80 Mozilla/5.0_(Windows_NT_6.1;_WOW64;_rv:29.0)_Gecko/20100101_Firefox/29.0 none /brutedet.html @40000000538e48af23f7c95c x.x.x.x 200 3467 santarago.org:80 Mozilla/5.0_(Windows_NT_6.1;_WOW64;_rv:29.0)_Gecko/20100101_Firefox/29.0 none /brutedet.html @40000000538e48e11a24871c x.x.x.x 200 3585 santarago.org:80 Mozilla/5.0_(Windows_NT_6.1;_WOW64;_rv:29.0)_Gecko/20100101_Firefox/29.0 none /brutedet.html
To extract the IP addresses and, for example, the user-agent data we pipe the preceding output through awk akin to:
$ stdbuf -i0 -o0 -e0 tail -f /var/log/fnord/current | \
awk -W interactive '{print $2 " " $6}'
x.x.x.x Mozilla/5.0_(Windows_NT_6.1;_WOW64;_rv:29.0)_Gecko/20100101_Firefox/29.0
x.x.x.x Mozilla/5.0_(Windows_NT_6.1;_WOW64;_rv:29.0)_Gecko/20100101_Firefox/29.0
x.x.x.x Mozilla/5.0_(Windows_NT_6.1;_WOW64;_rv:29.0)_Gecko/20100101_Firefox/29.0
x.x.x.x Mozilla/5.0_(Windows_NT_6.1;_WOW64;_rv:29.0)_Gecko/20100101_Firefox/29.0
The preceding data should now be piped into the tool. Let's settle for allowing a maximum of 5 requests from a specific IP every 10 seconds, a maximum of 10 requests every minute and a maximum of 20 requests every ten minutess. This means we run the tool with ./brutedet 5 10 20 <cmd>.
For the command which will be executed we need to output the IP address and the date as this is a fail2ban requirement. We specify the command as a long string. Before execution every occurence of the word KEY is replaced with the actual IP address. In the aforementioned fail2ban configuration we set up fail2ban such that it would read its input for the bruteforce detection jail from the file /var/log/brutedet.log. This means that the command needs to write out the IP and the date to that specific file. This results in a command like (printf 'KEY ' && date) >> /var/log/brutedet.log. The entire resulting command will then become the following:
$ stdbuf -i0 -o0 -e0 tail -f /var/log/fnord/current | \
awk -W interactive '{print $2 " " $6}' | \
./brutedet 5 10 20 "(printf 'KEY ' && date) >> /var/log/brutedet.log"
With the preceding command running we can now fire up a browser and request a url more than 5 times in 10 seconds. If everything went according to plan an entry will be written to /var/log/brutedet.log. This entry will be picked up by fail2ban which will automatically add a firewall rule using iptables and write a log entry to /var/log/fail2ban.log. After some time this IP will automatically become unbanned as per the fail2ban configuration:
2014-06-03 23:28:24,377 fail2ban.actions: WARNING [brutedet] Ban x.x.x.x 2014-06-03 23:38:24,797 fail2ban.actions: WARNING [brutedet] Unban x.x.x.xIf you want you can also have more advanced filters by inserting for example a
grep command with the appropriate filters in the command. Say that one has a webservice that has an API call which always takes 30 seconds to complete because it's very CPU/resource intensive. By filtering out these requests and only piping these through to the tool one has the ability to quickly do bruteforce detecting for these URL's without having to introduce this throttling functionality in the web application stack itself.
You can also use the tool to automatically detect SYN floods. By using someting like tshark one can filter out SYN packets quite easily and apply limits to how often a SYN packet will be accepted from a specific IP before the source IP will be blocked by fail2ban.
tshark -i eth0 -l -R "tcp.flags==0x02" "tcp" | \
awk -W interactive '{print $2 " " $2}' | \
./brutedet 5 10 20 "(printf 'KEY ' && date) >> /var/log/brutedet.log"
tar zxvf brutedet.tgz && cd brutedet && make && ./brutedetnot enough (or too many) arguments supplied ./brutedet [options] <t1> <t2> <t3> "echo added KEY" Parameters t1, t2, t3 are the integer tresholds for the 10 second-, 1 minute-, 10 minute-bucket respectively. The last argument is the command to execute once a treshold is passed. The word KEY will be replaced with the key in the bloomfilter. This will most likely be an IP address. Options: -c <capacity> bloom filter capacity (default: 100000) -e <error rate> allowed error rate(default: 0.010) -h help (this screen) The following example echo's a warning to a logfile and it will accept a maximum of 5 requests every 10 seconds, a maximum of 20 requests every minute and a maximum of 40 requests every 10 minutes. ./brutedet 5 20 40 "echo treshold reached for KEY >> /tmp/logfile.txt"