swatch - Perl-based system log watcher

While doing some webdesign, I noticed strange entries in lighttpd's access logs. After a short research, they turned out to be signatures of certain vulnerability scanners out there in the World Wild Web(TM).

But as every medal, this one has a frontside, too: one can use these log entries to blacklist the bad guys, and the following describes how I did that, using good old iptables and the nice and small swatch. But first things first.

Blacklisting a sender using iptables is as easy as entering

iptables -A INPUT -s <bad IP> -j DROP 

into one's command line. But for everything to be comfortable, it's better to use a dedicated chain for that:

iptables -A INPUT -j blacklist 

and add entries like so:

iptables -A blacklist -s <bad IP> -j DROP 

swatch is a small tool parsing (and optionally following) a file, triggering actions based on matching regular expressions.

A sample config will look like:

watchfor /::ffff:(.+) .* "GET \/w00tw00t\.at\.ISC\.SANS\.DFind:\) HTTP\/1\.1".*/
        exec sh /root/bin/blacklist_source.sh $1

watchfor /::ffff:(.+) .* "Morfeus Fucking Scanner".*/
        exec sh /root/bin/blacklist_source.sh $1

as easy as eatin' pancakes:

• watchfor specifies a new regular expression to match on
• exec makes swatch execute the given line

Using the power of regular expressions, one can specify groups (those enclosed in round quotes) and pass their value on using $N where N is a digit. As lighttpd also listens on IPv6, the matching for the source IP is a bit tricky here, as IPv6-mapped-IPv4 addresses are being used.

Finally, here comes the logic glueing it all together:

#!/bin/bash

# USAGE: blacklist_source.sh <IP>
# will add IP to the drop list of iptables

ipt="/sbin/iptables"
tbl="blacklist"
lock="/tmp/.blacklist_source.lock"

if ! dotlockfile -p -r 20 -l $lock; then
        logger -t "blacklist_source" "Aquiring lock failed, either there is something hanging or $lock is somehow broken"
        exit 1
fi

iplist="`$ipt -vnL $tbl | grep -v -e "^Chain" -e "^ pkts" | awk '{if ($9 == "0.0.0.0/0") print $8}'`"

[ -z $1 ] && exit 1

if [ -z "`echo $iplist | grep $1`" ]; then
        logger -t "blacklist_source" "Blacklisting IP $1"
        $ipt -A $tbl -s $1 -j DROP
        exit $?
fi

dotlockfile -u $lock
exit 0

There are some requirements the script has to fulfil, which is why it's a bit bigger than one might intuitively guess:

• swatch runs the script in the background, so when parsing the full log at once, several instances will be active in parallel. But as the script checks for already existing entries, this would break. So use dotlockfile to have it execute sequentially.
• the call to extract the blacklisted IPs is rather complicated, but it should check for most exactly just those entries the script added, not anything else to prevent troubles.

Actually, there are two ways of using the above:

• parse once, maybe on a regular base, over the whole file:

  smart -c <smart config> -f /var/log/lighttpd/access.log 

• continuous following the log, blacklisting IPs as they appear (RECOMMENDED):

  smart -c <smart config> -t /var/log/lighttpd/access.log --daemon 

  (note the -t instead of -f)

• Currently, I see no way of stopping a smart daemon without doing ugly hacks. This is because the original process runs tail internally and seems to exit only if this subprocess is being killed.
• Blacklist entries are never removed. There is an iptables extension adding an expire value to rules, but it's not (yet) available on my system. But it's pure existence prevents me from hacking something myself. ;)