Limiting brute-force attacks with IPTables

Update: a more comprehensive, tiered version of the procedure below is outlined in Escalating Consquences with ipTables. The material below is, however still applicable, and forms the basis for many more advanced implementations.

For the past few weeks, I have noticed that the scale of attempted intrusions, on ssh, pop3, and ftp ports has increased dramatically, reaching levels as high as 10000 attempts over the course of an hour.

The following chronicles a few observations about the attacks, and my approach towards a solution. It is worth noting that it would be preferable to use Amazon’s security groups for blocking as opposed to IPTables, however, the former does not support custom rules of any sort – it is limited to simply opening and closing ports.

Attack Description

The majority of these attacks have originated from the US, Germany, Spain, and China (in no particular order). Given the small scale of the sites running on this server, combined with a relatively ‘peaceful’ first few months, I did not expect quite the extent of attempts I have recently seen. Fortunately, all attempts so far have been futile.

Since this article is on IPTables, I will try to keep the deviations to a minimum, but a few points are worth a mention:

  • The attempted logins on pop3 greatly resemble a dictionary attack
    • One IP tried common usernames (adm, apache, bin, calendar, cyrus, daemon, …, www) 9 times each followed by an alphabetical listing of names appear to have been tried (aaron, abdiel, abdullah, abel, abraham, abram, adam, adan, addison, aden, aditya, adolfo, adonis, adrian, …, zion), with 4 attempts for each, followed by a list of words and less common names, 1 time each.
    • Another tried the several usernames (Admin01, Admin02, Administrador, Administrateur, Administrator, Admins, backup, guest, mail, postmaster) 1020 times each
  • SSH login attempts were typically at the rate of 2/second, with 500 attempts made per IP address per day.
  • FTP login attempts were on a much smaller scale – only about 85 attempts on a single site’s FTP with some common username variations tried.

Needless to say, uncommon usernames and strong passwords would go a long way towards preventing a successful intrusion of this sort; as well as the use of key/certificate based authentication on SSH.

Solutions Considered

The preferable approach to defending against such an attack appears to be blocking the attack by use of a firewall, although, the implementation varies considerably.

I briefly looked into the following options, but decided against them:

  • pam_shield:
    • Limitation: only works with PAM
  • fail2ban, daemon_shield, BlockHosts
    • Limitation: All of the above scan log files and as such do not appear to be instant acting

The options above all add a rule to IPTables based on specified triggers. Of all of them, PAM_shield does look the most appealing (its trigger is a login failure), it is even possible to customize the IPTables rules that PAM_shield executes. However, on my setup, SSH doesn’t authenticate through PAM, and I don’t currently wish to change that.

Adding rules to IPTables and using the recent module was the method I decided on. It however, does have the limitation of not distinguishing between successful and unsuccessful login attempts.

Solution Implemented

Objective: if an IP address attempts to initiate a connection to ssh, pop3, imap, or ftp, more than 5 times in 60 seconds, or more than 15 times in 10 minutes, ban them from accessing all of the above ports for an hour (and extend the time if they continue to try – so that an hour without any attempts is necessary).

Note: most edits/commands listed below must be run as root (sudo), and some parts are specific to Amazon’s Linux AMI.

Background Information

Firstly, a couple of quick points about IPTables on Amazon’s Linux AMI:

  • lsmod displays the recent module as xt_recent (instead of ipt_recent)
  • The recent module creates files under the folder /proc/net/xt_recent (instead of /proc/net/ipt_recent)
  • IPTables runs under the syslog facility user (instead of kern)

IPTables can be temporarily (until restart) modified using the command iptables, or modifications can be made permanent by saving the changes to /etc/sysconfig/iptables.

Keep in mind that incorrect IPTables rules may lock you out of your server – a good recommendation that I came across was to setup a cron task to flush (iptables –F) the IPTables rules every few minutes while testing.

The Rules

I added my rules directly to /etc/sysconfig/iptables (these are modifications, not a complete rule set):

:ATTACKED - [0:0]
:ATTK_CHECK - [0:0]

-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p tcp -m multiport --dports 21,22,110,143 -m recent --update --seconds 3600 --name BANNED --rsource -j DROP
-A INPUT -p tcp -m multiport --dports 21,22,110,143 -m state --state NEW -j ATTK_CHECK

#OTHER PRE-EXISTING RULES
#...

-A ATTACKED -m limit --limit 5/min -j LOG --log-prefix "IPTABLES (Rule ATTACKED): " --log-level 7
-A ATTACKED -m recent --set --name BANNED --rsource -j DROP
-A ATTK_CHECK -m recent --set --name ATTK –-rsource
-A ATTK_CHECK -m recent --update --seconds 600 --hitcount 16 --name ATTK --rsource -j ATTACKED
-A ATTK_CHECK -m recent --update --seconds 60 --hitcount 6 --name ATTK --rsource -j ATTACKED
-A ATTK_CHECK -j ACCEPT

After completing your edits, restart iptables:

service iptables restart

If you add the above rules using the iptables command:

  • you can view the current rules using: iptables –S (or a more ‘explained’ version with iptables –L)
  • you can save the rules to /etc/sysconfig/iptables using: iptables-save > /etc/sysconfig/iptables

You will need to specify the correct positions in order to have the first rules added to the top of the list.

Explanations

(Rules precede the explanation in each case)

:ATTACKED - [0:0]
:ATTK_CHECK - [0:0]

Create two new chains (ATTACKED and ATTK_CHECK) that will be referenced later.

-A INPUT -i lo -j ACCEPT

Accept all input on the loopback interface, so that local services (which might just make more requests than ‘permitted’ don’t get locked out); adding source (sport) and destination (dport) values of 127.0.0.1 might be desired, but seems redundant. Since rules are carried out in order, this ‘accept all’ rule must precede other rules that could drop the packets.

-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

Accept packets from connections that have already been established – this will save a bit of processing power, but perhaps, more importantly, will allow not close an already open connection if another connection (from the same source) exceeds the limit. One use is keeping an ssh session open for testing, and exceeding the limit by opening new ssh sessions. Without this rule, the original session will be locked out as soon at the rule is triggered, while with this, the successfully established sessions can continue, and only the one that exceeds the limit (and subsequent attempts) will be blocked.

-A INPUT -p tcp -m multiport --dports 21,22,110,143 -m recent --update --seconds 3600 --name BANNED --rsource -j DROP

Drop packets from IPs on the banned list (those that are recorded by the ATTACKED chain), to the selected ports, until 1 hour without any connections (using –update) have elapsed. We want this rule before other checks and accepts, so that banned IPs will be dealt with here.

-A INPUT -p tcp -m multiport --dports 21,22,110,143 -m state --state NEW -j ATTK_CHECK

For all new connections on the selected ports, jump to the ATTK_CHECK chain – we are going to check the new (i.e. syn set) packets against the recent connections list. If we have made it this far (not loopback and not banned), only then check the packet.

Chain: ATTACKED

All packets that enter this chain originate from IPs deemed ‘attackers’ (5+ connections/min; 15+ connections/10 min).

-A ATTACKED -m limit --limit 5/min -j LOG --log-prefix "IPTABLES (Rule ATTACKED): " --log-level 7

Log packets that have reached here (however, to prevent excess logging don’t exceed 5 log entries a minute), send to syslog with priority debug (log-level 7)

-A ATTACKED -m recent --set --name BANNED --rsource -j DROP

Drop all packets that have made it here and add IP to the banned list (so that further connection attempts are also dropped)

Chain: ATTK_CHECK

Check the packets that get here (new connections on specified ports) for repeated previous connections from the same IP.

-A ATTK_CHECK -m recent --set --name ATTK --rsource

Keep track of IPs (in file /proc/net/xt_recent/ATTK) to check for repeats

-A ATTK_CHECK -m recent --update --seconds 600 --hitcount 16 --name ATTK --rsource -j ATTACKED

If an IP has exceeded 16 checks in 10 minutes, treat it as an attack (goto the ATTACKED chain). This rule must precede the one below.

-A ATTK_CHECK -m recent --update --seconds 60 --hitcount 6 --name ATTK --rsource -j ATTACKED

If an IP has exceeded 5 checks in 1 minute, treat it as an attack

-A ATTK_CHECK -j ACCEPT

All packets we are not treating as an attack, we will accept

Setting up Logging

To setup the log for IPTables, add the following to /etc/syslog.conf:

user.debug          /var/log/iptables

and restart the syslog service:

service syslog restart

Note: Some other scripts/processes might also log as user.debug (for instance, Suhosin), so you could try a different log-level to see what works best for you.

Final Explanations and Results

A few final points:

  • The use of recent with –update as opposed to –rcheck is so that continued attempts to access the server, will keep resetting the time.
  • The use of DROP vs REJECT is seemingly frowned upon, but it doesn’t let the attacking machine know that the connection has been dropped – which will often slow down the attack somewhat, tie up the attacker’s resources a bit, and requires a slightly more sophisticated attack script to effectively overcome. If many people other than myself were legitimately logging onto the server, it might be better to use REJECT.

Following the implementation of these rules, I have seen a drop in the number of SSH and POP3 attempts, to about 10 per day. It appears that each IP tries once a day and once they are blocked, stop trying for that day.

Log Samples

For reference, sample log entries matching the ban and intrusion attempts are provided below:

/var/log/secure:

Jan 2 21:48:21 servername sshd[20071]: reverse mapping checking getaddrinfo for sub.example.com [aa.bb.cc.ddd] failed - POSSIBLE BREAK-IN ATTEMPT!
Jan 2 21:48:21 servername sshd[20072]: Received disconnect from aa.bb.cc.ddd: 11: Bye Bye

Attempts from the same IP were repeated at: 21:48:22 (twice), 21:48:23, 21:48:24 (total of 5 attempts logged)

/var/log/iptables:

Jan 2 21:48:24 servername klogd: [683891.966643] IPTABLES (Rule ATTACKED): IN=eth0 OUT= MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx SRC= aa.bb.cc.ddd DST=ee.fff.ggg.hhh LEN=60 TOS=0x00 PREC=0x00 TTL=46 DF PROTO=TCP SPT=35485 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0

This entry corresponds to a trigger of the ATTACKED chain, from the same IP above, on port 22 (SSH).

By cyberx86

Just a random guy who dabbles with assorted technologies yet works in a completely unrelated field.

11 comments

  1. My pop3 login attack history and analysis (January 2007 – March 2010):

    http://groups.google.com/group/alt.spam/browse_thread/thread/370eae8a2f00e5a4
    http://groups.google.com/group/alt.spam/browse_thread/thread/5b4329642e467fed
    http://www.webservertalk.com/archive209-2010-9-2293889.html

    Solution: Change your server’s pop3 port from 110 to something else (and make your user’s aware of the change). Or, if your server is behind a NAT router or modem, change your port-forwarding rules to forward your new pop3 port on your WAN side to port 110 on your LAN side.

    1. Thanks for the comment. The idea of changing the ports from a default port to a less common port does have merit. However, I don’t believe it is sufficient on its own – it is more a form of obscurity than security. If a high port number (>10k) is used, the probability of it being found by most scans is greatly lower, however, some client programs do not work well with non-standard ports; and some users would have difficulty if more steps are added to the setup. Frankly, I am of the opinion that if a popular internet service (e.g. Gmail), which is likely to be the target of far more attacks than any little sites can maintain the use of standard ports and protocols, then there must be a way for the rest of us to do the same. Since implementing the IPTables rules, the number of attacks attempted in a day rarely exceeds 5 (I believe that the highest has been 15 – 3 attack attempts from different IPs). Since I was getting attacks on FTP and SSH as well (of these, FTP is the most commonly used, although, SSH was the most commonly attacked), the IPTables rules have helped with attacks on all fronts. I would agree however, that a service like SSH (which most users do not have access to) would merit from a layered approach, where an obscure port number is used as a first line of defense, and IPTables as a second line against attacks.

  2. Minor typo:
    -A ATTK_CHECK -m recent –set –name ATTK –rsource
    (one dash in -rsource)

    should read
    -A ATTK_CHECK -m recent –set –name ATTK –-rsource
    (TWO dashes in –rsource)

    Thanks for the info. Works great! (with change)

    1. Glad you found it useful. Thanks for pointing out the typo, I have corrected it. I found that certain circumstances caused the single level rule to be ineffective (specifically a slower, prolonged attack). I have a revised version of the rules above, which thus far have proven to be be very effective. The updated rule set can be found at: Escalating Consequences with IPTables.

  3. I stumbled across this while researching iptables and what was needed to block brute force attacks. This is perfect for what I want to accomplish. When I was testing, after purposely locking myself out, I wanted to list the IPs blocked – I tried several variations of “iptables -L…”. While those variations did list the rules, it didn’t list what IPs where blocked. The only place I could actually find that was in /proc/net/xt_recent/BANNED and/or in /var/log/iptables. Should I be able to see the banned IPs via iptables? If so, what is the correct command to use?

    1. Good to know you found it useful. As you noted, iptables -L will only list the rules. As far as I know, there is no way to list the blocked IPs via iptables, and they can only be accessed via the the places you mentioned.

  4. Heres one for you when it comes to an attack – a timed attack. Basically one go every seventeen minutes that enables it to fly under the radar. Funny it stopped at 12.58pm. This was just as I was modifying one of your rules to catch it – that is if 3 scans in one hour its an attacker. I wish there was some better way of doing it, but hey-ho. But thanks for having this online, its good stuff.

  5. Hi there guys,

    could this also work for a webdav bruteforce protection?
    like:

    -A INPUT -p tcp -m multiport –dports 80,443,21,22,110,143 -m recent –update –seconds 3600 –name BANNED –rsource -j DROP
    -A INPUT -p tcp -m multiport –dports 80,443,21,22,110,143 -m state –state NEW -j ATTK_CHECK

    Or would that be to dream about?

Leave a comment

Your email address will not be published. Required fields are marked *