Installing and using ipset for blacklisting


#1

#Description
Ipset is an extension to iptables that allows for nearly constant-time ip blacklisting. Instead of managing thousands if individual iptables REJECT rules, you can simply reference an ipset with a single line like:

-A INPUT -i eth0 -m set --match-set blocklist src,dst -j DROP

It’s very fast compared to just iptables:

##Install ipset

apt-get update
apt-get install ipset

Try creating and viewing an ipset with:

ipset create test hash:net
ipset list

If it works without error (you should see ‘Name: test’ and a bunch of other info about your test ipset), you’re golden and you can continue. Otherwise, skip to Building the ipset module. As of this writing, the ipset kernel module (xt_set) is not included with the C1 kernels.

To get started, feel free to refer to the script I use to keep my blacklists up to date:

#!/bin/bash
BLACKLISTS=/data/blacklists/*

cd /data
python BlacklistDownloader.py
ipset create -exist blocklist hash:net
ipset flush blocklist
for f in $BLACKLISTS
do
  ipset restore < $f
done
iptables-restore < /data/iptables.rules

And its associated Python parsing script:
BlacklistDownloader.py

#Building the ipset module (on a C1):

This was written using a fresh install of the Ubuntu Wily (15.10) image. Adjust as needed for your distro of choice.

We need to get enough of the kernel source prepared to build the module.

##Install ipset and get build dependencies
This will install the ipset binaries and prepare you for building the kernel module.

apt-get install build-essential
apt-get build-dep ipset

##Get kernel source:
Figure out your kernel version: type ‘uname -r’. For me it was 4.1.6-249. Drop anything after the ‘-’. In this example, we get 4.1.6. Set KERNEL_RELEASE_VERSION to this number.

(Slightly modified from: https://github.com/scaleway/kernel-tools/blob/master/README.md#how-to-build-a-custom-kernel-module )

KERNEL_RELEASE_VERSION=4.1.6  # Set this to your kernel version
wget https://kernel.org/pub/linux/kernel/v4.x/linux-${KERNEL_RELEASE_VERSION}.tar.xz
tar xf linux-${KERNEL_RELEASE_VERSION}.tar.xz
mv linux-${KERNEL_RELEASE_VERSION} /usr/src
ln -s /usr/src/linux-${KERNEL_RELEASE_VERSION} /lib/modules/$(uname -r)/build
ln -s /lib/modules/$(uname -r) /lib/modules/${KERNEL_RELEASE_VERSION}  # For the actual module install

##Prepare kernel:

cd /lib/modules/$(uname -r)/build
zcat /proc/config.gz > .config
wget http://mirror.scaleway.com/kernel/$(uname -r)/Module.symvers

At this point, we need to remove something from the config that breaks the build process. Open .config with your favorite editor and look for the line that starts with ‘CONFIG_CROSS_COMPILE=’. Remove it or comment it out. Now type:

make prepare modules_prepare

It will prompt you with ‘Cross-compiler tool prefix (CROSS_COMPILE) [] (NEW)’. Just press enter.

At this point, the kernel source is ready for compiling modules.

##Get ipset source:
Now we need to get the ipset sources so we can build and install the kernel module. You can get it from Git (git://anonscm.debian.org/collab-maint/ipset.git), but I suggest simply using apt-get because it’s easier and you should get the version appropriate for your distro:

cd ~
apt-get source ipset
cd ipset-6.25.1

The ipset version will probably change as it gets updates, so change the version number according to what you download.

##Build ipset module:
From the directory containing the ipset sources, run the following commands:

./autogen.sh
./configure
make modules
make modules_install

##Fix depmod:
The depmod shipped in my image doesn’t look in the /extra directory, which is where ‘make modules_install’ puts the kernel module. If you get a message that looks like this:

modinfo: ERROR: Module ip_set_hash_ip not found.

!!! WARNING !!! WARNING !!! WARNING !!!

Your distribution seems to ignore the /lib/modules/<kernelrelease>/extra/
subdirectory, where the ipset kernel modules are installed.

Add the ‘extra’ directory to the search definition of your depmod
configuration (/etc/depmod.conf or /etc/depmod.d/) and re-run

depmod <kernelrelease>

Then neither does yours. I had to edit /etc/depmod.d/ubuntu.conf and add ‘extra’ to the end of the line, so it looked like:

search updates ubuntu built-in extra

Modify yours as needed, then run depmod <kernelrelease> as instructed.

At this point, if you run:

modprobe xt_set
lsmod

you should not get any errors, and you should see ‘xt_set’ as well as a few other related modules in the list. You’re now ready to follow the instructions at the beginning of this tutorial to set up your ipsets!

It’s possible that the version you compiled the kernel module for isn’t the same as the one you downloaded earlier from the repositories. If you’re having problems, remove the one that was installed earlier (apt-get purge ipset) and do ‘make && make install’ from the downloaded ipset folder.