This guide will show how to set up your server to request regular updates from ServerSnug. Be advised that this is shown for illustration purposes only. Your server might require a different approach.
In this guide you will learn how to set up a software firewall to filter IP, and to update it as new data is received
You may have ipset already installed in your server. You can check if it is already present by running the following command in your terminal:
sudo ipset -v
If ipset is not installed, you can install it using the server package manager. For instance, using apt from your terminal, you may type:
sudo apt-get install ipset
Once installed, you can check that it can be correctly invoked by running it as:
sudo ipset -v
You can now create the blocklist that ipset will be using to block IPs:
sudo ipset create blocklist hash:ip hashsize 2048 maxelem 1000000
The parameters hashsize and maxelem control the storage cost of the blocklist.
The first is the size of the hash and must be a power of 2. Smaller values reduce the storage requirements: the default value is 1024.
The second value is the maximum number of elements that can be stored in the list.
If your website uses a CDN, you can whitelist CDN IPs to avoid your website accidentally blacklisting the CDN IPs. If you are not using any CDN, you can skip this step.
The first step is to create two sets to store the CDN IPs:
# Create an IPv4 and IPv6 set with type hash:net.
sudo ipset create cdn_ipv4 hash:net family inet
sudo ipset create cdn_ipv6 hash:net family inet6
The next step is to create a list of IPs that can be loaded sequentially into the sets. The details of this step depend on the CDN being used. The following example
shows an implementation based on AWS Cloudfront. This implementation parses a JSON response using the jq that can be installed with:
sudo apt-get install jq
With the jq package installed, the response can be parsed to get the list of the CDN IPs:
JSON_FILE="ip-ranges.json"
JSON_URL="https://ip-ranges.amazonaws.com/${JSON_FILE}"
IPV4_FILE="ipv4_prefixes.txt"
IPV6_FILE="ipv6_prefixes.txt"
wget -N ${JSON_URL}
# Extract IPs
jq -r '.prefixes[] | select(.region=="GLOBAL" and .service=="CLOUDFRONT" and .ip_prefix) | .ip_prefix' "./${JSON_FILE}" > "${IPV4_FILE}"
jq -r '.ipv6_prefixes[] | select(.region=="GLOBAL" and .service=="CLOUDFRONT" and .ipv6_prefix) | .ipv6_prefix' "./${JSON_FILE}" > "${IPV6_FILE}"
The above bash code creates two files containing a list of IPs, which are now ready to be stored in the two whitelists
# Clear existing ipset entries
sudo ipset flush cdn_ipv4
sudo ipset flush cdn_ipv6
# Populate IPv4 set
while read -r ip; do
sudo ipset add cdn_ipv4 "$ip"
done < "${IPV4_FILE}"
# Populate IPv6 set
while read -r ip; do
sudo ipset add cdn_ipv6 "$ip"
done < "${IPV6_FILE}"
Since the CDN lists need to be updated regularly, the two code fragments above can be stored into a single script: cdn_update.sh which can be executed as root with
sudo bash cdn_update.sh
JSON_FILE="ip-ranges.json"
JSON_URL="https://ip-ranges.amazonaws.com/${JSON_FILE}"
IPV4_FILE="ipv4_prefixes.txt"
IPV6_FILE="ipv6_prefixes.txt"
wget -N ${JSON_URL}
# Extract IPs
jq -r '.prefixes[] | select(.region=="GLOBAL" and .service=="CLOUDFRONT" and .ip_prefix) | .ip_prefix' "./${JSON_FILE}" > "${IPV4_FILE}"
jq -r '.ipv6_prefixes[] | select(.region=="GLOBAL" and .service=="CLOUDFRONT" and .ipv6_prefix) | .ipv6_prefix' "./${JSON_FILE}" > "${IPV6_FILE}"
# Clear existing ipset entries
ipset flush cdn_ipv4
ipset flush cdn_ipv6
# Populate IPv4 set
while read -r ip; do
ipset add cdn_ipv4 "$ip"
done < "${IPV4_FILE}"
# Populate IPv6 set
while read -r ip; do
ipset add cdn_ipv6 "$ip"
done < "${IPV6_FILE}"
You can set up iptables explicitly, by allowing and denying access. The default set up for iptable is as follows:
# Reinit iptables
iptables -F
iptables -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
ip6tables -F
ip6tables -X
ip6tables -P INPUT ACCEPT
ip6tables -P FORWARD ACCEPT
ip6tables -P OUTPUT ACCEPT
You are expected to modify the above set-up according to your needs.
If you are behind a CDN, and you have set the cdn_ipv4 and cdn_ipv6 lists,
the lists can be added to iptables.
iptables -A INPUT -p tcp -m multiport --dports 80,443,22 -m set --match-set cdn_ipv4 src -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --sports 80,443,22 -m set --match-set cdn_ipv4 dst -j ACCEPT
ip6tables -A INPUT -p tcp -m multiport --dports 80,443,22 -m set --match-set cdn_ipv6 src -j ACCEPT
ip6tables -A OUTPUT -p tcp -m multiport --sports 80,443,22 -m set --match-set cdn_ipv6 dst -j ACCEPT
Finally, the blocklist can be added to iptables. The blocklist should be added after the CDN lists: as rules are checked in sequence, if an IP checks with the cdn lists, then the connection is accepted independently of whether the IP had been stored in the blocklist.
iptables -A INPUT -m set --match-set blocklist src -j DROP
Once this has been done, the ipset and iptables can be saved with the commands:
# Save ipset
ipset save > /etc/ipset.conf
# Save iptables rules
iptables-save > /etc/iptables/rules.v4
ip6tables-save > /etc/iptables/rules.v6