Tuning Linux firewall connection tracker ip_conntrack
Overview
If your Linux server should handle lots of connections, you can get into the problem with ip_conntrack iptables module. It limits number of simultaneous connections your system can have. Default value (in CentOS and most other distros) is 65536.
To check how many entries in the conntrack table are occupied at the moment:
Or you can dump whole table :
Conntrack table is hash table (hash map) of fixed size (8192 entries by default), which is used for primary lookup. When the slot in the table is found it points to list of conntrack structures, so secondary lookup is done using list traversal. 65536/8192 gives 8 – the average list length. You may want to experiment with this value on heavily loaded systems.
Modifying conntrack capacity
To see the current conntrack capacity:
You can modify it by echoing new value there:
Changes are immediate, but temporary – will not survive reboot.
Modifying number of buckets in the hash table
As mentioned above just changing this parameter will give you some relief, if your server was at the cap, but it is not ideal setup. For 1M connections average list becomes 1048576 / 8192 = 128, which is a bit too much.
To see current size of hash table:
which is read-only aliase for module parameter:
You can change it on the fly as well:
Persisting the changes
Making these changes persistent is a bit tricky.
For total number of connection just edit
Not so easy with hashtable size. You need to pass parameters to kerenl module at boot time. Edit add to
Memory usage
You can find how much kernel memory each conntrack entry occupies by grepping /var/log/messages :
1M connections would require 304MB of kernel memory.
Overview
If your Linux server should handle lots of connections, you can get into the problem with ip_conntrack iptables module. It limits number of simultaneous connections your system can have. Default value (in CentOS and most other distros) is 65536.
To check how many entries in the conntrack table are occupied at the moment:
cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
Or you can dump whole table :
cat /proc/net/ip_conntrack
Conntrack table is hash table (hash map) of fixed size (8192 entries by default), which is used for primary lookup. When the slot in the table is found it points to list of conntrack structures, so secondary lookup is done using list traversal. 65536/8192 gives 8 – the average list length. You may want to experiment with this value on heavily loaded systems.
Modifying conntrack capacity
To see the current conntrack capacity:
cat /proc/sys/net/ipv4/netfilter/ip_conntrack_max
You can modify it by echoing new value there:
# echo 131072 > /proc/sys/net/ipv4/netfilter/ip_conntrack_max
# cat /proc/sys/net/ipv4/netfilter/ip_conntrack_max
131072
Changes are immediate, but temporary – will not survive reboot.
Modifying number of buckets in the hash table
As mentioned above just changing this parameter will give you some relief, if your server was at the cap, but it is not ideal setup. For 1M connections average list becomes 1048576 / 8192 = 128, which is a bit too much.
To see current size of hash table:
cat /proc/sys/net/ipv4/netfilter/ip_conntrack_buckets
which is read-only aliase for module parameter:
cat /sys/module/ip_conntrack/parameters/hashsize
You can change it on the fly as well:
#echo 32768 > /sys/module/ip_conntrack/parameters/hashsize
# cat /sys/module/ip_conntrack/parameters/hashsize
32768
Persisting the changes
Making these changes persistent is a bit tricky.
For total number of connection just edit
/etc/sysctl.conf (CentOs, Redhat etc) and you are done:
# conntrack limits
net.ipv4.netfilter.ip_conntrack_max = 131072
Not so easy with hashtable size. You need to pass parameters to kerenl module at boot time. Edit add to
/etc/modprobe.conf:
options ip_conntrack hashsize=32768
Memory usage
You can find how much kernel memory each conntrack entry occupies by grepping /var/log/messages :
ip_conntrack version 2.4 (8192 buckets, 65536 max) - 304 bytes per conntrack
1M connections would require 304MB of kernel memory.
No comments:
Post a Comment