Reduce TIME_WAIT socket connections in Apache/Linux

Some time in your life you’ll run across an Apache server that always has tons of TIME_WAIT connections just seeming to hang out. While these don’t take up as many resources as an ESTABLISHED connection, why keep them around so long? This short article will show you how to identify how many you have, and how to tell your server to reduce them, reuse and recycle them (see, recycling IS a good thing).

First, SSH into your server and become root.

Next, let’s see how many TIME_WAITs you have hanging out:

netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n

1	netstat -nat \| awk '{print $6}' \| sort \| uniq -c \| sort -n

You should see something like:

      1 established)
      1 Foreign
      3 FIN_WAIT2
      5 LAST_ACK
      6 CLOSING
      9 SYN_RECV
     10 ESTABLISHED
     22 FIN_WAIT1
     26 LISTEN
    466 TIME_WAIT

1 established)

1 Foreign

3 FIN_WAIT2

5 LAST_ACK

6 CLOSING

9 SYN_RECV

10 ESTABLISHED

22 FIN_WAIT1

26 LISTEN

466 TIME_WAIT

So – let’s get that number smaller.

See what your current values are in these files by catting them to the screen:

cat /proc/sys/net/ipv4/tcp_fin_timeout
cat /proc/sys/net/ipv4/tcp_tw_recycle
cat /proc/sys/net/ipv4/tcp_tw_reuse

cat /proc/sys/net/ipv4/tcp_fin_timeout

cat /proc/sys/net/ipv4/tcp_tw_recycle

cat /proc/sys/net/ipv4/tcp_tw_reuse

If you have default settings, you’ll probably see values of 60, 0 and 0. Let’s change those values to 30, 1, 1.

echo 30 &gt; /proc/sys/net/ipv4/tcp_fin_timeout
echo 1 &gt; /proc/sys/net/ipv4/tcp_tw_recycle
echo 1 &gt; /proc/sys/net/ipv4/tcp_tw_reuse

echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout

echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle

echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse

Now, let’s make the change persistent by adding them to the sysctl.conf file. First however, let’s make sure there aren’t any entries in there yet for these settings.. cat the file and grep for the changes we’re about to make:

cat /etc/sysctl.conf |grep "net.ipv4.tcp_fin_timeout"
cat /etc/sysctl.conf |grep "net.ipv4.tcp_tw_recycle"
cat /etc/sysctl.conf |grep "net.ipv4.tcp_tw_reuse"

cat /etc/sysctl.conf |grep "net.ipv4.tcp_fin_timeout"

cat /etc/sysctl.conf |grep "net.ipv4.tcp_tw_recycle"

cat /etc/sysctl.conf |grep "net.ipv4.tcp_tw_reuse"

Make notes of what your settings are if you had any results..

Now, edit the /etc/sysctl.conf with your favorite editor and add these lines to the end of it (or edit the values you have in yours if they exist already):

# Decrease TIME_WAIT seconds
net.ipv4.tcp_fin_timeout = 30

# Recycle and Reuse TIME_WAIT sockets faster
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1

# Decrease TIME_WAIT seconds

net.ipv4.tcp_fin_timeout = 30

# Recycle and Reuse TIME_WAIT sockets faster

net.ipv4.tcp_tw_recycle = 1

net.ipv4.tcp_tw_reuse = 1

Now, let’s rerun that command from before and see where your TIME_WAITs are at:

netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n

1	netstat -nat \| awk '{print $6}' \| sort \| uniq -c \| sort -n

You may need to wait at least a minute or so (depending on what your old values were) to see a change here.

Be sure to comment below if you have any questions or other thoughts!

4 responses to Reduce TIME_WAIT socket connections

RoseHosting June 13, 2014 at 4:36 am

Nice write-up. I’m sure it will be helpful.

Anyway, I know it’s not a big deal, but you should avoid using useless use of cat.

for example, this:

cat /proc/sys/net/ipv4/tcp_fin_timeout cat /proc/sys/net/ipv4/tcp_tw_recycle cat /proc/sys/net/ipv4/tcp_tw_reuse

1
2
3

cat /proc/sys/net/ipv4/tcp_fin_timeout
cat /proc/sys/net/ipv4/tcp_tw_recycle
cat /proc/sys/net/ipv4/tcp_tw_reuse

can be done like this:

egrep 'net.ipv4.tcp_(tw_(reuse|recycle)|fin_timeout)' /etc/sysctl.conf

1

egrep 'net.ipv4.tcp_(tw_(reuse|recycle)|fin_timeout)' /etc/sysctl.conf

thanks

- Rob June 18, 2014 at 11:12 am
  
  Thanks for the comment.. Yeah – I hear you, but wanted to keep it as simple as possible and so the reader (if they weren’t seasoned) understood what was happening since they’re going to be root when running most of these commands… 😉
  
- m January 8, 2015 at 8:14 pm
  
  You are assuming that the config file matches reality, which may not be the case, as these options can be changed dynamically.
  
Hary May 25, 2015 at 4:09 am

Many Thanks Brother