[Openswan Users] kernel memory leak 2.4.7, 2.4.33 - misconfiguration or bug?

[Brad Langhorst]

I'm seeing a pretty big memory leak using openswan in a 1-1 vpn
deployment.

The leak is correlated with traffic over ipsec0.
see:  
https://development.coopmetrics.coop/munin/mcgruff/mcgruff.html

you can see that the free memory decreases quickly during the nightly
backup.

I have to reboot every few days or the machine runs out of ram and
becomes unstable.

The bad news is that this machine is in production, and I can't take it
down any time soon.  I also don't have  a tool chain in place to build
the packages for this system (bering 3.0 ulibc), but I think I'm going
to need to build one to solve this and will allocate some time to do
that.

I'm pretty sure that it's ipsec related because I don't lose memory when
doing a big scp transfer NOT via the vpn.

I have a similar system on the other side of the tunnel that does not
exhibit the lost memory problem.

LEAF Bering-uClibc 2.3 uClibc 0.9.20 Rev 2
Linux cujo 2.4.32 #1 Sat Mar 4 21:00:13 CET 2006 i686 unknown
ipsec           2.4.4 Rev 4    Openswan IPSEC 


What should I do to narrow down this problem?


memory and config below:


here's my config:

version 2.0     # conforms to second version of ipsec.conf specification

# basic configuration
config setup
        # plutodebug / klipsdebug = "all", "none" or a combation from
below:
        # "raw crypt parsing emitting control klips pfkey natt x509
private"
        # eg:
        # plutodebug="control parsing"
        #
        # Only enable klipsdebug=all if you are a developer
        #
        # NAT-TRAVERSAL support, see README.NAT-Traversal
        nat_traversal=yes
        virtual_private=%v4:10.0.0.0/8,%v4:172.16.0.0/12,%
v4:192.168.0.0/16,%v4:!192.168.3.0/24

conn cm-homeoffice
        right=vpn.coopmetrics.coop
        rightsubnet=192.168.3.0/24
        rightid="C=US, ST=NC, L=Carrboro, O=CoopMetrics, OU=VPN server,
CN=vpn.coopmetrics.coop"
        left=%defaultroute
        #left=192.168.0.2
        #leftnexthop=192.168.0.1
        leftsubnet=192.168.0.0/24
        leftcert=mcgruff_cert.pem
        leftsendcert=always
        rightsendcert=yes
        auto=start
        pfs=yes


Here's some memory info (note that the free memory decreases, but no
userpsace memory shows an increase in vmsize)

mcgruff# uname -a
Linux mcgruff 2.4.33 #1 Mon Sep 4 15:52:08 CEST 2006 i686 unknown

mcgruff# ps aux
  PID  Uid     VmSize Stat Command
    1 root        244 S   init [2]   
    2 root            SW  [keventd]
    3 root            SWN [ksoftirqd_CPU0]
    4 root            SW  [kswapd]
    5 root            SW  [bdflush]
    6 root            SW  [kupdated]
20110 root        268 S   /sbin/syslogd -m 240 
 9531 root        332 S   /sbin/klogd 
19053 root            SW  [khubd]
23869 root        244 S   /sbin/dhcpcd-bin -Y -N -R -d eth0 
16786 root        136 S   /usr/sbin/watchdog 
14975 root        232 S   /usr/sbin/inetd 
  914 root        272 S   /usr/sbin/ulogd -d 
 8127 root        956 S   /usr/sbin/sshd 
11645 root        420 S   /usr/sbin/ntpd -g 
10743 dnscache   1224 S   /usr/bin/dnscache 
 2076 root        288 S   /usr/bin/ez-ipupdate -c /etc/ez-ipupd.conf
-F /var/run/ez-ipupd.pid 
 1016 root        308 S   /usr/sbin/cron 
  254 root      13376 S   /usr/sbin/snmpd -Lsd -Lf /dev/null
-p /var/run/snmpd.pid 
16747 root        288 S   /sbin/getty 38400 tty1 
29709 root        288 S   /sbin/getty 38400 tty2 
31535 root        420 S   /usr/sbin/ntpd -g 
18574 root       1216 S   /usr/sbin/sshd: root at ttyp0
 5204 root        404 S   -sh 
22527 root        340 S   /bin/sh /usr/lib/ipsec/_plutorun --debug
--uniqueids yes --nocrsend  --strictcrlpolicy  --nat_trave
25116 root        296 S   logger -s -p daemon.error -t ipsec__plutorun 
30277 root        344 S   /bin/sh /usr/lib/ipsec/_plutorun --debug
--uniqueids yes --nocrsend  --strictcrlpolicy  --nat_trave
27023 root        340 S   /bin/sh /usr/lib/ipsec/_plutoload --wait no
--post  
32018 root        844 S   /usr/lib/ipsec/pluto --nofork
--secretsfile /etc/ipsec.secrets --ipsecdir /etc/ipsec.d --use-auto --
 2607 root        476 S N pluto helper  #  0
-nofork                                                                       
 6092 root        132 S   _pluto_adns 
 1810 root        284 R   ps aux 
mcgruff# free
              total         used         free       shared      buffers
  Mem:       119664       103964        15700            0           56
 Swap:            0            0            0
Total:       119664       103964        15700

mcgruff# cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  122535936 106496000 16039936        0    57344 14569472
Swap:        0        0        0
MemTotal:       119664 kB
MemFree:         15664 kB
MemShared:           0 kB
Buffers:            56 kB
Cached:          14228 kB
SwapCached:          0 kB
Active:           9680 kB
Inactive:         4660 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       119664 kB
LowFree:         15664 kB
SwapTotal:           0 kB
SwapFree:            0 kB


-- 
Brad Langhorst
CTO - CoopMetrics