[Openswan Users] MTU problems - Openswan to freeswan

[Cameron Davidson]

I'll top post, because the original part is now just background detail.

I have sorted out my MTU problems, and thought I might add some 
troubleshooting hints to the OpenSwan Wiki, but will set them out here 
first to see if anyone can spot mistakes.

kernel 2.6, with NETKEY.
tunnelled traffic with DF set cause the ESP packet to set DF also.
Provided the kernel is recent enough (some threads suggest after 2.6.12) 
then ESP packets that are blocked and replied to by ICMP (fragmentation 
needed) will cause Pluto to reduce the MTU on the tunnel. The MTU chosen 
is dependent on the encryption algorithm.
If the MTU is known in advance then it can be permanently defined for 
the remote gateway using the "ip route" command.
"ip route" can also be used to define the tunnel mtu but I suspect it is 
only after the route has been created, so it might need to go in the 
up-down script.

kernel 2.4 with Klips (snapgear)
tunnelled traffic with DF set gets converted to ESP packets with DF 
cleared, so  packets in between tunnel mtu and ethernet mtu just get 
fragmented. A linux client will reply with DF cleared, but windows-XP 
keeps DF set, so it won't get back.

If the tunnel MTU's are not equal in both directions then this can lead 
to strange behaviour: TCP traffic will usually negotiate MSS values 
based on the lower mtu, but only after the local TCP stack on the lan 
client has learnt the reduced mtu value. Ping traffic through the large 
MTU end can go through to the target machine, but the reply (which is 
the same size) cannot get back as it also has DF set.

Cameron.

I originally  wrote:
> I wrote:
>> I don't understand how MTU issues within openswan are supposed to be 
>> resolved, and the problem for me is that they are not.
>> A vpn over mtu=1500 connections reports a tunnel mtu of 1428 (via 
>> icmp reply to big ping packet),
>> while a link between an mtu=1500 site and one with mtu=1492 reports 
>> the tunnel mtu is 1444.
>>
>> For a long time I have had a 3-way vpn setup running between two adsl 
>> gateway boxes (call them A and B, which started as RH7.3) running 
>> various versions of superfreeswan and openswan v1, on various 2.4 
>> kernels. The third gateway (C) on cable is a Snapgear Lite+ v1.7.12.
>> Initially they all ran on links that were mtu=1500, and everything 
>> was fine.
>> These are all subnet-subnet tunnels.
>> Recently I upgraded A and B to Centos 4.3, and openswan to 2.4.5 
>> (using the netkey kernel IPSEC code).
>> I upgraded A first and it was able to establish a good vpn link to B, 
>> after a bit of tinkering with the config files. This link uses X509 
>> certificates.
>> Then I upgraded B and got that link working again.
>>
>> Now, the snapgear link has been unreliable because the cable 
>> operators have been changing IP address, even the nexthop address, at 
>> increasing frequencies, and the owner decided to go to adsl with a 
>> static IP, but is required to use PPPoE, using the client inside the 
>> snapgear. So the snapgear knows its wan port has mtu=1492, but the 
>> vpn mtu is set to 1444.
>> Any ping packet I send  from A to C (or B to C), with raw packet size 
>> between 1439 and 1444 gets encrypted as an ESP packet between 1493 
>> and 1500 bytes.  These cause an icmp fragmentation-needed reply from 
>> C's nexthop router, which seems to be ignored by  everything at end A.
>>
>> I suppose I could use the overridemtu option, without actually losing 
>> any capacity, but I don't understand why it is getting it wrong.
>>
>> One complicating factor was that I had to rebuild kernels on both 
>> Centos systems - A did not have my ethernet card, and B paniced in 
>> SMP mode.
>> System A now has a standard 2.6.16.18 kernel, and B has a rebuilt, 
>> stripped down rebuild of the Centos kernel 2.6.9-34.  There is always 
>> a chance I've left out something vital, but I can't see anything 
>> obvious.
>>
>
>
> Where I am so far...
> set overridemtu=1428 on the snapgear.
>
> To change mtu under OpenSwan v2+netkey I can make the "ip route 
> change" command work if I add all the existing parameters as well as 
> the new mtu.  Well, almost...
> "ip route" shows the new mtu on both OpenSwan v2 machines A and B. But 
> big pings get different replies. If I ping from machines on respective 
> subnets with a total packet length of over 1444 then:
>      gateway A replies saying too big, mtu is 1428,   but
>      gateway B replies saying too big, mtu is 1444.
>
> If wait long enough for the local machines to forget  their cached 
> path mtu values, and ping with a total packet length 1438 then:
>      gateway A replies saying too big, mtu is 1428,   but
>      gateway B passes it through to an ESP packet of 1496, which gets 
> a reply from the remote ISP's router saying too big, mtu is 1492. 
> Which is back where I started - Pluto and the kernel are both ignoring 
> path mtu.
>
> value of /proc/sys/net/ipv4/ip_no_pmtu_disc is zero on both machines A 
> and B.
>
> Surely there's an easier way, one that works.
>
> Cameron.
> _______________________________________________
> Users at openswan.org
> http://lists.openswan.org/mailman/listinfo/users
> Building and Integrating Virtual Private Networks with Openswan: 
> http://www.amazon.com/gp/product/1904811256/104-3099591-2946327?n=283155
>