[Openswan Users] Re: [Openswan dev] "auth failed on incoming packet" when using NAT-T

Markus Hanauska hanauska at equinux.de
Tue Dec 7 15:37:22 CET 2004

HAH! Nevermind, got it working!!! :)

Thanks to all people who tried to help me. The problem was indeed with 
my wrapper and Linux being to picky.

The UDP header field was set incorrectly and thus the package was 
truncated. This seems to be a problem with Linux and thus the Hash 
calculation failed. The other vendors are not that picky, they just 
strip out the UDP package, ignoring all data of the UDP header (they 
only care for the length in the IP header and the frame size).

You guys are great, keep up the good work and sorry for the headache. I 
hope to see a stable release with KLIPS for 2.6 kernel in the near 

Thanks a bunch!

On 07. Dez 2004, at 15:15 Uhr, Markus Hanauska wrote:

> On 07. Dez 2004, at 14:58 Uhr, Ken Bantoft wrote:
>> Try disabling compression on the racoon side, and 'compress=no' on the
>> Openswan side for your conn.
> It is disabled on the Racoon side by default, I disabled it on the 
> OpenSWAN side but no effect. Same error as before.
> The config ist like described in the readme, with %any as right and a 
> chosen address of as right net (right is remote in my 
> case). Left is bound to the address of one NIC of the Linux machine 
> and the net is the net behind the other NIC with the private IP. 
> Routes are created correctly.
> If I ping from Racoon to OpenSWAN side, packages are discarded with 
> the error message of before. If I ping from behind the OpenSWAN 
> Gateway, pings are going through the OpenSWAN gateway, become ESP and 
> are encapsulates in UDP packages port 4500, they arrive at my Racoon 
> side, where my wrapper decapsulates them (no error here), the ESP 
> layer decrypts them (no error either), the ping is answered (echo 
> request), the answer is again encrypted to ESP (no error), 
> encapsulated by my wrapper within UDP port 4500 and send out. Finally 
> it arrives at the OpenSWAN gateway and is thrown away, saying it can't 
> be authenticated.
> I checked the OpenSWAN code and from what I can see, this error 
> appears if the calculated hash does not match the hash within the 
> package. But how can NAT-T have an influence on the hash or how it is 
> calculated?
> Neither the hash, nor the hashed data is touched during en- and 
> de-capsulation. Both sides use HMAC-MD5 for hashing and they both use 
> the same HASH key methinks. At least on the Racoon side, that is easy 
> to find out:
> sudo setkey -D
> aa.aa.aa.aa bb.bb.bb.bb
>         esp mode=tunnel spi=3249018409(0xc1a81629) reqid=0(0x00000000)
>         E: 3des-cbc  878f3495 43dae82f b9b57a5c 3b3342c3 eaa9f326 
> 00b491ab
>         A: hmac-md5  ca24a073 d2477e05 010b6209 f03fa81f
>         replay=4 flags=0x00000000 state=mature seq=1 pid=8414
>         created: Dec  7 15:02:33 2004   current: Dec  7 15:11:02 2004
>         diff: 509(s)    hard: 28800(s)  soft: 23040(s)
>         last: Dec  7 15:11:02 2004      hard: 0(s)      soft: 0(s)
>         current: 68544(bytes)   hard: 0(bytes)  soft: 0(bytes)
>         allocated: 504  hard: 0 soft: 0
>         refcnt=2
> bb.bb.bb.bb aa.aa.aa.aa
>         esp mode=tunnel spi=127935886(0x07a0258e) reqid=0(0x00000000)
>         E: 3des-cbc  34f3247c e954aff4 5e32144a be9a1798 c510dfba 
> 136b7634
>         A: hmac-md5  4ded7705 2a4dac1e 6af2e490 6fadf12e
>         replay=4 flags=0x00000000 state=mature seq=0 pid=8414
>         created: Dec  7 15:02:33 2004   current: Dec  7 15:11:02 2004
>         diff: 509(s)    hard: 28800(s)  soft: 23040(s)
>         last:                           hard: 0(s)      soft: 0(s)
>         current: 0(bytes)       hard: 0(bytes)  soft: 0(bytes)
>         allocated: 0    hard: 0 soft: 0
>         refcnt=1
> bb.bb.bb.bb is the OpenSWAN side. Don't worry, this is just from a 
> test run, so no data of above is confidential, still I prefer to keep 
> the IP addresses secret.
> Any idea how to verify that OpenSWAN uses the same 3DES and HMAC keys 
> on its side? I can find the HMAC keys somewhere in the pluto log, but 
> any idea how to get this info for an already established tunnel?
>> On Mon, 2004-12-06 at 12:13 +0100, Markus Hanauska wrote:
>>> No reply on user list, maybe someone on dev list can help.
>>> Hi there!
>>> I have a little Problem with NAT-T, using OpensSWAN 2.2.0, KLIPS on
>>> Linux Kernel 2.4.28 (with NAT-T patch, that I had to partly patch in 
>>> by
>>> hand; but I read here that the bug is already known and fixed), the
>>> distribution is Fedora Core 1. OpenSWAN plays the server here, on the
>>> other side a patched version of Racoon (Kame project) with NAT-T
>>> support plays the client. It's a host to network connection.
>>> Without NAT-T I can make a tunnel and ping the private interface of 
>>> my
>>> Linux gateway. Everthing works like expected. Packages arive and 
>>> leave
>>> the interface as ESP. The NAT device in my test setup uses IPSec Pass
>>> Through, but that only works for one connection from behind that NAT
>>> device.
>>> For a second connection NAT-T is necessary. But the second I enable
>>> NAT-T, nothing works anymore.
>>> Pluto makes the connection as before, detecting NAT-T draft-2,
>>> switching to port 4500 during phase 1 (so does Racoon) and they both 
>>> do
>>> phase 1 and phase 2 like a charm. The tunnel is established, both 
>>> sides
>>> are sure about having a working tunnel. But all traffic is thrown 
>>> away
>>> by KLIPS with the message:
>>> "auth failed on incoming packet"
>>> The client side sends out the ESP packages, wrapped in UDP packages, 
>>> on
>>> port 4500. I know pretty well how  the client side works, after all
>>> I've written the wrapper all by myself.
>>> Okay, you may say that the problem is home-made, my wrapper might be
>>> wrong. Could be, but then why does it work with SonicWALL, Netscreen
>>> and Zyxel devices? And it works with old NAT-T (the one on port 500)
>>> and with new NAT-T (the one on port 4500) - e.g. the small SonicWALLs
>>> use the old one (RFC draft 0/1), the big ones use the new one (RFC
>>> draft 2); whereby the real new NAT-T starts with draft 3 (where the
>>> payload numbers have been altered).
>>> I've read all the RFCs and I do exactly what is written there. Add an
>>> UDP header in-between, correct the IP header (length, checksum, next
>>> protocol) and add a non-ESP marker (draft 2 and above) or a non-IKE
>>> marker (draft 0-1) where necessary. If this code were wrong, it
>>> wouldn't work with hardware solutions of other vendors.
>>> The next thing you may guess is, maybe the ESP implementation is
>>> broken. If this was the case, why does it work perfectly the second I
>>> disable NAT-T on the OpenSWAN side? After all the ESP implementation
>>> doesn't know that I wrap the packages (the packages are caught and
>>> altered between protocol and network layer, transparent for both
>>> layers), for the ESP layer, everything is like without NAT-T.
>>> What else can I tell you? Both sides show that they use 3des and MD5 
>>> on
>>> the tunnel and authentication during IKE was pre-shared key.
>>> On the client side, I have setkey and can thus easy access to the SPI
>>> or the key used to encrypt the packages, and so on. I don't know how 
>>> to
>>> get such output on the server (OpenSWAN) side.
>>> Maybe you can help me with that, give me some hints on how to get 
>>> more
>>> info useful for debugging the problem. Or maybe you have an idea what
>>> might cause the problem. Any advice will be greatly appreciated!
>>> Thanks in advance.
> -- 
> Best Regards,
>         Markus Hanauska
Markus Hanauska
equinux Aktiengesellschaft
Kirschstr. 35
80999 München - Germany
Tel. 089/520465-323
Fax. 089/520465-299
mailto:hanauska at equinux.de

More information about the Users mailing list