[Openswan Users] vista AuthIP

Jacco de Leeuw jacco2 at dds.nl
Mon Sep 10 07:26:04 EDT 2007

Marco Berizzi wrote:

> I have an interoperability problem with vista. The microsoft support has
> asked me if is it possible to change openswan configuration so that it does
> not respond to AuthIP

Paul wrote:

> Show us the logs on the openswan end.
> And preferable the OAKLEY.LOG on the windows end.

I can mail/post these if you want (openswan-dev?).

What I think happens is that Vista sends the payload type 133 and of course
Openswan responds with PAYLOAD_MALFORMED because 133 is a private payload.
Vista continues and ISAKMP SAs and IPsec SAs are established. But Vista
keeps on sending the payload 133's, possibly for Microsoft's second
authentication (AuthIP). This fails and Vista sends a Delete SA.

Note that this does not concern L2TP/IPsec, but IPsec VPNs configured with
the "Windows Firewall with Advanced Security" tool or "netsh advfirewall".

The Microsoft development team wrote:

>>> The 133 payload is sent under exchange type 243. Looks like what is 
>>> happening is that the linux implementation is accepting the exchange
>>> type 243 packet (it should drop it) and failing the negotiation when it
>>> finds a 133 payload in the packet.

Openswan responds to the 133 payload with PAYLOAD_MALFORMED. It does not
even get to check the exchange type. Is Microsoft saying that implementations
should first check the exchange type and only then the payload type? But
RFC 2408 says:

  "When an ISAKMP message is received, the receiving entity MUST do the

  1. Verify the Initiator and Responder "cookies".
  2. Check the Next Payload field
  3. Check the Major and Minor Version fields
  4. Check the Exchange Type field

(By the way, wireshark says they use exchange type 246, not 243).

So, why does Vista start with sending private payloads/exchange types?
Shouldn't it start with Vendor IDs first? If the receiving party does
not respond with the correct Vendor IDs, Vista should not send those
private payloads.

If you use a PSK instead of certs, Vista connects fine. It will not
do a second authentication (AuthIP) then.

Jacco de Leeuw                         mailto:jacco2 at dds.nl
Zaandam, The Netherlands           http://www.jacco2.dds.nl

More information about the Users mailing list