[Openswan Users] Leopard IPsec initial test - failed

[Jacco de Leeuw]

Paul Wouters wrote:

> Teh good news is that certifiacte imports are much much better, and 
> actually work. No more messing with Keychain.

You are probably referring to:
http://www.jacco2.dds.nl/networking/openswan-macosx.html#Certs

Importing certificates on Mac OS X 10.4 has always worked for me,
and other people confirmed it worked for them too. I don't know if
it was the GUI import method or the CLI import method, but I'm sorry
to hear that your Keychain was screwed up.

The thing is, there is zero documentation from Apple about this.
I had to come up with *something*... I think they intended to make
it reasonably easy to do: there is an option "Click to unlock the
System keychain" but if you do that and import a certificate the
Mac reports an error. Could be a bug, could be intentional.

> The bad news is, the IPsec is broken:

That is bad news indeed. (Doesn't Apple do any IPsec interop testing
at all?)

> ignoring unknown Vendor ID payload [8f8d83826d246b6fc7a8a6a428c11de8] 
> ignoring unknown Vendor ID payload [439b59f8ba676c4c7737ae22eab8f582] 
> ignoring unknown Vendor ID payload [4d1e0e136deafa34c4f3ea9f02ec7285] 
> ignoring unknown Vendor ID payload [80d0bb3def54565ee84645d4c85ce3ee] 
> ignoring unknown Vendor ID payload [9909b64eed937c6573de52ace952fa6b]

Hm, what could these be... We'll have to wait for the racoon source code
to show up on the Apple website. Come to think of it, they still haven't
released the racoon source code for 10.4.10.

> received Vendor ID payload [Dead Peer Detection]

Finally. I wonder if they implemented this feature themselves or
if they lifted it from ipsec-tools.

> STATE_MAIN_R3: sent MR3, ISAKMP SA established {auth=OAKLEY_RSA_SIG 
> cipher=oakley_3des_cbc_192 prf=oakley_sha group=modp1024}

I thought AES was supported?

> byte 2 of ISAKMP Hash Payload must be zero, but is not
> malformed payload in packet

The racoon source code would help in this case too.

> On OSX, we see:
> IPSec connection failed <IKE Error 22 (0x16) Invalid cert authority>
> 
> So hopefully it is just a simple keychain import bug, which we can work 
> around plus a bug in returning errors in IKE.

Are you using an Extended Key Usage (EKU) in your server certificate? It
turns out that Mac OS X does not like EKUs, unless you add a particular EKU:
http://www.jacco2.dds.nl/networking/openswan-macosx.html#Cert_ID
This results in INVALID_CERT_AUTHORITY which is very similar to what you get.

The problem was reported by Alexandre Ghisoli:
http://lists.openswan.org/pipermail/users/2007-August/012890.html
and was resolved by Daniel Bertolo over on the Strongswan mailinglist:
https://lists.strongswan.org/pipermail/users/2007-July/001949.html

Alexandre also posted info on getting more debug output from the Mac:
http://www.jacco2.dds.nl/networking/openswan-macosx.html#Troubleshooting
Could be useful in this case as well.

> It seems you need to still move/change the trust of the PKCS#12 CA-cert. 
> But it will not allow you to move it into the System Roots" (the old X509 
> Anchors)

What happens if you try to move it into the System root store? Do you get
CL_INVALID_FIELD_POINTER? Perhaps the "Click to unlock" method works now?

Jacco
-- 
Jacco de Leeuw                         mailto:jacco2 at dds.nl
Zaandam, The Netherlands           http://www.jacco2.dds.nl