[Openswan Users] next payload type of ISAKMP Hash Payload has an unknown value

[Brett Curtis]

A windows machine is unavailable to me right now. However I have
imported my p12 into my OSX keychain and I am getting different
results. I do plain on getting back to the windows machine.

I do get a SA Established however fail to finish the connection.

Feb  8 11:17:26 breakout pluto[10055]: "roadwarrior-osx-xp"[4]
172.17.187.76 #2: STATE_MAIN_R3: sent MR3, ISAKMP SA established
{auth=OAKLEY_RSA_SIG cipher=oakley_3des_cbc_192 prf=oakley_sha
group=modp1024}
Feb  8 11:17:26 breakout pluto[10055]: "roadwarrior-osx-xp"[4]
172.17.187.76 #2: ignoring informational payload, type
INVALID_CERT_AUTHORITYFeb  8 11:17:26 breakout pluto[10055]:
"roadwarrior-osx-xp"[4] 172.17.187.76 #2: received and ignored
informational message
Feb  8 11:17:28 breakout pluto[10055]: "roadwarrior-osx-xp"[4]
172.17.187.76 #2: retransmitting in response to duplicate packet;
already STATE_MAIN_R3
Feb  8 11:17:31 breakout pluto[10055]: "roadwarrior-osx-xp"[4]
172.17.187.76 #2: retransmitting in response to duplicate packet;
already STATE_MAIN_R3
Feb  8 11:17:34 breakout pluto[10055]: "roadwarrior-osx-xp"[4]
172.17.187.76 #2: discarding duplicate packet -- exhausted
retransmission; already STATE_MAIN_R3

Does this mean Something is incorrect with my cacert? When staring
openswan it claims it loads fine.

Thanks
On 2/6/07, Paul Wouters <paul at xelerance.com> wrote:
> On Mon, 5 Feb 2007, Brett Curtis wrote:
>
> > Sorry to reply to myself  (I switched email addresses for this list) anyways
> > I am still having the issue as described below. It is not mtu related as I
> > am now testing on subnets inside my LAN.
> >
> > What should I look into?
>
> Do you have a > 1024bit key in your certificate? that was cause IKE
> fragmentation that won't work with Openswan.
>
> Create/check the OAKLEY.LOG on windows to see what it is doing, and if
> that is what you think it hsould do.
>
> Paul
>
> > >Versions: openswan-2.4.7 / 2.6.18-gentoo-r3
> > >x86_64 arch
> >
> > >I have followed Nate's Guide and everything as far as creating the
> > >certs went well. My openswan loads all the correct certs and starts
> > >up fine.
> > >I imported my .p12 into windows both by hand and with the
> > >certimport.exe tool. This seemed to work fine in both cases.
> >
> > >However when I try to connect I get the windows 786 lt2p error and
> > >this in my openswan logs.
> >
> > >Jan 13 14:49:30 defender64 pluto[6562]: packet from
> > >74.65.156.181:500: ignoring Vendor ID payload [MS NT5 ISAKMPOAKLEY
> > >00000004]
> > >Jan 13 14:49:30 defender64 pluto[6562]: packet from
> > >74.65.156.181:500: ignoring Vendor ID payload [FRAGMENTATION]
> > >Jan 13 14:49:30 defender64 pluto[6562]: packet from
> > >74.65.156.181:500: received Vendor ID payload [draft-ietf-ipsec-nat-t-
> > >ike-02_n] method set to=106
> > >Jan 13 14:49:30 defender64 pluto[6562]: packet from
> > >74.65.156.181:500: ignoring Vendor ID payload [Vid-Initial-Contact]
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: responding to Main Mode from unknown peer
> > >74.65.156.181
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: transition from state STATE_MAIN_R0 to state
> > >STATE_MAIN_R1
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: STATE_MAIN_R1: sent MR1, expecting MI2
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: NAT-Traversal: Result using draft-ietf-ipsec-nat-t-
> > >ike-02/03: peer is NATed
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: transition from state STATE_MAIN_R1 to state
> > >STATE_MAIN_R2
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: STATE_MAIN_R2: sent MR2, expecting MI3
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: next payload type of ISAKMP Hash Payload has an
> > >unknown value: 51
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: malformed payload in packet
> > >Jan 13 14:49:30 defender64 pluto[6562]: | payload malformed after IV
> > >Jan 13 14:49:30 defender64 pluto[6562]: |   e7 12 22 63  76 fe 09 0c
> > >0e 2a b9 ec  7b 5e 1b 52
> > >Jan 13 14:49:30 defender64 pluto[6562]: |   9a c7 1f 66
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: sending notification PAYLOAD_MALFORMED to
> > >74.65.156.181:500
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: next payload type of ISAKMP Hash Payload has an
> > >unknown value: 39
> > >Jan 13 14:49:30 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: malformed payload in packet
> > >Jan 13 14:50:40 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181 #1: max number of retransmissions (2) reached
> > >STATE_MAIN_R2
> > >Jan 13 14:50:40 defender64 pluto[6562]: "roadwarrior-osx-xp"[1]
> > >74.65.156.181: deleting connection "roadwarrior-osx-xp" instance with
> > >peer 74.65.156.181 {isakmp=#0/ipsec=#0}
> > >
> > >I have searched these errors for a few hours now with no luck.
> > >
> > >Thanks for any help.
> >
>
> --
> Building and integrating Virtual Private Networks with Openswan:
> http://www.amazon.com/gp/product/1904811256/104-3099591-2946327?n=283155
>