[Openswan Users] WinXP -> OpenSwan problem

Toby Chamberlain toby at webtechservices.com.au
Fri Jul 22 13:57:30 CEST 2005 

Hi,

I've successfully configured a number of Free/OpenSwan boxes to work with Windows, both with the ipsec.exe tool and using the windows L2TPD, but I've recently come across a problem with a laptop that I cannot fix.

I'm using certificates, connecting to a Debian Sarge (2.4.27) with openswan 2.2.0-8. The server is setup correctly and I can connect to it from another Freeswan box as well as several windows PC's (all using ipsec.exe). The laptop in question connects fine to the a Freeswan box (RH 7.3 w/ the 2.4.27 freeswan module), but fails to connect to the openswan box using the exact same settings with only the server IP changed. It is connected to the net using a modem dialup (no NAT)

I have tried with 2 different certificates, with L2TPD, all firewalls off, everything I can think of. A standalone PC with the same certificate and the same ipsec.conf connects with no problem... I have monitored the traffic and there doesn't appear to be a firewall or fragmentation issue

The only difference between the machines that do connect and the laptop is that the working ones are all NATed and on a private LAN... the laptop doesn't have a LAN IP and isn't NATed. The relevant part of the openswan config looks like this:

conn Host-Linux
    rightsubnet=1.1.1.1/32 <host IP>
    also=WinXP-Linux

conn WinXP-Linux
    pfs=no
    leftsubnet=2.2.2.2/24 <linux side LAN>
    auto=add

The NAT'd boxes connect using the appropriate Host-Linux (there's one for each host that connects) and the laptop tries to connect using the generic WinXP-Linux (no rightsubnet)... as I said this same setup works perfectly on a freeswan box...  any ideas?


The OpenSWAN log says:

Jul 21 13:10:32 rw1 pluto[16238]: packet from 1.2.3.4:500: ignoring Vendor ID payload [MS NT5 ISAKMPOAKLEY 00000004]
Jul 21 13:10:32 rw1 pluto[16238]: packet from 1.2.3.4:500: ignoring Vendor ID payload [FRAGMENTATION]
Jul 21 13:10:32 rw1 pluto[16238]: packet from 1.2.3.4:500: received Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02_n] meth=106, but already using method 0
Jul 21 13:10:32 rw1 pluto[16238]: packet from 1.2.3.4:500: ignoring Vendor ID payload [26244d38eddb61b3172a36e3d0cfb819]
Jul 21 13:10:32 rw1 pluto[16238]: "WinXP-Linux"[9] 1.2.3.4 #13: responding to Main Mode from unknown peer 1.2.3.4
Jul 21 13:10:32 rw1 pluto[16238]: "WinXP-Linux"[9] 1.2.3.4 #13: transition from state (null) to state STATE_MAIN_R1
Jul 21 13:10:33 rw1 pluto[16238]: "WinXP-Linux"[9] 1.2.3.4 #13: transition from state STATE_MAIN_R1 to state STATE_MAIN_R2
Jul 21 13:11:36 rw1 pluto[16238]: "WinXP-Linux"[9] 1.2.3.4 #13: next payload type of ISAKMP Hash Payload has an unknown value: 228
Jul 21 13:11:36 rw1 pluto[16238]: "WinXP-Linux"[9] 1.2.3.4 #13: malformed payload in packet
Jul 21 13:11:36 rw1 pluto[16238]: "WinXP-Linux"[9] 1.2.3.4 #13: sending encrypted notification PAYLOAD_MALFORMED to 1.2.3.4:500
Jul 21 13:11:43 rw1 pluto[16238]: "WinXP-Linux"[9] 1.2.3.4 #13: max number of retransmissions (2) reached STATE_MAIN_R2
Jul 21 13:11:43 rw1 pluto[16238]: "WinXP-Linux"[9] 1.2.3.4: deleting connection "WinXP-Linux" instance with peer 1.2.3.4 {isakmp=#0/ipsec=#0}



The Oakley log says:
 7-21: 13:12:38:957:6ac Initialization OK
 7-21: 13:13:06:156:b00 Acquire from driver: op=0000000D src=1.2.3.4.0 dst=<Lan IP>.0 proto = 0, SrcMask=255.255.255.255, DstMask=255.255.254.0, Tunnel 1, TunnelEndpt=9.8.7.6 Inbound TunnelEndpt=1.2.3.4
 7-21: 13:13:06:156:53c Filter to match: Src 9.8.7.6 Dst 1.2.3.4
 7-21: 13:13:06:156:53c MM PolicyName: 2
 7-21: 13:13:06:156:53c MMPolicy dwFlags 2 SoftSAExpireTime 28800
 7-21: 13:13:06:156:53c MMOffer[0] LifetimeSec 28800 QMLimit 1 DHGroup 2
 7-21: 13:13:06:156:53c MMOffer[0] Encrypt: Triple DES CBC Hash: SHA
 7-21: 13:13:06:156:53c MMOffer[1] LifetimeSec 28800 QMLimit 1 DHGroup 2
 7-21: 13:13:06:156:53c MMOffer[1] Encrypt: Triple DES CBC Hash: MD5
 7-21: 13:13:06:156:53c MMOffer[2] LifetimeSec 28800 QMLimit 1 DHGroup 1
 7-21: 13:13:06:156:53c MMOffer[2] Encrypt: DES CBC Hash: SHA
 7-21: 13:13:06:156:53c MMOffer[3] LifetimeSec 28800 QMLimit 1 DHGroup 1
 7-21: 13:13:06:156:53c MMOffer[3] Encrypt: DES CBC Hash: MD5
 7-21: 13:13:06:156:53c Auth[0]:RSA Sig <CA Description> AuthFlags 0
 7-21: 13:13:06:156:53c QM PolicyName: Host-Remote filter action dwFlags 1
 7-21: 13:13:06:156:53c QMOffer[0] LifetimeKBytes 50000 LifetimeSec 3600
 7-21: 13:13:06:156:53c QMOffer[0] dwFlags 0 dwPFSGroup -2147483648
 7-21: 13:13:06:156:53c  Algo[0] Operation: ESP Algo: Triple DES CBC HMAC: MD5
 7-21: 13:13:06:156:53c Starting Negotiation: src = 1.2.3.4.0500, dst = 9.8.7.6.0500, proto = 00, context = 0000000D, ProxySrc = 1.2.3.4.0000, ProxyDst = <Lan Subnet>.0000 SrcMask = 255.255.255.255 DstMask = 255.255.254.0
 7-21: 13:13:06:156:53c constructing ISAKMP Header
 7-21: 13:13:06:156:53c constructing SA (ISAKMP)
 7-21: 13:13:06:156:53c Constructing Vendor MS NT5 ISAKMPOAKLEY
 7-21: 13:13:06:156:53c Constructing Vendor FRAGMENTATION
 7-21: 13:13:06:156:53c Constructing Vendor draft-ietf-ipsec-nat-t-ike-02
 7-21: 13:13:06:156:53c Constructing Vendor Vid-Initial-Contact
 7-21: 13:13:06:156:53c 
 7-21: 13:13:06:156:53c Sending: SA = 0x000EFA70 to 9.8.7.6:Type 2.500
 7-21: 13:13:06:156:53c ISAKMP Header: (V1.0), len = 276
 7-21: 13:13:06:156:53c   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:06:156:53c   R-COOKIE 0000000000000000
 7-21: 13:13:06:156:53c   exchange: Oakley Main Mode
 7-21: 13:13:06:156:53c   flags: 0
 7-21: 13:13:06:156:53c   next payload: SA
 7-21: 13:13:06:156:53c   message ID: 00000000
 7-21: 13:13:06:156:53c Ports S:f401 D:f401
 7-21: 13:13:06:437:53c 
 7-21: 13:13:06:437:53c Receive: (get) SA = 0x000efa70 from 9.8.7.6.500
 7-21: 13:13:06:437:53c ISAKMP Header: (V1.0), len = 84
 7-21: 13:13:06:437:53c   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:06:437:53c   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:06:437:53c   exchange: Oakley Main Mode
 7-21: 13:13:06:437:53c   flags: 0
 7-21: 13:13:06:437:53c   next payload: SA
 7-21: 13:13:06:437:53c   message ID: 00000000
 7-21: 13:13:06:437:53c processing payload SA
 7-21: 13:13:06:437:53c Received Phase 1 Transform 1
 7-21: 13:13:06:437:53c      Encryption Alg Triple DES CBC(5)
 7-21: 13:13:06:437:53c      Hash Alg SHA(2)
 7-21: 13:13:06:437:53c      Oakley Group 2
 7-21: 13:13:06:437:53c      Auth Method RSA Signature with Certificates(3)
 7-21: 13:13:06:437:53c      Life type in Seconds
 7-21: 13:13:06:437:53c      Life duration of 28800
 7-21: 13:13:06:437:53c Phase 1 SA accepted: transform=1
 7-21: 13:13:06:437:53c SA - Oakley proposal accepted
 7-21: 13:13:06:437:53c ClearFragList
 7-21: 13:13:06:437:53c constructing ISAKMP Header
 7-21: 13:13:06:467:53c constructing KE
 7-21: 13:13:06:467:53c constructing NONCE (ISAKMP)
 7-21: 13:13:06:467:53c 
 7-21: 13:13:06:467:53c Sending: SA = 0x000EFA70 to 9.8.7.6:Type 2.500
 7-21: 13:13:06:467:53c ISAKMP Header: (V1.0), len = 184
 7-21: 13:13:06:467:53c   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:06:467:53c   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:06:467:53c   exchange: Oakley Main Mode
 7-21: 13:13:06:467:53c   flags: 0
 7-21: 13:13:06:467:53c   next payload: KE
 7-21: 13:13:06:467:53c   message ID: 00000000
 7-21: 13:13:06:467:53c Ports S:f401 D:f401
 7-21: 13:13:06:737:53c 
 7-21: 13:13:06:737:53c Receive: (get) SA = 0x000efa70 from 9.8.7.6.500
 7-21: 13:13:06:737:53c ISAKMP Header: (V1.0), len = 180
 7-21: 13:13:06:737:53c   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:06:737:53c   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:06:737:53c   exchange: Oakley Main Mode
 7-21: 13:13:06:737:53c   flags: 0
 7-21: 13:13:06:737:53c   next payload: KE
 7-21: 13:13:06:737:53c   message ID: 00000000
 7-21: 13:13:06:737:53c processing payload KE
 7-21: 13:13:06:767:53c processing payload NONCE
 7-21: 13:13:06:767:53c ClearFragList
 7-21: 13:13:06:767:53c constructing ISAKMP Header
 7-21: 13:13:06:767:53c constructing ID
 7-21: 13:13:06:767:53c Received no valid CRPs.  Using all configured
 7-21: 13:13:06:767:53c Looking for IPSec only cert
 7-21: 13:13:06:777:53c Cert Trustes.  0 100
 7-21: 13:13:06:777:53c Cert SHA Thumbprint <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
 7-21: 13:13:06:777:53c 67316577
 7-21: 13:13:06:777:53c CertFindExtenstion failed with 0
 7-21: 13:13:06:777:53c Entered CRL check
 7-21: 13:13:06:777:53c Left CRL check
 7-21: 13:13:06:777:53c Cert SHA Thumbprint <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
 7-21: 13:13:06:777:53c 67316577
 7-21: 13:13:06:777:53c SubjectName: <My Certificate>
 7-21: 13:13:06:777:53c Cert Serialnumber 06
 7-21: 13:13:06:777:53c Cert SHA Thumbprint <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
 7-21: 13:13:06:777:53c 67316577
 7-21: 13:13:06:777:53c SubjectName: <CA Description>
 7-21: 13:13:06:777:53c Cert Serialnumber 00
 7-21: 13:13:06:777:53c Cert SHA Thumbprint dcd5270722a99e1c1ae3dd86a5fb9de9
 7-21: 13:13:06:777:53c 9d0d9561
 7-21: 13:13:06:777:53c Not storing My cert chain in SA.
 7-21: 13:13:06:777:53c MM ID Type 9
 7-21: 13:13:06:777:53c MM ID <lots of numbers>
 <SNIP LOTS OF NUMBERS>
 7-21: 13:13:06:777:53c constructing CERT
 7-21: 13:13:06:777:53c Construct SIG
 7-21: 13:13:06:817:53c Constructing Cert Request
 7-21: 13:13:06:817:53c <CA Description>
 7-21: 13:13:06:817:53c 
 7-21: 13:13:06:817:53c Sending: SA = 0x000EFA70 to 9.8.7.6:Type 2.500
 7-21: 13:13:06:817:53c ISAKMP Header: (V1.0), len = 1876
 7-21: 13:13:06:817:53c   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:06:817:53c   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:06:817:53c   exchange: Oakley Main Mode
 7-21: 13:13:06:817:53c   flags: 1 ( encrypted )
 7-21: 13:13:06:817:53c   next payload: ID
 7-21: 13:13:06:817:53c   message ID: 00000000
 7-21: 13:13:06:817:53c Ports S:f401 D:f401
 7-21: 13:13:07:959:a78 retransmit: sa = 000EFA70 centry 00000000 , count = 1
 7-21: 13:13:07:959:a78 
 7-21: 13:13:07:959:a78 Sending: SA = 0x000EFA70 to 9.8.7.6:Type 2.500
 7-21: 13:13:07:959:a78 ISAKMP Header: (V1.0), len = 1876
 7-21: 13:13:07:959:a78   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:07:959:a78   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:07:959:a78   exchange: Oakley Main Mode
 7-21: 13:13:07:959:a78   flags: 1 ( encrypted )
 7-21: 13:13:07:959:a78   next payload: ID
 7-21: 13:13:07:959:a78   message ID: 00000000
 7-21: 13:13:07:959:a78 Ports S:f401 D:f401
 7-21: 13:13:09:952:a78 retransmit: sa = 000EFA70 centry 00000000 , count = 2
 7-21: 13:13:09:952:a78 
 7-21: 13:13:09:952:a78 Sending: SA = 0x000EFA70 to 9.8.7.6:Type 2.500
 7-21: 13:13:09:952:a78 ISAKMP Header: (V1.0), len = 1876
 7-21: 13:13:09:952:a78   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:09:952:a78   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:09:952:a78   exchange: Oakley Main Mode
 7-21: 13:13:09:952:a78   flags: 1 ( encrypted )
 7-21: 13:13:09:952:a78   next payload: ID
 7-21: 13:13:09:952:a78   message ID: 00000000
 7-21: 13:13:09:952:a78 Ports S:f401 D:f401
 7-21: 13:13:13:958:a78 retransmit: sa = 000EFA70 centry 00000000 , count = 3
 7-21: 13:13:13:958:a78 
 7-21: 13:13:13:958:a78 Sending: SA = 0x000EFA70 to 9.8.7.6:Type 2.500
 7-21: 13:13:13:958:a78 ISAKMP Header: (V1.0), len = 1876
 7-21: 13:13:13:958:a78   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:13:958:a78   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:13:958:a78   exchange: Oakley Main Mode
 7-21: 13:13:13:958:a78   flags: 1 ( encrypted )
 7-21: 13:13:13:958:a78   next payload: ID
 7-21: 13:13:13:958:a78   message ID: 00000000
 7-21: 13:13:13:958:a78 Ports S:f401 D:f401
 7-21: 13:13:16:712:53c 
 7-21: 13:13:16:712:53c Receive: (get) SA = 0x000efa70 from 9.8.7.6.500
 7-21: 13:13:16:712:53c ISAKMP Header: (V1.0), len = 180
 7-21: 13:13:16:712:53c   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:16:712:53c   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:16:712:53c   exchange: Oakley Main Mode
 7-21: 13:13:16:712:53c   flags: 0
 7-21: 13:13:16:712:53c   next payload: KE
 7-21: 13:13:16:712:53c   message ID: 00000000
 7-21: 13:13:16:712:53c received an unencrypted packet when crypto active
 7-21: 13:13:16:712:53c GetPacket failed 35ec
 7-21: 13:13:21:959:a78 retransmit: sa = 000EFA70 centry 00000000 , count = 4
 7-21: 13:13:21:959:a78 
 7-21: 13:13:21:959:a78 Sending: SA = 0x000EFA70 to 9.8.7.6:Type 2.500
 7-21: 13:13:21:959:a78 ISAKMP Header: (V1.0), len = 1876
 7-21: 13:13:21:959:a78   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:21:959:a78   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:21:959:a78   exchange: Oakley Main Mode
 7-21: 13:13:21:959:a78   flags: 1 ( encrypted )
 7-21: 13:13:21:959:a78   next payload: ID
 7-21: 13:13:21:959:a78   message ID: 00000000
 7-21: 13:13:21:959:a78 Ports S:f401 D:f401
 7-21: 13:13:36:710:53c 
 7-21: 13:13:36:710:53c Receive: (get) SA = 0x000efa70 from 9.8.7.6.500
 7-21: 13:13:36:710:53c ISAKMP Header: (V1.0), len = 180
 7-21: 13:13:36:710:53c   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:36:710:53c   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:36:710:53c   exchange: Oakley Main Mode
 7-21: 13:13:36:710:53c   flags: 0
 7-21: 13:13:36:710:53c   next payload: KE
 7-21: 13:13:36:710:53c   message ID: 00000000
 7-21: 13:13:36:710:53c received an unencrypted packet when crypto active
 7-21: 13:13:36:710:53c GetPacket failed 35ec
 7-21: 13:13:37:962:a78 retransmit: sa = 000EFA70 centry 00000000 , count = 5
 7-21: 13:13:37:962:a78 
 7-21: 13:13:37:962:a78 Sending: SA = 0x000EFA70 to 9.8.7.6:Type 2.500
 7-21: 13:13:37:962:a78 ISAKMP Header: (V1.0), len = 1876
 7-21: 13:13:37:962:a78   I-COOKIE ea6373899ccc7f64
 7-21: 13:13:37:962:a78   R-COOKIE 9c720bd1092af40f
 7-21: 13:13:37:962:a78   exchange: Oakley Main Mode
 7-21: 13:13:37:962:a78   flags: 1 ( encrypted )
 7-21: 13:13:37:962:a78   next payload: ID
 7-21: 13:13:37:962:a78   message ID: 00000000
 7-21: 13:13:37:962:a78 Ports S:f401 D:f401
 7-21: 13:14:09:968:a78 retransmit exhausted: sa = 000EFA70 centry 00000000, count = 6
 7-21: 13:14:09:968:a78 SA Dead. sa:000EFA70 status:35ed
 7-21: 13:14:09:968:a78 isadb_set_status sa:000EFA70 centry:00000000 status 35ed

Thanks
Toby
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openswan.org/pipermail/users/attachments/20050722/017d7f22/attachment-0001.htm

More information about the Users mailing list