[Openswan Users] OpenSWAN to PIX woes

[Ted Kaczmarek]

On Tue, 2004-08-17 at 21:41 +0000, arno van der walt wrote:
> Hey guys
> 
> I'm testing something in my lab before putting in into production and I'm 
> stuck.
> 
> >From my debugs this must be an ipsec proposal issue but for the life of me 
> everything looks fine.
> 
> I've been at this for 18 hours straight...so I'm possibly missing the 
> obvious.
> 
> I have copied the ipsec.barf here ===> http://65.96.55.101/ipsec.barf
> 
> My pix is configured as follows:
> sysopt connection permit-ipsec
> crypto ipsec transform-set myset esp-3des esp-md5-hmac
> crypto map mymap 10 ipsec-isakmp
> crypto map mymap 10 match address FREESWAN-VPN
> crypto map mymap 10 set pfs group2
> crypto map mymap 10 set peer 10.5.1.2
> crypto map mymap 10 set transform-set myset
> crypto map mymap interface intf2
> isakmp enable intf2
> isakmp key ******** address 10.5.1.2 netmask 255.255.255.255 no-xauth 
> no-config-mode
> isakmp peer ip 10.5.1.2 no-xauth no-config-mode
> isakmp identity address
> isakmp policy 5 authentication pre-share
> isakmp policy 5 encryption 3des
> isakmp policy 5 hash md5
> isakmp policy 5 group 5
> isakmp policy 5 lifetime 28800
> 
> 
> 
> 
> Here is an excerpt from the pix debug:
> 
> 1
> crypto_isakmp_process_block:src:10.5.1.2, dest:172.16.1.1 spt:500 dpt:500
> OAK_QM exchange
> oakley_process_quick_mode:
> OAK_QM_IDLE
> ISAKMP (0): processing SA payload. message ID = 2808959208
> 
> ISAKMP : Checking IPSec proposal 0
> 
> ISAKMP: transform 0, ESP_3DES
> ISAKMP:   attributes in transform:
> ISAKMP:      encaps is 1
> ISAKMP:      SA life type in seconds
> ISAKMP:      SA life duration (basic) of 28800
> ISAKMP:      authenticator is HMAC-MD5IPSEC(validate_proposal): invalid 
> transform proposal flags -- 0x4
> 
> ISAKMP (0): atts not acceptable. Next payload is 3
> ISAKMP: transform 1, ESP_3DES
> ISAKMP:   attributes in transform:
> ISAKMP:      encaps is 1
> ISAKMP:      SA life type in seconds
> ISAKMP:      SA life duration (basic) of 28800
> ISAKMP:      authenticator is HMAC-SHAIPSEC(validate_proposal): transform 
> proposal (prot 3, trans 3, hmac_alg 2) not supported
> 
> ISAKMP (0): atts not acceptable. Next payload is 0
> ISAKMP (0): SA not acceptable!
> ISAKMP (0): sending NOTIFY message 14 protocol 0
> return status is IKMP_ERR_NO_RETRANS
> crypto_isakmp_process_block:src:10.5.1.2, dest:172.16.1.1 spt:500 dpt:500
> ISAKMP: phase 2 packet is a duplicate of a previous packet
> ISAKMP: resending last response
> ISAKMP (0): retransmitting phase 2 (0/0)... mess_id 0xa76d50e8
> crypto_isakmp_process_block:src:10.5.1.2, dest:172.16.1.1 spt:500 dpt:500
> ISAKMP: phase 2 packet is a duplicate of a previous packet
> 
> Any help is appreciated!!! It must be the transform set...right? But what is 
> wrong on it??? I'm stumped.
> 
> Thanks
> 
> Arno
> 
> _________________________________________________________________


Your transform on the Pix is ok, at least that one works fine for me
with 6.3.3 on the pix and Openswan 2.1.4.

Is your FREESWAN-VPN a match to your ipsec.conf?

Also, I would recomend using a numeric value in the acl name unless you
are planning on having only 1 tunnel. 

Also, wr mem on the pix and reload will also save a lot of headaches. I
can't tell you how much time I spent trying to figure out what was wrong
with Openswan when the Pix was misbehaving. I suspect clearing would do
the same, but they don't have a clear all function. 

By the grace of God a  friend of mine gave me his Netscreen 5XT,  and I
use it for sanity checks, sure saves a lot of wasted time with Pix
quirks.

Ted