[Openswan Users] [NEWBIE] Help needed - Openswan 2.2 - Sarge 2.4.27 <-> Cisco Pix

[Peter McGill]

> -----Original Message-----
> From: Mathieu Chappuis [mailto:mathieu.chappuis.lists at gmail.com] 
> Sent: November 28, 2006 1:09 AM
> 
> Now, using 3DES on both sides for IKE&ESP, and it's better, but I'm
> stuck on I3 phase :
> 
> # /usr/local/sbin/ipsec auto --up vpn
> 104 "vpn" #1: STATE_MAIN_I1: initiate
> 106 "vpn" #1: STATE_MAIN_I2: sent MI2, expecting MR2
> 003 "vpn" #1: received Vendor ID payload [Cisco-Unity]
> 003 "vpn" #1: received Vendor ID payload [Dead Peer Detection]
> 003 "vpn" #1: ignoring unknown Vendor ID payload
> [14dff993135b9d66a29e5a9ba5b1763b]
> 108 "vpn" #1: STATE_MAIN_I3: sent MI3, expecting MR3
> 010 "vpn" #1: STATE_MAIN_I3: retransmission; will wait 20s 
> for response
> 010 "vpn" #1: STATE_MAIN_I3: retransmission; will wait 40s 
> for response
> 031 "vpn" #1: max number of retransmissions (2) reached STATE_MAIN_I3.
>  Possible authentication failure: no acceptable response to our first
> encrypted message

> Any ideas ?

I did a quick search of the list history at: http://dir.gmane.org/gmane.network.openswan.user
There wasn't much there relating to this, but what was seemed to indicate a problem with a NAT.

Is either your server or the cisco going through a NAT'ing router?

> On Netfilter, I work in full open mode with the rightside peer.
> Faq, talk about firewall problem ??

I doubt it, I would have expected it to stop on STATE_MAIN_I1 in that case.
Just in case you need to allow the following:
iptables -A INPUT -i eth0 -p udp --dport isakmp -j ACCEPT # isakmp = 500
iptables -A INPUT -i eth0 -p udp --dport 4500 -j ACCEPT # 4500 is used w/ nat-t
iptables -A INPUT -i eth0 -p esp -j ACCEPT # esp = 50
# most of us don't use ah, so you can probably leave the next line out, I've included it for completeness
iptables -A INPUT -i eth0 -p ah -j ACCEPT # ah = 51
You also need to allow outbound ipsec (I usually allow all outbound (I trust my own server): iptables -A OUTPUT -j ACCEPT)
You also need to allow the tunnel traffic before and after encryption:
When using KLIPS, which you probably are with kernel 2.4.x, this is easily done by:
iptables -A INPUT -i ipsec0 -j ACCEPT
iptables -A FORWARD -i ipsec0 -j ACCEPT
iptables -A FORWARD -o ipsec0 -j ACCEPT
You can be more restrictive on the above three lines if you want, these just allow any traffic that comes encrypted through openswan
(How much you trust your peers is up to you).
NETKEY gets trickier because there is no ipsec0 interface.

> Wrong PSK ?

Maybe but I would have expected to see a clear error, rather than no response.
Check anyway, double check all your connection settings with the remote host.
Don't overlook that Aggressive Mode should be off, and your pfs=, (Perfect
Forward Secrecy) settings should match, pfs=yes (On) is best, but if your
Unsure what the cisco has set, then pfs=no, should allow on or off, depending
Which the Cisco has choosen.

I cc'd the list, in case someone else has another suggestion.

Peter