[Openswan Users] stuck in STATE_MAIN_I3/STATE_MAIN_R2

[Oskar Liljeblad]

I have a simple OpenS/WAN setup between two linux 2.6.7 boxes, both running
2.2.0-4 (debian). They both have public static IPs. I use X509 certificates
for initial authentication.

It appears that the box which initiates the VPN, alpha, gets stuck in state
STATE_MAIN_I3. The other end, beta, is in STATE_MAIN_R2:

beta #1: responding to Main Mode
beta #1: transition from state (null) to state STATE_MAIN_R1
beta #1: transition from state STATE_MAIN_R1 to state STATE_MAIN_R2
beta #1: max number of retransmissions (2) reached STATE_MAIN_R2

alpha #1: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
alpha #1: discarding duplicate packet; already STATE_MAIN_I3
alpha #1: discarding duplicate packet; already STATE_MAIN_I3
alpha #1: max number of retransmissions (2) reached STATE_MAIN_I3.  Possible authentication failure: no acceptable response to our first encrypted message
alpha #1: starting keying attempt 2 of an unlimited number

and so it repeats.

With tcpdump I'm seeing these packets on alpha:

17:31:29.274391 IP alpha.isakmp > beta.isakmp: isakmp: phase 1 I ident
17:31:29.292350 IP beta.isakmp > alpha.isakmp: isakmp: phase 1 R ident
17:31:29.320873 IP alpha.isakmp > beta.isakmp: isakmp: phase 1 I ident
17:31:29.396066 IP beta.isakmp > alpha.isakmp: isakmp: phase 1 R ident
17:31:29.518108 IP alpha.isakmp > beta.isakmp: isakmp: phase 1 I ident[E]
[more packets come 10 seconds later]

and on beta:

17:31:29.282465 IP alpha.isakmp > beta.isakmp: isakmp: phase 1 I ident
17:31:29.283566 IP beta.isakmp > alpha.isakmp: isakmp: phase 1 R ident
17:31:29.328918 IP alpha.isakmp > beta.isakmp: isakmp: phase 1 I ident
17:31:29.384198 IP beta.isakmp > alpha.isakmp: isakmp: phase 1 R ident
[more packets come 20 seconds later]

The setup worked perfectly with 2.1.3, but recently I
upgraded to 2.2.0 and some time later it stopped working.
However, downgrading to 2.1.3 did not help (same issue).
I have no idea what else changed. I even reinstalled openswan
on both machines. Since it stopped working all of a sudden,
I'm starting to believe that one of the ISPs is blocking vital
VPN traffic. Or the kernel on one of the machines...?

Regards,

Oskar Liljeblad (oskar at osk.mine.nu)