[Openswan Users] Problem: interroperability between Linux and Windows Server 2003

[Ilia Sotnikov]

On 12/9/06, Denis Roy <droy at lb3.ca> wrote:
> I use Openswan 2.4.7 with kernel 2.6.18-gentoo-r3. The setup is very simple
> and consist of a bidirectional ipsec encryption in mode=transport between
> two hosts on the same subnet. When I first launch OpenSwan, everything is
> alright but it stops working one hour or so later. The strange thing is that
> it asks to delete an old SA (ISAKMP State#1), while it was using the eighth
> and deleted the seventh already. Shortly after that, I get an error
> "Informational Exchange is for an unknown (expired?) SA" and then, I cannot
> successfully ping the host until I restart OpenSwan. During that time, I can
> see that my outgoing traffic is still encrypted but the Windows Server does
> not reply.

The problem is that under some conditions Openswan will have only
Quick Mode established when working with Microsoft Windows as the
peer. So looks like you're experiencing the same problem. After
QuickMode will expire without Main Mode available IPSec will not have
enougn information to encrypt the traffic. Also I've seen asymmetrical
SA in such a  situation (looking by setkey -D, setkey -DP) - there was
only outgoing SA, without symmetrical incoming (if I remember
correctly).

Also Windows IPSec has QuickMode limit by traffic. Notice that using 0
bytes there will not switch that limit off (Windows will use default
value). To make the configuration more predictable we switched the
limit off using highest available value (2 147 483 636 bytes)

You could also try to set Main and Quick Mode limits higher on the
Windows side, thus forcing it to always be the responder and Openswan
will always be the initiator.


-- 
 Ilia Sotnikov