[Openswan Users] Problem: interroperability between Linux and Windows Server 2003

[Denis Roy]

Hello M. Sotnikov,

I got it working this weekend while trying different configurations. So far, it has been working continuously for the past 28hours or so, even after rigorous files transfer test (to exceed the 2GB limit you told me about).

So, to get interoperability between OpenSwan 2.4.7/Linux 2.6.18 and Windows 2003 SBS Sp1, here's what I have now: 

	  type=transport
        authby=secret
        left=192.168.16.1
        right=192.168.16.2
        pfs=yes
        keylife=1h
        ikelifetime=1h
        compress=yes
        keyingtries=5
        rekey=no
        failureshunt=passthrough
        auth=esp
        auto=start
        ike=3des-sha1-modp1024
        esp=3des-sha1
        pfsgroup=modp1024


In other words, I made Windows the initiator and shortened the key lifetime from 8h to 1h (3600 seconds). Now, all that is left for me is to switch from PSK to .X509. :) 

Have a nice week,

Denis Roy.


-----Original Message-----
From: Ilia Sotnikov [mailto:hostcc at gmail.com] 
Sent: 9 décembre 2006 10:06
To: Denis Roy
Cc: users at openswan.org
Subject: Re: [Openswan Users] Problem: interroperability between Linux and Windows Server 2003

On 12/9/06, Denis Roy <droy at lb3.ca> wrote:
> I use Openswan 2.4.7 with kernel 2.6.18-gentoo-r3. The setup is very simple
> and consist of a bidirectional ipsec encryption in mode=transport between
> two hosts on the same subnet. When I first launch OpenSwan, everything is
> alright but it stops working one hour or so later. The strange thing is that
> it asks to delete an old SA (ISAKMP State#1), while it was using the eighth
> and deleted the seventh already. Shortly after that, I get an error
> "Informational Exchange is for an unknown (expired?) SA" and then, I cannot
> successfully ping the host until I restart OpenSwan. During that time, I can
> see that my outgoing traffic is still encrypted but the Windows Server does
> not reply.

The problem is that under some conditions Openswan will have only
Quick Mode established when working with Microsoft Windows as the
peer. So looks like you're experiencing the same problem. After
QuickMode will expire without Main Mode available IPSec will not have
enougn information to encrypt the traffic. Also I've seen asymmetrical
SA in such a  situation (looking by setkey -D, setkey -DP) - there was
only outgoing SA, without symmetrical incoming (if I remember
correctly).

Also Windows IPSec has QuickMode limit by traffic. Notice that using 0
bytes there will not switch that limit off (Windows will use default
value). To make the configuration more predictable we switched the
limit off using highest available value (2 147 483 636 bytes)

You could also try to set Main and Quick Mode limits higher on the
Windows side, thus forcing it to always be the responder and Openswan
will always be the initiator.


-- 
 Ilia Sotnikov