[Openswan Users] Problem with VM as VPN client under Vista

[P Li]

The remote openswan server is a Linux box with a public IP
(abc.def.45.22) and ports 500 and 4500 open. It is NOT behind a NAT
router.

The client is a guest Linux virtual machine (192.168.117.128) running
in NAT mode under Windows.

Incoming UDP port forwarding was enabled for ports 500 and 4500 for
NAT in VMware. Under Windows, IPSec/IKEEXT services were disabled to
release UDP ports 500 and 4500, Otherwise the VMware NAT service would
not start after port forwarding was set up. I tried both VMware Server
2.0 Beta and VMware Workstation 6.0.2 on Windows XP and Vista.

The host machine was connected to a broadband router or to a cable
modem directly. In both cases, if the host OS is XP, the VPN worked on
the VM.

However, the VPN did not work if the host OS was Vista, no matter the
machine was directly connected to the cable modem or not. Using
Wireshack, I could see the responses from the server were received by
the host machine, The VM even received a few packets from the server
through ports 500 and 4500 at the beginning. But then the packets were
not forwarded to the VM any more.

The capture below shows the initialization process of VPN. Frames
10-12 were three pings from the client, the responses were received by
the Vista host but not forwarded to the VM. When the host was XP,
there was no problem.

No.     Time        Source                Destination           Src
Port Dest Port Protocol Info
      1 0.000000    192.168.117.128       abc.def.45.22         500
  500       ISAKMP   Identity Protection (Main Mode)
      2 0.032642    abc.def.45.22         192.168.117.128       500
  500       ISAKMP   Identity Protection (Main Mode)
      3 0.040566    192.168.117.128       abc.def.45.22         500
  500       ISAKMP   Identity Protection (Main Mode)
      4 0.086090    abc.def.45.22         192.168.117.128       500
  500       ISAKMP   Identity Protection (Main Mode)
      5 0.127976    192.168.117.128       abc.def.45.22         4500
  4500      ISAKMP   Identity Protection (Main Mode)
      6 0.187260    abc.def.45.22         192.168.117.128       4500
  4500      ISAKMP   Identity Protection (Main Mode)
      7 0.195701    192.168.117.128       abc.def.45.22         4500
  4500      ISAKMP   Quick Mode
      8 0.275112    abc.def.45.22         192.168.117.128       4500
  4500      ISAKMP   Quick Mode
      9 0.312014    192.168.117.128       abc.def.45.22         4500
  4500      ISAKMP   Quick Mode
     10 4.460707    192.168.117.128       abc.def.45.22         4500
  4500      ESP      ESP (SPI=0x494bd498)
     11 5.402257    192.168.117.128       abc.def.45.22         4500
  4500      ESP      ESP (SPI=0x494bd498)
     12 6.414442    192.168.117.128       abc.def.45.22         4500
  4500      ESP      ESP (SPI=0x494bd498)
     13 19.289616   192.168.117.128       abc.def.45.22         4500
  4500      UDPENCAP
     14 19.289959   192.168.117.128       abc.def.45.22         4500
  4500      UDPENCAP


The VPN worked when the VM was in the bridged mode. But I need to make
it work with NAT. The confusing part was that there was no problem
under Windows XP.

I did not have any third-party proxy/firewall programs on Vista. I
disabled the Windows Firewall (It might not be necessary to disable
the stateful firewall but I just wanted to try) and UAC, and the
problem persisted.

Here is ipsec.conf. Any suggestions? Thanks.

version 2.0

config setup
        # NAT-TRAVERSAL support, see README.NAT-Traversal
         nat_traversal=yes
conn testuser
        type=tunnel
        left=%defaultroute
        leftid=@testuser
        leftsubnet=10.0.0.12/32
        leftrsasigkey= (deleted)
        right=abc.def.45.22
        rightid=@vpnserver
        rightsubnet=10.28.0.254/24
        rightrsasigkey= (deleted)
        authby=rsasig
        auto=start

#Disable Opportunistic
include /etc/ipsec.d/examples/no_oe.conf