[Openswan Users] Ubuntu Linux 2.6.15, OpenSwan and Checkpoint FW-1 NG

Gerard Toonstra gtoonstra at gmail.com
Sat Dec 2 14:20:54 EST 2006

Hi guys,

I hope that the maintainers of openswan would consider including this howto
in the openswan documentation website.

After 2 days of hard work figuring out how to connect a standard Ubuntu box
to a
CP FW-1 server, I can document here the fruits of that work. It's a tutorial
on getting
this setup working (also depending on config of FW-1 of course!).




Ubuntu Linux 2.6.15 (just set to 'latest' in aptitude), OpenSwan and
Checkpoint FW-1 NG (55 I think)

This is a guide to connect your Ubuntu linux (or any other more recent I
to a checkpoint firewall, which may use a hybrid mode for connecting. Some
firewalls are setup that way and they don't use RSA public keys for
or shared keys for all roadwarriors.

This guide assumes Ubuntu, but it probably works on Debian as well and
some wiz's should be able to make it work on SuSE, RedHat and other


DISCLAIMER:  I am not an ipsec or openswan wiz, nor a security export.
I'm just a regular Linux guy and need to connect with VPN to work. Since we
some clients with a lot of influence, the config of the firewall is not
really easy to
change into public key auth...(bureaucracy). So there you go, I had to use
xauth and
CP style user authentication.

Since CP only provides that RedHat 2.4 SecureClient code, I was
not prepared to downgrade to 2.4, nor leave another box running, wasting
just because some guys are too protective about their source code.

Working this out took me about 2 days in total, trying out every different
option in the
book and reading miles of docs on ipsec, klips, netkey, routers and other
stuff, including
the examples in the openswan source code ('testing/pluto' or somewhere).

I hope it is useful for people with similar configurations.


Prerequisites for this howto:

    * Linux 2.6.15 (my kernel is the plain, vanilla Ubuntu kernel. no
modifications. This has the netkey module, not klips).
    * aptitude
    * openswan 2.4.4 sources
    * The OpenSCclient patch to connect to CP FW-1 in one of the links.
    * gcc, bison, flex, gawk, vi
    * ethereal probably
    * A firewall vpn login (this gives you a username & password)
    * I recommend a direct connection to connect first to eliminate any
router NAT problems. For ADSL:
      ifconfig eth0 up
      sudo pppoeconfig
      pon dsl-provider
      ( turn off = "poff" )
    * The public certificate of the Certificate Authority that signed the
public key of the firewall server. (we'll authenticate the server through
it's RSA public certicate, actually identify the key that it should have).
    * The public certificate of the FW-1 server to extract the RSA sig key.
(see openSCclient website).

The bit about not using a router has to do with Network Address Translation.
This is where the IP of your machine is in your own LAN and the
communication all goes through your router which strips of the IP and puts
its own in place, but later replaces the IP and routes the packet to the
proper terminal behind this router. This is fine for regular inet traffic,
but when you depend on source IP's and destination IP's to send packets
through encrypted tunnels it doesn't work too good.

Now, secureclient and openswan both have options called "NAT-T", which is
NAT traversal. This encapsulates the packets in UDP, but I cannot confirm
this at home since my router is one of those crap routers where the VPN
connection doesn't work. It basically does not allow for the connection to
be set up. Check out the last link here in the end to see some of the
problems people have both on windows and naturally on Linux, since the
problem is the router, not the software or local computer.

Please read up a bit first on the concepts of openswan before attempting
this tutorial. It helps in troubleshooting :) Here are some interesting

Very good troubleshooting and config guide, step by step:


explanation how IKE works:






OpenSCclient, the core of the whole thing that allows openswan to work with
but allegedly breaks other things, which is the hesitation in making this
part of
openswan, afaik.

You definitely need this:


useful commands for debugging throughout this howto:
vi /etc/ipsec.conf
/etc/init.d/ipsec restart
ipsec auto --up mycompanyvpn
ipsec auto --status
ipsec barf
ipsec verify
tcpdump -i ethX
ethereal &
/etc/init.d/ipsec restart (yup, you may be doing this a lot)
tail -f /var/log/auth.log
vi /var/log/syslog
vi /var/log/kern.log
vi /var/log/messages
route -n
ip route
Here we go. First we remove openswan and ipsec-tools, because we will build
them with a small patch file later and install that on ubuntu.

The rest are commands to download source from ubuntu (patched files from
ubuntu distrib) and some other things to get it installed. Get the necessary
patch file from the attachment to this article. (see all the way below).

apt-get remove openswan
apt-get remove ipsec-tools
cd /usr/src
apt-get source openswan
mv openswan-2.4.4 openswan-2.4.0
patch -p0 < openswan-2.4.0-SecureClient.diff
( changes should merge successfully )
mv openswan-2.4.0 openswan-2.4.4
cd openswan-2.4.4
( If you did not have gawk, flex, bison or gcc installed, this is where you
will have to resolve those dependencies ).
cd ..
( rm /etc/ipsec.d/private/****key.pem? )
( rm /etc/ipsec.d/certs/****cert.pem? )
dpkg -i openswan_2.4.4-3ubuntu1_i386.deb

The files /usr/lib/ipsec/auto and /usr/libexec/ipsec/auto are incorrect.
It uses a function called "default" which is a keyword. vi those files and

default("cpsc", "no")


set_os_default("cpsc", "no")

( the patch is old and this was a problem at 2.4.0 still. In 2.4.4 this was
already fixed )

Now, let's edit ipsec.conf

This is my /etc/ipsec.conf:

version 2.0

include /etc/ipsec.d/examples/no_oe.conf

config setup
        # plutodebug="raw crypt parsing emitting control klips all private
        # enable this later when it all works
        # plutodebug="none"
        # set this log line in ipsec.conf during connection debug. disable
later when it works.
        # enable the next for nat traversal, (using router?) not done this
        # nat_traversal=yes

conn mycompanyvpn
        # right = your company's gateway (firewall IP?)
        # rightsubnet = your company's subnet that you want to reach behind
the gateway
        rightrsasigkey=0x1234567890ABCDEF....... (truncated)
        # might help for NAT resolutions later on. not tried yet.
        # forceencaps=yes
        # The following settings for ike= and esp= may need some
modifications, depending on the
        # firewall settings and the version. These worked for my
        # If you get "NO_PROPOSAL_CHOSEN" in the auth.log (Ubuntu), this is
when you look at these settings.
        # adds this route to the ipsec config, but does not bring it up yet.


The rightrsasigkey needs to be populated with the value as per the
openscclient website instructions.
It is the RSA public key signature from the Checkpoint FW-1 server.

This line would already take care of things, but the problem is that we need
to start "trusting" that
sigkey and public certificate. You need to put the CA public certificate
into /etc/ipsec.d/cacerts, also
according to the instructions. That should theoretically take care of things
and if you have your login,
you can now attempt to connect.

open a new "log" terminal as root and:

tail -f /var/log/auth.log
Now, from the main terminal as root we will bring up our tunnel:

/etc/init.d/ipsec restart
ipsec verify
ipsec auto --up mycompanyvpn
( wait for terminal to prompt for your username and password, if the
negotiation gets that far!)
ping  ( = any server in your company's network that you
normally connect to.

if you get problems about "unknown connections", try:

ipsec auto --add mycompanyvpn

If this did not work, errors are printed in the logs and on the console.

see /var/log/auth.log

correct log output:
104 "mycompanyvpn" #1: STATE_MAIN_I1: initiate
106 "mycompanyvpn" #1: STATE_MAIN_I2: sent MI2, expecting MR2
108 "mycompanyvpn" #1: STATE_MAIN_I3: sent MI3, expecting MR3
004 "mycompanyvpn" #1: STATE_MAIN_I4: ISAKMP SA established
{auth=OAKLEY_RSA_SIG cipher=oakley_3des_cbc_192 prf=oakley_md5
003 "mycompanyvpn" #1: discarding duplicate packet; already STATE_MAIN_I4
003 "mycompanyvpn" #1: discarding duplicate packet; already STATE_MAIN_I4
041 "mycompanyvpn" #1: mycompanyvpn prompt for Username:
Name enter:   <username here>
004 "mycompanyvpn" #1: STATE_CPSC_I1: CP SecureClient - awaiting password
040 "mycompanyvpn" #1: mycompanyvpn prompt for Password:
004 "mycompanyvpn" #1: STATE_CPSC_I2: CP SecureClient - awaiting
authentication status
004 "mycompanyvpn" #1: STATE_MAIN_I4: ISAKMP SA established
117 "mycompanyvpn" #2: STATE_QUICK_I1: initiate
003 "mycompanyvpn" #2: ignoring informational payload, type
004 "mycompanyvpn" #2: STATE_QUICK_I2: sent QI2, IPsec SA established
{ESP=>0x73ef1a8e <0x5d044af4 xfrm=3DES_0-HMAC_SHA1 NATD=none DPD=none}

Some common troubleshooting:

    * NO_PROPOSAL_CHOSEN : The server did not like the proposed encryption
algorithms either in ike= or in esp=. If this happens in STATE_MAIN_I1 or
MAIN_I*, it is the IKE negotiation. If it happens in STATE_QUICK_I1, it is
    * Cannot see other servers on the network: Does your /etc/resolv.conf
contain nameservers of your company's network?
    * received packet with more than 20 payloads; ignored; : broken NAT-T
implementation or invalid router. I have not been able yet to play with
routers enough to see if NAT works with openswan. Resolution: get a
different router (not sure which one) or connect directly to the Internet. I
have a NETGear WGR614 and it doesn't seem to like VPN's that much. This is
why a direct connection first eliminates those possibilities.
    * hanging at authentication:  check auth.log for any errors, especially
    * Established tunnel, but no traffic: The kernel needs to determine how
to route traffic. It does this by the source IP and dest IP combination and
the subnets. Maybe the kernel has a bug, you have ip forwarding on or off or
some other filtering occurs in the kernel. Check this out. Are you using
iptables?  Check through some logs if any packets are being dropped in the
ipsec kernel code because it can't find a route. Check out ip route
In case you have klips and netkey loaded at same time in the kernel... ugh!
I had this and the packets on return all get dropped without a trace. so...
don't do it and fix your kernel config before you do anything else.
    * For routers: You need to check out which ports to open up on the
router. I think you need 4500 and 500 for Checkpoint FW-1, but some people
at forums are opening more (Checkpoint users forum). You should probably
configure "port forwarding" on your router before things even remotely start
to work.

Good luck!

Gerard Toonstra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openswan.org/pipermail/users/attachments/20061202/ad89311b/attachment-0001.html 

More information about the Users mailing list