[Openswan Users] [OT] Kerberos not tunneled between openswan and linsys client

John A. Sullivan III jsullivan at opensourcedevel.com
Fri Dec 2 22:26:37 CET 2005

We've been struggling to get a client who wanted to use the native
Microsoft IPSec stack managed through the linsys front end
(http://sourceforge.net/projects/lsipsectool).  The tunnel worked great
but windows kerberos authentication usually failed.  To our great
surprise, we found out the Microsoft excepts Kerberos traffic from an
IPSec tunnel by default.  We need a registry hack to change the default
behavior as follows:

1. Start Registry Editor (Regedt32.exe).

2. Click the following registry key:

3. On the Edit menu, click Add Value, and then add the following value:
Value Name: NoDefaultExempt (note that this name is case sensitive)
Data Type: REG_DWORD
Data Value: 0 or 1

• 0: Default exemptions apply (default)
• 1: RSVP and Kerberos are not exempted (only IKE, Multicast, and
Broadcast are exempted)

We need to set it to "1".  There is also a value of "2" for XP which
allows broadcast and multicast packets to be tunneled.

We also needed to use the overridemtu value to accommodate PPPoE
Ethernet connections using NAT-T.  We weren't sure what size to set it
to.  It seems that the ESP header size changes depending on the
encryption algorithm (someone please correct me if I'm wrong) so we set
it to 1400 to accommodate any algorithm.  We could probably have
squeezed another six to 22 bytes out of it but weren't sure.

Finally, we're struggling with the Microsoft network browsing nightmare.
We are using WINS but, the windows clients do not appear to be querying
the WINS servers even if we force them to p nodes.  We'll post that when
we resolve it in hopes that someone else struggling to use Openswan and
linsys this way will not have to reinvent the wheel.  If anyone can
guide us on the browsing issue, we'd appreciate it.  We are suspecting
it has to do either with needing to enable file and print sharing on the
clients or forcing them to not be browse list servers.  Thanks - John
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan at opensourcedevel.com

If you would like to participate in the development of an open source
enterprise class network security management system, please visit

More information about the Users mailing list