[Openswan Users] multiple tunnel fails after upgrade, kernel 2.6.8 bug!?

[Vik Heyndrickx]

Hi,

Problem symptom: When two ipsec tunnels are up between the same gateways (kernel 2.6.8 + openswan 2.1.5), only one tunnel will work, although both tunnels come up. The IPSec gateway looses the detunneled packets for the second tunnel just before they leave the box, as detunneled packets for the first tunnel just leave the box without any problem. It strongly looks like a kernel bug to me, but I'm not sure what to check of where to look.

The full description of the problem with a hopefully nice comprehensive picture, and some test results are attached (it is a plain text file with CR's ;-) ), so I am praying that those greedy virus filters don't strip it off.

I am absolutely willing to contribute to a solution, but I would not have a clue where to look at this moment.

Best regards all,

-- 
Vik
-------------- next part --------------
Problem symptom: I can ping from left host through the IPSec tunnel to right subnet 10.222.223.0/24, but
I can not reach right subnet 10.222.222.0/24 when both tunnels are up at the same time. The de-tunneled
ping packet from left host to right subnet 10.222.222.254 never leaves right gateway.

	  left subnet                       "public" network                    right subnets
         10.111.111.0/24                     192.168.77.0/24                   10.222.222.0/24
                                                                               10.222.223.0/24
+------------+     +-----------------------------+     +---------------------------+     +--------------+
| left host  |     |        left gateway         |     |       right gateway       |     |  right host  |
|    eth0    |-----|      eth1        eth0       |-----|      eth0        eth1     |-----|     eth0     |
|10.111.111.1|     |10.111.111.254 192.168.77.127|     |192.168.77.128 10.222.222.1|     |10.222.222.254|
|            |     |                             |     |               10.222.223.1|     |10.222.223.254|
+------------+     +-----------------------------+     +---------------------------+     +--------------+

                                              IPSEC tunnel
                            10.111.111.0/24 ================ 10.222.222.0/24
                                              IPSEC tunnel
                            10.111.111.0/24 ================ 10.222.223.0/24


All hosts run Fedora Core 2 with kernel 2.6.8, and Openswan 2.1.5 for the IPSec keying. No firewalling or
NAT is applied.

Left host will reach right subnet via left gateway, and right host will reach left subnet via right
gateway.
Left host and right host have set their default gateway to an IP address of a non-existing host, i.e.
10.111.111.69 and 10.222.222.69 respectively.

All other routing entries "ip route show table all" look good on all four hosts. All routing entries for
the networks 10.222.222.0/24 and 10.222.223.0/24 are symmetrical.

Note that Openswan code (actually a shell script _updown) takes care of adding deleting a route on right
gateway to left subnet, and two routes on left gateway to the right subnets.

I can ping from left host to left gateway (and the other way around)
I can ping from left gateway to right gateway (and the other way around)
I can ping from right gateway to right host on both IP addresses (and the other way around)
If the tunnels are down, I can NOT ping from left host to right gateway (this is intentional)
If the tunnels are down, I can NOT ping from left gateway to right host (this is intentional)

If only the tunnel 10.111.111.0/24 - 10.222.222.0/24 is up, I can ping from left host to 10.222.222.254
If only the tunnel 10.111.111.0/24 - 10.222.223.0/24 is up, I can ping from left host to 10.222.223.254

If both tunnels are up, I can ping from left host to 10.222.223.254, but NOT to 10.222.222.254.

If I ping from left host to 10.222.223.69 (a non existing host), I see arp queries for 10.222.223.69
emerging from right gateway on the eth1 side every second (the ping interval).
If I ping from left host to 10.222.222.69 (also non existing), I see NO arp queries for 10.222.222.69
emerging (on any interface, not only eth1). Although the ping packet is received encrypted by the right
gateway, and I even can see the decrypted ping packet when I do "tcpdump -i eth0" on right gateway, the
de-tunnelled ping packet never leaves right gateway. If I do at this moment a ping directly from right
gateway to 10.222.222.254 it responds, and all routing table entries look OK for the packet going out 
on the eth1 interface. But this never happens.

All options for both tunnels are set the same, the tunnels are absolutely certainly up (the fact that
the de-tunnelled packet at right gateway can be seen with tcpdump before it disappears proves that).

Compression on or off does not make a difference. 
The above labo tests is the most simple configuration that I could reduce it to, although the original
problem was noted with more than two networks that are not directly connected to right gateway, and 
which has the Internet between left gateway and right gateway.

For me the mistery is why the de-tunneled ping packet never leaves right gateway. I do not know if this
bug, if it is a bug, is within the kernel code or the openswan code.