[Openswan Users] Linux (debian lenny) client to Checkpoint Firewall NGx R65 using certificates - secureclient ok, openswan ko - PAYLOAD_MALFORMED

[Paul Wouters]

On Mon, 10 May 2010, Luca Arzeni wrote:

> Hi there,
> I'm trying to connect a debian lenny client (Linux 2.6.26-2-amd64 #1 SMP x86_64 GNU/Linux) to a Checkpoint
> Firewall 1 (NGx R65).

They do some funky stuff like hybrid mode. It might be that your particular configuration
is not supported with openswan. There is an unsupported patch in contrib/ that could
help you.

> I'm using openswan 2.4.12 (debian lenny revision is openswan  2.4.12+dfsg-1.3+lenny2)

Update to the latest 2.4.x.

> I followed various how to, but in the and I'm stuck at this point:
> 
> May 10 16:49:14 gadara pluto[9253]: | p15 state object #1 found, in STATE_MAIN_I2
> May 10 16:49:14 gadara pluto[9253]: | processing connection checkpoint-vpn
> May 10 16:49:14 gadara pluto[9253]: | processing informational PAYLOAD_MALFORMED (16)
> May 10 16:49:14 gadara pluto[9253]: "checkpoint-vpn" #1: received 1 malformed payload notifies
> May 10 16:49:14 gadara pluto[9253]: | complete state transition with STF_IGNORE
> May 10 16:49:14 gadara pluto[9253]: | next event EVENT_RETRANSMIT in 10 seconds for #1

A plutodebug=all might be required to check out the malformed payload.

Paul

> Is there anyone successfull connecting a Checkpoint R65 to an openswan client?
> Connection data and log follows... any hint will be appreciated!
> 
> Thanks, Luca
> 
> ############################
> ### this is the /etc/ipsec.conf ###
> # basic configuration
> config setup
>         plutodebug=control
>         nat_traversal=yes
> 
> conn checkpoint-vpn
>         # left is my lenny client
>         left=%defaultroute
>         leftcert=/etc/ipsec.d/certs/client-cert.pem
>         leftrsasigkey=%cert
>         # right is the checkpoint firewall
>         right=checkpoint-fw
>         rightsubnet=192.168.255.0/24
>         rightcert=/etc/ipsec.d/certs/checkpoint-cert.pem
>         rightrsasigkey=%cert
>         auto=start
> 
> #Disable Opportunistic Encryption
> include /etc/ipsec.d/examples/no_oe.conf
> 
> #################################
> ### this is the /var/log/syslog output ###
> May 10 16:49:13 gadara ipsec_setup: NETKEY on eth0 192.168.144.162/255.255.255.0 broadcast 192.168.144.255
> May 10 16:49:13 gadara ipsec_setup: ...Openswan IPsec started
> May 10 16:49:13 gadara ipsec_setup: Starting Openswan IPsec U2.4.12/K2.6.26-2-amd64...
> May 10 16:49:14 gadara ipsec__plutorun: 104 "checkpoint-vpn" #1: STATE_MAIN_I1: initiate
> May 10 16:49:14 gadara ipsec__plutorun: ...could not start conn "checkpoint-vpn"
> 
> ###################################
> ### this is the /var/log/auth.log output ###
> 
> May 10 17:17:07 gadara pluto[9858]: "checkpoint-vpn" #3: transition from state STATE_MAIN_I1 to state
> STATE_MAIN_I2
> May 10 17:17:07 gadara pluto[9858]: | sending reply packet to x.y.z.w:500 (from port=500)
> May 10 17:17:07 gadara pluto[9858]: | sending 292 bytes for STATE_MAIN_I1 through eth0:500 to x.y.z.w:500:
> May 10 17:17:07 gadara pluto[9858]: | inserting event EVENT_RETRANSMIT, timeout in 10 seconds for #3
> May 10 17:17:07 gadara pluto[9858]: "checkpoint-vpn" #3: STATE_MAIN_I2: sent MI2, expecting MR2
> May 10 17:17:07 gadara pluto[9858]: | modecfg pull: noquirk policy:push not-client
> May 10 17:17:07 gadara pluto[9858]: | phase 1 is done, looking for phase 1 to unpend
> May 10 17:17:07 gadara pluto[9858]: | next event EVENT_RETRANSMIT in 10 seconds for #3
> May 10 17:17:07 gadara pluto[9858]: |
> May 10 17:17:07 gadara pluto[9858]: | *received 40 bytes from x.y.z.w:500 on eth0 (port=500)
> May 10 17:17:07 gadara pluto[9858]: |  processing packet with exchange type=ISAKMP_XCHG_INFO (5)
> May 10 17:17:07 gadara pluto[9858]: | ICOOKIE:  62 d9 1d c3  7c 2c b2 a9
> May 10 17:17:07 gadara pluto[9858]: | RCOOKIE:  fd 0b ef 4b  36 d6 7d 9b
> May 10 17:17:07 gadara pluto[9858]: | peer:  x.y.z.w
> May 10 17:17:07 gadara pluto[9858]: | state hash entry 29
> May 10 17:17:07 gadara pluto[9858]: | peer and cookies match on #3, provided msgid 00000000 vs 00000000/00000000
> May 10 17:17:07 gadara pluto[9858]: | p15 state object #3 found, in STATE_MAIN_I2
> May 10 17:17:07 gadara pluto[9858]: | processing connection checkpoint-vpn
> May 10 17:17:07 gadara pluto[9858]: | processing informational PAYLOAD_MALFORMED (16)
> May 10 17:17:07 gadara pluto[9858]: "checkpoint-vpn" #3: received 1 malformed payload notifies
> May 10 17:17:07 gadara pluto[9858]: | complete state transition with STF_IGNORE
> May 10 17:17:07 gadara pluto[9858]: | next event EVENT_RETRANSMIT in 10 seconds for #3
> 
> 
> --
> Luca Arzeni
> Tel.: 339 8350298
> mailto: l.arzeni at gmail.com
> 
> === Start-of Internet E-mail Confidentiality Footer ===
> 
> L'uso non autorizzato di questo messaggio o dei suoi allegati e' vietato e potrebbe costituire reato.
> Se ha ricevuto per errore questo messaggio, La prego di informarmi e di distruggerlo immediatamente coi suoi
> allegati. Le dichiarazioni contenute in questo messaggio o nei suoi allegati non impegnano Luca Arzeni nei
> confronti del destinatario o di terzi. Luca Arzeni non si assume alcuna responsabilita' per eventuali
> intercettazioni, modifiche o danneggiamenti del presente messaggio.
> 
>