[Openswan Users] Wierd ISAKMP message

[Shaheen Ali]

Our testers have come up with a really wierd IPSEC security association
message.  It has two proposals in it, both for the same protocol ISAKMP.

In each proposal is the exact same transform, 3des and sha1.

I think pluto (openswan-1.0.7) rejects the message with a notification
malformed payload.

I have a snippet of the logs and unfortunately, it was not on full pluto
debugging.  So, the log messages said this:

Is this correct?

Jan 3 05:50:57 dunder pluto[862]: "dunce" #101: responding to Main Mode
Jan 3 05:50:57 dunder pluto[862]: "dunce" #101: protocol ID of ISAKMP
Proposal Payload has an unknown value: 0
Jan 3 05:50:57 dunder pluto[862]: "dunce" #101: sending notification
PAYLOAD_MALFORMED to 192.168.0.1:500.

I have looked at the code and I think I know how the packet can get
mis-interpreted.

In my own shorthand, the packet looks like this:

isakmp hdr
{
   type = ID protection
   np   = SA
   len  = 120
   SA payload
   {
        np = none
        len = 92
        doi = ipsec
        sit = identity
        Proposal payload 1
        {
            np = Proposal
            len = 40
            proposal # = 1
            proto = ISAKMP
            ...
            Transform 0
            {
                np = none
                len = 32
                trans # = 0
                id = key_ike
                auth = PSK
                hash = SHA
                encryption = 3DES-CBC
                group-desc = default 768-bit modp
                life-type = seconds
                duration = 1800
             }
         }
        Proposal payload 2
        {
            np = Proposal
            len = 40
            proposal # = 2
            proto = ISAKMP
            ...
            Transform 1
            {
                np = none
                len = 32
                trans # = 1
                id = key_ike
                auth = PSK
                hash = SHA
                encryption = 3DES-CBC
                group-desc = default 768-bit modp
                life-type = seconds
                duration = 1800
             }
         }
      }

The lengths seem to line up and the values seem correct, but notice that
other than the proposal and the transform numbers everything is the same.

Is this legit?

Here is my theory on how this packet got marked as malformed by pluto:

Proposal one payload has a next-payload(np) field set to Proposal rather
than none.  Because of that, rather than assume that the next item is a
transform, it starts processing the transform as a new proposal and
completely chokes since the transform number is aliased to the protocol id
of a proposal payload.

That is why the in_struct routine barfs on the payload and causes a
Malformed_payload notification.

So, my question is still does this packet make any kind of sense and
should openswan have to handle it?

Shaheen