[Openswan dev] FC3/FC4 builds and openswan 2.x

[Paul Wouters]

On Fri, 12 Aug 2005, Michael Richardson wrote:

> We had many reports in the last months, usually from FCx users about
> crashes in des_set_key(), cbc_encrypt, etc. Often they had *IMPOSSIBLE*
> call traces,

> Well, I should have clued in that FCx kernels are built without frame
> pointers, so the call trace is really just a guess. On x86 we use
> assembly versions of AES and 3DES --- and that code makes assumptions
> about the ABI (frame pointers vs not). The assembly code can be
> adjusted, and can be conditionally compiled, but medium to long term, we
> wish to just use the cryptoapi code for software crypto anyway.
>
> HEAD as of 10:15am today has a patch to the AES and 3DES Makefile's that
> will use C versions of the code if frame pointers have been omitted. We
> detect that from the .config file definitions.
>
> Paul is testing the automatic part of the patch to confirm that it makes
> the right decision.

I have done some (limiting) testing so far. The latest FC3 kernel, being
kernel-2.6.12-1.1372_FC3, cannot use a KLIPS module made for a kernel
without CONFIG_FRAME_POINTERS yielding obscure SEGV's:

ipsec_setup: /usr/local/lib/ipsec/_startklips: line 324:  6360 Segmentation fault      ipsec klipsdebug --none

However, using 2.6.11-1.35_FC3, which also was not compiled with CONFIG_FRAME_POINTERS, now correctly works!

Both kernels have CONFIG_CRYPTO_* options (CryptoAPI) enabled.

I have not yet performed tests for FC4 kernels or AMD64 kernels.

So it seems one major problem for running KLIPS on Fedora/RedHat kernels has
been solved, but that in the latest kernel, RedHat changed something else.
I've attached a diff of config-2.6.11-1.35_FC3 and config-2.6.12-1.1372_FC3,
but I do not think it is a kernel option that is responsible for the changed
behaviour.

Paul