[Openswan dev] pluto crashes on rsa key

[Paul Wouters]

On Thu, 20 Apr 2006, sasa wrote:

> [root at fw pluto]# gdb -c /var/tmp/core.13540 /usr/libexec/ipsec/pluto

> Program terminated with signal 8, Arithmetic exception.
> Reading symbols from /usr/lib/sse2/libgmp.so.3...done.
> Loaded symbols for /usr/lib/sse2/libgmp.so.3

> #0  0x0065e887 in __gmp_exception (error_bit=2) at ../errno.c:40
> 40      ../errno.c: No such file or directory.
>        in ../errno.c
> (gdb) bt
> #0  0x0065e887 in __gmp_exception (error_bit=2) at ../errno.c:40
> #1  0x0065e8d2 in __gmp_divide_by_zero () at ../errno.c:56
> #2  0x0067030d in __gmpz_tdiv_r (rem=0xfefb3310, num=0xfefb3310,
> den=0xa000970) at ../../mpz/tdiv_r.c:41
> #3  0x0066ba78 in __gmpz_mod (rem=0xfefb3310, dividend=0xfefb3310,
> divisor=0xa000970) at ../../mpz/mod.c:45
> #4  0x080766e7 in RSA_private_key_sanity (k=0xa000954) at keys.c:160
> #5  0x080778f9 in process_rsa_keyfile (rsak=0xa000954, whackfd=10) at
> keys.c:772
> #6  0x0807852b in process_secrets_file (file_pat=0xfefb45e6 "RSA
> vpngateway_key.pem \"brontolo221271\" {", whackfd=-1)
>    at keys.c:1028
> #7  0x080719e4 in whack_handle (whackctlfd=10) at rcv_whack.c:390
> #8  0x0805c268 in call_server () at server.c:1173
> #9  0x080592bb in main (argc=9, argv=0xfefb5cf4) at plutomain.c:783

Hmm, so it crashes in the gmp library. Did you compile that yourself? Which
version of gmp is that? Is there a newer version?
Can you give us more information about this particular RSA key/cert?

Paul