[Openswan dev] nss/nspr4 warning and PLUTO_SENDS_VENDORID vs HAVE_LIBNSS

[Paul Wouters]

Hi Avesh,

I was looking at a bug reported that we do not compile with
USE_VENDORID=false. I disentangled some ifdef's around a
check with Pluto_IsFIPS() to resolve this issue.

It looks like you had wanted to log (or do?) something different when
running in fips mode, but actually did not log the fips mode.  This got
interweaved with PLUTO_SENDS_VENDORID.

Can you verify that I am not missing some hidden assumption.  For
instance, it might be possible you actualy do not want to initiate the
sending vendorid code on purpose in fips mode, but if so, it was not
clear to me if this was done by accident or on purpose. I repaired it
to honour PLUTO_SENDS_VENDORID regardless of the HAVE_LIBNSS setting.

If you actually wanted to not send the vendorid in fips mode, I'd rather
set PLUTO_SENDS_VENDORID based on USE_FIPSCHECK, instead of through some
undocumented/uncommented nested ifdef clause.

While testing this code and enabling HAVE_LIBNSS. with or without
USE_FIPSCHECK, I get the following warning/error:

cc -c -DHAVE_LIBNSS -g -Werror -DKLIPS -DSCANDIR_HAS_CONST  -I/vol/git/openswan.git/ports/linux/include  -I/vol/git/openswan.git/ports/linux/include  -I/vol/git/openswan.git/ports/linux/include  -I/vol/git/openswan.git/ports/linux/include  -I/usr/include/nspr4 -I/usr/include/nss3 -I/vol/git/openswan.git -I/vol/git/openswan.git/linux/include -I/vol/git/openswan.git/include  -Wall -Wpointer-arith -Wcast-qual -Wstrict-prototypes -Wbad-function-cast  -DKLIPS -DNAT_TRAVERSAL -DNAT_TRAVERSAL -DKERNEL_ALG -DIKE_ALG -DFINALCONFDIR=\"/etc\" -DFINALCONFDDIR=\"/etc/ipsec.d\" -DFINALCONFFILE=\"/etc/ipsec.conf\" -DFINALVARDIR=\"/var\" -Werror /vol/git/openswan.git/programs/showhostkey/showhostkey.c
cc1: warnings being treated as errors
In file included from /usr/include/nss3/secport.h:48,
                  from /usr/include/nss3/seccomon.h:63,
                  from /usr/include/nss3/nss.h:78,
                  from /vol/git/openswan.git/include/oswconf.h:25,
                  from /vol/git/openswan.git/programs/showhostkey/showhostkey.c:43:
/usr/include/nspr4/prlink.h:52: error: function declaration isn’t a prototype
In file included from /usr/include/nss3/secport.h:48,
                  from /usr/include/nss3/seccomon.h:63,
                  from /usr/include/nss3/nss.h:78,
                  from /vol/git/openswan.git/include/oswconf.h:25,
                  from /vol/git/openswan.git/programs/showhostkey/showhostkey.c:43:

This happens in showhostkey and newrsakey. Do you not have this problem?
This is on Fedora 12 with nspr-4.8.2-1.fc12.x86_64 and
nss-3.12.4-14.fc12.x86_64

Paul