[openssl-users] Problem building Linux shared library with static FIPS capable OpenSSL

[Nathan Glasser]

Hello,

We are using openssl-fips 2.0.14 with OpenSSL 1.0.2j.

We have a shared library on both Linux and Windows which uses static OpenSSL
libraries. We'd like it to use static FIPS-capable OpenSSL libraries.

On Windows, everything is fine. On Linux, I have a problem. I am
doing my tests on RedHat 6.0.

I am able to make standalone executables just fine, but shared library (.so)
building does not work. I am linking using supplied the fipsld script.

The script gets error 139, which means a segmentation fault. Modifying
the fipsld script to uncomment the "set -x" at the top shows me that
the following is where the segmentation fault is occurring.

	# generate signature...
	SIG=`"${TARGET}"`

It is attempting to run ${TARGET}, which is the .so file that has just been
generated in the first link step. (It's not suprising to me that this results
in a segmentation fault.) If I run the file which is left after the building
aborts, I also get a segmentation fault.

I can see that there is another case - when the filename matches
lib*|*.dll, which it does not.

If I try renaming the target to have "lib" at the start of the name,
then when it runs this part

	# generate signature...
	SIG=`"${PREMAIN_DSO}" "${TARGET}"`

it fails because there is no fips_premain_dso program. Nor can I find
this anywhere in the openssl-fips or openssl packages. Should this have
gotten built automatically in an earlier step?

I created a simplified test which consists of the fips_hmac sample (included
in the OpenSSL Fips 2.0 manual), with main renamed to something else.

Can someone on this list please point me in the right direction for
getting this to work? Thanks. Below are my makefile and build log.


Here is the makefile I am using. The directory names included reflect
my local environment, but they are all using the versions I stated.

--------------------

CC = gcc
OPENSSLDIR = ../openssl/export_openssl/
FIPSDIR=export_fips/linux/4.4
LIBCRYPTO = $(OPENSSLDIR)/linux/4.4/libcrypto.a
INCLUDES = -I$(OPENSSLDIR)/include
CMD     = fips_hmac
OBJS = $(CMD).o

$(CMD).so: $(OBJS)
	FIPSLD_CC=$(CC) $(FIPSDIR)/bin/fipsld -m32 -shared -Wl,-Bsymbolic -o $(CMD).so $(OBJS) $(LIBCRYPTO) -ldl

$(OBJS): $(CMD).c
	$(CC) -c $(CMD).c $(INCLUDES) -m32

clean:
	rm $(OBJS)

--------------------

Here is the output from the build using the modified fipsld to include
"set -x".

--------------------

gcc -c fips_hmac.c -I../openssl/export_openssl//include -m32
FIPSLD_CC=gcc export_fips/linux/4.4/bin/fipsld -m32 -shared -Wl,-Bsymbolic -o fips_hmac.so fips_hmac.o ../openssl/export_openssl//linux/4.4/libcrypto.a -ldl
+ CC=gcc
+ '[' -n gcc ']'
+ '[' x-m32 '!=' x -a x-m32 '!=' x-c -a x-m32 '!=' x-E ']'
+ shift
+ '[' x-shared '!=' x -a x-shared '!=' x-c -a x-shared '!=' x-E ']'
+ shift
+ '[' x-Wl,-Bsymbolic '!=' x -a x-Wl,-Bsymbolic '!=' x-c -a x-Wl,-Bsymbolic '!=' x-E ']'
+ shift
+ '[' x-o '!=' x -a x-o '!=' x-c -a x-o '!=' x-E ']'
+ shift
+ '[' xfips_hmac.so '!=' x -a xfips_hmac.so '!=' x-c -a xfips_hmac.so '!=' x-E ']'
+ shift
+ '[' xfips_hmac.o '!=' x -a xfips_hmac.o '!=' x-c -a xfips_hmac.o '!=' x-E ']'
+ shift
+ '[' x../openssl/export_openssl//linux/4.4/libcrypto.a '!=' x -a x../openssl/export_openssl//linux/4.4/libcrypto.a '!=' x-c -a x../openssl/export_openssl//linux/4.4/libcrypto.a '!=' x-E ']'
+ shift
+ '[' x-ldl '!=' x -a x-ldl '!=' x-c -a x-ldl '!=' x-E ']'
+ shift
+ '[' x '!=' x -a x '!=' x-c -a x '!=' x-E ']'
+ '[' 0 -ge 1 ']'
++ '[' x-m32 '!=' x -a x-m32 '!=' x-o ']'
++ shift
++ '[' x-shared '!=' x -a x-shared '!=' x-o ']'
++ shift
++ '[' x-Wl,-Bsymbolic '!=' x -a x-Wl,-Bsymbolic '!=' x-o ']'
++ shift
++ '[' x-o '!=' x -a x-o '!=' x-o ']'
++ echo fips_hmac.so
+ TARGET=fips_hmac.so
+ case `basename "${TARGET}"` in
++ basename fips_hmac.so
+ case "$*" in
+ '[' -n fips_hmac.so ']'
+ '[' x-m32 '!=' x -a x-m32 '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ shift
+ '[' x-shared '!=' x -a x-shared '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ shift
+ '[' x-Wl,-Bsymbolic '!=' x -a x-Wl,-Bsymbolic '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ shift
+ '[' x-o '!=' x -a x-o '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ shift
+ '[' xfips_hmac.so '!=' x -a xfips_hmac.so '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ shift
+ '[' xfips_hmac.o '!=' x -a xfips_hmac.o '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ shift
+ '[' x../openssl/export_openssl//linux/4.4/libcrypto.a '!=' x -a x../openssl/export_openssl//linux/4.4/libcrypto.a '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ shift
+ '[' x-ldl '!=' x -a x-ldl '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ shift
+ '[' x '!=' x -a x '!=' x-DDEBUG_FINGERPRINT_PREMAIN ']'
+ '[' 0 -ge 1 ']'
++ echo export_fips/linux/4.4/bin/fipsld
++ sed -e 's|[^/]*$||'
+ THERE=export_fips/linux/4.4/bin/..
++ '[' x-m32 '!=' x ']'
++ case "$1" in
++ shift
++ '[' x-shared '!=' x ']'
++ case "$1" in
++ shift
++ '[' x-Wl,-Bsymbolic '!=' x ']'
++ case "$1" in
++ shift
++ '[' x-o '!=' x ']'
++ case "$1" in
++ shift
++ '[' xfips_hmac.so '!=' x ']'
++ case "$1" in
++ shift
++ '[' xfips_hmac.o '!=' x ']'
++ case "$1" in
++ shift
++ '[' x../openssl/export_openssl//linux/4.4/libcrypto.a '!=' x ']'
++ case "$1" in
++ shift
++ '[' x-ldl '!=' x ']'
++ case "$1" in
++ shift
++ '[' x '!=' x ']'
+ CANISTER_O=
+ '[' -z '' ']'
+ '[' -n '' ']'
+ '[' -f export_fips/linux/4.4/bin/../fips/fipscanister.o ']'
+ '[' -f export_fips/linux/4.4/bin/../lib/fipscanister.o ']'
+ CANISTER_O=export_fips/linux/4.4/bin/../lib/fipscanister.o
+ CANISTER_O_CMD=export_fips/linux/4.4/bin/../lib/fipscanister.o
+ '[' -f export_fips/linux/4.4/bin/../lib/fipscanister.o ']'
++ dirname export_fips/linux/4.4/bin/../lib/fipscanister.o
+ PREMAIN_C=export_fips/linux/4.4/bin/../lib/fips_premain.c
+ HMAC_KEY=etaonrishdlcupfm
+ case "`(uname -s) 2>/dev/null`" in
+ case "${TARGET}" in
+ TARGET=./fips_hmac.so
+ case `basename "${TARGET}"` in
++ basename ./fips_hmac.so
+ '[' -x export_fips/linux/4.4/bin/../fips/fips_standalone_sha1 ']'
+ '[' -x export_fips/linux/4.4/bin/../bin/fips_standalone_sha1 ']'
+ FINGERTYPE=export_fips/linux/4.4/bin/../bin/fips_standalone_sha1
+ export_fips/linux/4.4/bin/../bin/fips_standalone_sha1 export_fips/linux/4.4/bin/../lib/fipscanister.o
+ sed 's/(.*\//(/'
+ diff -w export_fips/linux/4.4/bin/../lib/fipscanister.o.sha1 -
+ export_fips/linux/4.4/bin/../bin/fips_standalone_sha1 export_fips/linux/4.4/bin/../lib/fips_premain.c
+ sed 's/(.*\//(/'
+ diff -w export_fips/linux/4.4/bin/../lib/fips_premain.c.sha1 -
+ /bin/rm -f ./fips_hmac.so
+ gcc export_fips/linux/4.4/bin/../lib/fipscanister.o export_fips/linux/4.4/bin/../lib/fips_premain.c -m32 -shared -Wl,-Bsymbolic -o fips_hmac.so fips_hmac.o ../openssl/export_openssl//linux/4.4/libcrypto.a -ldl
+ '[' x '!=' x ']'
++ ./fips_hmac.so
+ SIG=
make: *** [fips_hmac.so] Error 139

--------------------

Thanks,
Nathan