OpenSSL 1.1.1b tests fail on Solaris - solution and possible fix

Wed May 15 20:08:29 UTC 2019

>> Issue #pragma weak for a symbol only in the files that define that symbol, not in the ones that merely reference it.

The problem is that those pragmas are in .h files. They contain inline functions that use those symbols. The pragmas were added because of problems with apps that used the .h files (hard to avoid since they are basic crypto header files) but did not link with libcrypto.so. That library was explicitly loaded by the app, but the app would not start because of the missing symbols.

See issues 6912 and 8102.

Regards,
John.

John Unsworth |Meta-Directory Engineering and Support
Mobile: +44 777.557.2643

-----Original Message-----
From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of Jakob Bohm via openssl-users
Sent: 15 May 2019 16:11
To: openssl-users at openssl.org
Subject: Re: OpenSSL 1.1.1b tests fail on Solaris - solution and possible fix

CAUTION: This email originated from outside of Synchronoss.

Alternative suggestion (from my understanding of the documentation quoted
below):

Issue #pragma weak for a symbol only in the files that define that symbol, not in the ones that merely reference it.

The hoped effect would be:

1. Object files that merely reference a symbol will contain regular UNDEF
   entries
2. Object files that do define such a symbol will contain a WEAK defined
   entry
3. When seeing the UNDEF entries, the linker will look for an actual
   definition
4. If finding multiple WEAK defined entries, the linker will not complain,
   just use the first found.

I suspect the specified linker semantics of not trying to satisfy "WEAK UNDEF" entries would be suitable for optional link-time selected callbacks or data where a referencing library do runtime tests to see if the library- using program has provided a definition or not.  For example, it could be used for a symbol holding a list of global C++ constructors/destructors with libc code invoking that list if present without requiring dummy lists in programs without.

The .so case mentioned probably works "by chance" because libssl.so references non-weak libcrypto.so symbols, forcing the definition to be included in the runtime process anyway.

On 14/05/2019 11:29, John Unsworth wrote:
> Because of the #pragma weak directive the functions are defined multiple times in both libcrypto.a and libssl.a:
>
> libcrypto.a
>
> Many UNDEF:
> ct_log.o
> [47]    |                   0|                   0|FUNC |WEAK |0    |UNDEF  |OPENSSL_sk_new_null
> ... and more
>
> One definition from stack.c as you'd expect:
> stack.o
> [33]  |                1232|                  20|FUNC |WEAK |2    |2      |OPENSSL_sk_new_null
>
> libssl.a has multiple instances also, all UNDEF:
> d1_srtp.o
> [43]  |                   0|                   0|FUNC |WEAK |0    |UNDEF  |OPENSSL_sk_new_null
> s3_lib.o
> [107] |                   0|                   0|FUNC |WEAK |0    |UNDEF  |OPENSSL_sk_new_null
> and so on.
>
>  From the linker docs:
> B.2.17.1 #pragma weak name
> In the form #pragma weak name, the directive makes name a weak symbol. The linker will not complain if it does not find a symbol definition for name. It also does not complain about multiple weak definitions of the symbol. The linker simply takes the first one it encounters.
>
> So when linking with libcrypto.a and libssl.a the first definition is taken which is UNDEF in both cases leading to the error.
> This can be fixed in libcrypto.a by making stack.o, lh_stats.o and lhash.o the first files included. It could also be fixed by not defining those pragmas when compiling stack.c, lh_stats.c and lhash.c because they will then be GLOB instead of WEAK and will override all the WEAK definitions.
>
> However there is no suitable fix for libssl.a because those files are not part of that library.
>
> Doing that means that executables that link libcrypto.a before libssl.s are built correctly. However if the link is libssl.a followed by libcrypto.a the UNDEF symbols from libssl.a are still used.
>
> In the shared libraries I see:
>
> libcrypto.so only instance:
> [3537]        |             2409672|                  20|FUNC |LOCL |2    |10     |OPENSSL_sk_new_null
>
> Shared libssl.so also one instance:
> [2091]        |                   0|                   0|FUNC |WEAK |0    |UNDEF  |OPENSSL_sk_new_null
>
> and it seems that the linker will resolve those correctly.
>
> So it seems to me that the only clean solution is to not define those pragmas when compiling stack.c, lh_stats.c and lhash.c by including a suitable define in the makefiles when building the libraries that is used in safestack.h and lhash.h to omit them.
>
> I see there is also
> # elif defined(__SUNPRO_C)
> #pragma weak getisax
> in crypto\sparcv9cap.c
> so maybe that needs consideration too.
>
> Regards,
> John.
>
> -----Original Message-----
> From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of
> John Unsworth
> Sent: 10 May 2019 16:23
> To: openssl-users at openssl.org
> Subject: RE: OpenSSL 1.1.1b tests fail on Solaris - solution
>
> CAUTION: This email originated from outside of Synchronoss.
>
>
> This seems to be caused by the ongoing saga documented in issues 6912 and 8102. These functions were declared as weak in 1.1.1b.
>
> safestack.h
> #  pragma weak OPENSSL_sk_num
> #  pragma weak OPENSSL_sk_value
> #  pragma weak OPENSSL_sk_new
> #  pragma weak OPENSSL_sk_new_null
> ...
>
> lhash.h
> #  pragma weak OPENSSL_LH_new
> #  pragma weak OPENSSL_LH_free
> #  pragma weak OPENSSL_LH_insert
> #  pragma weak OPENSSL_LH_delete
> ...
>
> Removing these lines allows all the tests to work. I am building static libraries (no-shared) which is presumably why it works for some people out of the box if they are using shared libraries.
>
> Please advise as to the way forward. I note that one other user (see 8102) has reported this too:
>
> rammishr commented on Mar 26
> Hi,
> Sorry if this should have been posted separately.
>
> When i build openssl1.1.1b on solaris , it is successful. But while verifying the version " openssl version" it fails with below error.
> ld.so.1: openssl: fatal: relocation error: file
> /vobs_tools/prgs/src/openssl/solaris64/bin/openssl: symbol
> OPENSSL_sk_new_null: referenced symbol not found Killed
>
> The same machine works fine for openssl1.1.1.
> So, in 1.1.1b I can observe that OPENSSL_sk_new_null has been defined as below in safestack.h.
> pragma weak OPENSSL_sk_new_null
> Can this be related ? Am I missing anything while Configure ?
> <<<<<<<<<<<<<<<
>
> Regards,
> John.
>
> -----Original Message-----
> From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of
> John Unsworth
> Sent: 09 May 2019 10:13
> To: openssl-users at openssl.org
> Subject: RE: OpenSSL 1.1.1b tests fail on Solaris
>
> CAUTION: This email originated from outside of Synchronoss.
>
>
> This looks like the problem:
>
> ld.so.1: sanitytest: fatal: relocation error: file
> ../../test/sanitytest: symbol OPENSSL_sk_new_null: referenced symbol
> not found ../../util/shlib_wrap.sh ../../test/sanitytest => 137 not ok
> 1 - running sanitytest
>
> #   Failed test 'running sanitytest'
> #   at /home/metabld/OpenSSL/openssl-1.1.1b/test/../util/perl/OpenSSL/Test/Simple.pm line 77.
> # Looks like you failed 1 test of 1.
> Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests
>
> This results in the same error:
> sol-mds-build-01 $ cd apps
> sol-mds-build-01 $ ./openssl version
> ld.so.1: openssl: fatal: relocation error: file openssl: symbol
> OPENSSL_sk_new_null: referenced symbol not found
>
> I have built static libraries.
>
> John
>
> -----Original Message-----
> From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of
> Matt Caswell
> Sent: 09 May 2019 09:38
> To: openssl-users at openssl.org
> Subject: Re: OpenSSL 1.1.1b tests fail on Solaris
>
> CAUTION: This email originated from outside of Synchronoss.
>
>
> What is the output from:
>
> $ make V=1 TESTS=test_sanity test
>
> Matt
>
> On 08/05/2019 19:22, John Unsworth wrote:
>> I have build OpenSSL 1.1.1b 64 bit on Solaris SunOS 5.10
>> Generic_Virtual sun4v sparc SUNW,T5140.
>>
>>
>>
>> ./Configure -lrt solaris64-sparcv9-cc no-shared -m64 -xcode=pic32
>> -xldscope=hidden
>>
>>
>>
>> It builds fine but all the tests fail, with or without no-asm. Can
>> anyone help please? Here is the start of the test run:
>>
>>
>>
>> $ make test
>>
>> make depend && make _tests
>>
>> ( cd test; \
>>
>>    mkdir -p test-runs; \
>>
>>    SRCTOP=../. \
>>
>>    BLDTOP=../. \
>>
>>    RESULT_D=test-runs \
>>
>>   PERL="/opt/perl-5.26.1/bin/perl" \
>>
>>    EXE_EXT= \
>>
>>    OPENSSL_ENGINES=`cd .././engines 2>/dev/null && pwd` \
>>
>>    OPENSSL_DEBUG_MEMORY=on \
>>
>>      /opt/perl-5.26.1/bin/perl .././test/run_tests.pl  )
>>
>> ../test/recipes/01-test_abort.t .................... ok
>>
>> ../test/recipes/01-test_sanity.t ................... Dubious, test
>> returned 1 (wstat 256, 0x100)
>>
>> Failed 1/1 subtests
>>

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20190515/d268d694/attachment-0001.html>