OpenSSL 1.1.1b tests fail on Solaris - solution and possible fix

Thu May 16 18:53:46 UTC 2019

Thanks. So in the meantime we can remove the DEFINE macros and the pragmas (or just the pragmas as we have done) to build the no-shared libraries.

Regards,
John.

-------- Originalmeddelande --------
Från: Richard Levitte <levitte at openssl.org>
Skickat: 16 maj 2019 08:34:06 GMT-07:00
Till: John Unsworth <John.Unsworth at synchronoss.com>
Ämne: RE: OpenSSL 1.1.1b tests fail on Solaris - solution and possible fix

The actual problem is the call of DEFINE macros in safestack.h. They should not be there.

The only reason those lines haven't been removed us that we fear someone might have use of them. I plan to submit a PR for 3.0 that does remove those lines.

Cheers
Richard

John Unsworth <John.Unsworth at synchronoss.com> skrev: (16 maj 2019 03:46:49 GMT-07:00)
>In the absence of any steer from openssl gurus we will proceed by 
>removing the #pragmas in safestack.h and lhash.h while we build the 
>no-shared libraries on solaris. Hopefully someone will come up with a 
>proper fix at some point.
>
>Regards,
>John
>
>From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of 
>John Unsworth
>Sent: 15 May 2019 21:08
>To: openssl-users at openssl.org
>Subject: OpenSSL 1.1.1b tests fail on Solaris - solution and possible 
>fix
>
>>> Issue #pragma weak for a symbol only in the files that define that
>symbol, not in the ones that merely reference it.
>The problem is that those pragmas are in .h files. They contain inline 
>functions that use those symbols. The pragmas were added because of 
>problems with apps that used the .h files (hard to avoid since they are 
>basic crypto header files) but did not link with libcrypto.so. That 
>library was explicitly loaded by the app, but the app would not start 
>because of the missing symbols.
>
>See issues 6912 and 8102.
>
>Regards,
>John.
>
>-----Original Message-----
>From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of 
>Jakob Bohm via openssl-users
>Sent: 15 May 2019 16:11
>To: openssl-users at openssl.org
>Subject: Re: OpenSSL 1.1.1b tests fail on Solaris - solution and 
>possible fix
>
>Alternative suggestion (from my understanding of the documentation 
>quoted
>below):
>
>Issue #pragma weak for a symbol only in the files that define that 
>symbol, not in the ones that merely reference it.
>
>The hoped effect would be:
>
>1. Object files that merely reference a symbol will contain regular 
>UNDEF
>   entries
>2. Object files that do define such a symbol will contain a WEAK 
>defined
>   entry
>3. When seeing the UNDEF entries, the linker will look for an actual
>   definition
>4. If finding multiple WEAK defined entries, the linker will not 
>complain,
>   just use the first found.
>
>I suspect the specified linker semantics of not trying to satisfy "WEAK 
>UNDEF" entries would be suitable for optional link-time selected 
>callbacks or data where a referencing library do runtime tests to see 
>if the library- using program has provided a definition or not.  For 
>example, it could be used for a symbol holding a list of global C++ 
>constructors/destructors with libc code invoking that list if present 
>without requiring dummy lists in programs without.
>
>The .so case mentioned probably works "by chance" because libssl.so 
>references non-weak libcrypto.so symbols, forcing the definition to be 
>included in the runtime process anyway.
>
>On 14/05/2019 11:29, John Unsworth wrote:
>> Because of the #pragma weak directive the functions are defined
>multiple times in both libcrypto.a and libssl.a:
>>
>> libcrypto.a
>>
>> Many UNDEF:
>> ct_log.o
>> [47]    |                   0|                   0|FUNC |WEAK |0
>|UNDEF  |OPENSSL_sk_new_null
>> ... and more
>>
>> One definition from stack.c as you'd expect:
>> stack.o
>> [33]  |                1232|                  20|FUNC |WEAK |2    |2
>    |OPENSSL_sk_new_null
>>
>> libssl.a has multiple instances also, all UNDEF:
>> d1_srtp.o
>> [43]  |                   0|                   0|FUNC |WEAK |0
>|UNDEF  |OPENSSL_sk_new_null
>> s3_lib.o
>> [107] |                   0|                   0|FUNC |WEAK |0
>|UNDEF  |OPENSSL_sk_new_null
>> and so on.
>>
>>  From the linker docs:
>> B.2.17.1 #pragma weak name
>> In the form #pragma weak name, the directive makes name a weak
>symbol. The linker will not complain if it does not find a symbol 
>definition for name. It also does not complain about multiple weak 
>definitions of the symbol. The linker simply takes the first one it 
>encounters.
>>
>> So when linking with libcrypto.a and libssl.a the first definition is
>taken which is UNDEF in both cases leading to the error.
>> This can be fixed in libcrypto.a by making stack.o, lh_stats.o and
>lhash.o the first files included. It could also be fixed by not 
>defining those pragmas when compiling stack.c, lh_stats.c and lhash.c 
>because they will then be GLOB instead of WEAK and will override all 
>the WEAK definitions.
>>
>> However there is no suitable fix for libssl.a because those files are
>not part of that library.
>>
>> Doing that means that executables that link libcrypto.a before
>libssl.s are built correctly. However if the link is libssl.a followed 
>by libcrypto.a the UNDEF symbols from libssl.a are still used.
>>
>> In the shared libraries I see:
>>
>> libcrypto.so only instance:
>> [3537]        |             2409672|                  20|FUNC |LOCL
>|2    |10     |OPENSSL_sk_new_null
>>
>> Shared libssl.so also one instance:
>> [2091]        |                   0|                   0|FUNC |WEAK
>|0    |UNDEF  |OPENSSL_sk_new_null
>>
>> and it seems that the linker will resolve those correctly.
>>
>> So it seems to me that the only clean solution is to not define those
>pragmas when compiling stack.c, lh_stats.c and lhash.c by including a 
>suitable define in the makefiles when building the libraries that is 
>used in safestack.h and lhash.h to omit them.
>>
>> I see there is also
>> # elif defined(__SUNPRO_C)
>> #pragma weak getisax
>> in crypto\sparcv9cap.c
>> so maybe that needs consideration too.
>>
>> Regards,
>> John.
>>
>> -----Original Message-----
>> From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of 
>> John Unsworth
>> Sent: 10 May 2019 16:23
>> To: openssl-users at openssl.org
>> Subject: RE: OpenSSL 1.1.1b tests fail on Solaris - solution
>>
>> This seems to be caused by the ongoing saga documented in issues 6912
>and 8102. These functions were declared as weak in 1.1.1b.
>>
>> safestack.h
>> #  pragma weak OPENSSL_sk_num
>> #  pragma weak OPENSSL_sk_value
>> #  pragma weak OPENSSL_sk_new
>> #  pragma weak OPENSSL_sk_new_null
>> ...
>>
>> lhash.h
>> #  pragma weak OPENSSL_LH_new
>> #  pragma weak OPENSSL_LH_free
>> #  pragma weak OPENSSL_LH_insert
>> #  pragma weak OPENSSL_LH_delete
>> ...
>>
>> Removing these lines allows all the tests to work. I am building
>static libraries (no-shared) which is presumably why it works for some 
>people out of the box if they are using shared libraries.
>>
>> Please advise as to the way forward. I note that one other user (see
>8102) has reported this too:
>>
>> rammishr commented on Mar 26
>> Hi,
>> Sorry if this should have been posted separately.
>>
>> When i build openssl1.1.1b on solaris , it is successful. But while
>verifying the version " openssl version" it fails with below error.
>> ld.so.1: openssl: fatal: relocation error: file
>> /vobs_tools/prgs/src/openssl/solaris64/bin/openssl: symbol
>> OPENSSL_sk_new_null: referenced symbol not found Killed
>>
>> The same machine works fine for openssl1.1.1.
>> So, in 1.1.1b I can observe that OPENSSL_sk_new_null has been defined
>as below in safestack.h.
>> pragma weak OPENSSL_sk_new_null
>> Can this be related ? Am I missing anything while Configure ?
>> <<<<<<<<<<<<<<<
>>
>> Regards,
>> John.
>>
>> -----Original Message-----
>> From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of 
>> John Unsworth
>> Sent: 09 May 2019 10:13
>> To: openssl-users at openssl.org
>> Subject: RE: OpenSSL 1.1.1b tests fail on Solaris
>>
>> This looks like the problem:
>>
>> ld.so.1: sanitytest: fatal: relocation error: file
>> ../../test/sanitytest: symbol OPENSSL_sk_new_null: referenced symbol 
>> not found ../../util/shlib_wrap.sh ../../test/sanitytest => 137 not
>ok
>> 1 - running sanitytest
>>
>> #   Failed test 'running sanitytest'
>> #   at
>/home/metabld/OpenSSL/openssl-1.1.1b/test/../util/perl/OpenSSL/Test/Sim
>ple.pm
>line 77.
>> # Looks like you failed 1 test of 1.
>> Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests
>>
>> This results in the same error:
>> sol-mds-build-01 $ cd apps
>> sol-mds-build-01 $ ./openssl version
>> ld.so.1: openssl: fatal: relocation error: file openssl: symbol
>> OPENSSL_sk_new_null: referenced symbol not found
>>
>> I have built static libraries.
>>
>> John
>>
>> -----Original Message-----
>> From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of 
>> Matt Caswell
>> Sent: 09 May 2019 09:38
>> To: openssl-users at openssl.org
>> Subject: Re: OpenSSL 1.1.1b tests fail on Solaris
>>
>> What is the output from:
>>
>> $ make V=1 TESTS=test_sanity test
>>
>> Matt
>>
>> On 08/05/2019 19:22, John Unsworth wrote:
>>> I have build OpenSSL 1.1.1b 64 bit on Solaris SunOS 5.10 
>>> Generic_Virtual sun4v sparc SUNW,T5140.
>>>
>>>
>>>
>>> ./Configure -lrt solaris64-sparcv9-cc no-shared -m64 -xcode=pic32 
>>> -xldscope=hidden
>>>
>>>
>>>
>>> It builds fine but all the tests fail, with or without no-asm. Can 
>>> anyone help please? Here is the start of the test run:
>>>
>>>
>>>
>>> $ make test
>>>
>>> make depend && make _tests
>>>
>>> ( cd test; \
>>>
>>>    mkdir -p test-runs; \
>>>
>>>    SRCTOP=../. \
>>>
>>>    BLDTOP=../. \
>>>
>>>    RESULT_D=test-runs \
>>>
>>>   PERL="/opt/perl-5.26.1/bin/perl" \
>>>
>>>    EXE_EXT= \
>>>
>>>    OPENSSL_ENGINES=`cd .././engines 2>/dev/null && pwd` \
>>>
>>>    OPENSSL_DEBUG_MEMORY=on \
>>>
>>>      /opt/perl-5.26.1/bin/perl .././test/run_tests.pl  )
>>>
>>> ../test/recipes/01-test_abort.t .................... ok
>>>
>>> ../test/recipes/01-test_sanity.t ................... Dubious, test 
>>> returned 1 (wstat 256, 0x100)
>>>
>>> Failed 1/1 subtests
>>>
>
>Enjoy
>
>Jakob
>--
>Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com 
>Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10 This 
>public discussion message is non-binding and may contain errors.
>WiseMo - Remote Service Management for PCs, Phones and Embedded

--
Skickat från min Android-enhet med K-9 Mail. Ursäkta min fåordighet.
--
Skickat från min Android-enhet med K-9 Mail. Ursäkta min fåordighet.