[openssl-users] [openssl-dev] [openssl.org #3804] BUG: OpenSSL 1.0.2 Solaris 32 bit build is broken

[Andy Polyakov via RT]

Hi,

> I have an application that runs quite happily using OpenSSL 1.0.1h on Solaris 32 bit. I want to upgrade but neither 1.0.2 nor 1.0.2a work.
> 
> Solaris 10
> Solaris Studio 12.4
> 
> Make test log attached.
> 
> 1 When building 1.0.2 using
> 
> ./Configure solaris-sparcv9-cc no-shared -m32 -xcode=pic32 -xldscope=hidden
> 
> openssl s_client crashes on start:
> 
> -bash-3.00$ ./openssl s_client -connect eos.es.cpth.ie:4250
> Segmentation Fault (core dumped)
> -bash-3.00$ pstack core
> core 'core' of 468:     ./openssl s_client -connect eos.es.cpth.ie:4250
> 000e9ce8 sha1_block_data_order (2ed490, 2ed4ec, 4, ffbfebc0, ffbfebc4, 44) + 8

Well, combining no-shared and -xcode=pic32 results in ambiguity in the
very beginning of this and few other subroutines. For code to be
compiled as position-independent __PIC__ macro is required to be
defined, but I don't think passing -xcode=pic32 cuts it. Question is
what is your objective? I mean why is it such contradicting mixture of
no-shared and pic options? If the goal is to build static library that
can be later linked into another shared library, then I'd suggest
no-shared -KPIC, which supposedly can be followed by -xcode=pic32
-xlscope=hidden. Keyword here is that -KPIC ought to pre-define __PIC__.
If it doesn't, then complement it with -DOPENSSL_PIC, i.e. './Configure
solaris-sparcv9-cc no-shared -KPIC -DOPENSSL_PIC ...'

> 2 So I then rebuilt adding no-asm flag. It manages to connect but negotiation fails with an error:
> 
> 4280581268:error:140943FC:SSL routines:ssl3_read_bytes:sslv3 alert bad record mac:s3_pkt.c:1456:SSL alert number 20
> 4280581268:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
> 
> This is against the server that is still running 1.0.1h and can be successfully connected with openssl s_client built with 1.0.1h.
> 
> The 64 bit build seems to work perfectly. The 32 bit builds that we use on Windows and Linux also work perfectly.
> 
> 1.0.2a build fails in the same way. gcc build fails in the same way.

So what you are saying is that everything works, but 32-bit SPARC 1.0.2
build and it doesn't matter if it's compiled with vendor compiler or
gcc. H-m-m-m... Did you run 'make clean' in between? Thing is that
normal reaction in such situation is to suspect compiler bug, but two
compilers are highly unlikely to share bug. So try 'make clean' in
between tries, or better yet, try builds in separate directories. If
both still fail, play with -cipher option to identify faulty algorithm...