[openssl-dev] [openssl.org #3714] OpenSSL 1.0.2 "make test" bus error in evp_test (Solaris 10 Sparc, sun4u)

[Rainer Jung via RT]

I updated gcc from 4.9.1 to 4.9.2 and the crash is gone.

Details maybe useful to others running into this problem:

It does not occur with gcc 4.8.4, but it occurs every time with gcc 4.9.1.

The crash also does not occur as soon as I add a debug fprintf() for the 
ctx->cmac, scratch and temp variable addresses.

The crash does not occur, if I compile ccm128.c without gcc -O flag or 
with -O0. It does always occur with O1, O2 and O3 (combined with gcc 4.9.1).

gcc flags:

gcc
-I..
-I../..
-I../modes
-I../asn1
-I../evp
-I../../include
-fPIC
-DOPENSSL_PIC
-DOPENSSL_THREADS
-D_REENTRANT
-DDSO_DLFCN
-DHAVE_DLFCN_H
-fPIC
-mcpu=v9
-g
-Wall
-fno-strict-aliasing
-m32
-mcpu=ultrasparc
-O3
-fomit-frame-pointer
-Wall
-DB_ENDIAN
-DBN_DIV2W
-DOPENSSL_BN_ASM_MONT
-DOPENSSL_BN_ASM_GF2m
-DSHA1_ASM
-DSHA256_ASM
-DSHA512_ASM
-DMD5_ASM
-DAES_ASM
-DGHASH_ASM
-c
-o
ccm128.o
ccm128.c

The u64 type seems to get correctly translated into unsigned long long.

Somehow the temp variable is not aligned correctly if kept in registers 
only.

The assembler diff between 4.9.1 and 4.9.2 around the suspect code is:

         ldub    [%i5+8], %g1
         add     %g1, 1, %g1
         stb     %g1, [%i5+8]
-       ldd     [%i4], %i2
+       ldub    [%i4+12], %g1
         add     %i4, 16, %i4
+       ldub    [%i4-3], %g4
+       sll     %g1, 24, %i1
+       ldub    [%i4-12], %g3
+       ldub    [%i4-11], %g2
+       sll     %g4, 16, %g1
+       ldub    [%i4-7], %i0
+       sll     %g3, 24, %o7
+       or      %g1, %i1, %g1
+       ldub    [%i4-2], %g4
+       sll     %g2, 16, %g2
+       ldub    [%i4-8], %i1
+       sll     %i0, 16, %i0
+       or      %g2, %o7, %g3
+       ldub    [%i4-10], %g2
+       sll     %g4, 8, %g4
+       sll     %i1, 24, %i1
+       or      %g4, %g1, %g4
+       ldub    [%i4-16], %o7
+       ldub    [%i4-15], %g1
+       sll     %g2, 8, %g2
+       or      %i0, %i1, %i0
+       ldub    [%i4-6], %i1
+       sll     %o7, 24, %o7
+       or      %g2, %g3, %g2
+       ldub    [%i4-14], %g3
+       sll     %g1, 16, %g1
+       sll     %i1, 8, %i1
+       or      %g1, %o7, %g1
+       ldub    [%i4-1], %o3
+       sll     %g3, 8, %g3
+       or      %i1, %i0, %i1
+       ldub    [%i4-5], %i0
+       ldub    [%i4-9], %o7
+       or      %g3, %g1, %g1
+       or      %o3, %g4, %o5
+       ldub    [%i4-13], %g3
+       or      %i0, %i1, %o4
+       ldd     [%fp-8], %i0
+       or      %o7, %g2, %i3
+       or      %g3, %g1, %i2
         ldd     [%fp-16], %g2
-       ldd     [%i4-8], %o4
-       xor     %i2, %g2, %i0
-       xor     %i3, %g3, %i1
-       ldd     [%fp-8], %g2
-       std     %i0, [%fp-16]
-       xor     %o4, %g2, %g2
-       xor     %o5, %g3, %g3
+       xor     %o4, %i0, %i0
+       xor     %o5, %i1, %i1
         ldd     [%i5+16], %o4
-       std     %g2, [%fp-8]
-       xor     %o4, %i0, %i2
-       xor     %o5, %i1, %i3
-       ldd     [%i5+24], %o4
-       std     %i2, [%i5+16]
+       xor     %i2, %g2, %g2
+       xor     %i3, %g3, %g3
+       std     %i0, [%fp-8]
         xor     %o4, %g2, %i2
         xor     %o5, %g3, %i3
+       ldd     [%i5+24], %o4
+       std     %i2, [%i5+16]
+       xor     %o4, %i0, %i2
+       xor     %o5, %i1, %i3
+       std     %i2, [%i5+24]
         call    memcpy, 0
-        std    %i2, [%i5+24]
+        std    %g2, [%fp-16]
         mov     %l5, %o0
         mov     %l5, %o1
         call    %l1, 0

I didn't find a corresponding fix in the 4.9.2 changelog. I think you 
can nevertheless close this issue here, because it is very likely a gcc 
bug introduced in the latest gcc branch and already fixed in the latest 
gcc release.

Sorry for the noise.

Regards,

Rainer