Intel CET in OpenSSL 3.x and VIA C3 Nehemiah - Illegal instruction

[Hamish McWilliam]

Hi folks,

Not sure if this is intended behaviour or not...

The use of Intel Control-flow Enforcement Technology (CET) in OpenSSL 3.0.2
is giving me issues on a VIA C3 Nehemiah based system. Turns out that while
the VIA C3 Nehemiah is a i686 class x86 processor, it does not implement
all the nop opcodes added in the Pentium Pro. So I get 'Illegal
instruction' when trying to run OpenSSL programs due to the presence of the
endbr32 instructions:

$ LD_LIBRARY_PATH=`pwd` apps/openssl version
Illegal instruction

Testing with the endbr32 generation in crypto/perlasm/x86asm.pl commented
out generates a working binary:

$ LD_LIBRARY_PATH=`pwd` apps/openssl version -a
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
built on: Sat Apr 30 10:20:30 2022 UTC
platform: linux-x86
options:  bn(64,32)
compiler: gcc -fPIC -pthread -m32 -Wa,--noexecstack -Wall -O3
-fomit-frame-pointer -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC
-DOPENSSL_BUILDING_OPENSSL -DNDEBUG
OPENSSLDIR: "/usr/local/ssl"
ENGINESDIR: "/usr/local/lib/engines-3"
MODULESDIR: "/usr/local/lib/ossl-modules"
Seeding source: os-specific
CPUINFO: OPENSSL_ia32cap=0x381bf3f:0x0

>From a bit of experimentation it appears that the endbr32 instruction is
always included in the generated x86 32-bit assembler code, even when using
the 386 assembler option.

So, not sure if OpenSSL is dropping support for older 32-bit x86
processors, and the documentation hasn't caught up, or if this is just an
oversight?

FWIW the gcc folks have limited generation of endbr32 instructions to at
least i686 (cmov) and SSE2 (see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98667), to limit the use of
the endbr32 instruction to target systems where it is handled as a nop.

All the best,

Hamish
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20220501/aa615d92/attachment.htm>