Application segfaults after upgrade from 3.0.11 to 3.0.13

[Victor Wagner]

Hi!

I'm using osslsigncode application on Debian 12 system (amd64) to sign
stuff with RSA key stored on hardware token with PKCS11 interface.

osslsigncode (https://github.com/mtrojnar/osslsigncode) seems to be
well-behaved openssl application, which uses digest BIO and PKCS7 API,
does no poking into opaque structures etc.

Application was compiled from source in February, when openssl version
in Debian was 3.0.11-1~deb12u1

Unfortunately, when security update of libssl3 (debian package for
openssl libraries) version 3.0.13-1~deb12u1 was installed, osslsigncode
begin to crash with SIGSEGV.

Quick debugging session shows that application is able to initialize
token and correctly obtain private key handle and certificate for it.
But when trying to sign, it receives invalid pointer to
PKCS11_CTX_private structure. (segfault happens inside pkcs11.so)
This pointer is contained in PKCS11_SLOT_private structure, which
has refcount field before this pointer, and this field also seems to be
filled with garbage (i expect refcount to be less than 10 in so small
program, which handles just one signature and it is some 32-bit value
with second high order bit set).

Downgrade to previous version of openssl libraries fixes the problem.

I suspect that problem is in application, which somehow misuses openssl
API but have no idea how to look for problem. Really, it seems to to be
good idea to track memory writes to PKCS11_SLOT object, but it is
hidden inside so many levels of opaque structures.

I've thought about checking what change in openssl may affect problem,
but don't see anything appropriate in changelog between 3.0.11 and
3.0.13 (and debian maintainers seems to add nothing new over upstream
changes).
--