Application segfaults after upgrade from 3.0.11 to 3.0.13

[Neil Horman]

Can you post the stack trace of the segv here?

On Tue, Jul 16, 2024 at 12:43 PM Victor Wagner <vitus at wagner.pp.ru> wrote:

> Hi!
>
> I'm using osslsigncode application on Debian 12 system (amd64) to sign
> stuff with RSA key stored on hardware token with PKCS11 interface.
>
> osslsigncode (https://github.com/mtrojnar/osslsigncode) seems to be
> well-behaved openssl application, which uses digest BIO and PKCS7 API,
> does no poking into opaque structures etc.
>
> Application was compiled from source in February, when openssl version
> in Debian was 3.0.11-1~deb12u1
>
> Unfortunately, when security update of libssl3 (debian package for
> openssl libraries) version 3.0.13-1~deb12u1 was installed, osslsigncode
> begin to crash with SIGSEGV.
>
> Quick debugging session shows that application is able to initialize
> token and correctly obtain private key handle and certificate for it.
> But when trying to sign, it receives invalid pointer to
> PKCS11_CTX_private structure. (segfault happens inside pkcs11.so)
> This pointer is contained in PKCS11_SLOT_private structure, which
> has refcount field before this pointer, and this field also seems to be
> filled with garbage (i expect refcount to be less than 10 in so small
> program, which handles just one signature and it is some 32-bit value
> with second high order bit set).
>
> Downgrade to previous version of openssl libraries fixes the problem.
>
> I suspect that problem is in application, which somehow misuses openssl
> API but have no idea how to look for problem. Really, it seems to to be
> good idea to track memory writes to PKCS11_SLOT object, but it is
> hidden inside so many levels of opaque structures.
>
> I've thought about checking what change in openssl may affect problem,
> but don't see anything appropriate in changelog between 3.0.11 and
> 3.0.13 (and debian maintainers seems to add nothing new over upstream
> changes).
> --
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20240716/be0e896d/attachment.htm>