[openssl-dev] Question about commit 222333cf01e2fec4a20c107ac9e820694611a4db

[Dr. Stephen Henson]

On Tue, Apr 11, 2017, Michael Reilly wrote:

> Hi,
> 
> commit 222333cf01e2fec4a20c107ac9e820694611a4db added a check that the size
> returned by EVP_PKEY_size(ctx->pkey) in M_check_autoarg() in
> crypto/evp/pmeth_fn.c is != 0.
> 
> We are in the process of upgrading from 1.0.2j to 1.0.2k and discovered that the
> if (pksize == 0) check added in 1.0.2k breaks some of our applications.
> 
> We use an engine for the RSA sign operation.  The applications do not know
> anything about the keypair being used.  The keypair is kept private by the
> engine so the application couldn't determine the attributes of the keypair if it
> wanted to do so.
> 
> If this check is necessary is there a way to bypass it when the application does
> not have the keypair but the engine being used is holding the keypair?
> 
> I know we can simply remove this line from our copy of the code but we like to
> avoid modifying the openssl distributed code if at all possible.
> 

Well the point of that code is so an application knows how large a buffer to
allocate for the signature. If it returns zero I can't see how applications
can do that.

Note that you don't have to return the *precise* length of the signature just
an upper bound is sufficient.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org