EVP_PKEY_fromdata_init returns unsupported.

[Matt Caswell]

On 25/02/2022 22:07, William Roberts wrote:
> Hello,
> 
> In openssl 3.0.1 the following code hits the ctx->keymgt is null check
> and thus returns -2
> in pmeth_gn.c:
> static int fromdata_init(EVP_PKEY_CTX *ctx, int operation)
> {
>      if (ctx == NULL || ctx->keytype == NULL)
>          goto not_supported;
> 
>      evp_pkey_ctx_free_old_ops(ctx);
>      if (ctx->keymgmt == NULL)
>          goto not_supported;
> 
> The callpath comes in from EVP_PKEY_fromdata_init:
> 
> libctx = OSSL_LIB_CTX_new()
> genctx = EVP_PKEY_CTX_new_from_name(libctx, "RSA", NULL);


My guess is EVP_PKEY_CTX_new_from_name() is finding a default engine 
implementation for RSA. You might like to step through 
EVP_PKEY_CTX_new_from_name in the debugger (actually int_ctx_new in 
crypto/evp/pmeth_lib.c) and see if the "e" variable ever gets associated 
with an engine.

If an engine is being found then the EVP_PKEY_CTX will use that engine 
implementation for all subsequent RSA operations. EVP_PKEY_fromdata will 
only work with provider based implementations (we should make that 
explicit in the documentation) - hence it will fail.

Matt



> int rc = EVP_PKEY_fromdata_init(genctx);
> 
> I have no idea why it returns unsupported... any ideas?
> I also tried replacing EVP_PKEY_CTX_new_from_name  with
> EVP_PKEY_CTX_new_id, same error.
> 
> Thanks,
> Bill
>