[openssl-users] How to override methods in EVP_PKEY_METHOD structure that is attached to a EVP_PKEY_CTX?

[Dr. Stephen Henson]

On Mon, Feb 27, 2017, Stephan M?hlstrasser wrote:

> Steve,
> 
> Am 25.02.17 um 05:53 schrieb Dr. Stephen Henson:
> >On Fri, Feb 17, 2017, Stephan M?hlstrasser wrote:
> >...
> >>Is it possible to override methods in an EVP_PKEY_METHOD structure,
> >>or would it be necessary to implement a whole OpenSSL engine to do
> >>what I want?
> >>
> >
> >It should be possible yes, though AFAIK no one has yet tried to do this so
> >there may be some pieces missing.
> >
> >In outline you'd retrieve the appropriate EVP_PKEY_METHOD for the algorithm of
> >interest, make a copy of it and then set the operation you wish to override,
> >you can also retrieve the original operation in case you sometimes wish to
> >call that.
> 
> thanks for confirming that this should be possible in principle.
> 
> I guess my problem was that I thought one must retrieve the
> EVP_PKEY_METHOD from the EVP_PKEY_CTX pointer. As you are saying it
> must be retrieved for the algorithm, I think I understood now that
> it must be fetched via EVP_PKEY_meth_find().
> 
> Is the following sketch roughly appropriate?
> 
> int my_sign_init_function(EVP_PKEY_CTX *ctx);
> int my_sign_function(EVP_PKEY_CTX *ctx, unsigned char *sig, size_t
> *siglen, const unsigned char *tbs, size_t tbslen);
> 
> const EVP_PKEY_METHOD *rsa_meth = EVP_PKEY_meth_find(EVP_PKEY_RSA);
> EVP_PKEY_METHOD *new_rsa_meth = EVP_PKEY_meth_new(EVP_PKEY_RSA, 0);
> EVP_PKEY_meth_copy(new_rsa_meth, rsa_meth);
> EVP_PKEY_meth_set_sign(new_rsa_meth, my_sign_init_function,
> my_sign_function);
> EVP_PKEY_meth_add0(new_rsa_meth);
> 
> What is still unclear to me is how to retrieve the original function
> pointers from the EVP_PKEY_METHOD. EVP_PKEY_METHOD is an opaque
> structure, and I could not find a getter counterpart for
> EVP_PKEY_meth_set_sign().
> 
> How is it supposed to be possible to retrieve the original
> operations from an EVP_PKEY_METHOD pointer?
> 

Ah I see you're using OpenSSL 1.0.2. There isn't a way to get the existing
function pointers in 1.0.2, the "getter" functions are only in 1.1.0.

There shouldn't be any need to add the method to the list: it should be
possible to associate an EVP_PKEY with a non-default method (e.g. explicitly
or implemented in an ENGINE). I say *should* because there doesn't seem to be
currently a way to do that without changing EVP_PKEY internal fields (which
isn't possible in 1.1.0 anyway).

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org