[openssl-users] How to override methods in EVP_PKEY_METHOD structure that is attached to a EVP_PKEY_CTX?

[Stephan Mühlstrasser]

Hi,

we use OpenSSL 1.0.2 together with PKCS#11 tokens by plugging methods 
into the RSA_METHOD structure that interface with the PKCS#11 token, and 
this works fine so far. However, for creating RSA signatures with PSS 
padding this strategy doesn't work anymore, because OpenSSL wants to 
directly encrypt with the private key in this case, which is not 
possible in PKCS#11.

Therefore my idea is to override the function pkey_rsa_sign() and plug a 
wrapper around it into the EVP_PKEY_METHOD structure that is associated 
with the EVP_PKEY_CTX to handle this special situation.

The header evp.h declares the following functions among others:

EVP_PKEY_METHOD* EVP_PKEY_meth_new(int id, int flags);
void EVP_PKEY_meth_copy(EVP_PKEY_METHOD *dst, const EVP_PKEY_METHOD *src);

void EVP_PKEY_meth_set_sign(EVP_PKEY_METHOD *pmeth,
	int (*sign_init)(EVP_PKEY_CTX *ctx),
	int (*sign)(EVP_PKEY_CTX *ctx, unsigned char *sig,
             size_t *siglen, const unsigned char *tbs, size_t tbslen));

But I can't figure out how to use these functions to achieve what I 
want, because the following pieces seem to be missing:

- Retrieve the EVP_PKEY_METHOD pointer from a EVP_PKEY_CTX pointer
- Set the EVP_PKEY_METHOD pointer for a EVP_PKEY_CTX pointer
- Retrieve the existing "sign_init" and "sign" function pointers from an 
initialized EVP_PKEY_METHOD pointer for being able to wrap them

Is it possible to override methods in an EVP_PKEY_METHOD structure, or 
would it be necessary to implement a whole OpenSSL engine to do what I want?

Thanks.

-- 
Stephan