OpenSSL 1.0.2k EVP_PKEY decryption failing

[Viktor Dukhovni]

On Tue, Mar 28, 2023 at 04:34:51PM +0000, Viken Kondakji wrote:

> My objective is to encrypt data using RSA public key, with SHA-256 as
> hash function and MGF1 as mask generating function.

You almost never want to do that.  RSA is encryption is very difficult
to use correctly as a raw primitive.  You probably should be using
CMS, in encrypt-then-sign mode, with RSA used only to encrypt the
symmetric key that in turn encrypts the data.

All the RSA operations will be handled internally by the CMS (modernised
S/MIME) API.

-- 
    Viktor.