[openssl-dev] confusion with rsa_meth_st in a custom RSA engine

[Brett R. Nicholas]

> The rsa_mod_exp function is only called for private key operations. You can't
> tell if it is a private encrypt or a private decrypt though but that
> shouldn't matter because the operation performed at that level is the same for
> both.

Ah, I see. So to clarify (pls correct me if I'm wrong):

  *   rsa_mod_exp() is the modular exponentiation function that openSSL will attempt to use for all private key operations (if RSA_FLAG_EXT_PKEY is set, or the private parameters of that method are non-null), before defaulting to bn_mod_exp().
  *   And bn_mod_exp() is the modular exponentiation function used by all public key operations


So in my case, I should set RSA_FLAG_EX_PKEY in my engine's RSA_METHOD struct, and then implement my own versions of rsa_mod_exp (for the private key encryption/decryption), and bn_mod_exp (for the public key encryption/decryption).


Brett

________________________________
From: openssl-dev <openssl-dev-bounces at openssl.org> on behalf of Dr. Stephen Henson <steve at openssl.org>
Sent: Sunday, August 27, 2017 6:45 PM
To: openssl-dev at openssl.org
Subject: Re: [openssl-dev] confusion with rsa_meth_st in a custom RSA engine

On Sun, Aug 27, 2017, Brett R. Nicholas wrote:

>
> This makes sense to me, and it seems that is the desired behavior. However,
> if I *only* reimplement the rsa_mod_exp() function, and leave the
> encrypt/decrypt functions to the default openSSL implementations, how can my
> engine know which of the four of those functions called its rsa_mod_exp
> function()? To put it another way: my accelerator will need to know whether
> it is meant to perform public key or private key operations, since the
> inputs will be written to different memory addresses. From within
> rsa_mod_exp(), the only way I can think of determining whether the function
> has been called by private_encrypt/decrypt() or public_encrypt/decrypt would
> be to check if the  p and q fields of the RSA *rsa struct passed into the
> function are NULL? But I can't tell from the source code if this will be
> guaranteed just by having  RSA_FLAG_EXT_PKEY set in the "flags" field of my
> engine's RSA_METHOD.
>

The rsa_mod_exp function is only called for private key operations. You can't
tell if it is a private encrypt or a private decrypt though but that
shouldn't matter because the operation performed at that level is the same for
both.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.openssl.org&data=02%7C01%7Cbrett.r.nicholas.th%40dartmouth.edu%7C5374bb23132c4377080c08d4edb671de%7C995b093648d640e5a31ebf689ec9446f%7C0%7C0%7C636394815232517002&sdata=AdSj9Ys%2FG%2FQ1VJIIaD5kZhAwbki5mcF0AL8b2UBxic4%3D&reserved=0
--
openssl-dev mailing list
To unsubscribe: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmta.openssl.org%2Fmailman%2Flistinfo%2Fopenssl-dev&data=02%7C01%7Cbrett.r.nicholas.th%40dartmouth.edu%7C5374bb23132c4377080c08d4edb671de%7C995b093648d640e5a31ebf689ec9446f%7C0%7C0%7C636394815232517002&sdata=zzQB9AlLiORZC2btJ6zi1MZQxVzoB8pgWHTtAC8LHtM%3D&reserved=0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170828/3eca45b3/attachment.html>