[openssl-users] Padding for RSA signatures

[Matt Caswell]

Some comments inserted below.

Matt

On 29/12/17 15:20, Gelareh Taban wrote:
> Hi all,
> 
> Any help would be *much* appreciated. I am playing around with RSA
> signatures with different padding options and I have some questions.
> 
> I am trying to define different padding options and so am defining and
> using a EVP_PKEY_CTX . However I am not sure if this padding is getting
> used in the signature since  my Verify outputs OK regardless of which
> option my Sign uses. Which leads to:
> 
> 1 - Do I need to use a EVP_PKEY_CTX with the same options when doing
> verify? If so, I assume I can't reuse the same PKey_Ctx and I have to
> define another one. Right now even when I don't use any EVP_PKEY_CTX in
> Verify, I still verify OK, which makes me question if the padding option
> has been set.

It hasn't. The call to EVP_DigestSignInit() expects an EVP_PKEY_CTX **.
This is because that function creates its *own* EVP_PKEY_CTX * and
stores it in the location you provide (if you give one). In your code
the EVP_PKEY_CTX you are creating is being overwritten by the one
created by EVP_DigestSignInit(). From the documentation:

  EVP_DigestSignInit() sets up signing context ctx to use digest type
  from ENGINE impl and private key pkey. ctx must be created with
  EVP_MD_CTX_new() before calling this function. If pctx is not NULL the
  EVP_PKEY_CTX of the signing operation will be written to *pctx: this
  can be used to set alternative signing options.

Try removing the creation of your own EVP_PKEY_CTX *, and moving the
EVP_PKEY_CTX_set_rsa_padding() call to after EVP_DigestSignInit().

> 
> 2 - Is there a way to figure out what padding/hashing/etc option was
> used for the Sign/verify operation? This way I can be sure what
> algorithm or standard is being used. 
> 
> 3 - Do I need to set the hash function I am using in both EVP_PKEY_CTX 
> as well as EVP_MD_CTX ? Or the latter is what defines this for the
> signing option?

You only need to specify the hash function in
EVP_DigestSignInit()/EVP_DigestVerifyInit().

To answer your question in the code, there is no need to call
EVP_PKEY_CTX_set_signature_md() directly in this scenario. It is called
for you by EVP_DigestSignInit()/EVP_DigestVerifyInit().

> 
> 4 - In general, is there a way of making the Signature/Encryptions in
> OpenSSL be deterministic for debugging/testing purposes? >
> 5 - I noticed that there are two ways of determining the signature size:
> (a) by calling EVP_PKEY_size(rsaKeypair) as I am doing below, as well as
> (b) calling EVP_DigestSignFinal(md_ctx, nil, &sig_len) . Is one better
> than the other? 

The former gives you a maximum bound on the size of the signature before
the signature is created. The latter gives you the *actual* size of the
signature that is generated (which could be smaller).


> 
> My sample code is below for reference. It's in Swift (but it should
> still be close enough to C to be readable). Also in Swift, some of the
> complex macros in OpenSSL have to be broken down to be compilable hence
> my usage of EVP_DigestUpdate instead of EVP_DigestVerifyUpdate .
> 
> Thanks in advance for any insight in the above.
> 
> cheers!
> Gelareh
> 
> 
>         let md_ctx = EVP_MD_CTX_create()
> 
>         let md_ctx_verify = EVP_MD_CTX_create()
> 
>         
> 
>         // To define padding option used in signature
> 
>         let pkey_ctx = EVP_PKEY_CTX_new(rsaKeypair, nil)
> 
>         
> 
>         // EVP_PKEY_CTX_set_rsa_padding(pkey_ctx, RSA_PKCS1_PADDING) -
> complex macro needs to be replaced
> 
>         EVP_PKEY_CTX_ctrl(pkey_ctx, EVP_PKEY_RSA, -1,
> EVP_PKEY_CTRL_RSA_PADDING, RSA_X931_PADDING, nil)
> 
> 
>         // EVP_PKEY_CTX_set_signature_md() When should this be set?
> 
>         
> 
>         //  SIGN
> 
>         var rc = EVP_DigestSignInit(md_ctx, &pkey_ctx, EVP_sha256(),
> nil, myRSA.rsaKeypair)
> 
>         // EVP_DigestSignUpdate(md_ctx, message, message.count)
> 
>         // Complex macro needs to be replaced
> 
>         rc = EVP_DigestUpdate(md_ctx, message, message.count)
> 
>         
> 
>         // allocate memory for signature
> 
>         var sig_len: Int = Int(EVP_PKEY_size(rsaKeypair))
> 
>         let sig = UnsafeMutablePointer<UInt8>.allocate(capacity: sig_len)
> 
> 
>         rc = EVP_DigestSignFinal(md_ctx, sig, &sig_len)
> 
>       
> 
>         
> 
>         // VERIFY
> 
>         rc = EVP_DigestVerifyInit(md_ctx_verify, nil, EVP_sha256(), nil,
> rsaKeypair)
> 
> 
>         //        rc = EVP_DigestVerifyUpdate(md_ctx_verify, message,
> message.count)
> 
>         rc = EVP_DigestUpdate(md_ctx_verify, message, message.count)
> 
>        
> 
>         rc = EVP_DigestVerifyFinal(md_ctx_verify, sig, sig_len)
> 
>         print("signature verified = \(rc == 1? "OK": "FAIL")")
> 
>         
> 
> 
> 
> 
> 
> 
>