OpenSSL 1.1.1: How to get signature algorithm id? (no EVP_MD_FLAG_PKEY_METHOD_SIGNATURE)

[Tomas Mraz]

You can use OBJ_find_sigid_by_algs() and pass the
EVP_PKEY_base_id(pkey) and EVP_MD_type(md) values to it. It should find
the signature algorithm id.

Tomas Mraz, OpenSSL

On Thu, 2023-02-09 at 15:21 +0000, Andrew Lynch via openssl-users
wrote:
> Hi,
>  
> I have some old code that determines the signature algorithm OID
> given a combination of EVP_PKEY *pkey and EVP_MD *digest.  It
> contains an #ifdef EVP_MD_FLAG_PKEY_METHOD_SIGNATURE.  If true, it
> uses OBJ_find_sigid_by_algs() with the digest and pkey->ameth-
> >pkey_id to get the OID.  The else case uses EVP_MD_pkey_type() with
> only the digest.
>  
> Given an ECDSA pkey and a SHA256 digest the application using OpenSSL
> 1.0.2 outputs the expected ecdsa-with-SHA256.  With OpenSSL 1.1.1 it
> incorrectly outputs sha256WithRSAEncryption.
>  
> OpenSSL 1.0.2 has the flag defined whereas it does not exist at all
> in 1.1.1.  As EVP_MD_pkey_type() only has the digest to work with it
> will always return NID_sha256WithRSAEncryption regardless of the type
> of EVP_PKEY that is actually used with the digest to create a
> signature.
>  
> Which API calls can I use in OpenSSL 1.1.1 to get the correct
> signature algorithm id given some combination of EVP_PKEY and EVP_MD?
>  
> Regards,
> Andrew.
>  

-- 
Tomáš Mráz, OpenSSL