Migrating to 3.x - how to handle PKEY types and Sig Algorithms?

Tue Aug 22 10:46:51 UTC 2023

On 22/08/2023 06:20, Dr. Pala wrote:
> Hello OpenSSL Community,
> 
> We are in the process of upgrading the LibPKI project to use the OSSL 
> 3.x instead of the OSSL 1.1.1x that we are using today ... and we can 
> use some help. Specifically, we noticed some inconsistent behavior when 
> porting from the 1.1.1x version and we hope you can */_help us finding 
> what are we doing wrong and what is the right way of doing things_/* in 
> the new "providers" world :)
> 
> The original code, when compiled against the OpenSSL 1.1.1x branch, 
> works correctly... However, with the 3.x version, things do not behave 
> the same way and our applications fail when compiled/linked against it. 
> Our setting is OpenSSL 3.x with the OQS provider installed. The 
> application successfully loads the 'default' provider first, then the 
> 'legacy', and finally the 'oqsprovider' one
> 
> Here's some of my initial questions:
> 
>   * *EVP_PKEY_id() and EVP_PKEY_type()* are not reliable anymore with
>     providers (e.g., for "dilithium2" from the OQS provider they all
>     return '0'). What shall we use to get the type of a PKEY structure
>     (i.e., this is an "RSA" key or a "Falcon512" one) ? Currently, to
>     address the issue with *EVP_PKEY_id*(), we are getting the name from
>     the pkey (pkey_name = *EVP_PKEY_get0_type_name*(pkey) - Example
>     Code:
>     https://github.com/openca/libpki/blob/00b1fbb434dbf1ffd09fcad5ddfb7ceb7eeaba6a/src/openssl/pki_keypair.c#L350) and then converting that into an ID (*OBJ_sn2nid*(pkey_name)). If that is the supposed way of doing things (I really doubt it, we just used it as an hack), /_how do we get the type of key from the ID_ (i.e., the functionality provided via*EVP_PKEY_type()*/) ? Shall we... string/mem compare when *EVP_PKEY_type*() returns 0? In other words, how do I know an EVP_KEY is an RSA one instead of a dilithium or falcon one (e.g., EVP_PKEY_type() vs. EVP_PKEY_EC, EVP_PKEY_RSA, etc.)? Example Code: https://github.com/openca/libpki/blob/28c510193735e6a68671bfb4cb4e66589a1fd9d0/src/drivers/hsm_main.c#L530

We use EVP_PKEY_is_a() extensively internally to check whether a pkey is 
a particular type.

https://www.openssl.org/docs/man3.1/man3/EVP_PKEY_is_a.html

>   * *OBJ_add_sigid() and OBJ_find_sigid_by_algs()* do not seem to work
>     as usual. Specifically, with the above configuration (i.e., with the
>     {default, legacy, oqsprovider} providers loaded), during the
>     initialization of the library/application, we try to add a bunch of
>     OIDs for new signature types that sometimes seem to persist and
>     sometimes they do not. For example,  at startup _/we add a series of
>     OIDs for hash-n-sign with dilithium2/_. We do this by creating the
>     new signature OIDs with *OBJ_create*() first, and then we use those
>     new IDs with the *OBJ_add_sigid*() with different hash identifiers.
>     When we use the *OBJ_find_sigid_by_algs()* function right after
>     adding the signature ID and, for some algorithms, the function fails
>     to find it. For example, we can add a DILITHIUM2-SHA3_256 signature
>     correctly, but the same code fails to retrieve the
>     DILITHIUM2-SHA3_512. Even more strange is the fact that if we try to
>     find sigid that were found during initialization, we now fail for no
>     apparent reason. Example Code:
>     https://github.com/openca/libpki/blob/28c510193735e6a68671bfb4cb4e66589a1fd9d0/src/openssl/pki_oid_defs.c#L622

This sounds like it should work, so I don't know why you are seeing odd 
behaviour. Perhaps raise a github issue for this?

One thing that does spring to mind is that I know that oqsprovider 
already registers various sigid OIDs - so plausibly there is some kind 
of issue caused by that:

https://github.com/open-quantum-safe/oqs-provider/blob/b6da1e97b7180a779981d1b0bfa0ab2cd723066d/oqsprov/oqsprov.c#L592-L619

https://github.com/open-quantum-safe/oqs-provider/blob/b6da1e97b7180a779981d1b0bfa0ab2cd723066d/oqsprov/oqsprov.c#L45-L71

> 
>   * *How to replace PKEY/ASN1_PKEY methods.* In our library, we
>     currently provide a new type of algorithm (i.e., Composite Crypto)
>     via a PKEY/ASN1_PKEY combination that we inject at initialization
>     time via the *EVP_PKEY_meth_add0*() and *EVP_PKEY_asn1_add0*(). What
>     is the equivalent functions to use under the providers' paradigm? In
>     other words, do we need to generate a separate provider object or
>     can we inject it programmatically? Example Code:
>     https://github.com/openca/libpki/blob/28c510193735e6a68671bfb4cb4e66589a1fd9d0/src/openssl/composite/composite_init.c#L326

Yes, you would need a new provider to add a new algorithm (although the 
old approach should still *work* - even though it is deprecated). You 
can just add a "built-in" one if you want (so no need to create a 
separate .so file for it - it can just live in your application code)

https://www.openssl.org/docs/man3.1/man3/OSSL_PROVIDER_add_builtin.html

Start by looking at the page for general information for provider authors:

https://www.openssl.org/docs/man3.1/man7/provider.html

Richard Levitte has a "toy" provider example which you can look at here:

https://github.com/provider-corner/vigenere/blob/main/vigenere.c

To implement an algorithm yourself you need to first determine what kind 
of algorithm it is (e.g. cipher, signature, key-exchange, etc) and then 
implement the appropriate provider functions for that algorithm. To 
implement a new signature algorithm you would need to implement the 
signature functions as well as the key management functions. E.g. see

https://www.openssl.org/docs/man3.1/man7/provider-signature.html

and

https://www.openssl.org/docs/man3.1/man7/provider-keymgmt.html

> 
>   * *EVP_PKEY_get0() fails when called from within the pkey_bits()
>     function of the EVP_PKEY_ASN1_METHOD*. Specifically, when the
>     pkey_bits() function is called with the parameter of type `const
>     EVP_PKEY *pk`, the EVP_PKEY_get0(pk) function always returns NULL.
>     What are we supposed to use to retrieve the internal key structure?
>     Example Code:
>     https://github.com/openca/libpki/blob/28c510193735e6a68671bfb4cb4e66589a1fd9d0/src/openssl/composite/composite_ameth.c#L963
> 

EVP_PKEY_get0() returns a pointer to the legacy key structure (e.g. RSA, 
DSA, DH, EC_KEY, etc). It won't work if you call it on a provider based 
EVP_PKEY (returns NULL) but you can call the typed equivalents, e.g. 
EVP_PKEY_get0_RSA(). In that case it gives you a cached *copy* of the 
legacy key (the real key is held in the provider and is inaccessibly 
directly to the application).

It's unclear to me from your question whether you are talking about your 
*own* EVP_PKEY type, i.e. one you've defined yourself that does not use 
a provider. If so then this should still work as it did before, i.e. as 
long as data has been assigned to the key via EVP_PKEY_assign() then it 
should be available via a subsequent call to EVP_PKEY_get0().

Matt

> The migration to the 3.x has some interesting differences that are quite 
> challenging, thanks everybody for any help you can provide :)
> 
> Cheers,
> Max
> 
> -- 
> Best Regards,
> Massimiliano Pala, Ph.D.
> OpenCA Labs Director
> OpenCA Logo