Question about migrating from d2i_ECPrivateKey() to d2i_PrivateKey(EVP_PKEY_EC, ...)

[Viktor Dukhovni]

On Fri, Nov 18, 2022 at 11:33:08PM -0600, Nico Williams wrote:
> On Fri, Nov 18, 2022 at 04:53:44PM -0600, Nico Williams wrote:
> > I can't use d2i_PrivateKey() because that requires an existing
> > EVP_PKEY * that has the group already set.
> 
> Although, that's just what's documented.  From code inspection, if the
> parameters are found in the encoded private key, then the group will be
> set internally and no error will be returned.

Often, if you want a clear example of OpenSSL API usage, one place to
look is the Postfix "tls" library.  In this case:

    https://github.com/vdukhovni/postfix/blob/master/postfix/src/tls/tls_certkey.c#L245-L266
    https://github.com/vdukhovni/postfix/blob/master/postfix/src/tls/tls_certkey.c#L363-L370

Postfix does not do much with low-level crypto, but it exercises a
non-trivial chunk of the certificate and TLS API surface, ECDH/DH
setup and digests.

Generally, I would expect d2i_... to automatically detect the algorithm
when tagged with a suitable OIDs, and so d2i_AutoPrivateKey() could
often work, but if you know the expected key type, you can ask for
that explicitly with d2i_PrivateKey().

You don't need to pass an existing key.  Just pass NULL for
(EVP_PKEY **) pointer, and let OpenSSL return a freshly allocated
key:

    EVP_PKEY *key;

    key = d2i_PrivateKey(type, NULL, ...);
    key = d2i_AutoPrivateKey(NULL, ...);

I strive to also check that the buffer pointer advanced by the expected
length (no "left-over" data):

    https://github.com/vdukhovni/postfix/blob/master/postfix/src/tls/tls_certkey.c#L293-L306

-- 
    Viktor.