EVP_PKEY_get_raw_public_key fails with OpenSSL 3.0

[Matt Caswell]

On 11/11/2022 00:49, James Muir wrote:
> On 2022-11-10 18:35, fus at plutonium24.de wrote:
>> I have been using EVP_PKEY_get_raw_public_key with OpenSSL 1.1.1 
>> without any problems to extract a raw public key (secp521r1, NIST 
>> curve P-521). With OpenSSL 3.0 this fails. I'm using this call to get 
>> the raw public key and to compare it with a reference value I have and 
>> I also check that the group name is "secp521r1".
> 
> That doesn't work in 3.0.
> 
> Quoting from 
> https://www.openssl.org/docs/manmaster/man3/EVP_PKEY_new.html  :
> 
> <quote>
> EVP_PKEY_get_raw_public_key() fills the buffer provided by pub with raw 
> public key data. The size of the pub buffer should be in *len on entry 
> to the function, and on exit *len is updated with the number of bytes 
> actually written. If the buffer pub is NULL then *len is populated with 
> the number of bytes required to hold the key. The calling application is 
> responsible for ensuring that the buffer is large enough to receive the 
> public key data. This function only works for algorithms that support 
> raw public keys. Currently this is: EVP_PKEY_X25519, EVP_PKEY_ED25519, 
> EVP_PKEY_X448 or EVP_PKEY_ED448.
> </quote>
> 

That text exists even in the 1.1.1 version of the man page:

https://www.openssl.org/docs/man1.1.1/man3/EVP_PKEY_get_raw_public_key.html

I am surprised that this was working in 1.1.1....from code inspection I 
can't see how it would since EC keys seem to lack the necessary support. 
I threw together some test code to check this using 1.1.1:

#include <openssl/ec.h>
#include <openssl/evp.h>
#include <openssl/bio.h>
#include <openssl/err.h>
#include <stdio.h>
#include <stdlib.h>

int main(void)
{
     EC_KEY *key = EC_KEY_new_by_curve_name(NID_secp521r1);
     EVP_PKEY *pkey = EVP_PKEY_new();
     unsigned char rawkey[1024];
     size_t keylen = sizeof(rawkey);

     if (key == NULL || pkey == NULL) {
         printf("Failed to allocate keys\n");
         goto err;
     }

     if (!EC_KEY_generate_key(key)) {
         printf("Failed to generate key\n");
         goto err;
     }

     if (!EVP_PKEY_assign_EC_KEY(pkey, key)) {
         printf("Failed to assign EC_KEY\n");
         goto err;
     }

     if (!EVP_PKEY_get_raw_public_key(pkey, rawkey, &keylen)) {
         printf("Failed to get raw public key\n");
         goto err;
     }

     printf("Raw key is:\n");
     BIO_dump_fp(stdout, rawkey, keylen);
     printf("\n");

     return EXIT_SUCCESS;
  err:
     ERR_print_errors_fp(stdout);
     return EXIT_FAILURE;
}

Running this I get:

$ openssl version
OpenSSL 1.1.1t-dev  xx XXX xxxx
$ ./eckeygen
Failed to get raw public key
140164760770368:error:060CB096:digital envelope 
routines:EVP_PKEY_get_raw_public_key:operation not supported for this 
keytype:crypto/evp/p_lib.c:309:

So, I don't understand how this ever worked for you. There must be 
something slightly strange about your key/setup??

Matt

> You were reading the P521 public-key previously (with 1.1.1), but in 
> what format was it sent you?  Do you want just the bytes of the public 
> EC point?
> 
> -James M
>