Unable to load RSA public key for signature verification

[Juan Pablo Garibotti Arias]

Hello,

I'm attempting to use libssl to verify a RSA with SHA256 signature, but I
can't get the public key to be read. I'm using a couple of Ubuntu systems,
package libssl-dev 1.0.2 and 3.0.2, both experiencing the issue.

To ensure I'm working with good keys, I ran the following steps to set up
the test:

$ openssl genrsa -out key.pem 4096
$ openssl rsa -in key.pem -pubout > key.pub
$ echo "The message to sign" > msg
$ openssl dgst -sign key.pem -keyform PEM -sha256 -out msg.sign -binary msg
$ openssl dgst -verify key.pub -keyform PEM -sha256 -signature msg.sign
-binary msg
Verified OK

At this point I have a pair of rsa keys that can be used to sign and verify
a message.

I tried to use PEM_read_RSAPublicKey to read the key from file,
and PEM_read_bio_RSAPublicKey to first read the contents and then use a BIO
to read it, but in both cases I get the following error message:
error:0480006C:PEM
routines::no start line

Here's a sample c++ program that experiences the issue

#include <iostream>
#include <memory>
#include <cstdio>

#include <openssl/rsa.h>
#include <openssl/pem.h>
#include <openssl/err.h>

struct RsaDeleter{ void operator()(RSA* ptr){RSA_free(ptr);}};
using RsaPtr = std::unique_ptr<RSA, RsaDeleter>;

struct BioDeleter{ void operator()(BIO* ptr){BIO_free(ptr);}};
using BioPtr = std::unique_ptr<BIO, BioDeleter>;

struct FileDeleter{ void operator()(FILE* ptr){fclose(ptr);}};
using FilePtr = std::unique_ptr<FILE, FileDeleter>;

RsaPtr PublicKey(const char* key, size_t keySz) {
  auto keyBio = BioPtr{BIO_new_mem_buf(key, keySz)};
  char pass[] = "";
  return RsaPtr{PEM_read_bio_RSAPublicKey(keyBio.get(), nullptr, nullptr,
pass)};
}

RsaPtr PublicKey(FILE* keyFile) {
  char pass[] = "";
  return RsaPtr{PEM_read_RSAPublicKey(keyFile, nullptr, nullptr, pass)};
}

int main() {
  constexpr auto pubKey = "-----BEGIN PUBLIC KEY-----\n"
"MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAmeleGL76annhN96BaxhZ\n"
"iZXZ+6r9Ya+qNH9ulQcKOk6/T8S1T1/cdMDM6GDAb98quemk0bc8yTgTmvHl+JLu\n"
"hNea0zYSfr0mgEcG0Yjjc/jG2V9M5xejRkgwexP/FuMY3P621k2wBLLbne0QI/k2\n"
"REi+ryd6CXoY8JpURVY75t3BlKpnPc6wttWhyn7ScnMYU8rIXQI/4ppLX1SDgHQw\n"
"Mwbp2Vqou9QZBEeZ4cUNvZJ0bHMghz0CCpxihzNFqPfuGkZ6kRC7TUFybSydSU2V\n"
"+a9WdEf63q1Vj09pyq6zvnIYHaIzfTsMfL1j6t3TMeRMV75ieTYvYXIHCW+U4UuI\n"
"kwCkP4b1/ScJ17MkkPIH1TMiTqUyXQSWIJCMfHjZ3pZ1LfAJpYG0f0rfbXAY0XBj\n"
"EO+mm/zwH77/8stKkFKUCoiXi0duq99oEvMrIU+s0mJrvQdOhmA7U3k4UKImLRs8\n"
"JLWGfI4GYes40ly9ld1D52l0HWGE9wXVtjIY4ZDDtmf5SyvF9QOctQ6IQeuX4hHj\n"
"1ewjX6offF6t4TqpaVMJKLNZRG/rnXX/pomhm+Sm2BEaNcJHfhGIRLTV3dxV26X5\n"
"FDxhY6mEz8W7f2tucc1D9FbA9iemfTsCfs5KAPZYZ9ytc9ADcUQGMN7xByzVYR7S\n"
"7/KGw3T3A+/gduseXjvmxV8CAwEAAQ==\n"
"-----END PUBLIC KEY-----\n";
  auto memKey = PublicKey(pubKey, sizeof(pubKey)-1);
  if ( !memKey )
    std::cout << "ERROR : " << ERR_error_string(ERR_get_error(), nullptr)
<< '\n';
  auto filePtr = FilePtr{std::fopen("key.pub", "r")};
  auto fileKey = PublicKey(filePtr.get());
  if ( !fileKey )
    std::cout << "ERROR : " << ERR_error_string(ERR_get_error(), nullptr)
<< '\n';
  return 0;
}

The output from calling this program is:

$ g++ -o read-pubkey -std=c++17 -DOPENSSL_API_COMPAT=10101 main.cpp -lssl
-lcrypto
$ ./read-pubkey
ERROR : error:0480006C:PEM routines::no start line
ERROR : error:0480006C:PEM routines::no start line

The hardcoded pubKey string holds the contents of the key.pub I created
before. I try to load the key both from file and from memory and both
return the "no start line" error I described above.

I am able to load the private key generated above with
PEM_read_bio_RSAPrivateKey
which is why I am trying to use the corresponding RSAPublicKey, but I've
tried with the other pubkey functions and they all have similar errors.
Even when attempting to load it into a EVP with PEM_read_bio_PUBKEY to
create the RSA from it I get error:1E08010C:DECODER routines::unsupported

So at this point I'm pretty stumped, and was hoping someone could indicate
what it is I am doing wrong.

Thanks,
Juan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20230413/8a7ba7d4/attachment.htm>