[openssl-dev] Upgrading OpenSSL

Mon Aug 28 14:09:19 UTC 2017

The upgrade is now working fine in one of the applications which make TLS connections. I can see the engine functions being called when some action (sign/verify) which require the privatekey.

However, this engine is also used in a patched version of Racoon2.
In one of the files (crypto_openssl.c) a function is called by the IKE daemon (iked) during setup with:
            :
EVP_SignInit(&ctx, md);
EVP_SignUpdate(&ctx, octets->v, octets->l);
EVP_SignFinal(&ctx, (unsigned char*)sig->v, &siglen, pkey);
:
With the upgraded OpenSSL v1.0.2, the last function now fails with the error:
2017-08-28 15:44:14 [INTERNAL_ERR]: crypto_openssl.c:1238:eay_rsassa_pkcs1_v1_5_sign(): RSA_sign failed: 30972:error:0606B06E:digital envelope routines:EVP_SignFinal:wrong public key type:p_sign.c:123:

Q: I assume it is looking for one of the missing parameters (p) in the RSA structure - correct?
Q: If so, how did it work in v0.9.8?

If I change the first command to:
EVP_SignInit_ex(&ctx, md, engine);
Then it segfaults in the "SignFinal" command :(

The engine is dynamically loaded the same way in both my TLS connection application and in the Racoon2 application.

Thanks for your time - any help is appreciated!

Leon Brits
System Engineer
Mobile: +27 84 250 2855

[cid:image001.png at 01D32016.91CF07F0]

76 Regency Drive Route 21 Corporate Park Irene 0157

Tel +27 12 678 9740 (ext. 9767) | Fax +27 12 345 2561

www.parsec.co.za<http://www.parsec.co.za>

[cid:image002.png at 01D32016.91CF07F0]

From: openssl-dev [mailto:openssl-dev-bounces at openssl.org] On Behalf Of Leon Brits
Sent: 28 August 2017 08:08 AM
To: openssl-dev at openssl.org
Subject: Re: [openssl-dev] Upgrading OpenSSL

Thanks for the help.

I've come to learn that my problem is the HSM. It removes the RSA values p,q and d from the EVP key before returning it. This is normal since it is protecting the key by keeping it in the HSM - duh. Anyway so, I cannot use it as a normal key. "Live and learn"

So this bring me to the next question: Is there any changes I need to make in the OpenSSL Engine for my upgrade (0.9.8 -> 1.0.2) to be complete?

Regards,

Leon Brits
System Engineer
Mobile: +27 84 250 2855

[cid:image001.png at 01D32016.91CF07F0]

76 Regency Drive Route 21 Corporate Park Irene 0157

Tel +27 12 678 9740 (ext. 9767) | Fax +27 12 345 2561

www.parsec.co.za<http://www.parsec.co.za>

[cid:image002.png at 01D32016.91CF07F0]

From: openssl-dev [mailto:openssl-dev-bounces at openssl.org] On Behalf Of Leon Brits
Sent: 23 August 2017 11:52 AM
To: openssl-dev at openssl.org<mailto:openssl-dev at openssl.org>
Subject: [openssl-dev] Upgrading OpenSSL

Hi all,

I am task to update two machines from v0.9.8z to v1.0.2 (since it is LTS).

With the minimal changes, I've been able to get the application on the machines to compile with the newer version and generate RSA 4096 key pairs. The applications are able to successfully use their respective private keys and certificates to establish TLS connection between them. However, when I used the CLI to check a dumped privatekey i got the following output:

% openssl rsa -check -in privkey.pem
unable to load Private Key
1995859152:error:0D078079:asn1 encoding routines:ASN1_ITEM_EX_D2I:field missing:tasn_dec.c:489:Field=d, Type=RSA
1995859152:error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121:
1995859152:error:0606F091:digital envelope routines:EVP_PKCS82PKEY:private key decode error:evp_pkey.c:92:
1995859152:error:0907B00D:PEM routines:PEM_READ_BIO_PRIVATEKEY:ASN1 lib:pem_pkey.c:141:

Any suggestions at what is wrong with the key?
Note that an ID is stored in the RSA extended data since the private key may be stored in HSM.

Thanks for your time
LJB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170828/70eadff2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 14099 bytes
Desc: image001.png
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170828/70eadff2/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 9367 bytes
Desc: image002.png
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170828/70eadff2/attachment-0003.png>