<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta name=Title content=""><meta name=Keywords content=""><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Arial;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body bgcolor=white lang=EN-US link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt'>Working on pkcs11 engine, I discovered a bug in crypto/rsa/rsa_pmeth.c in pkey_rsa_encrypt() and pkey_rsa_decrypt().<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'>They cause a crash when called with out==NULL. Normally it should not happen – but when an engine is called, and it cannot process the padding – it reverts to the original OpenSSL-provided pkey_rsa_encrypt() or pkey_rsa_decrypt() (as appropriate). OpenSSL pkeyutl makes two calls when the key is not directly available (aka not presented in a disk file), and the first call with out==NULL crashes when RSA_private_decrypt() or RSA_public_encrypt() tries to copy the result to out.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'>The fix should be adding something like<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'> if (out == NULL) {<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'> int klen = RSA_size(ctx->pkey->pkey.rsa);<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'> *outlen = klen;<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'> return 1;<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'> }<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'>right before the call to RSA_public_encrypt().<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'>P.S. It’s more critical in pkey_rsa_encrypt(), because it’s more likely that the engine would handle the decryption operation completely by itself.<o:p></o:p></span></p><div><p class=MsoNormal><span style='font-size:10.5pt;color:black'>--<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;color:black'>Regards,<o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:10.5pt;color:black'>Uri Blumenthal</span><o:p></o:p></p></div></body></html>