<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta name=Title content=""><meta name=Keywords content=""><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body bgcolor=white lang=EN-US link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal style='margin-left:.5in'><span style='font-size:11.0pt'>OpenSSL implementation of OAEP wrongly refuses to set the hash algorithm, preventing one from using SHA-2 family:</span><o:p></o:p></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:11.0pt'><br>You'll probably need to pick up master and its -rsa_mgf1_md argument to pkeyutl.<br><br><o:p></o:p></span></p><p class=MsoNormal><b><span style='font-size:11.0pt;color:#7030A0'>Thank you – better with “</span></b><b><span style='font-size:11.0pt;font-family:"Courier",serif;color:#7030A0'>-pkeyopt rsa_mgf1_md:sha256</span></b><b><span style='font-size:11.0pt;color:#7030A0'>”. But still broken – as it affects only the MGF1 setting, but <u>not</u> the hash setting. I’d say it still needs to allow “</span></b><b><span style='font-size:11.0pt;font-family:"Courier",serif;color:#7030A0'>-pkeyutl digest:xxx</span></b><b><span style='font-size:11.0pt;color:#7030A0'>” parameter.</span></b><b><span style='font-size:11.0pt'><o:p></o:p></span></b></p><p class=MsoNormal><span style='font-size:11.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>$ ~/openssl-1.1/bin/openssl version<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>OpenSSL 1.1.1-dev xx XXX xxxx<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>$ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_mgf1_md:sha256<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>$ yhsm2-tool --decrypt -m RSA-PKCS-OAEP --id 0301 -i t1264.dat.enc2.oaep -o t1264.dat.dec2 --hash-algorithm SHA256<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Using slot 0 with a present token (0x0)<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Logging in to "YubiHSM".<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Please enter User PIN: <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Using decrypt algorithm RSA-PKCS-OAEP<o:p></o:p></span></p><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Courier",serif'>OAEP parameters: hashAlg=SHA256, mgf=MGF1-SHA256</span></b><span style='font-size:11.0pt;font-family:"Courier",serif'>, source_type=0, source_ptr=0x0, source_len=0<o:p></o:p></span></p><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Courier",serif'>error</span></b><span style='font-size:11.0pt;font-family:"Courier",serif'>: PKCS11 function C_Decrypt failed: rv = CKR_FUNCTION_FAILED (0x6)<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Aborting.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>$ yhsm2-tool --decrypt -m RSA-PKCS-OAEP --id 0301 -i t1264.dat.enc2.oaep -o t1264.dat.dec2 <b>--hash-algorithm SHA-1</b> --mgf MGF1-SHA256<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Using slot 0 with a present token (0x0)<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Logging in to "YubiHSM".<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Please enter User PIN: <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>Using decrypt algorithm RSA-PKCS-OAEP<o:p></o:p></span></p><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Courier",serif'>OAEP parameters: hashAlg=SHA-1, mgf=MGF1-SHA256</span></b><span style='font-size:11.0pt;font-family:"Courier",serif'>, source_type=0, source_ptr=0x0, source_len=0<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>$ cmp t1264.dat t1264.dat.dec2<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Courier",serif'>$</span><span style='font-size:11.0pt'><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt'> <o:p></o:p></span></p></div></body></html>