<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi Michal,</p>
<p>CMS has limited backward compatibility with PKCS#7. This is
discussed in <a moz-do-not-send="true"
href="https://tools.ietf.org/html/rfc5652#section-2">RFC 5652</a>,
and includes some suggestions as to how to some issues that might
crop up. At least the old draft of SCEP very specifically does
NOT specify CMS, but that might not have stopped someone from
using it. These docs tend to be quite sloppy. For example, I
know one that uses both PEM and DER encodings on the wire. Drives
me batty.</p>
<p>For enrolment this is particularly embarrassing, considering its
sensitivity. As a sysadmin, you might try wrapping this stuff
in
PHP and just trying both the pkcs_decrypt and cms_decrypt calls
and seeing if either work.</p>
<p>Eliot<br>
</p>
<div class="moz-cite-prefix">On 12.04.21 14:13, Michal Moravec
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:014A6238-B9C5-498B-995B-46E3CCE2C2E3@logicworks.cz">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div class="" style="word-wrap:break-word;
line-break:after-white-space">Hello,
<div class=""><br class="">
</div>
<div class="">I am a system administrator trying to integrate
two pieces of software using the SCEP protocol (more on that
later in the More Context section) .</div>
<div class="">Integration was working but one side (SaaS
product) changed the algorithms it uses for signing and
encryption within the PKCS7 message.</div>
<div class="">After that I am unable to decrypt the PKCS7
messages on my end and get the error: "ERROR
139701710696896:error:06065064:digital envelope
routines:EVP_DecryptFinal_ex:bad
decrypt:../crypto/evp/evp_enc.c:597:<br class="">
message_static_functions.c:221: decryption failed"</div>
<div class=""><br class="">
</div>
<div class="">From my current state of investigation it looks
highly likely there is a problem with PKCS7_decrypt method
which might be unable decrypt the PKCS7/CMS payload if the RSA
is used together with OAEP padding.</div>
<div class=""><br class="">
</div>
<div class="">1. What is a diffrences between pkcs7 <a
href="https://github.com/openssl/openssl/tree/681618cfc18b4f01f2c07e823308d30f6f47504b/crypto/pkcs7"
class="" moz-do-not-send="true">https://github.com/openssl/openssl/tree/681618cfc18b4f01f2c07e823308d30f6f47504b/crypto/pkcs7</a> and
CMS <a
href="https://github.com/openssl/openssl/tree/681618cfc18b4f01f2c07e823308d30f6f47504b/crypto/cms"
class="" moz-do-not-send="true">
https://github.com/openssl/openssl/tree/681618cfc18b4f01f2c07e823308d30f6f47504b/crypto/cms</a> implementations?</div>
<div class=""><br class="">
</div>
<div class="">If I understand correctly CMS is just newer name
for PKCS7 adopted by newer RFCs?</div>
<div class="">Do these two implementations overlap?</div>
<div class="">Or are there some kind of major diffrence like
PKCS7 module being there for compatibility with older code and
CMS is basically replacement to use for the future?</div>
<div class=""><br class="">
</div>
<div class="">2. Is PKCS7_decrypt method in <a
href="https://github.com/openssl/openssl/blob/681618cfc18b4f01f2c07e823308d30f6f47504b/crypto/pkcs7/pk7_smime.c"
class="" moz-do-not-send="true">
https://github.com/openssl/openssl/blob/681618cfc18b4f01f2c07e823308d30f6f47504b/crypto/pkcs7/pk7_smime.c</a> capable
of decrypting the PKCS7 payload whebf OAEP padding is used
together with RSA to encrypt it? If it is indeed not capable
of doing that would it be a bug or desired behavior?</div>
<div class=""><br class="">
</div>
<div class="">If found a few pointers:</div>
<div class=""><br class="">
</div>
<div class="">- <a
href="https://stackoverflow.com/questions/56941480/how-to-set-padding-oaep-for-pkcs7-decrypt-function-using-openssl"
class="" moz-do-not-send="true">https://stackoverflow.com/questions/56941480/how-to-set-padding-oaep-for-pkcs7-decrypt-function-using-openssl</a> <-
Developer is unable to use PKCS7_decrypt with RSA OAEP and
forced to switch to CMS_decrypt method.</div>
<div class="">- <a
href="https://stackoverflow.com/questions/34304570/how-to-resolve-the-evp-decryptfinal-ex-bad-decrypt-during-file-decryption"
class="" moz-do-not-send="true">https://stackoverflow.com/questions/34304570/how-to-resolve-the-evp-decryptfinal-ex-bad-decrypt-during-file-decryption</a> <-
Padding being a source problem of EVP_DecryptFinal_ex:bad
decrypt error message.</div>
<div class=""><br class="">
</div>
<div class="">3. How to replace PKCS7_decrypt method CMS_decrypt
as easily as possible?</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">I have no hard proof of the PKCS7_decrypt being
the culprit here so I started working on simple C program to
reproduce the issue. </div>
<div class="">Problem is my C skills are very rusty and I have
no experience working with OpenSSL C library.</div>
</div>
<div class=""><br class="">
</div>
<blockquote type="cite" class="">
<div class="">int PKCS7_decrypt(PKCS7 *p7,
EVP_PKEY *pkey, X509 *cert, BIO *data,
int flags);</div>
<div class="">
<div class="">int CMS_decrypt (CMS_ContentInfo
*cms,
EVP_PKEY *pkey, X509 *cert, BIO *dcont, BIO *out, unsigned
int flags);</div>
</div>
</blockquote>
<div class=""><br class="">
</div>
<div class="">Methods are very similiar but input data format it
different. It there an easyway to convert PKCS7 struct into
CMS_ContentInfo struct?</div>
<div class=""><br class="">
</div>
<div class="">Original usage of PKCS7_decrypt here -> <a
href="https://github.com/openxpki/libscep/blob/4067eae283ce0b3025d414e9d3b6af30def8c093/src/message_static_functions.c"
class="" moz-do-not-send="true">https://github.com/openxpki/libscep/blob/4067eae283ce0b3025d414e9d3b6af30def8c093/src/message_static_functions.c</a> </div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<div class=""># More Context</div>
</div>
<div class="">
<div class=""><br class="">
</div>
<div class="">My goal is to obtain signed client certificate
using the SCEP protocol.</div>
<div class=""><br class="">
</div>
<div class="">SCEP server: OpenXPKI <a
href="https://github.com/openxpki/openxpki" class=""
moz-do-not-send="true">
https://github.com/openxpki/openxpki</a> (Perl) with
libscep <a href="https://github.com/openxpki/libscep"
class="" moz-do-not-send="true">https://github.com/openxpki/libscep</a> library
(C). libscep used the OpenSSL.</div>
<div class="">SCEP client: VMware Workspace ONE UEM SaaS
(WSO). Black box. From the error codes client produces I
strongly suspect they use OpenSSL within .Net code.<br
class="">
<br class="">
Originally WSO SCEP client used these three algorithms for
PKCS7 operation: SHA-1, RSA (PKCS 1.5 padding or no padding
<- can't tell which) and 3DES-CBC.</div>
<div class="">They made some sort of change (unannouced) and
since the last SaaS upgdate WSO client uses SHA-2 (256bit),
RSA with OAEP padding and AES-256-CBC.<br class="">
Compare the attached PKCS7 messages (libscep_wso*.p7b). I
use <a href="https://lapo.it/asn1js" class=""
moz-do-not-send="true">https://lapo.it/asn1js</a> to
decode them easily.</div>
<div class="">After the change I am unable to decrypt the
PKIOperation message on the server side.</div>
<div class=""><br class="">
</div>
<div class="">I suspect the RSA with OAEP is the cause of the
problem because using different client with SHA-256,
AES-256-CBC and RSA decryption works without any problem
(libscep_sscep_ok.p7b).</div>
<div class=""><br class="">
</div>
<div class="">Best Regards,</div>
<div class="">Michal Moravec</div>
<div class=""><br class="">
</div>
<div class=""> </div>
</div>
</div>
<div style="word-wrap:break-word; line-break:after-white-space">
<meta content="text/html; charset=us-ascii">
<div>
</div>
</div>
<div style="word-wrap:break-word; line-break:after-white-space">
<meta content="text/html; charset=us-ascii">
<div>
</div>
</div>
<div class="" style="word-wrap:break-word;
line-break:after-white-space">
<div class="">
</div>
</div>
</blockquote>
</body>
</html>