[openssl-dev] Inconsistency between implementation and docs in openssl cms

[Blumenthal, Uri - 0553 - MITLL]

Manual page for “openssl cms” says:

	If the -decrypt option is used without a recipient certificate then an
attempt is made
	to locate the recipient by trying each potential recipient in turn using
the supplied
	private key. 

	To thwart the MMA attack (Bleichenbacher's attack on PKCS #1 v1.5 RSA
padding) all 
	recipients are tried whether they succeed or not and if no recipients
match the message
	is "decrypted" using a random key which will typically output garbage.
The -debug_decrypt
	option can be used to disable the MMA attack protection and return an
error if no 
	recipient can be found: this option should be used with caution.


The first paragraph does not seem to be true - from what I observed, when
no recipient is specified, the decryption always fails - in contradiction
to the above.

This is how I created an encrypted SMIME:

$ openssl version
OpenSSL 1.0.2h  3 May 2016
$ openssl cms -encrypt -aes256 -inform SMIME -in Cyph_Bot_test.eml
-outform SMIME -out Cyph_Bot_test.smime.eml -subject SMIME_ECC
~/Documents/Certs/me_mouse_yubi_9d_.pem


Decryption with explicitly specified -recip works:

$ openssl cms -engine pkcs11 -keyform engine -decrypt -aes256 -inform
SMIME -in Cyph_Bot_test.smime.eml -outform SMIME -out
Cyph_Bot_test.decrypt.eml -recip ~/Documents/Certs/me_mouse_yubi_9d_.pem
-inkey "pkcs11:object=KEY%20MAN%20key;object-type=private"
engine "pkcs11" set.
PKCS#11 token PIN:
$ tail Cyph_Bot_test.decrypt.eml
Message-id: <FBD81819-1B51-48CA-939B-03E6403F5A9A at us.army.mil>
Date: Sun, 02 Jun 2013 00:56:22 -0400
To: Cloud Mouse <mouse07410 at me.com>
MIME-version: 1.0 (1.0)
X-Mailer: iPad Mail (10B329)

4DFJ3ECyu3XQmJJtPTXp1HJXeCSFnmL8euXcOSc1NGmDm9fqgR0RU+s0Rl1oggUJ

But the same decryption fails when -recip is omitted:


$ openssl cms -engine pkcs11 -keyform engine -decrypt -aes256 -inform
SMIME -in Cyph_Bot_test.smime.eml -outform SMIME -out
Cyph_Bot_test.decrypt1.eml -inkey
"pkcs11:object=KEY%20MAN%20key;object-type=private"
engine "pkcs11" set.
PKCS#11 token PIN:
Error decrypting CMS structure
140735083847760:error:06065064:digital envelope
routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:529:
$


Adding -debug_decrypt flag reveals the problem:

$ openssl cms -engine pkcs11 -keyform engine -decrypt -debug_decrypt
-aes256 -inform SMIME -in Cyph_Bot_test.smime.eml -outform SMIME -out
Cyph_Bot_test.decrypt1.eml -inkey
"pkcs11:object=KEY%20MAN%20key;object-type=private"
engine "pkcs11" set.
PKCS#11 token PIN:
Error decrypting CMS using private key
140735083847760:error:2E072084:CMS routines:CMS_decrypt_set1_pkey:no
matching recipient:cms_smime.c:661:
$


Either the decryptor fails to properly determine the match (and should be
fixed), or the documentation is wrong (ad should be edited).
-- 
Regards,
Uri Blumenthal
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4324 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20160603/25f515b7/attachment-0001.bin>