[openssl-users] PKCS7->signerInfo->encryptedDigest not type X509_SIG

[Michael Heide]

Hi,

I'm using OpenSSL to verify a (proprietary?) timestamp in Microsoft Authenticode via PKCS7_verify() (in pk7_smime.c).

Those Timestamps are inside a PKCS7 SignerInfo Structure (OpenSSL type PKCS7_SIGNER_INFO). I put those inside a PKCS7 to be able to use PKCS7_verify().

Most of them are verified fine. In most cases the encryptedDigest structure inside (pkcs7_signer_info_st->enc_digest) corresponds to an OpenSSL type X509_SIG after decryption, which is an ASN.1 encoded Identifier+Hash (sha1,sha256,...). OpenSSL can parse them and use the hash aka message digest inside. 

But sometimes there is only the plain hash (sha1 usually) inside the decrypted "encryptedDigest". OpenSSL doesn't detect this. int_rsa_verify() simply tries to decode it via d2i_X509_SIG() which fails. 

The solution is simple. If there's only a hash inside, then after decryption and removing the rsa-padding the resulting structure has exactly the size of the expected hash (-> use the hash for comparison). In all other cases the decrypted ASN.1 structure must be bigger than the hash (including headers). This test is simple and fast, at least if it fails in the normal case. 

I've patched OpenSSL and it works. See Attachment. 

I don't think this is a bug inside OpenSSL. In fact, rfc2315 states the encrypted digest has to be a DigestInfo structure:

  DigestInfo ::= SEQUENCE {
     digestAlgorithm DigestAlgorithmIdentifier,
     digest Digest }

This corresponds to OpenSSLs X509_SIG.

So I do not report it as a bug. Probably those Authenticode Timestamps are not standards compliant. Or the PKCS7 structure I'm creating around those SignerInfos is wrong/broken/incomplete. Or maybe my approach to use PKCS7_verify() for this is wrong? Thus I would like to share my experiences here. 

Nevertheless, if there are no regular cases where i equals m_len in int_rsa_verify(), then it shouldn't do any harm to include those changes in OpenSSL. (with added handling of rm!=NULL which is ignored so far, doesn't get used by RSA_verify(), simple to implement if needed)

Regards
Michael
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: openssl-1.0.2d-non-enveloped-hash-patch.txt
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20150911/a13808e7/attachment.txt>