openssl smime/cms unable to handle binary encoded message? (header too long:asn1)

[Carlos Maynard]

Hello gentlemen,

I am troubleshooting an issue with an AS2 setup, the error I'm getting is
AS1_get_obect:header too long:asn1. Playing with another system, I was able
to replicate the issue by switching the encoding from Base64 to Binary.

Before i get a smart response... the trading partner can't switch the
encoding from binary to base64,.. so here I am ;-)

Apparently openssl assumes that all SMIME messages are base64 encoded, and
balks when this is not the case?

Basically the transmission is an encrypted and signed. The decryption goes
well, but then the signature verification fails, the full error message is
below.

139666245117592:error:0D07207B:asn1 encoding
routines:ASN1_get_object:header too long:asn1_lib.c:157:
139666245117592:error:0D0D106E:asn1 encoding routines:B64_READ_ASN1:decode
error:asn_mime.c:192:
139666245117592:error:0D0D40CC:asn1 encoding routines:SMIME_read_ASN1:asn1
sig parse error:asn_mime.c:490:

It looks like I'm not alone in this...
https://superuser.com/questions/1097544/error-when-decompress-smime-file

According to the link above... I should be able to discard the the headers
and handle the binary body with openssl. Copying/Pasting from the link
above...

$ sed '1,/^\r$/d' <suA97544.mime >suA97544.body

$ openssl cms -uncompress -inform der -in suA97544.body

Now, when I try this suggestion... sed does strip the MIME headers, but
still when attempting to uncompress with openssl there is an error.

Error reading S/MIME message
140081090963096:error:0D0680A8:asn1 encoding
routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1217:
140081090963096:error:0D07803A:asn1 encoding
routines:ASN1_ITEM_EX_D2I:nested asn1
error:tasn_dec.c:386:Type=CMS_ContentInfo

I am hoping I am just missing something here... obviously openssl is not my
forte. :-)

For illustration purposes, here is the original structure:

content-type: multipart/signed;
protocol="application/pkcs7-signature"; micalg=SHA-1;
        boundary="_=4094798051677677Sterling4094798051677677MOKO"

--_=4094798051677677Sterling4094798051677677MOKO
content-type: application/pkcs7-mime; smime-type=compressed-data; name=smime.p7m

.... (Binary)
--_=4094798051677677Sterling4094798051677677MOKO
content-type: application/pkcs7-signature; name=EDIINTSIG.p7s

.... (Binary)


After sed... the structure looks like this... which fails to uncompress
with openssl

--_=4094798051677677Sterling4094798051677677MOKO
content-type: application/pkcs7-mime; smime-type=compressed-data; name=smime.p7m

.... (Binary)
--_=4094798051677677Sterling4094798051677677MOKO
content-type: application/pkcs7-signature; name=EDIINTSIG.p7s

... (Binary)

openssl pkcs7 -in 5d7aa60750796.dat.3 -inform DER


Last but not least, if I split the two parts, remove the boundary and
content-type line... I can use pkcs7 to convert the binary content for each
individual file, and I get this, but I'm not sure that this is what I think
it is (the data what the next step would be. Talk about shooting in the
dark. LoL.

openssl pkcs7 -in 5d7aa60750796.dat.3 -inform DER
-----BEGIN PKCS7-----
MIHOBgsqhkiG9woBCRABCaCBvjCAAgEAMAoGCyqGSIb3CgEJEAMIMIAGCSqGSIb3
CgEHAaCAJIAEAnicBIGMc87PK0nNK9EtqSxItVJwLCjIyUxOLMnMz9N3dfHUjTA0
4uVKhirJTLFSsPH19/Z3dnMGSeblp6Qa6hqaJRsYGicaJ5oY6hoZWRgZmJuaWTi7
uTsGGzlAFNrxcvFyKQOBQoiHZ7ACEDkqhLgGhyj4ugYHO7q7KoDlXPwV/PxDFAKC
/J2BwmAxABhSKacAAAAAAAA=
-----END PKCS7-----


I suspect this is a compressed version of the actual message, but I can't
get any further. Any help, recommendations, thoughts would be greatly
appreciated!!

Is there a solid openssl implementation that can handle the binary encoded
smime message?

Thanks in advance!

CM.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20190915/d30eb27b/attachment-0001.html>