SMIME signed message verification

[Harald Koch]

Hello,

my task is to sign a message in C for SMIME exchange, which works as expected and openSSL is self-fulfilling with itself in successful verification (and unsuccessful in produced errors as expected). I've tested PKCS7 SMIME functions, as well as CMS ones, leading to the same result: the reference software endpoints (both written in Java; at least one uses BuncyCastle) are unable to verify the signature. See below the BASE64 blocks of a successful reference signature, and an unsuccessful openSSL variant of the same message, both signed with the same certificate and private key. The error message extracted from the Java implementations are:
- "Unable to verify content integrity: Missing data"
- "The system is unable to find out the sign algorithm of the inbound message"

I digged a bit deeper into the ASN1 data („cat signature.base64 | base64 -d | openssl asn1parse -inform DER" ), leading to my assumption that the algorithm provided for signature contained differs:
- openSSL indicates „rsaEncryption"
- Java indicates „sha512WithRSAEncryption"

Since message digest and signing timestamp are available in both signatures, this is my only point of difference. Is there something I can do in openSSL to change the message algorithm to be provided in a form which Java accepts?

Here the two base64 encoded signatures. Reference message (which works in all directions):

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCAMIID
ljCCAn6gAwIBAgICBF0wDQYJKoZIhvcNAQELBQAwgY0xEjAQBgNVBAMTCWRlZGljYXRlZDEXMBUG
A1UEChMOVElIIEl0YWxpYSBTUkwxDDAKBgNVBAsTA2VkaTEPMA0GA1UEBxMGUGFkb3ZhMQ4wDAYD
VQQIEwVJdGFseTELMAkGA1UEBhMCUEQxIjAgBgkqhkiG9w0BCQEWE2FzMkBkZWRpY2F0ZWQud29y
bGQwHhcNMTcxMTAyMTMyNzE3WhcNMjIxMTAxMTMyNzE3WjCBjTESMBAGA1UEAxMJZGVkaWNhdGVk
MRcwFQYDVQQKEw5USUggSXRhbGlhIFNSTDEMMAoGA1UECxMDZWRpMQ8wDQYDVQQHEwZQYWRvdmEx
DjAMBgNVBAgTBUl0YWx5MQswCQYDVQQGEwJQRDEiMCAGCSqGSIb3DQEJARYTYXMyQGRlZGljYXRl
ZC53b3JsZDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALZv/pezb4MuezuMVb40qyCD
z20xtPzWtjchXqeJnPo6b2VPHloSr2zGQ8mG9fsyfFj/WLz6d06ErB09Fr6M53Y7UjmMmik1BThL
VXFKEZThlWTtfTwlhinoIjbcZ+sr4BKsVPvUShIEE5p4E3xHEcbx/8cMovt5s1gWiIakiwB+I8XV
gg0+fr2IBxf9FvzNPyZZGN65sqdhNDHEyfkl65SFZoL8iDhmIgmeCB1ebW5syccFmAS984iBx0jk
RWGACZaWge5X0kP50wk1WnEaHcz/Uay+3pC/j4ZUZGVL2jw44fd1BUXelaj+XcqG341r6xN0fJlR
Xc0RhwmCcKINgFECAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAJGJhUUcu/LTzRb+D7xCqwVi9h4EN
WTxv81uFBOtebUjdQkNSsaXD5eHOr//tvqgNti3uGVdIfJarGSbaM32RrTSzbQvo+6r9MUM+qkx5
o/KTB2GuG+yyKEYbgrRsTluxu0KcsX44t+HyZ2VJnt9u44E2e34So3ioC02B4CoTzOFtT+DsWznd
EWfB5r9GsY3IEfAOHtTgMFDPjb8pDfl/fTk6Pv3c+drSK/Os8qTa1gbe3GKOwIhXh01AsG2kaIss
g7tBXtHYlnZgBqoRuIJus+kmUkJ4s9VDr+HCy4sVg1Q/h0Y6voUdTkBECRWR5zK5eb6VHSZRD2i9
uJSoqQmjNQAAMYICgjCCAn4CAQEwgZQwgY0xEjAQBgNVBAMTCWRlZGljYXRlZDEXMBUGA1UEChMO
VElIIEl0YWxpYSBTUkwxDDAKBgNVBAsTA2VkaTEPMA0GA1UEBxMGUGFkb3ZhMQ4wDAYDVQQIEwVJ
dGFseTELMAkGA1UEBhMCUEQxIjAgBgkqhkiG9w0BCQEWE2FzMkBkZWRpY2F0ZWQud29ybGQCAgRd
MA0GCWCGSAFlAwQCAwUAoIG/MBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkF
MQ8XDTIwMDEyNDE5NDQzNlowNAYJKoZIhvcNAQkPMScwJTAKBggqhkiG9w0DBzAOBggqhkiG9w0D
AgICAIAwBwYFKw4DAgcwTwYJKoZIhvcNAQkEMUIEQEuNaTyufm0EkVoVX1YXteYJnQk+ViLhR9YL
lz5P8+s8PVPtY1bOazDhEC4nL2U4Cj99zlaq+FQrKRu58uOjZ9QwDQYJKoZIhvcNAQENBQAEggEA
RngLYRDUXYCYRb1Xlht8Ywl7MWawKvJqYMgH+YFJA2Wpsgi2soSrjA35TmvqOQtm1cJBJrqx8OU7
NdnE8y5rrJPNDp4QMnwzWAzWWyf6zZWGFQGGEMn729zAPjl170Ka+WqLjRbVIC9Yvq4fYktfZKpg
WosLs1zvLzJYAivWeFxmbuAmaUii/FkhFlZNEp/EM3FVbAG5GlqG/2vciE6uwcuW+Jk4/D+0deh5
N8ib3Dar9vBOlqfcqwUyvIY7RJVCrIa7xuwfJY/nrG/EK2p3uzHX542AKs0sBYk9Bm5DS0cXvONe
UiQl/4Cj3pOivvmGIdhy5o7N5pwCCJh6cZv14QAAAAAAAA==


openSSL’s version, which is not accepted by Java applications:
MIILuAYJKoZIhvcNAQcCoIILqTCCC6UCAQExDTALBglghkgBZQMEAgEwCwYJKoZI
hvcNAQcBoIII0zCCCM8wggS3oAMCAQICClLgJv3GpnHBcM0wDQYJKoZIhvcNAQEL
BQAwgYsxCzAJBgNVBAYTAkRFMRYwFAYDVQQHDA1Ib2x6Z2VybGluZ2VuMRUwEwYD
VQQKDAxjLXdvcmtzIEdtYkgxFTATBgNVBAsMDENlcnRpZmljYXRlczERMA8GA1UE
AwwIT0ZUUDIgQ0ExIzAhBgoJkiaJk/IsZAEZFhNvZnRwMmNhLmMtd29ya3MubmV0
MB4XDTE4MDQxMDEzMTQwMloXDTIwMDMxMDEzMTQwMlowcDELMAkGA1UEBhMCSVQx
DzANBgNVBAcMBlBhZG92YTEdMBsGA1UECgwUZEVESWNhdGVkIEl0YWxpYSBzcmwx
ETAPBgNVBAsMCEVESSBUZWFtMR4wHAYDVQQDDBVvZnRwMi5kZWRpY2F0ZWQud29y
bGQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBHryKO7KqVZdIPrnB
ArhEjBqeNfBUFeM5n1lmSU+x5pYfAp5MYGMxPnFpH7Q4Vf+XK/6yTn3VI+hT/bcW
NiealiyRBjRQEX/was7bxDzY9y/Ms+tWGqhq0H6qN5q8gJ/15H6/B3jSV5h/gOAX
hdXUuh3NuC2aXubWVlF+IEWtFj9xTVf6V+6MHoEpIw+C9LCbVvTipasjxB8O8dFk
N3bvFhFfHUkXA2OlPxZZxBMFa26DARibS7YO5vjoCpofwBtsonxkUEps4vn4QFu3
0klEqRohhZnNzEdj9LqW6qdKPf3ZyVIgaOQa+WMaVDve2yZ9mh/T9lypM6NEBUDV
WT11AgMBAAGjggJNMIICSTAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIGwDAO
BgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMDAG
CWCGSAGG+EIBDQQjFiFPRlRQMiBDZXJ0aWZpY2F0ZSBvZiBjLXdvcmtzIEdtYkgw
UQYDVR0RBEowSIYWb2Z0cDovL08wMTc3WDBZTTAwMDAwMYIVb2Z0cDIuZGVkaWNh
dGVkLndvcmxkgRdzdXBwb3J0QGRlZGljYXRlZC53b3JsZDAdBgNVHQ4EFgQUtcZm
yFMAlxg78z9qTJsX7grfEV4wHwYDVR0jBBgwFoAU2/OITatwz/A2aA5O8GYErWWT
oNwwUAYIKwYBBQUHAQEERDBCMEAGCCsGAQUFBzAChjRodHRwOi8vb2Z0cDJjYS5j
LXdvcmtzLm5ldC9wa2kvcHViL2NhY2VydC9jYWNlcnQuY2VyMB4GA1UdEgQXMBWC
E29mdHAyY2EuYy13b3Jrcy5uZXQwPwYJYIZIAYb4QgEEBDIWMGh0dHA6Ly9vZnRw
MmNhLmMtd29ya3MubmV0L3BraS9wdWIvY3JsL2NhY3JsLmNybDA/BglghkgBhvhC
AQMEMhYwaHR0cDovL29mdHAyY2EuYy13b3Jrcy5uZXQvcGtpL3B1Yi9jcmwvY2Fj
cmwuY3JsMEEGA1UdHwQ6MDgwNqA0oDKGMGh0dHA6Ly9vZnRwMmNhLmMtd29ya3Mu
bmV0L3BraS9wdWIvY3JsL2NhY3JsLmNybDANBgkqhkiG9w0BAQsFAAOCBAEAE3KU
a89sGS4Cz7EFqmC2cjawfeaKM0ryihuLqaiMlm1dhuqiUaBTDDXZyoK/Z908uHGA
YQGkHsyDL50Rn7FOemf4vgkw+FJ5Jz8AVUNQ0Ai8lFNOirvbXbOjI9BjIIdWrfu+
cTrqw746ZUl+gsfeLtuneWhJLbAbvOJxQIczzRkHW/VkBSzRzgbLDoaQFEKNJyfg
Gj7uZRDZBFjaVMuObyNQgDLyv167+zjwRXsjzGGk2q830nM2qxH3jsWXjBGr5h0H
mtrZu4/x+ek7wXqr+i2azE9iN7cV55ni5tbcSXmQOT3bHd+8gQ+lG6ZB5a4u94pS
LU4LsbNFjprQmMQJ+HvgjiWLS3cAXo6TA6Kc/ojrmRVinkPzD7fEXAg/MxROAnRI
VGCYiXipMPputi9h1FdOnIakwOHxgW7mAVFh1kdhsJ4Sjv42sbbUwgPkkxww9dG0
Uk78dCALUfZCSjTFVYXpOh0OxO8SgY9CCHVPjcPQV0e7k/N3Je3erTCNw20pLXtF
wINtpxp0SdCRl3NsYG+6YJ1WTK4TF4DOw9Hi8rKjnU6Fz/aYqVPD2I4IfIXter5B
f26+2utLMr0VG+QV9cXkoRaYEgXZOiuEY+AmHn+tSL9/ICcRG1ya/4xRNI5USP5K
mzeh5AZXIFN4R6Q8cctGn4Bl1LZqgPoIja029N8PEiqefzSFv/fcf6GMkl9GKQRy
WaPCLsjRYMHUsn5IjYnb2FS34LtrVIzHAZ/W+kiG5gX/hz13B1yFMnFj4p65XFWN
Bfp8UKuPgUshFMKA8CaOTDmHLsIzZDC3NMbcVJAxhFWGz8FhXoKh47x+VzR+E5oL
cxnV1z3P1pTGyRn1ro+1sBqmiwPZsuyA80z3W1sfUs1XPMSeB1rDUsX0EFivGKnT
++s9nykHA/vysY1kerh5/UJ7O/72uGFCj6BB1DsXniexC1KW5Z41HmlvIlwvDA6g
DopNII6Kej2kDw2VWVoCLYNFo7xV/qoRcTonS5bvGdYC/Id4PZ6XA89WT5PyvXyK
Ptl1pOLd0GNLreyGGeCMhEYM6vQAXGB6yC/ugR2bhUc+MSUbkey4WXFDrXiPHqGo
QWIbgiRf51Y5+fHEoUFGNWGcxAg7UMUZv7OctGEH2tx4A+7SC2ZGO6rnuGUwTe0z
ack4+ypFSih6O59Y8wgeesbTCAFFZsSyzH/aggPTuydz8ARiZgN2td33iSDDw3Ec
5mhpBJf08l/H4abGKV1t3VNdQEkOFSn+N7/ksTsSXmcC24ONBDN+Ra0MwdCidO7B
/yc5jz75mwHf5AKd34ToO2RivgQxCHoCnTEBUE/kt4+8cNtv0crLTS04C97ByAod
29b75rnqhnmcO6kZlzGCAqswggKnAgEBMIGaMIGLMQswCQYDVQQGEwJERTEWMBQG
A1UEBwwNSG9semdlcmxpbmdlbjEVMBMGA1UECgwMYy13b3JrcyBHbWJIMRUwEwYD
VQQLDAxDZXJ0aWZpY2F0ZXMxETAPBgNVBAMMCE9GVFAyIENBMSMwIQYKCZImiZPy
LGQBGRYTb2Z0cDJjYS5jLXdvcmtzLm5ldAIKUuAm/camccFwzTALBglghkgBZQME
AgGggeQwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcN
MjAwMTI0MTkzODUzWjAvBgkqhkiG9w0BCQQxIgQgQBCQAoZud7bmLBC3cTB621kM
lRfJ4pkbKN52sbAELHoweQYJKoZIhvcNAQkPMWwwajALBglghkgBZQMEASowCwYJ
YIZIAWUDBAEWMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgIC
AIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZI
hvcNAQEBBQAEggEAsz2Lq/bBiX20RCodqm1k+DmNRMsNNvb2maZnL8//lcJkz73t
A38rICeH53CYDXjk/5SD6T7ogfMgxBu7tdNpFaWXMgXALBrmYiE6X0vk9Mvc6bvW
4Hrqc8FY0WLgUOExMS9LJ4zqbCNxGZGJmhPCkuFGAK0cd+ycijSNCY3WbwSmNTIP
Qvhhh5b+y252efMDDWcxl831JWmDBBTjsNAzk+mISDTm+asAJQjtUNbZMWIWgoQ/
rs7BhD+UKLbzlS7OyCh2yJAT472CXlJmA6Wv2ULhDCTqAC21ZXhs8JNIKNrLWoqO
1NGDfyQiKEk+91070jO28hXTvwol2wDyvvH4dg==

Any input is welcome.

Regards,
Harald