[openssl-users] OpenSSL cannot decrypt RC4-encrypted CMS object

[Stephan Mühlstrasser]

Hi,

I create a self-signed certificate, encrypt some data as a CMS message 
with "-rc4", and try to decrypt it. This fails with an error message 
(tested with OpenSSL 1.0.2):

$ echo "abcdefg" >data.txt
$ openssl req -x509 -newkey rsa:2048 -keyout key.pem -nodes -out 
cert.pem -days 100 -subj "/CN=RC4 SMIME Test"
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
Generating a 2048 bit RSA private key
....................................+++
.......................+++
writing new private key to 'key.pem'
-----
$ openssl cms -rc4 -encrypt -binary -in data.txt -out data.txt.cms 
-outform DER cert.pem
$ openssl cms -decrypt -in data.txt.cms -inform DER -out data2.txt 
-inkey key.pem -recip cert.pem
Error decrypting CMS structure
140735291474768:error:2E078066:CMS 
routines:cms_EncryptedContent_init_bio:cipher parameter initialisation 
error:cms_enc.c:128:

With other encryption algorithms this works as expected. Is there 
something special about RC4 and PKCS#7/CMS objects? Is this a bug?

-- 
Stephan