[openssl-users] RSA engine is not using the functions mentioned in struct

[Richard Levitte]

In message <KL1PR01MB1334EA721A8064B36C7062F3DC590 at KL1PR01MB1334.apcprd01.prod.exchangelabs.com> on Mon, 13 Feb 2017 10:57:27 +0000, khurram ashraf <khurramashraf_786 at hotmail.com> said:

khurramashraf_786> Here the engine loads but when i try to encrypt a text file by using
khurramashraf_786> following command
khurramashraf_786> 
khurramashraf_786> openssl pkeyutl -encrypt -in message.txt -pubin -inkey pubkey-B.pem -engine rsa_engine -out cipher.bin
khurramashraf_786> openssl pkeyutl -decrypt -in cipher.bin -inkey privkey-B.pem -engine rsa_engine -out rec.txt
khurramashraf_786> 
khurramashraf_786> It seems that it is not using the functions which I defined in the
khurramashraf_786> struct_rsa. It is also not giving the output from the printf in the
khurramashraf_786> function.

You need to tell it where to find the engine as well.  libcrypto looks
in the standard system places (typically /usr/lib or so) and the path
given by the environment variable OPENSSL_ENGINES.  Try these lines
and see if that makes a difference:

OPENSSL_ENGINES=.
export OPENSSL_ENGINES
openssl pkeyutl -encrypt -in message.txt -pubin -inkey pubkey-B.pem -engine rsa_engine -out cipher.bin
openssl pkeyutl -decrypt -in cipher.bin -inkey privkey-B.pem -engine rsa_engine -out rec.txt

Cheers,
Richard

-- 
Richard Levitte         levitte at openssl.org
OpenSSL Project         http://www.openssl.org/~levitte/