RSAerr(PADDING CHECK) called in success path?
Thomas.Uban at VERIFONE.com
Wed Sep 25 18:32:17 UTC 2019
I am using the openssl 3.0.0 library in an embedded environment and added a unit test for RSA based on test/rsa_mp_test.c:test_rsa_mp().
When the test is run, I see:
1. crypto/rsa/rsa_pk1.c:RSA_padding_check_PKCS1_type_2() calls RSAerr() just before returning at the bottom of the function in what appears to be the success path.
2. crypto/rsa/rsa_ossl.c:rsa_ossl_private_decrypt() calls RSAerr() near the bottom of the function, regardless of the value of ‘r’ – so also in what appears to be the success path.
Both errors indicate padding problems for what are presumably good test data. If I comment out the rsa_pk1.c case and add if (r < 0) to the rsa_ossl.c case, the final comparison in the test indicates that the encrypt/decrypt resulted in the original data.
What am I missing here?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the openssl-users