RSAerr(PADDING CHECK) called in success path?

[Thomas Uban]

I am using the openssl 3.0.0 library in an embedded environment and added a unit test for RSA based on test/rsa_mp_test.c:test_rsa_mp().
When the test is run, I see:
1.      crypto/rsa/rsa_pk1.c:RSA_padding_check_PKCS1_type_2() calls RSAerr() just before returning at the bottom of the function in what appears to be the success path.
2.      crypto/rsa/rsa_ossl.c:rsa_ossl_private_decrypt() calls RSAerr() near the bottom of the function, regardless of the value of ‘r’ – so also in what appears to be the success path.

Both errors indicate padding problems for what are presumably good test data. If I comment out the rsa_pk1.c case and add if (r < 0) to the rsa_ossl.c case, the final comparison in the test indicates that the encrypt/decrypt resulted in the original data.

What am I missing here?

--tnx
--tom



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20190925/954d619a/attachment.html>