RSA test vectors, etc.

[Philip Prindeville]

Hi,

I've been trying to rewrite the res_crypto.so support in Asterisk to use Openssl-1.1.x and the EVP_PKEY interface, rather than the AES_* and RSA_* stuff.

The AES stuff uses ECB and 128 bit keys... That's a larger issue of redesigning the entire API and the client apps to support GCM and stronger keys.  Yes, I'm aware... but I'm focusing on baby steps for now.

To make sure I'm not breaking anything, I'm trying to add test coverage (including test vectors) for both.

AES-ECB is easy, because it's 100% reproducible.

RSA is turning out to be trickier, because of OAEP and PKCSv1.5 randomness.

As I see it, I have two choices:

(1) test RSA as an end-to-end pipeline, encrypting, then decrypting, and verifying that there's agreement on the plaintext message at both ends--this gives no visibility into the intermediate crypt text results... for all I know, the text is going through unchanged;

(2) mess with the randomness/seeding of OAEP and PSS to force it to always generate the same results--this is ideal from a reproducibility point of view, but cryptographically a nightmare;

As a test, I tried to generate my crypt text from the CLI to paste into my C code as:

% echo -n "Mary had a littl" | openssl rsautl -inkey tests/keys/rsa_key1.pub -pubin -encrypt -oaep -rand /dev/zero | xxd --include -c 8 

But repeating this command gets me different output every time, so faking out the random-number generator with something that always generates the same value doesn't seem to be sufficient.

How do other people deal with this?

The other tests I need to do are RSA signing and verifying.  Verifying is easy because I can use a canned signature (and key, of course).  Signing is more problematic, because of the non-determinism/reproducibility.

Same question: how do other people deal with this?

Thanks,

-Philip