[openssl-dev] RSA key generation in FIPS mode

[rsteck at symsysresearch.com]

This is potentially a question with openssl-dev/openssl-users crossover.

In the FIPS Security Policy there are listed two functions for 
generating RSA keys:
FIPS_rsa_generate_key_ex()      (renamed from RSA_generate_key_ex())
and
FIPS_rsa_x931_generate_key_ex() (renamed from 
RSA_X931_generate_key_ex())

The later is a complete implementation, according to X9.31, and an 
approved method in FIPS 186-2.

The former is a wrapper function for either a "custom" keygen contained 
in the RSA struct, or the static "rsa_builtin_keygen".  This builtin 
function does not conform to X9.31 (and therefor is not acceptable under 
186-2).

In testing, it appears when running in FIPS mode and calling the wrapper 
function, the non-approved builtin function is the one that is called.  
The default RSA struct creation function, defined in 
"fips/fips_rsa_lib.c:FIPS_rsa_new()" sets a mechanism parameter 
("RSA_PKCS1_SSLeay()") that doesn't specify any key construction method 
(see cryupto/rsa/rsa_eay.c).  Without this specified in the struct, the 
default (builtin; non-approved) method is used.

Thus, it appears that there is a function in the FIPS API that allows 
for the creation of RSA keys in a non-approved manner.

Am I missing something?

Assuming I was to remediate this for one of my clients, the wrapper 
function within the canister should replace the call to the builtin 
function with a call to the RSA_X931_generate_key_ex() function, and/or 
the struct creation function should explicitly set the rsa_keygen 
method.  Correct?

Thanks,
Randy