[openssl-users] RSA key generation in FIPS mode

[Randy Steck]

I posted this to openssl-dev, but didn't get a reply.  Perhaps it's more 
appropriate here.

In the FIPS Security Policy there are listed two functions for
generating RSA keys:
FIPS_rsa_generate_key_ex()      (renamed from RSA_generate_key_ex())
and
FIPS_rsa_x931_generate_key_ex() (renamed from 
RSA_X931_generate_key_ex())

The later is a complete implementation, according to X9.31, and an
approved method in FIPS 186-2.

The former is a wrapper function for either a custom keygen contained
in the RSA struct, or the static "rsa_builtin_keygen".  This builtin
function does not conform to X9.31 (and therefor is not acceptable under
186-2).

In testing, it appears when running in FIPS mode and calling the wrapper
function, the non-approved builtin function is the one that is called.
The default RSA struct creation function, defined in
"fips/fips_rsa_lib.c:FIPS_rsa_new()" sets a mechanism parameter
("RSA_PKCS1_SSLeay()") that doesn't specify any key construction method
(see cryupto/rsa/rsa_eay.c).  Without this specified in the struct, the
default (builtin; non-approved) method is used.

Thus, it appears that there is a function in the FIPS API that allows
for the creation of RSA keys in a non-approved manner.

Am I missing something?  Is this by design, or is it a bug?

Assuming I was to remediate this for one of my clients (hardware 
validation),
the wrapper function within the canister should replace the call to the
builtin function with a call to the RSA_X931_generate_key_ex() function,
and/or the struct creation function should explicitly set the rsa_keygen
method.  Correct?

Thanks,
Randy