[openssl-dev] FIPS / RSA / ENGINE bug?

[Hong Cho]

​FYI, I played with "no-rsax" without "no-engine", and it ​works.

Without RSAX, the engine is NULL, and the default method (in this case,
FIPS) weren't getting overridden in RSA_new_method().

I think I will go with this.

Hong.

On Fri, Feb 27, 2015 at 12:56 PM, Hong Cho <hongcho at gmail.com> wrote:

> Steve,
>
> Thank you for the response.
>
> Here is what ENGINE_get_id() is saying.
>
> ------
> XXX RSA_new_method: ret->engine = 0x800e28100
> XXX RSA_new_method: ENGINE_get_id(ret->engine) = rsax
> XXX RSA_new_method: ret->meth = 0x800b2ee40
> ------
>
> Hong.
>
> On Fri, Feb 27, 2015 at 10:16 AM, Dr. Stephen Henson <steve at openssl.org>
> wrote:
>
>> On Fri, Feb 27, 2015, Hong Cho wrote:
>>
>> > Hi,
>> >
>> > I generated OpenSSL libcrypto (1.0.1l) with the OpenSSL FIPS crypto
>> module
>> > (2.0.8) on FreeBSD 8.4 amd64.
>> >
>> > It seems to build fine, and with OPENSSL_FIPS, it seems to behave
>> correctly
>> > (e.g., MD5 is refused, DH with 512-bit key is refused, etc.).
>> >
>> > However, genrsa(1) is failing with the following message.
>> >
>> > ------
>> > # /usr/bin/openssl genrsa -out rsa1 2048
>> > Generating RSA private key, 2048 bit long modulus
>> > 34374116264:error:0409B09D:rsa routines:RSA_generate_key_ex:non fips rsa
>> > method:[...]/openssl/crypto/rsa/rsa_gen.c:88:
>> > ------
>> >
>> > So, I put some debugging printf's, and this is what I found.
>> >
>> > ------
>> > # /usr/bin/openssl genrsa -out rsa1 2048
>> > XXX MAIN: engine = 0x0
>> > Generating RSA private key, 2048 bit long modulus
>> > XXX MAIN: e = 0x0
>> > XXX RSA_get_default_method: returning FIPS_rsa_pkcs1_ssleay()
>> > XXX RSA_new_method: ret->meth = 0x800b4a7e0
>> > XXX RSA_new_method: engine = 0x0
>> > XXX RSA_new_method: ret->engine = 0x800e28100
>> > XXX RSA_new_method: ret->meth = 0x800b2ee40
>> > XXX MAIN: rsa->meth = 0x800b2ee40
>> > XXX RSA_generate_key_ex: rsa->meth = 0x800b2ee40
>> > 34374116264:error:0409B09D:rsa routines:RSA_generate_key_ex:non fips rsa
>> >
>> method:/usr/home/hongch/ns_depot/TOT/usr.src/crypto/openssl/crypto/rsa/rsa_gen.c:88:
>> > ------
>> >
>> > ???So, it seems like the FIPS RSA_METHOD gets overridden by the default
>> > ENGINE (the machine does not have any hardware crypto module), which
>> does
>> > not have??? the RSA_FLAG_FIPS_METHOD flag set.
>> >
>> > I understand that I can rebuild the library with "./config no-engine",
>> but
>> > I haven't seen this mentioned in the FIPS guide.
>> >
>> > Is this the right way to do it? Or can the interaction between FIPS and
>> > (the default) ENGINE be better?
>> >
>>
>> Can you work out which ENGINE it is that is doing that? If you print out
>> ENGINE_get_id(engine) that will show it. I'm guessing it's the cryptodev
>> ENGINE?
>>
>> Using no-engine is the only way I can immediately think of to address this
>> without modifying OpenSSL or making some API calls to disable the ENGINE.
>>
>> Steve.
>> --
>> Dr Stephen N. Henson. OpenSSL project core developer.
>> Commercial tech support now available see: http://www.openssl.org
>> _______________________________________________
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20150227/f5ed402e/attachment.html>