[openssl-users] Enable the FIPS mode in the library level

[Dr. Matthias St. Pierre]

Am 05.03.2018 um 20:39 schrieb Alan Dean:
> Thanks Matthias for your response.
>
> I have a different question:
>
> Per your suggestion in the previous email, FIPS_mode_set() can be
> moved inside of OPENSSL_init(), in order to force the FIPS mode
> enabled in the library level.
>
> However currently OPENSSL_init() is actually invoked from within
> FIPS_mode_set(). If we put FIPS_mode_set() inside OPENSSL_init, then
> it makes OPENSSL_init to call itself recursively. In that case would
> it be ok to just remove OPENSSL_init from within FIPS_mode_set?
>

It is not a problem if OPENSSL_init() is reentered, because the 'done'
flag at the beginning of the function is set on first call to prevent
the body from being executed more than once. (Well, there is a potential
race condition if the first call to OPENSSL_init() happens concurrently,
but let's assume that on your first call to OPENSSL_init() your
application is still single-threaded.) So you can put the
FIPS_mode_set() call where I indicated without risking an infinite
recursion.

Having said that, I consider this change an interesting 'hack' if you
want to see what happens, but not more. I would like to emphasize once
more that it is a quite ambitious and almost impossible task to make all
these third party applications you were talking about conforming to the
FIPS regulations. Turning on FIPS mode is the smallest part of the
problem, you have to make the entire applications conform to the FIPS
rules. It's not only that you have to convert all the calls into low
level algorithms to using the EVP interface. Also, you have to prevent
the applications from using non-fipsed algorithms, like e.g. md5.  So in
many cases you will have to make deep changes to the application's logic.

Matthias


void OPENSSL_init(void)
{
    static int done = 0;
    if (done)
        return;
    done = 1;
#ifdef OPENSSL_FIPS
    FIPS_set_locking_callbacks(CRYPTO_lock, CRYPTO_add_lock);
# ifndef OPENSSL_NO_DEPRECATED
    FIPS_crypto_set_id_callback(CRYPTO_thread_id);
# endif
    FIPS_set_error_callbacks(ERR_put_error, ERR_add_error_vdata);
    FIPS_set_malloc_callbacks(CRYPTO_malloc, CRYPTO_free);
    RAND_init_fips();
    FIPS_mode_set(1);   <<< ENABLE FIPS MODE HERE <<<
#endif
#if 0
    fprintf(stderr, "Called OPENSSL_init\n");
#endif

}