[openssl-users] DH_generate_key Hangs

[Michael Wojcik]

> From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf
> Of Jeffrey Walton
> Sent: Thursday, October 05, 2017 13:33
> To: Jason Qian; OpenSSL Users
> Subject: Re: [openssl-users] DH_generate_key Hangs
> 
> 
> You should avoid calls to RAND_poll altogether on Windows. Do so by
> explicitly seeding the random number generator yourself.

As a starting point, try something like this:

-----
static ENGINE *rdrand;

void init_prng(void) {
    /* Try to seed the PRNG with the Intel RDRAND on-chip PRNG */
    OPENSSL_cpuid_setup();
    ENGINE_load_rdrand();
    rdrand = ENGINE_by_id("rdrand");
    if (rdrand) {
        int success = 0;
        if (ENGINE_init(rdrand)) {
            success = ENGINE_set_default(rdrand, ENGINE_METHOD_RAND);
        }

        /***
        Per OpenSSL wiki, call ENGINE_free here regardless of whether we're
        successfully using rdrand. The "functional reference" to rdrand will
        be released when we call ENGINE_finish.
        ***/
        ENGINE_free(rdrand);
        if (! success) ENGINE_finish(rdrand), rdrand = NULL;
    }

    if (!rdrand && !RAND_status()){
      RAND_screen();   /* this isn't really emough entropy, but it's a start */
      if (!RAND_status()) {
         RAND_poll();      /* try to gather additional entropy */
      }
   }
}

void terminate_engines(void) {
   if (rdrand) ENGINE_finish(rdrand), rdrand = NULL;
   /* similarly for any other engines you use */
   ENGINE_cleanup();
}
-----

Call init_prng after your OpenSSL initialization code (e.g. after calling OpenSSL_add_all_algorithms), and terminate_engines when you're done using OpenSSL (e.g. just before process exit).

Note that this code uses RAND_screen if RDRAND isn't available. RAND_screen is really not a very good idea; it may be OK on workstations, but rarely provides much entropy on servers because they typically aren't doing much screen output. And if you still need entropy after the RAND_screen call, you'll end up in RAND_poll anyway. The alternative is to write your own code that harvests entropy from some source (or sources).

Other people may have better suggestions.

-- 
Michael Wojcik 
Distinguished Engineer, Micro Focus