[openssl-users] Working with large DH parameters

[Kurt Roeckx]

On Tue, Apr 28, 2015 at 09:26:25AM -0500, jack seth wrote:
> Ok I have been doing some experiments with OpenVPN and I can connect using 10000 bit DH parameters.  Any bigger than that up to at least 13824 I get the following 'modulus too large' error on the client log:
> 
> TLS_ERROR: BIO read tls_read_plaintext error: error:05066067:Diffie-Hellman routines:COMPUTE_KEY:modulus too large: error:14098005:SSL routines:SSL3_SEND_CLIENT_KEY_EXCHANGE:DH lib
> Wed Apr 22 07:08:58 2015 TLS Error: TLS object -> incoming plaintext read error
> Wed Apr 22 07:08:58 2015 TLS Error: TLS handshake failed
> 
> Something interesting/weird also happened.  I tried to test 10001, 10002, and 10004 bit DH to find the exact place I would get the 'modulus too large' error.  But the server log reported the DH parameters being 10008 instead.  I did a test at 15104 that gave the same error but then I tried two more times and the client just sat at the 'initial packet point' like it does with the 16384 bit parameters.  So somewhere between 13824 and 16384 it switches between the error above and just sitting there 'frozen'.
> 
> Questions: 1. Can the modulus error be cured?  2. Do you think the same modulus error is going on when the client appears to freeze with parameters larger than 13824 or is something else going (i.e. why does it freeze instead of giving the 'modulus error')?  3. Why does the server log report 10001, 10002, 10004 bit DH as 10008? 		 	   		  

There is a limit of 10000:
#define OPENSSL_DH_MAX_MODULUS_BITS    10000

I suggest you do not change this.  It just gets slower without
adding security.

I have no idea why it would freeze with something larger than
13824.

I'm not sure what is logging the size, but it might be using
DH_size()*8 to log it.  I don't think their currently is an API
that returns it in bits.


Kurt