Best way of preventing denial of service attacks by way of secure client-initiated renegotiation

[tim.j.culhane at gmail.com]

Hi Matt,

Many thanks for your informative reply.

So it seems the best approach is to  upgrade to a version of OpenSSL  supporting the SSL_OP_NO_RENGOTIATION option.

If this option is enabled will it  still allow server-initiated secure  renegotiations if TLS 1.3 is being used?

The docs suggests that only  client side renegotiation requests are disabled in TLS 1.3.

Tim


-----Original Message-----
From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of Matt Caswell
Sent: Monday 15 April 2019 13:44
To: openssl-users at openssl.org
Subject: Re: Best way of preventing denial of service attacks by way of secure client-initiated renegotiation



On 15/04/2019 09:35, tim.j.culhane at gmail.com wrote:
> I'm not sure if this means  renegotiation has failed?  Either way the 
> connection remains open.  Presumably if a client issued a large number 
> of renegotiations like this the server could become overwhelmed.

No - renegotiation was successful.

> Note that I got the same results if I remove the -legacy_renegotiation 
> option, so I don't think  this has any impact?

The legacy_renegotiation option does something different to what you think it does. This option allows "insecure" renegotiation as opposed to the later (and
default) "secure" renegotiation. This dates back to 2009 when a flaw in the TLS protocol for renegotiation was discovered.

> 
> So, I suppose I firstly need to know if the results from testssl.sh 
> and from my own investigations point to a potential security risk by 
> way of a DoS attack?

Over the years there have been many attacks against renegotiation. They've all been fixed, however since this is a common attack vector and many applications don't need this feature it is often recommended that it is disabled.


> If so, what is the best way to prevent this.

The best way is to upgrade to a recent version of OpenSSL and use the SSL_OP_NO_RENGOTIATION option for this purpose (available from 1.1.0h and above).

If you *must* use OpenSSL 1.0.2 then there is a way to do it but it is undocumented and unfortunately this method is no longer possible in OpenSSL 1.1.0+ due to the opacity changes.

You can mark a particular SSL object (call it "s") so that it should not do renegotiation like this:

s->s3->flags |= SSL3_FLAGS_NO_RENEGOTIATE_CIPHERS;


> From what I've read online it isn't possible to disable 
> client-initiated secure renegotiation in openssl.
> Indeed, it could be argued that there are circumstances when it is 
> perfectly valid for a client to renegotiate a connection  especially 
> if it is a long-running connection.
> 
> The only  way I could find of limiting such an attack was to track the 
> number of renegotiation requests over a time and if we get a high 
> number  in a short period  then close the connection.
> I believe this can be done  in a callback function  set up via a call to:
> 
> SSL_CTX_set_info_callback

I'd recommend against this approach. A number of applications took this route due to a lack of a good alternative. However it can have unexpected consequences if you later upgrade to OpenSSL 1.1.1 and start using TLSv1.3 (where a number of legitimate interactions happen post-handshake that can be mistaken for renegotiations).

Matt