Best way of preventing denial of service attacks by way of secure client-initiated renegotiation

[tim.j.culhane at gmail.com]

Hi all,

A customer of ours was recently running security checks against our mail
server.  To do this they were running the testssl.sh script available at:

https://testssl.sh/

The test tool reports a potential DoS thread as a result of client-initiated
secure renegotiation as shown from the following line from the test tool's
output:


 Secure Client-Initiated Renegotiation     VULNERABLE (NOT
ok), potential DoS threat

I have also reproduced  their findings in-house.

Our mail server is using version 1.0.2g of openssl  and  the client openssl
version was 1.0.2j.

In the testssl.sh script  it connects to the ssl port using TLS1.2  and the
legacy_renegotiation option.

When I do a similar  test in my test environment  I  get results like the
below:

 Issue  a connect request, pause for 1 second and then Send 'R'  to start a
renegotiation:

(echo -n; sleep 1; echo R) | openssl s_client -CAfile NightlyBuild-CA.cert
-crlf -tls1_2 -legacy_renegotiation -connect <hostname>:465

Output is as follows:



CONNECTED(00000003)
---
Certificate chain
 0 s:/C=ie/CN=something/CN=devimpala13.es.cpth.ie
   i:/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC0DCCAjmgAwIBAgIBATANBgkqhkiG9w0BAQQFADBFMQswCQYDVQQGEwJBVTET
...
-----END CERTIFICATE-----
subject=/C=ie/CN=something/CN=devimpala13.es.cpth.ie
issuer=/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd
---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms:
RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SH
A256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA
+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms:
RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SH
A256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:RSA+SHA1:DSA
+SHA1:ECDSA+SHA1
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1297 bytes and written 445 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID:
A1E559EAFC571033874C52C715E8CE3452751FB80AD843F4E051E77664F20FFE
    Session-ID-ctx: 
    Master-Key:
BE06E684D71B9B3349DBB057C433BB0C0C44717EF2157EAA912A896F43DF8E6E912A69B8258E
C9031CF2219F0F031C1B
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 61 3a da 77 02 6c 7e 26-c1 98 84 ae 26 21 8f 1d
a:.w.l~&....&!..
...
    Start Time: 1555315404
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
220 devimpala13.es.cpth.ie ESMTP Service ready
RENEGOTIATING
depth=1 C = AU, ST = Some-State, O = Internet Widgits Pty Ltd
verify return:1
depth=0 C = ie, CN = something, CN = devimpala13.es.cpth.ie
verify return:1
DONE

So, as you can see the  initial connection is successful.   The
renegotiation  begins and  it returns:

verify return:1

I'm not sure if this means  renegotiation has failed?  Either way the
connection remains open.  Presumably if a client issued a large number of
renegotiations like this the server could become overwhelmed.

Note that I got the same results if I remove the -legacy_renegotiation
option, so I don't think  this has any impact?

So, I suppose I firstly need to know if the results from testssl.sh and from
my own investigations point to a potential security risk by way of a DoS
attack?

If so, what is the best way to prevent this.
>From what I've read online it isn't possible to disable client-initiated
secure renegotiation in openssl.
Indeed, it could be argued that there are circumstances when it is perfectly
valid for a client to renegotiate a connection  especially if it is a
long-running connection.

The only  way I could find of limiting such an attack was to track the
number of renegotiation requests over a time and if we get a high number  in
a short period  then close the connection.
I believe this can be done  in a callback function  set up via a call to:

SSL_CTX_set_info_callback


So, is the above information correct or is there a better way of preventing
this sort of attack.  Do we really need to allow a limited form of
client-based secure renegotiation?

If preventing an attack by way of monitoring the number of renegotiations
over time is the only good approach, do you have any code examples in C as
to how to implement this?

Many thanks,

Tim