SSL_TLSEXT_ERR_NOACK not working as expected

[Matt Caswell]

On 10/03/2022 11:33, Tal Dery wrote:
> I am developing a MITM proxy server,
> but in the case of some SNI I am interested in transferring the ClientHello as it is to the target server and actually making a transparent proxy.
> Therefore, I cannot send ServerHello to the client.
> 

I see. There isn't a supported way to abort the handshake without 
sending an alert (because in the normal course of events you're really 
not supposed to do that).

There *is* actually an undocumented way to achieve this - but I would 
consider this an accident of the implementation rather than behaviour 
that you can rely on. Populate the int pointed at by "al" with the value 
-1 before returning SSL_TLSEXT_ERR_ALERT_FATAL. This is an internal-only 
value meaning "no alert".

Another way to achieve the same goal in a probably more robust way would 
be to use the null BIO (BIO_s_null). In the tlsext_servername callback, 
change the write BIO for the SSL object (SSL_set0_wbio) to the null BIO 
causing all subsequent writes to just "disappear".

Matt



> 
> -----Original Message-----
> From: Matt Caswell <matt at openssl.org>
> Sent: Thursday, 10 March 2022 13:25
> To: Tal Dery <tal at awrora.io>; openssl-users at openssl.org
> Subject: Re: SSL_TLSEXT_ERR_NOACK not working as expected
> 
> 
> 
> On 10/03/2022 11:21, Tal Dery wrote:
>> Hi Matt,
>> Yes, I want to abort the session without notifying the client.
>> SSL_TLSEXT_ERR_ALERT_FATAL sent " 15 03 03 00 02 02 00" to client.
>> What can I do to not send the message?
> 
> You want to abort the handshake without sending an alert? That would be a protocol violation. Don't do that.
> 
> Matt
> 
> 
> 
>>
>> Thanks
>>
>>
>> -----Original Message-----
>> From: Matt Caswell <matt at openssl.org>
>> Sent: Thursday, 10 March 2022 12:54
>> To: Tal Dery <tal at awrora.io>; openssl-users at openssl.org
>> Subject: Re: SSL_TLSEXT_ERR_NOACK not working as expected
>>
>>
>>
>> On 10/03/2022 10:26, Tal Dery wrote:
>>> Hi,
>>>
>>> I am implementing an SSL server.
>>>
>>> Using SSL_CTX_set_tlsext_servername_callback I'm checking the SNI.
>>>
>>> When SNI meets my requirements (for example does not contain
>>> offensive
>>> words) I allow the handshake by returning SSL_TLSEXT_ERR_OK.
>>>
>>> When there is an offensive word, I do not want to send Server Hello
>>> message. I try to do this by returning SSL_TLSEXT_ERR_NOACK.
>>
>> If you don't want the ServerHello to be sent then you are aborting the handshake. In that case you should return SSL_TLSEXT_ERR_ALERT_FATAL. By comparison SSL_TLSEXT_ERR_NOACK is a non-fatal return code. The SNI request is not acknowledged by the server (i.e. it acts the same way as if SNI was not configured on the server at all), but no alerts are sent so the handshake proceeds as normal.
>>
>> Matt
>>
>>
>>
>>>
>>> For some reason, the server is still sending the message, and I wonder why?
>>>
>>> SSL_TLSEXT_ERR_ALERT_WARNING works as expected.
>>>
>>> I'm using OpenSSL 1.1.1f and Wireshark to verify what I say.
>>>
>>> Thanks
>>>
>>
>>
>>
> 
> 
>