SSL_TLSEXT_ERR_NOACK not working as expected

[Tal Dery]

Both solutions work, thanks!

-----Original Message-----
From: Matt Caswell <matt at openssl.org> 
Sent: Thursday, 10 March 2022 13:51
To: Tal Dery <tal at awrora.io>; openssl-users at openssl.org
Subject: Re: SSL_TLSEXT_ERR_NOACK not working as expected



On 10/03/2022 11:33, Tal Dery wrote:
> I am developing a MITM proxy server,
> but in the case of some SNI I am interested in transferring the ClientHello as it is to the target server and actually making a transparent proxy.
> Therefore, I cannot send ServerHello to the client.
> 

I see. There isn't a supported way to abort the handshake without sending an alert (because in the normal course of events you're really not supposed to do that).

There *is* actually an undocumented way to achieve this - but I would consider this an accident of the implementation rather than behaviour that you can rely on. Populate the int pointed at by "al" with the value
-1 before returning SSL_TLSEXT_ERR_ALERT_FATAL. This is an internal-only value meaning "no alert".

Another way to achieve the same goal in a probably more robust way would be to use the null BIO (BIO_s_null). In the tlsext_servername callback, change the write BIO for the SSL object (SSL_set0_wbio) to the null BIO causing all subsequent writes to just "disappear".

Matt



> 
> -----Original Message-----
> From: Matt Caswell <matt at openssl.org>
> Sent: Thursday, 10 March 2022 13:25
> To: Tal Dery <tal at awrora.io>; openssl-users at openssl.org
> Subject: Re: SSL_TLSEXT_ERR_NOACK not working as expected
> 
> 
> 
> On 10/03/2022 11:21, Tal Dery wrote:
>> Hi Matt,
>> Yes, I want to abort the session without notifying the client.
>> SSL_TLSEXT_ERR_ALERT_FATAL sent " 15 03 03 00 02 02 00" to client.
>> What can I do to not send the message?
> 
> You want to abort the handshake without sending an alert? That would be a protocol violation. Don't do that.
> 
> Matt
> 
> 
> 
>>
>> Thanks
>>
>>
>> -----Original Message-----
>> From: Matt Caswell <matt at openssl.org>
>> Sent: Thursday, 10 March 2022 12:54
>> To: Tal Dery <tal at awrora.io>; openssl-users at openssl.org
>> Subject: Re: SSL_TLSEXT_ERR_NOACK not working as expected
>>
>>
>>
>> On 10/03/2022 10:26, Tal Dery wrote:
>>> Hi,
>>>
>>> I am implementing an SSL server.
>>>
>>> Using SSL_CTX_set_tlsext_servername_callback I'm checking the SNI.
>>>
>>> When SNI meets my requirements (for example does not contain 
>>> offensive
>>> words) I allow the handshake by returning SSL_TLSEXT_ERR_OK.
>>>
>>> When there is an offensive word, I do not want to send Server Hello 
>>> message. I try to do this by returning SSL_TLSEXT_ERR_NOACK.
>>
>> If you don't want the ServerHello to be sent then you are aborting the handshake. In that case you should return SSL_TLSEXT_ERR_ALERT_FATAL. By comparison SSL_TLSEXT_ERR_NOACK is a non-fatal return code. The SNI request is not acknowledged by the server (i.e. it acts the same way as if SNI was not configured on the server at all), but no alerts are sent so the handshake proceeds as normal.
>>
>> Matt
>>
>>
>>
>>>
>>> For some reason, the server is still sending the message, and I wonder why?
>>>
>>> SSL_TLSEXT_ERR_ALERT_WARNING works as expected.
>>>
>>> I'm using OpenSSL 1.1.1f and Wireshark to verify what I say.
>>>
>>> Thanks
>>>
>>
>>
>>
> 
> 
>