[openssl-users] SSL_read() returns -1, and SSL_read_ex does not update readbytes where a record containing a session ticket is being read (TLS 1.3)

[Matt Caswell]

>> If no application data is available then you get the -1 (for SSL_read()) or
>> 0 (for SSL_read_ex()) return code. You also get that return code for other
>> types of issues, and you are supposed to call SSL_get_error() to interpret
>> it.
> 
> In this case SSL_get_error() returns 2 (SSL_ERROR_WANT_READ) with either
> SSL_MODE_AUTO_RETRY on or /off.
> 
>> Note that by default OpenSSL sets SSL_MODE_AUTO_RETRY in 1.1.1 (which it
>> didn't in 1.1.0). This should mean that it automatically tries again to
>> read application data without returning an error. Have you switched 
>> SSL_MODE_AUTO_RETRY off?
> 
> It was left at defaults for the initial tests.

If SSL_MODE_AUTO_RETRY is on, and you're getting SSL_ERROR_WANT_READ this means
there is no application data available from the underlying BIO yet. OpenSSL may
have processed intervening session tickets, but if so it has then immediately
tried to read application data and there wasn't any there.

> 
>>> It seems to be that SSL_read() should return a positive integer
>>> representing the number of bytes read from the BIO whilst processing the
>>> session tickets, and SSL_read_ex should update readbytes to the number of
>>> bytes read from the BIO whilst processing the session tickets, as is done
>>> with other handshake messages.
>> 
>> This is not correct. SSL_read()/SSL_read_ex() only ever provide the number
>> of application data bytes that have been read, irrespective of how many
>> bytes were read from the underlying BIO.
> 
> Would call BIO_pending() before/after the call to SSL_read() be the best way
> to determine the number of bytes consumed by SSL_read()?
> 
> We could use this to determine what SSL_ERROR_WANT_READ is indicating.  As it
> seems  SSL_ERROR_WANT_READ could indicate two conditions in this scenario:
> 
> 1) No pending bytes - Additional handshake messages were processed, there's
> an expectation of additional application_data, but no hint that more
> application_data will be forthcoming.

> 2) Pending bytes - There is an
> incomplete record that needs processing.  Additional data should be fed into
> the BIO.

Probably you want to use SSL_pending() and/or SSL_has_pending(). From the docs:

SSL_pending() returns the number of bytes which have been processed, buffered
and are available inside B<ssl> for immediate read.

...

SSL_has_pending() returns 1 if B<s> has buffered data (whether processed or
unprocessed) and 0 otherwise.

See the following for full details:

https://www.openssl.org/docs/man1.1.1/man3/SSL_pending.html

Matt