[openssl-users] ssl_pending returns 0 despite having data to read

[Matt Caswell]

On 11/01/17 20:07, Nadia Lapkovskaya wrote:
> Hi,
> 
> We are using openssl-1.0.2j. Noticed, that for http protocol everything is working fine, but when we are using our own binary protocol ssl_pending returns 0 all the time. We are using blocking socket. Tried with SSL_CTX_set_read_ahead set and unset.
> 
> Out test server sends back any info received from the client.
> 
> Test code looks like this:
> bool write(const uint64_t* data, int count)
> {
>   int rc = SSL_write(_ssl, data, count * sizeof(uint64_t));
>   return rc > 0 ? true : false;
> }
> 
> bool read(uint64_t* data, int count)
> {
>   do {
>       int rc = SSL_read(_ssl, data, count * sizeof(uint64_t));
>       if (rc <= 0) {
>           int err = SSL_get_error(_ssl, rc);
>           std::string errs = ERR_error_string(err, nullptr);
>           return false;
>       }
>   } while (SSL_pending(_ssl));
>   return true;
> }
> 
> During first ssl_read we received eight bytes, and after that ssl_pending returns 0. If we continue reading despite having no pending data, ssl_read returns the rest of the data. 
> Could you please suggest what is wrong here.

There are three levels of buffered data that you need to consider:

- Data that is buffered at the network level
- Data that is buffered in OpenSSL but not yet processed (i.e. decrypted)
- Data that is buffered in OpenSSL that has been processed

SSL_pending() only tells you about the last type of data. TLS delivers
blocks of data in records and OpenSSL will decrypt an entire record in
one go. If your application only then reads some of that record then
SSL_pending() will tell you how many bytes of data it still has
available. If you always read an entire record in one go (i.e. if the
size of the buffer that you pass to SSL_read() is equal to or greater
than the amount of data in the record) then SSL_pending() will always
return 0.

Normally OpenSSL will only read one record at a time, so there isn't any
data of the second type. However if you set read_ahead then it will
attempt to read as much data as the network can give it, until the
internal buffer is filled. If that means it has read more than one
record (which could include partial records) then you will get data of
the second type. In 1.0.2 there is no way to get OpenSSL to tell you
whether it has any of that data buffered. In 1.1.0 you can find out
about this data using the new function SSL_has_pending():

https://www.openssl.org/docs/man1.1.0/ssl/SSL_pending.html

For data buffered at the network level you should query this yourself
using something like select() or poll().

Matt