SSL_read() fails when using manually managed buffers

[Christian Schmidt]

Hi Christian,

On 04/01/2023 16:55, Christian Mazakas wrote:
> Hey everyone,
> 
> I'm trying to get libssl to work without using any built-in I/O so for this,
> I'm using manually managed BIO pairs.
> 
> It seems like I can get a TLS Client Hello written to a user-provided buffer
> but I'm having trouble getting the server to parse this data and generate
> its own Server Hello in response.
> 
> I have a Compiler Explorer link that demonstrates my issue here:
> https://godbolt.org/z/ncxezWfqG <https://godbolt.org/z/ncxezWfqG>
> 
> Is there something plainly obvious that I'm missing? I've been scouring the
> docs so I'm wondering if there's something easy I missed.

Since I am doing something similar (I am wrapping all of my sockets, 
some of which may be encrypted using openssl, others not, through one 
select() statement and thus can't hand off the sockets to openssl 
directly), you'll want something like

...
             result = SSL_read_ex (session->cryptoparams->ssl,
                                   (void *) buffer_data_start 
(&session->ibuf),
                                   buffer_data_space (&session->ibuf),
                                   &rsize);
             if (result) {
                 log_msg (V_DEBUG3, "crypto: read %zu bytes", rsize);
                 // ...
             } else {
                 error = SSL_get_error (session->cryptoparams->ssl, result);
                 switch (error) {
                 // Not enough data
                 case SSL_ERROR_WANT_READ:
                 case SSL_ERROR_WANT_WRITE:
// I am handling writes from server to client outside this; it's part of 
a loop
                     return 0;
                 // Intended shutdown of TLS session
                 case SSL_ERROR_ZERO_RETURN:
                     log_msg (V_INFO, "crypto: TLS session closed");
                     SSL_shutdown (session->cryptoparams->ssl);
                     return 1;
                 // read error: shutdown
                 default:
                     log_msg (V_ERROR, "crypto: failed to read: %s",
                              SSL_reason_error_string (error));
                     SSL_shutdown (session->cryptoparams->ssl);
                     return -1;
                 }
             }
...

I am manually handling the SSL_accept() for the server with state 
tracking though, with similar error checking & handling. I left some 
references to my internal data structures and functions in and hope they 
are self explaining.

If you remove the direct I/O handling from openssl, you need to handle 
the SSL handshake data exchange, too, and won't be able to only act on 
the decrypted data, as data needs to be exchanged between client and 
server with potentially extra round trips, which will result in the 
SSL_ERROR_WANT_* which are not terminal errors, but you'll need to act 
on them. The 16,3,1 is a good start though as it's a TLS 1.0 handshake, 
which SHOULD be used up to and including TLS 1.3.

Good luck,
Christian

> - Christian