[openssl-dev] [openssl.org #4572] SSL_set_bio and friends

[David Benjamin via RT]

I recently made some changes around BoringSSL's SSL_set_bio, etc. which you
all might be interested in. The BIO management has two weird behaviors
right now:

1. The existence of bbio is leaked in the public API when it should be an
implementation detail. (Otherwise you're stuck with it for DTLS where it's
really messy.)  SSL_get_wbio will return it, and SSL_set_bio messes up when
the bbio is active.

2. SSL_set_bio's object ownership story is a mess. It also doesn't quite
work. This crashes:
   SSL_set_fd(ssl, 1);
   SSL_set_rfd(ssl, 2);
But this does not:
   SSL_set_fd(ssl, 1);
   SSL_set_wfd(ssl, 2);
Not that anyone would do such a thing, but the asymmetry is off.

For 1, I made this change:
https://boringssl.googlesource.com/boringssl/+/2f87112b963fe9dee6a75b23a8dae45000001063%5E%21/
SSL_get_wbio now always returns the true wbio. BIO_set_bio is aware of bbio
and behaves accordingly.

Then for 2, I wrote this test:
https://boringssl.googlesource.com/boringssl/+/5c0fb889a1348ecaa5691f6139f9d60a610f2129%5E%21/
And then made this change:
https://boringssl.googlesource.com/boringssl/+/f715c423224a292d79ba0e3df373c828fbae29f7%5E%21/
[Plus
comment typo fix]
Releasing ssl->{rbio,wbio} is now much more straight-forward. All the
ownership quirks are left in SSL_set_bio. It's messy, but it's the best
option I found which preserves the existing calling patterns. The different
cases reflect the desired behavior inherently having a lot of cases.

For OpenSSL master, I believe it'd also work to add an s->rbio != s->wbio
check to SSL_set_rbio, but I think those are worse semantics for
SSL_set_{rbio,wbio}. They are new APIs, so, before it's too late, give them
clear semantics like "SSL_set_rbio takes ownership of its argument",
consistent with "set0" functions, rather than a mix of "set0" and "set1".

David

-- 
Ticket here: http://rt.openssl.org/Ticket/Display.html?id=4572
Please log in as guest with password guest if prompted