[openssl-dev] [openssl-team] Discussion: design issue: async and -lpthread

[Nico Williams]

On Tue, Nov 24, 2015 at 11:32:32AM +1000, Peter Waltenberg wrote:
> I wasn't saying there was anything wrong with mmap(), just that guard pages
> only work if you can guarantee your overrun hits the guard page (and
> doesn't just step over it). Large stack allocations increase the odds of
> 'stepping over' the guard pages. It's still better than not having guard
> pages, but they aren't a hard guarantee that you won't have mysterious bugs
> still.
> 
> You obviously realize that, but bn_prime() is the classic example of
> allocating very large chunks of memory on the stack.

Sure.  Rich Salz claims this is all fixed in master, so guard pages
strike me as a plus, not a wash.

> As for fibre's, I doubt it'll work in general, the issue there is simply
> the range of OS's OpenSSL supports. If you wire it in you still have to run
> with man+dog+world in the process, that's a hard ask. One of the good
> points about OpenSSL up until now, it tends to not break those big messy
> apps where a whole lot of independly developed code ends up in the same
> process.

That's... a joke, right?

OpenSSL very much does "break those big messy apps ...".  I don't see
the fibers project making that worse, nor better -- it's neutral.

Let me give you some examples.

Using Heimdal's libgssapi from Java JGSS (with the JNI GSS wrapper)
blows up due to no one initializing OpenSSL's lock callbacks.  Heimdal,
of course, uses OpenSSL.  Who should be initializing the lock callbacks?
Not the JVM -- why should it know/assume that some library used via
dlopen() will want OpenSSL?

But it's not a library's place to initialize OpenSSL's lock callbacks
either!  Suppose a Java program were loading via dlopen() *two*
libraries that use OpenSSL, and suppose different threads are racing to
do this.

This happens, and it happens because OpenSSL is used in so many
*libraries*, not just *programs*.  OpenSSL is a prime case of how a
library meant for use by programs comes to be used by libraries.
Another example is PKCS#11.  That's no excuse.  Use by libraries must be
supported.

And historically OpenSSL has been very bad at keeping its ABI
backwards-compatible, so DLL Hell cases often involve OpenSSL.

The more layers: the higher the likelihood of breakage involving
OpenSSL.  All threaded pluggable-software-using software (name services
switch, PAM, JNI, ...) is vulnerable to these problems.

If the OpenSSL team finally decides to do something about sane locking
by default, then it will be a huge improvement.  If this thread provides
the impetus, so much the better.

Nico
--