[openssl-dev] [openssl-team] Discussion: design issue: async and -lpthread

Tue Nov 24 01:32:32 UTC 2015

I wasn't saying there was anything wrong with mmap(), just that guard pages
only work if you can guarantee your overrun hits the guard page (and
doesn't just step over it). Large stack allocations increase the odds of
'stepping over' the guard pages. It's still better than not having guard
pages, but they aren't a hard guarantee that you won't have mysterious bugs
still.

You obviously realize that, but bn_prime() is the classic example of
allocating very large chunks of memory on the stack.

As for fibre's, I doubt it'll work in general, the issue there is simply
the range of OS's OpenSSL supports. If you wire it in you still have to run
with man+dog+world in the process, that's a hard ask. One of the good
points about OpenSSL up until now, it tends to not break those big messy
apps where a whole lot of independly developed code ends up in the same
process.

Peter

From:	Nico Williams <nico at cryptonector.com>
To:	openssl-dev at openssl.org
Date:	24/11/2015 10:42
Subject:	Re: [openssl-dev] [openssl-team] Discussion: design issue:
            async and -lpthread
Sent by:	"openssl-dev" <openssl-dev-bounces at openssl.org>

On Mon, Nov 23, 2015 at 09:53:15PM +1000, Peter Waltenberg wrote:
>
> "
> Please do.  It will make this much safer.  Also, you might want to run
> some experiments to find the best stack size on each platform.  The
> smaller the stack you can get away with, the better.
> "
>
> It does, but it also requires code changes in a few places.
probable_prime
> () in bn_prime.c being far and away the worst offender. We instrumented
our
> test code so we could find out what the stack usage was, for libcrypto
you
> can get it under 4k for 32 bit and under 8k for 64 bit code on x86 Linux.

Are you saying that using mmap() would be onerous?  Something else?

> FYI, nothing elegant there, just have your code allocate and fill a large
> stack array then add check points further down to see how far you've
eaten
> into it.

Sure.

> "
> > > A guard page
> > > would allow one to safely tune down fiber stack size to the whatever
> > > OpenSSL actually needs for a given use.
> "
>
> Unless someone allocates a stack array larger than the size of the guard
> page and scribbles over another threads stack. This is another reason to
> never use large arrays on the stack.

alloca() and VLAs aren't safe for allocating more bytes than fit in a
guard page.  One should not use alloca()/VLAs for anything larger than
that.

This is no reason not to have a guard page!

This is a reason to have coding standards that address alloca()/VLAs.

> "
> Is there something wrong with that that I should know?  I suppose the
> test could use threads to make real sure that it's getting thread-
> locals, in case the compiler is simply ignoring __thread.  Are there
> compilers that ignore __thread??
> "
>
> Only that it's a compile time choice and OpenSSL is currently 'thread
> neutral' at runtime, not at compile time ?.

OpenSSL is "thread-neutral" at run-time as to locks and thread IDs
because of the lock/threadid callbacks.  But here we're talking about a
new feature (fibers) that uses thread-locals, and here using pthread
thread locals (pthread_getspecific()) clearly means no longer being
"thread-neutral" -- if I understand your definition of that term
anyways.

It's perfectly fine to use __thread in compiled code regardless of what
threading library is used, provided -of course- that __thread was
supported to begin with and that the compiler isn't lying.

> Compile time is easy, making this work at runtime is hard and
occasionally
> is really valuable - i.e. way back in the dim distant path when Linux had
> multiple thread packages available.

If the compiler accepts __thread but allows it to break at run-time
depending on the available threading libraries, then the compiler is
broken and should now have allowed __thread to begin with.  I can't find
anything describing such brokenness.  If you assert such brokenness
exists then please post links or instructions for how to reproduce it.

BTW, https://en.wikipedia.org/wiki/Thread-local_storage#C_and_C.2B.2B
says that even Visual Studio supports thread-locals.  Though there's a
caveat that requires some care at configuration time:

  On Windows versions before Vista and Server 2008, __declspec(thread)
  works in DLLs only when those DLLs are bound to the executable, and
  will not work for those loaded with LoadLibrary() (a protection fault
  or data corruption may occur).[9]

There must, of course, be compilers that don't support thread locals
(pcc?).  Wouldn't it be fair to say that OpenSSL simply doesn't support
fibers on those compilers?  I think so.

Nico
--
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20151124/3be4c65b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20151124/3be4c65b/attachment.gif>