[openssl-dev] [openssl-team] Discussion: design issue: async and -lpthread

[Peter Waltenberg]

"
Please do.  It will make this much safer.  Also, you might want to run
some experiments to find the best stack size on each platform.  The
smaller the stack you can get away with, the better.
"

It does, but it also requires code changes in a few places. probable_prime
() in bn_prime.c being far and away the worst offender. We instrumented our
test code so we could find out what the stack usage was, for libcrypto you
can get it under 4k for 32 bit and under 8k for 64 bit code on x86 Linux.

FYI, nothing elegant there, just have your code allocate and fill a large
stack array then add check points further down to see how far you've eaten
into it.

"
> > A guard page
> > would allow one to safely tune down fiber stack size to the whatever
> > OpenSSL actually needs for a given use.
"

Unless someone allocates a stack array larger than the size of the guard
page and scribbles over another threads stack. This is another reason to
never use large arrays on the stack.

"
Is there something wrong with that that I should know?  I suppose the
test could use threads to make real sure that it's getting thread-
locals, in case the compiler is simply ignoring __thread.  Are there
compilers that ignore __thread??
"

Only that it's a compile time choice and OpenSSL is currently 'thread
neutral' at runtime, not at compile time ?.
Compile time is easy, making this work at runtime is hard and occasionally
is really valuable - i.e. way back in the dim distant path when Linux had
multiple thread packages available.

Peter






From:	Nico Williams <nico at cryptonector.com>
To:	openssl-dev at openssl.org
Date:	24/11/2015 06:49
Subject:	Re: [openssl-dev] [openssl-team] Discussion: design issue:
            async and -lpthread
Sent by:	"openssl-dev" <openssl-dev-bounces at openssl.org>



On Mon, Nov 23, 2015 at 08:34:29PM +0000, Matt Caswell wrote:
> On 23/11/15 17:49, Nico Williams wrote:
> > On a slightly related note, I asked and Viktor tells me that fiber
> > stacks are allocated with malloc().  I would prefer that they were
> > allocated with mmap(), because then you get a guard page.  A guard page
> > would allow one to safely tune down fiber stack size to the whatever
> > OpenSSL actually needs for a given use.
>
> Interesting. I'll take a look at that.

Please do.  It will make this much safer.  Also, you might want to run
some experiments to find the best stack size on each platform.  The
smaller the stack you can get away with, the better.

> > Still, if -lpthread avoidance were still desired, you'd have to find an
> > alternative to pthread_key_create(), pthread_getspecific(), and
friends.
>
> Just a point to note about this. The async code that introduced this has
> 3 different implementations:
>
> - posix
> - windows
> - null
>
> The detection code will check if you have a suitable posix or windows
> implementation and use that. Otherwise the fallback position is to use
> the null implementation. With "null" everything will compile and run but
> you won't be able to use any of the new async functionality.
>
> Only the posix implementation uses the pthread* functions (and only for
> thread local storage). Part of the requirement of the posix detection
> code is that you have "Configured" with "threads" enabled. This is the
> default. However it is possible to explicitly configure with
> "no-threads". This suppresses stuff like the "-DRENENTERANT" flag. It
> now will also force the use of the null implementation for async and
> hence will not use any of the pthread functions.

Ah, I see.  I think that's fine.  Maybe Viktor misunderstood this?

> One other option we could pursue is to use the "__thread" syntax for
> thread local variables and avoid the need for libpthread altogether. An
> earlier version of the code did this. I have not found a way to reliably
> detect at compile time the capability to do this and my understanding is
> that this is a lot less portable.

I use this in an autoconf project (I know, OpenSSL doesn't use autoconf):

  dnl Thread local storage
  have___thread=no
  AC_MSG_CHECKING(for thread-local storage)
  AC_LINK_IFELSE([AC_LANG_SOURCE([
  static __thread int x ;
  int main () { x = 123; return x; }
  ])], have___thread=yes)
  if test $have___thread = yes; then
     AC_DEFINE([HAVE___THREAD],1,[Define to 1 if the system supports
__thread])
  fi
  AC_MSG_RESULT($have___thread)

Is there something wrong with that that I should know?  I suppose the
test could use threads to make real sure that it's getting thread-
locals, in case the compiler is simply ignoring __thread.  Are there
compilers that ignore __thread??

Nico
--
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20151123/fd584764/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20151123/fd584764/attachment.gif>