[openssl-dev] [openssl.org #4614] pthread_once and malloc failures

[Richard Levitte via RT]

On Tue Jul 19 15:26:58 2016, matt wrote:
>
>
> On 19/07/16 16:23, Richard Levitte via RT wrote:
> > On Mon Jul 11 16:20:29 2016, kurt at roeckx.be wrote:
> >> Hi,
> >>
> >> When trying to check what happens if we simulate malloc()
> >> returning NULL I'm running into a problem that I'm not sure how to
> >> deal with.
> >>
> >> We have CRYPTO_THREAD_run_once(), which takes an init() function
> >> that returns void, so it can't return failures. At least the
> >> pthread_once() function also has it as void.
> >>
> >> But if those functions call malloc() and that returns NULL, we now
> >> don't catch that error, and later just try to use a NULL pointer.
> >>
> >> Anybody a good idea how to solve this?
> >
> > Rethinking this...
> >
> > Most of all, we use CRYPTO_THREAD_run_once() internally to initiate
> > the first
> > locks, so pretty much in an initial state of the library (not
> > entirely true,
> > since we do these inits opportunistically, but it's probable that
> > they happen
> > very early on). If they are having memory allocation, the running app
> > is
> > probably in deep shit anyway, so a hard assert in our diverse inits
> > would
> > probably be appropriate either way.
>
> You are assuming that the application loads and inits OpenSSL early
> and
> that it is critical to its function. It may not be.

True, I made that assumption.

Ok, so the other way I can think of is a bit ugly, but... using oob data, in
this case a variable that the init routine must set to signal that everything
went well. I'm thinking a variable in CRYPTO_THREAD_run_once() that gets passed
by reference to the init routine

To be noted is that we never check the value CRYPTO_THREAD_run_once()
returns... Should we make it __owur?

--
Richard Levitte
levitte at openssl.org

-- 
Ticket here: http://rt.openssl.org/Ticket/Display.html?id=4614
Please log in as guest with password guest if prompted