Regression in 1.1.1 against 1.1.0 in SSL_CTX_new

[Tomas Mraz]

On Thu, 2020-04-16 at 17:32 +0200, Harald Koch wrote:
> > Am 16.04.2020 um 17:07 schrieb Tomas Mraz <tmraz at redhat.com>:
> > 
> > On Thu, 2020-04-16 at 15:42 +0200, Harald Koch wrote:
> > > Hello list,
> > > 
> > > I have a TLS server which is started on demand in a multithreaded
> > > (pthread) application. The TLS server is one thread which is
> > > being
> > > started and stopped. At first start, the TLS server initialized
> > > with
> > > SSL_CTX_new with TLS_server_method works as expected, after
> > > cleaning
> > > up, eliminating the thread and starting it again at a later time
> > > in
> > > the same process, SSL_CTX_new returns NULL. I’ve been digging
> > > deeper
> > > into the initialization code, and found out that in
> > > crypto/threads_pthread.c, function CRYPTO_THREAD_set_local the
> > > call
> > > to pthread_setspecific returns a value != 0 (in my case: 22). The

This is EINVAL - meaning most probably that the pthread_setspecific()
is called on uninitialized key.

> > > error queue of openSSL stays empty. The same code works with
> > > openSSL
> > > 1.1.0 in all versions.
> > > Some posts googled state that before usage, be sure to run
> > > OPENSSL_init_ssl (which I do, even if not required to my analysis
> > > since it’s already called in one of the called functions deeper
> > > in
> > > the library).
> > > Am I missing something in a multithreaded environment?
> > 
> > Is this pure old 1.1.1 version or a current release from the 1.1.1
> > branch (i.e. 1.1.1f)?
> It’s 1.1.1f, also tested 1.1.1c. In 1.1.0t it works. I can test
> against other versions if you want to. As a speciality, I compile
> openSSL with gzip support („./config enable-zlib ...“, for support of
> compressed SMIME contents in other application).
> 
> > Do you call the OPENSSL_init_ssl from the main thread or from the
> > TLS
> > server thread?
> 
> I call it from the TLS server thread (created by pthread_create):
> 
> if (!OPENSSL_init_ssl(OPENSSL_INIT_LOAD_SSL_STRINGS, NULL))
> 	return;
> I tried to do it only once (instead of every started thread): no
> difference.

I do not see how this error could really happen unless
OPENSSL_cleanup() is called.

Could you try to set a breakpoint on that function and see if it is
somehow called inadvertently?

> In addition, I load random data via /dev/urandom (also tested only
> once or every time the server thread starts):
> 	RAND_load_file("/dev/urandom", 256);

That call should not be necessary.

-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
                                              Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]