[openssl-users] 1.1.1a: crash in CRYPTO_THREAD_lock_free

[Viktor Dukhovni]

On Tue, Nov 27, 2018 at 04:11:06AM -0800, Claus Assmann wrote:

> I'm trying OpenSSL 1.1.1a on FreeBSD 11.2-RELEASE-p4 and got the following
> crash in one of my test programs (I compiled OpenSSL with -g after the
> first time this happened to get at least some debug info):
> 
> #0  __je_huge_salloc (tsdn=<value optimized out>, ptr=<value optimized out>) at extent.h:114
> #1  0x000000080122d01e in ifree (tsd=<value optimized out>) at arena.h:141468
> #2  0x000000080122d5b1 in __free (ptr=0x800000000) at tsd.h:716
> #3  0x0000000801535abb in _pthread_rwlock_destroy (rwlock=<value optimized out>)
>    4at /usr/src/lib/libthr/thread/thr_rwlock.c:127
> #4  0x0000000800e67c28 in CRYPTO_THREAD_lock_free (lock=0x801a27298) at crypto/threads_pthread.c:102
> #5  0x0000000800dcb760 in EVP_PKEY_free (x=0x801a7b370) at crypto/evp/p_lib.c:601
> #6  0x00000008008affce in ssl3_free (s=0x801bbd000) at ssl/s3_lib.c:3321
> #7  0x0000000800904c91 in tls1_free (s=0x801bbd000) at ssl/t1_lib.c:115
> #8  0x00000008008c085c in SSL_free (s=0x801bbd000) at ssl/ssl_lib.c:1204
> #9  0x00000000004133d3 in sm_tlsbio_close (fp=0x6612e0, flags=0) at ../../mta/libmta/tlsbio.c:391
> ...
> (gdb) p (pthread_rwlock_t)0x801a27298
> $2 = (struct pthread_rwlock *) 0x801a27298
> (gdb) p *$2
> $3 = {lock = {rw_state = 1, rw_flags = 0, rw_blocked_readers = 1, rw_blocked_writers = 0, rw_spare = 0x801a272a8}, 
>   owner = 27402512}

I also use FreeBSD 11.2-RELEASE-p4, and have a /usr/src tree:

    110 int
    111 _pthread_rwlock_destroy (pthread_rwlock_t *rwlock)
    112 {
    113         pthread_rwlock_t prwlock;
    114         int ret;
    115
    116         prwlock = *rwlock;
    117         if (prwlock == THR_RWLOCK_INITIALIZER)
    118                 ret = 0;
    119         else if (prwlock == THR_RWLOCK_DESTROYED)
    120                 ret = EINVAL;
    121         else if (prwlock == THR_PSHARED_PTR) {
    122                 *rwlock = THR_RWLOCK_DESTROYED;
    123                 __thr_pshared_destroy(rwlock);
    124                 ret = 0;
    125         } else {
    126                 *rwlock = THR_RWLOCK_DESTROYED;
    127                 free(prwlock);
    128                 ret = 0;
    129         }
    130         return (ret);
    131 }

The first thing to note is that the function tkaes a *pointer* to
a pthread_rwlock_t, but the thing pointed to is itself a pointer.
It has three "magic" values, and otherwise points to allocated
storage, freed on line 127 (matching your stack trace).

The magic values are:

    #define   THR_RWLOCK_INITIALIZER          ((struct pthread_rwlock *)NULL)
    #define   THR_RWLOCK_DESTROYED            ((struct pthread_rwlock *)1)
    #define   THR_PSHARED_PTR                                         \
	((void *)(uintptr_t)((1ULL << (NBBY * sizeof(long) - 1)) | 1))

But I think that your gdb commands to display the object are not
right.  We have

 lock = (pthread_rwlock_t *)0x801a27298
 *lock = prwlock = (struct pthread_rwlock_t *) 0x80000000

So you'd need to look at (**lock) not (*lock) to see the underlying
structure, but the address may be invalid.  I don't know how it
came to be 0x800000000 (2^35)...  I seems that something zeroed
the low 32 bits of the pointer.

More interesting that the lock might be the content of the EVP_PKEY
in frame #5:

    #5  0x0000000800dcb760 in EVP_PKEY_free (x=0x801a7b370) at crypto/evp/p_lib.c:601

    p ((struct evp_pkey_st *)0x801a7b370)[0]

Knowing what type of key this is might help to narrow the search.

-- 
	Viktor.