Need additional control over async stack allocation

[Arran Cudbard-Bell]

In our application we use the OpenSSL ASYNC_* API to jump out of verification and session load/store callbacks.

On the POSIX side, when creating a new context OpenSSL calls the standard OPENSSL_malloc and
OPENSSL_free functions to allocate memory for the stack passed into makecontext.

https://github.com/openssl/openssl/blob/1c0eede9827b0962f1d752fa4ab5d436fa039da4/crypto/async/arch/async_posix.c

There are several issues with this approach:

1.  On some systems stack memory must be page aligned.  The user provided allocation functions
     don't know the context in which the memory is being requested so can't ensure this.
2.  glibc recommends the stack be allocated with mmap and set to executable to allow trampoline
     functions to work correctly (unsure if this is relevant).
     https://www.gnu.org/software/libc/manual/html_node/System-V-contexts.html
3. Using heap memory for the stack means there's no guard page.  Overflowing the stack appears
     to cause silent memory corruption on both macOS and Linux.

The stack size is also currently fixed and requires OpenSSL recompilation to change.

I'd like to make the stack allocation and free functions configurable.

Ideally the malloc function would have a signature similar to this:

	void *(*stack_alloc)(size_t *len);

and free would be

	void (*stack_free)(void *stack);

The idea being that the allocation function returns the memory it allocated for the stack, and the size of the stack via *len.

If user stack alloc/free functions are not configured then we'd fall back to the standard OPENSSL_malloc() and OPENSSL_free()
functions with a static stack size.

For our particular use case we're planning to have stack_alloc create a new thread.  We'll then return the stack allocated
for that thread, which we believe in most cases fixes the issues described above.

The free function will then signal/join the thread.

-Arran


Arran Cudbard-Bell <a.cudbardb at freeradius.org>
FreeRADIUS Development Team

FD31 3077 42EC 7FCD 32FE 5EE2 56CF 27F9 30A8 CAA2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20220222/1e0c423f/attachment.sig>