Memory leak issue with TLSv1.3 usage - OpensSSL-3.1.0

[Viktor Dukhovni]

On Wed, Aug 16, 2023 at 10:57:38PM +0530, Ishani wrote:

> After changing the TLS protocol to TLSv1.3, we are observing some
> memory leaks that indicates that the memory allocated for new session
> in new_session_cb() is not freed properly. However, I have cleaned up
> the memory as part of remove_session_cb() but still that memory leaks
> are visible.

There's little point in storing sessions in the client-side internal
cache.  Since the client-side internal cache is not automatically
consulted, it is best simply effectively disabled.  Management of your
external cache is your own code, independent of OpenSSL.

For a sensible implementation, see:

    https://github.com/vdukhovni/postfix/blob/v3.8.1/postfix/src/tls/tls_client.c#L836-L872

the key ingredient is:

	SSL_CTX_set_session_cache_mode(client_ctx,
				       SSL_SESS_CACHE_CLIENT |
				       SSL_SESS_CACHE_NO_INTERNAL_STORE |
				       SSL_SESS_CACHE_NO_AUTO_CLEAR);

Which avoids storing sessions in the *internal* cache, and avoids the
need to register remove callbacks that might attempt to "synchronise"
the internal and external caches.  Just manage your external cache
as you see fit, and remove any stale entries per your retention policy.

-- 
    Viktor.