memory leak debug options

[Chetan, Sethi (Sasken; LEADER ; ADITJ/SWG)]

Hello,

Currently we are seeing below 3 patterns of memory leak issue with Openssl 1.1.1i and 1.1.1k version using address santizer option as below:
Direct leak of 208 byte(s) in 2 object(s) allocated from:
    #0 0x9ef57b in __interceptor_malloc (/usr/local/bin/autoai/custom_binary+0x9ef57b)
    #1 0xffffac627743 in BN_MONT_CTX_new (/usr/lib64/libcrypto.so.1.1+0xaf743)
    #2 0xffffac627baf in BN_MONT_CTX_set_locked (/usr/lib64/libcrypto.so.1.1+0xafbaf)
    #3 0xffffac6ec77b  (/usr/lib64/libcrypto.so.1.1+0x17477b) rsa_ossl_public_decrypt
    #4 0xffffac6f0577  (/usr/lib64/libcrypto.so.1.1+0x178577) RSA_verify_ASN1_OCTET_STRING
    #5 0xffffac6eef1b  (/usr/lib64/libcrypto.so.1.1+0x176f1b) pkey_rsa_verify
    #6 0xffffac6b44cf in EVP_DigestVerifyFinal (/usr/lib64/libcrypto.so.1.1+0x13c4cf)
    #7 0xffffac8d95c7  (/usr/lib64/libssl.so.1.1+0x515c7) tls_process_key_exchange ->EVP_DigestVerify->EVP_DigestVerifyFinal
    #8 0xffffac8d9b33  (/usr/lib64/libssl.so.1.1+0x51b33) ossl_statem_client_process_message
    #9 0xffffac8d3acb  (/usr/lib64/libssl.so.1.1+0x4bacb) state_machine -> read_state_machine -> *process_change->ossl_statem_client_process_message
    #10 0xffffac8c01fb in SSL_do_handshake (/usr/lib64/libssl.so.1.1+0x381fb) via SSL_connect
    #11 0xffffac858f3b  (/usr/lib64/libcurl.so.4+0x4ff3b) ossl_connect_step2
    #12 0xffffac85afeb  (/usr/lib64/libcurl.so.4+0x51feb) ossl_connect_common
    #13 0xffffac85c1ef  (/usr/lib64/libcurl.so.4+0x531ef) Curl_ssl_connect_nonblocking
    #14 0xffffac82c96f  (/usr/lib64/libcurl.so.4+0x2396f) https_connecting

Indirect leak of 2064 byte(s) in 2 object(s) allocated from:
    #0 0x9ef57b in __interceptor_malloc (/usr/local/bin/autoai/custom_binary+0x9ef57b)
    #1 0xffffac6bf173 in CRYPTO_zalloc (/usr/lib64/libcrypto.so.1.1+0x147173)
    #2 0xffffac625fd7  (/usr/lib64/libcrypto.so.1.1+0xadfd7)bn_expand2
    #3 0xffffac6262af in BN_set_bit (/usr/lib64/libcrypto.so.1.1+0xae2af)
    #4 0xffffac627a0b in BN_MONT_CTX_set (/usr/lib64/libcrypto.so.1.1+0xafa0b)
    #5 0xffffac627bc3 in BN_MONT_CTX_set_locked (/usr/lib64/libcrypto.so.1.1+0xafbc3)
<Curl backtrace is same as above>

Indirect leak of 1536 byte(s) in 5 object(s) allocated from:
    #0 0x9ef57b in __interceptor_malloc (/usr/local/bin/autoai/custom_binary+0x9ef57b)
    #1 0xffffac6bf173 in CRYPTO_zalloc (/usr/lib64/libcrypto.so.1.1+0x147173)
    #2 0xffffac625fd7  (/usr/lib64/libcrypto.so.1.1+0xadfd7) bn_expand2
    #3 0xffffac626143 in BN_copy (/usr/lib64/libcrypto.so.1.1+0xae143)
    #4 0xffffac6278a3 in BN_MONT_CTX_set (/usr/lib64/libcrypto.so.1.1+0xaf8a3)
    #5 0xffffac627bc3 in BN_MONT_CTX_set_locked (/usr/lib64/libcrypto.so.1.1+0xafbc3)
<Curl backtrace is same as above>

After statical analysis we noted that the crypto_malloc and crypto_free functions are further called.
In these functions we could see debug flags OPENSSL_NO_CRYPTO_MDEBUG and OPENSSL_NO_CRYPTO_MDEBUG_BACKTRACE which might provide useful information in debugging this leak issue.

In this regard we have below queries:-
1. Could you please let us know how to enable/check the logs generated by above debug flags ?
2. We tried one experiment wherein we have disabled flags RSA_FLAG_CACHE_PUBLIC, RSA_FLAG_CACHE_PRIVATE from rsa_ossl_init() function and we can see that above leak does not happen.
   In this regards, are there any known issue/fixes available which we could try.
2. Any other suggesstion for debugging this memory leak from openssl point of view, would be welcome.

Regards,
Chetan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20210428/6dd4a2a1/attachment.html>