[openssl-users] Apparent Memory Leak When Executing EVP_PKEY_derive (MS vc14, 32 bit, OpenSSL 1.0.2h/m)

Wed Nov 22 10:19:15 UTC 2017

Dear OpenSSL community,

in a C++ (vc14, OpenSSL 1.0.2h/m) project, I'm trying to negotiate a secret between two parties, using EC crypto. I've used
https://wiki.openssl.org/index.php/EVP_Key_Agreement
as a recipe and wrapped memory allocations into unique_ptrs. The code fulfills its purpose of deriving a shared secret, but memory leak detection with
	_CrtSetDbgFlag( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );
	_CrtSetReportMode( _CRT_WARN, _CRTDBG_MODE_FILE );
	_CrtSetReportFile( _CRT_WARN, _CRTDBG_FILE_STDOUT );
produces a warning:
	Detected memory leaks!
	Dumping objects ->
	{10316} normal block at 0x00307680, 12 bytes long.
	 Data: < v0         > F0 76 30 00 00 00 00 00 0D 00 00 00
	{10315} normal block at 0x00351068, 16 bytes long.
	 Data: <                > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
	{10314} normal block at 0x00350F68, 20 bytes long.
	 Data: <    h 5         > 00 00 00 00 68 10 35 00 00 00 00 00 04 00 00 00
	{10313} normal block at 0x003076F0, 12 bytes long.
	 Data: <    h 5     > 0D 00 00 00 68 0F 35 00 00 00 00 00
	{10312} normal block at 0x00310A70, 64 bytes long.
	 Data: <                > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
	{10311} normal block at 0x00309E98, 96 bytes long.
	 Data: <p 1  A    '     > 70 0A 31 00 90 41 20 0F A0 0D 27 0F 08 00 00 00
	Object dump complete.

Let me note two things:
- The line without which the memory leak disappears is the line triggering the actual secret derivation:
  if(EVP_PKEY_derive(ctx.get(), skey.get(), &skeylen) <= 0)
- The size of the leak is independent of how often I run the procedure of generating two sets of keys and then calling DeriveSecret().

***
Therefore, I'm wondering whether I've missed a call to free that I must trigger to deallocate this memory.
***

I have added an allocation hook to check, as an example, when the 96-byte allocation occurs:

	[Name]					[Language]
	libeay32.dll!default_malloc_ex(unsigned int num, const char * file, int line) Line 79	C
 	libeay32.dll!CRYPTO_malloc(int num, const char * file, int line) Line 342	C
 	libeay32.dll!lh_new(unsigned long(*)(const void *) h, int(*)(const void *, const void *) c) Line 120	C
 	libeay32.dll!ex_data_check() Line 279	C
 	libeay32.dll!def_get_class(int class_index) Line 316	C
 	libeay32.dll!int_new_ex_data(int class_index, void * obj, crypto_ex_data_st * ad) Line 421	C
 	libeay32.dll!CRYPTO_new_ex_data(int class_index, void * obj, crypto_ex_data_st * ad) Line 583	C
 	libeay32.dll!ECDH_DATA_new_method(engine_st * engine) Line 165	C
 	libeay32.dll!ecdh_data_new() Line 178	C
 	libeay32.dll!ecdh_check(ec_key_st * key) Line 215	C
 	libeay32.dll!ECDH_compute_key(void * out, unsigned int outlen, const ec_point_st * pub_key, ec_key_st * eckey, void *(*)(const void *, unsigned int, void *, unsigned int *) KDF) Line 77	C
 	libeay32.dll!pkey_ec_derive(evp_pkey_ctx_st * ctx, unsigned char * key, unsigned int * keylen) Line 237	C
 	libeay32.dll!pkey_ec_kdf_derive(evp_pkey_ctx_st * ctx, unsigned char * key, unsigned int * keylen) Line 252	C
 	libeay32.dll!EVP_PKEY_derive(evp_pkey_ctx_st * ctx, unsigned char * key, unsigned int * pkeylen) Line 345	C
 	[.]DeriveSecret() Line 802	C++

Please see below for how I call EVP_PKEY_derive. Can you tell me what I'm missing? Do you need more info?

Thank you for looking into this and helping!

Regards,
Holger

template<typename T> using deleted_unique_ptr = std::unique_ptr<T,std::function<void(T*)>>;

deleted_unique_ptr<EVP_PKEY> m_pOwnKey;
deleted_unique_ptr<EVP_PKEY> m_pPeerKey;

void [...]::DeriveSecret()
{
  if (!(m_pOwnKey && m_pOwnKey->pkey.ec))
  {
    throw KeyUninitializedError("Secret cannot be derived if own key is missing");
  }
  if (!m_pPeerKey)
  {
    throw KeyUninitializedError("Secret cannot be derived if peer key is missing");
  }

  deleted_unique_ptr<EVP_PKEY_CTX> ctx(EVP_PKEY_CTX_new(m_pOwnKey.get(), NULL), [](EVP_PKEY_CTX* obj)
  {
    EVP_PKEY_CTX_free(obj);
  });

  size_t skeylen;
  if(!ctx)
  {
    /* Error occurred */
    throw OpenSslError("Error creating EVP_PKEY_CTX");
  }
  if(EVP_PKEY_derive_init(ctx.get()) <= 0)
  {
    /* Error */
    throw OpenSslError("Error in EVP_PKEY_derive_init");
  }
  if(EVP_PKEY_derive_set_peer(ctx.get(), m_pPeerKey.get()) <= 0)
  {
    /* Error */
    throw OpenSslError("Error setting peer key in key negotiation");
  }
  /* Determine buffer length */
  if(EVP_PKEY_derive(ctx.get(), NULL, &skeylen) <= 0)
  {
    /* Error */
    throw OpenSslError("Error computing length of shared secret");
  }

  if (skeylen != secretSize)
  {
    throw OpenSslError("Shared secret has unexpected size");
  }

  deleted_unique_ptr<unsigned char> skey((unsigned char*)OPENSSL_malloc(skeylen), [](unsigned char* obj)
  {
    OPENSSL_free(obj);
  });

  /// @note The derivation of the secret seems to leak memory within OpenSSL
  if(EVP_PKEY_derive(ctx.get(), skey.get(), &skeylen) <= 0)
  {
    /* Error */
    throw OpenSslError("Error deriving shared secret");
  }

  [...]
}

Dr. Holger Seiberlich
Software Developer
Bruker Optik GmbH
Rudolf-Plank-Str. 27
76275 Ettlingen, Germany 

Phone: +49 7243 504-2626
Fax: +49 7243 504-42626
Holger.Seiberlich at bruker.com
www.bruker.com 

Sitz der Gesellschaft ist Ettlingen, HRB 36 2608 Amtsgericht Mannheim
Geschäftsführer: Urban Faeh, Marc Beisel, Juergen Srega

Diese E-Mail und alle Anlagen können Betriebs- oder Geschäftsgeheimnisse, oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail und aller Anlagen ausdrücklich untersagt. Bitte benachrichtigen Sie den Absender und löschen/vernichten Sie die empfangene E-Mail und alle Anlagen. Vielen Dank. 

This message and any attachments may contain trade secrets or privileged, undisclosed or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying or distribution of it and its attachments is strictly prohibited. Please inform the sender immediately and delete/destroy the original message and any copies. Thank you. 