[openssl-users] compilation error with openssl-1.1.0 and DH_get0_key

[Matt Caswell]

On 21/02/18 16:20, Benjamin Kaduk via openssl-users wrote:
> On 02/21/2018 10:16 AM, Robert Watson wrote:
>> I'm trying to update a crypto library for crtmpserver to work with
>> openssl 1.1.0.  The software is no longer actively maintained and my
>> c++ skills are somewhat rudimentary but I keep getting a compilation
>> error for something that seems trivial.
>>
>> Here's the code snippet:
>> bool DHWrapper::CopyPublicKey(uint8_t *pDst, int32_t dstLength) {
>>     if (_pDH == NULL) {
>>         FATAL("DHWrapper not initialized");
>>         return false;
>>     }
>>     BIGNUM *_keyPublic,*_keyPrivate;
>>     _keyPublic = BN_new();
>>     _keyPrivate = BN_new();
>>     DH_get0_key( _pDH, *_keyPublic, *_keyPrivate );
> 
> Use '&' instead of '*'

Yes, this is the problem. In addition to that though you have a memory
leak. DH_get0_key() will overwrite the values pointed to by _keyPublic
and_keyPrivate. So don't initialise them first with the BN_new() calls.

Matt