need help with EVP_PKEY_derive function to generate shared key

[Srinivas, Saketh (c)]

Hi,
 i am trying to modify a function which earlier used openss1 to compute shared key the aruguments to the function are:

 rc_vchar_t *pub , rc_vchar_t *priv ;  '// public and private keys.

 if (eay_v2bn(&dh->pub_key, pub) < 0)
        goto end;
    if (eay_v2bn(&dh->priv_key, priv) < 0)
        goto end;



 DH_compute_key(v, dh_pub, dh))


For openssl3 we are trying to use (EVP_PKEY_derive) the below code is how i implemented:

 rc_vchar_t *pub,  rc_vchar_t *priv,  // parameters to the function

 BIGNUM *dh_pub_key = NULL;
 BIGNUM *dh_priv_key = NULL;

if (eay_v2bn(&dh_pub_key, pub) < 0)
         goto end;
if (eay_v2bn(&dh_priv_key, priv) < 0)
         goto end;

 pub_key_buf = (unsigned char*) malloc( BN_num_bytes(dh_pub_key));

  if (!pub_key_buf)
            goto end;
  BN_bn2nativepad(dh_pub_key, pub_key_buf, BN_num_bytes(dh_pub_key));

  priv_key_buf = (unsigned char*) malloc( BN_num_bytes(dh_priv_key));

   if (!priv_key_buf)
            goto end;
  BN_bn2nativepad(dh_priv_key, priv_key_buf, BN_num_bytes(dh_priv_key));

  pklen = strlen((char*)priv_key_buf);
  peerklen = strlen((char*)pub_key_buf);

  const unsigned char *pub_key = ( const unsigned char *)pub_key_buf;
  const unsigned char *priv_key = ( const unsigned char *)priv_key_buf;

  pkey = d2i_PrivateKey(EVP_PKEY_RSA, &pkey, &priv_key, pklen);

 peerkey = d2i_PublicKey(EVP_PKEY_RSA, &peerkey, &pub_key, peerklen);

ctx = EVP_PKEY_CTX_new(pkey, NULL);


The problem its failing at  Pkey and PeerKey creation.  Does anyone have any idea how to get it working. Also can anyone suggest how to know the correct replacemant for  EVP_PKEY_RSA, OR any way to know what is the type for my pub and priv keys


Thanks,
Saketh.


Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. and its Affiliates that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20220125/03d3be7c/attachment.htm>