Issue in DH Algorithm Keys Generation in OpenSSL 3.3.0

[Vishal Kevat]

Hi OpenSSL users,

I am using OpenSSL source version 3.3.0 and facing an issue in key generation part of Diffie Hellman (DH) Algorithm. Below are the APIs I am using for generating Public and Private Keys:

static unsigned char DH_PRIME_128[] = { /* 128 bit prime */
      0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
      0xc9, 0x0f, 0xda, 0xa2, 0x21, 0x68, 0xc2, 0x34
};

static unsigned char dh_g[] = {
      0x02,
};

dhContext->handle = DH_new(); // Allocate DH Context

BIGNUM *dhp_bn, *dhg_bn;

dhp_bn = BN_bin2bn(DH_PRIME_128, sizeof(DH_PRIME_128), NULL);
dhg_bn = BN_bin2bn(dh_g, sizeof(dh_g), NULL);
DH_set0_pqg(dhContext->handle, dhp_bn, NULL, dhg_bn)

int ret = DH_generate_key(dhContext->handle);  // Generate DH Pub and Priv keys

const BIGNUM* pub_key = DH_get0_pub_key(dhContext->handle);
const BIGNUM* pri_key = DH_get0_priv_key(dhContext->handle);


When I set the prime number DH_PRIME_128 as above, the API DH_generate_key() is failing to generate the Public and Private Keys and returning 0.
Using the above prime number Mocana Crypto Library is able to generate the DH Pub and Priv keys but OpenSSL is failing to do so.



When I use the below prime number instead:
static unsigned char DH_PRIME_128[] = { /* 128 bit prime */
      0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
      0x34, 0xC2, 0x68, 0x21, 0xA2, 0xDA, 0x0F, 0xC9
};

The DH_generate_key() API is successfully able to generate the keys.

Please let me know, what is the reason of this failing API when I use the first prime number for DH algorithm key generation or I am doing wrong anything here?

Regards,
Vishal Kevat




General
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20240531/9b1fdc07/attachment.htm>