Issue in DH Algorithm Keys Generation in OpenSSL 3.3.0

[Vishal Kevat]

Hi Viktor,

Is there any way to make this prime number work by doing some modifications in the openssl source code.

Like bypassing the OpenSSL DH prime check?

Regards,
Vishal


General
-----Original Message-----
From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of Viktor Dukhovni
Sent: Friday, May 31, 2024 03:01 PM
To: openssl-users at openssl.org
Subject: Re: Issue in DH Algorithm Keys Generation in OpenSSL 3.3.0

[External email: Use caution with links and attachments]

________________________________



On Fri, May 31, 2024 at 07:47:40AM +0000, Vishal Kevat via openssl-users wrote:

> I am using OpenSSL source version 3.3.0 and facing an issue in key
> generation part of Diffie Hellman (DH) Algorithm. Below are the APIs I
> am using for generating Public and Private Keys:
>
> static unsigned char DH_PRIME_128[] = { /* 128 bit prime */
>       0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
>       0xc9, 0x0f, 0xda, 0xa2, 0x21, 0x68, 0xc2, 0x34 };
>
> static unsigned char dh_g[] = {
>       0x02,
> };

That number is clearly not prime, it is, for a start obviously divisible by 4!  And of course "2" is then clearly not a generator of the multiplicative group of residues that are coprime to it, indeed the group is not cyclic:

    https://en.wikipedia.org/wiki/Multiplicative_group_of_integers_modulo_n#Cyclic_case

so it has no generator.

FWIW, the factors of your "prime" are:

    4, 31, 2347, 439409, 1327715723, 2004151850481839419

As can be confirmed via:

    $ echo '4 31 * 2347 * 439409 * 1327715723 * 2004151850481839419 * 16o p' | dc
    FFFFFFFFFFFFFFFFC90FDAA22168C234

Bottom line you should not expect this "prime" to yield a viable DH group.

--
    Viktor.