Issue in DH Algorithm Keys Generation in OpenSSL 3.3.0

[Viktor Dukhovni]

On Fri, May 31, 2024 at 07:47:40AM +0000, Vishal Kevat via openssl-users wrote:

> I am using OpenSSL source version 3.3.0 and facing an issue in key
> generation part of Diffie Hellman (DH) Algorithm. Below are the APIs I
> am using for generating Public and Private Keys:
> 
> static unsigned char DH_PRIME_128[] = { /* 128 bit prime */
>       0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
>       0xc9, 0x0f, 0xda, 0xa2, 0x21, 0x68, 0xc2, 0x34
> };
> 
> static unsigned char dh_g[] = {
>       0x02,
> };

That number is clearly not prime, it is, for a start obviously divisible
by 4!  And of course "2" is then clearly not a generator of the
multiplicative group of residues that are coprime to it, indeed the
group is not cyclic:

    https://en.wikipedia.org/wiki/Multiplicative_group_of_integers_modulo_n#Cyclic_case

so it has no generator.

FWIW, the factors of your "prime" are:

    4, 31, 2347, 439409, 1327715723, 2004151850481839419

As can be confirmed via:

    $ echo '4 31 * 2347 * 439409 * 1327715723 * 2004151850481839419 * 16o p' | dc
    FFFFFFFFFFFFFFFFC90FDAA22168C234

Bottom line you should not expect this "prime" to yield a viable DH
group.

-- 
    Viktor.