[openssl/openssl] 91ddeb: DH_check(): Do not try checking q properties if it...

[Tomáš Mráz]

  Branch: refs/heads/OpenSSL_1_1_1-stable
  Home:   https://github.com/openssl/openssl
  Commit: 91ddeba0f2269b017dc06c46c993a788974b1aa5
      https://github.com/openssl/openssl/commit/91ddeba0f2269b017dc06c46c993a788974b1aa5
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-31 (Mon, 31 Jul 2023)

  Changed paths:
    M crypto/dh/dh_check.c

  Log Message:
  -----------
  DH_check(): Do not try checking q properties if it is obviously invalid

If  |q| >= |p| then the q value is obviously wrong as q
is supposed to be a prime divisor of p-1.

We check if p is overly large so this added test implies that
q is not large either when performing subsequent tests using that
q value.

Otherwise if it is too large these additional checks of the q value
such as the primality test can then trigger DoS by doing overly long
computations.

Fixes CVE-2023-3817

Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Matt Caswell <matt at openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21551)


  Commit: 34d0f5cb93680a5286d1eb59125631ec8fd6dc81
      https://github.com/openssl/openssl/commit/34d0f5cb93680a5286d1eb59125631ec8fd6dc81
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-31 (Mon, 31 Jul 2023)

  Changed paths:
    M test/dhtest.c

  Log Message:
  -----------
  dhtest.c: Add test of DH_check() with q = p + 1

This must fail with DH_CHECK_INVALID_Q_VALUE and
with DH_CHECK_Q_NOT_PRIME unset.

Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Matt Caswell <matt at openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21551)


  Commit: 2330fdfb84961a1e49ada4dc335868a0831122ce
      https://github.com/openssl/openssl/commit/2330fdfb84961a1e49ada4dc335868a0831122ce
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-31 (Mon, 31 Jul 2023)

  Changed paths:
    M CHANGES
    M NEWS

  Log Message:
  -----------
  Add CHANGES/NEWS for CVE-2023-3817

Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Matt Caswell <matt at openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21551)


Compare: https://github.com/openssl/openssl/compare/eec805ee7135...2330fdfb8496