[openssl/openssl] 9002fd: DH_check(): Do not try checking q properties if it...

[Tomáš Mráz]

  Branch: refs/heads/openssl-3.0
  Home:   https://github.com/openssl/openssl
  Commit: 9002fd07327a91f35ba6c1307e71fa6fd4409b7f
      https://github.com/openssl/openssl/commit/9002fd07327a91f35ba6c1307e71fa6fd4409b7f
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-27 (Thu, 27 Jul 2023)

  Changed paths:
    M crypto/dh/dh_check.c

  Log Message:
  -----------
  DH_check(): Do not try checking q properties if it is obviously invalid

If  |q| >= |p| then the q value is obviously wrong as q
is supposed to be a prime divisor of p-1.

We check if p is overly large so this added test implies that
q is not large either when performing subsequent tests using that
q value.

Otherwise if it is too large these additional checks of the q value
such as the primality test can then trigger DoS by doing overly long
computations.

Fixes CVE-2023-3817

Reviewed-by: Matt Caswell <matt at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Tom Cosgrove <tom.cosgrove at arm.com>
Reviewed-by: Todd Short <todd.short at me.com>
(Merged from https://github.com/openssl/openssl/pull/21550)

(cherry picked from commit 1c16253f3c3a8d1e25918c3f404aae6a5b0893de)
(cherry picked from commit 6a1eb62c29db6cb5eec707f9338aee00f44e26f5)


  Commit: 2255f6c74e6c8b702adcf352b04c5d3e6c759745
      https://github.com/openssl/openssl/commit/2255f6c74e6c8b702adcf352b04c5d3e6c759745
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-27 (Thu, 27 Jul 2023)

  Changed paths:
    M test/dhtest.c

  Log Message:
  -----------
  dhtest.c: Add test of DH_check() with q = p + 1

This must fail with DH_CHECK_INVALID_Q_VALUE and
with DH_CHECK_Q_NOT_PRIME unset.

Reviewed-by: Matt Caswell <matt at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Tom Cosgrove <tom.cosgrove at arm.com>
Reviewed-by: Todd Short <todd.short at me.com>
(Merged from https://github.com/openssl/openssl/pull/21550)

(cherry picked from commit ad5d35572695d7b5748b2bd4fb1afaa189b29e28)
(cherry picked from commit 1478ffad3f123550ec1014642d5c880dfbe270ef)


  Commit: f777f9ff6750929a219c4ec63fa865909abb840b
      https://github.com/openssl/openssl/commit/f777f9ff6750929a219c4ec63fa865909abb840b
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-27 (Thu, 27 Jul 2023)

  Changed paths:
    M CHANGES.md
    M NEWS.md

  Log Message:
  -----------
  Add CHANGES.md and NEWS.md entries for CVE-2023-3817

Reviewed-by: Matt Caswell <matt at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Tom Cosgrove <tom.cosgrove at arm.com>
Reviewed-by: Todd Short <todd.short at me.com>
(Merged from https://github.com/openssl/openssl/pull/21550)

(cherry picked from commit 4b29762802c05fa871f0e1efcf804e86db0ddaa2)
(cherry picked from commit fb54f415b9981adebb03997304ac77d4d0cc520a)


Compare: https://github.com/openssl/openssl/compare/e648db50d9a6...f777f9ff6750