[openssl/openssl] 6a1eb6: DH_check(): Do not try checking q properties if it...

[Tomáš Mráz]

  Branch: refs/heads/openssl-3.1
  Home:   https://github.com/openssl/openssl
  Commit: 6a1eb62c29db6cb5eec707f9338aee00f44e26f5
      https://github.com/openssl/openssl/commit/6a1eb62c29db6cb5eec707f9338aee00f44e26f5
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-27 (Thu, 27 Jul 2023)

  Changed paths:
    M crypto/dh/dh_check.c

  Log Message:
  -----------
  DH_check(): Do not try checking q properties if it is obviously invalid

If  |q| >= |p| then the q value is obviously wrong as q
is supposed to be a prime divisor of p-1.

We check if p is overly large so this added test implies that
q is not large either when performing subsequent tests using that
q value.

Otherwise if it is too large these additional checks of the q value
such as the primality test can then trigger DoS by doing overly long
computations.

Fixes CVE-2023-3817

Reviewed-by: Matt Caswell <matt at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Tom Cosgrove <tom.cosgrove at arm.com>
Reviewed-by: Todd Short <todd.short at me.com>
(Merged from https://github.com/openssl/openssl/pull/21550)

(cherry picked from commit 1c16253f3c3a8d1e25918c3f404aae6a5b0893de)


  Commit: 1478ffad3f123550ec1014642d5c880dfbe270ef
      https://github.com/openssl/openssl/commit/1478ffad3f123550ec1014642d5c880dfbe270ef
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-27 (Thu, 27 Jul 2023)

  Changed paths:
    M test/dhtest.c

  Log Message:
  -----------
  dhtest.c: Add test of DH_check() with q = p + 1

This must fail with DH_CHECK_INVALID_Q_VALUE and
with DH_CHECK_Q_NOT_PRIME unset.

Reviewed-by: Matt Caswell <matt at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Tom Cosgrove <tom.cosgrove at arm.com>
Reviewed-by: Todd Short <todd.short at me.com>
(Merged from https://github.com/openssl/openssl/pull/21550)

(cherry picked from commit ad5d35572695d7b5748b2bd4fb1afaa189b29e28)


  Commit: fb54f415b9981adebb03997304ac77d4d0cc520a
      https://github.com/openssl/openssl/commit/fb54f415b9981adebb03997304ac77d4d0cc520a
  Author: Tomas Mraz <tomas at openssl.org>
  Date:   2023-07-27 (Thu, 27 Jul 2023)

  Changed paths:
    M CHANGES.md
    M NEWS.md

  Log Message:
  -----------
  Add CHANGES.md and NEWS.md entries for CVE-2023-3817

Reviewed-by: Matt Caswell <matt at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: Tom Cosgrove <tom.cosgrove at arm.com>
Reviewed-by: Todd Short <todd.short at me.com>
(Merged from https://github.com/openssl/openssl/pull/21550)

(cherry picked from commit 4b29762802c05fa871f0e1efcf804e86db0ddaa2)


Compare: https://github.com/openssl/openssl/compare/06bbee97aaa6...fb54f415b998