[openssl/openssl] d4a8a5: CMS_decrypt*(): fix misconceptions and mem leak

[David von Oheimb]

  Branch: refs/heads/openssl-3.1
  Home:   https://github.com/openssl/openssl
  Commit: d4a8a5307b56c289ec6d6d8cb2decd2e03d58688
      https://github.com/openssl/openssl/commit/d4a8a5307b56c289ec6d6d8cb2decd2e03d58688
  Author: Dr. David von Oheimb <David.von.Oheimb at siemens.com>
  Date:   2023-02-24 (Fri, 24 Feb 2023)

  Changed paths:
    M crypto/cms/cms_env.c
    M crypto/cms/cms_smime.c
    M doc/man3/CMS_EncryptedData_decrypt.pod
    M doc/man3/CMS_decrypt.pod

  Log Message:
  -----------
  CMS_decrypt*(): fix misconceptions and mem leak

Also document CMS_decrypt_set1_password() and fix CMS_EnvelopedData_create.pod.

Reviewed-by: Tomas Mraz <tomas at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb at siemens.com>
(Merged from https://github.com/openssl/openssl/pull/20209)

(cherry picked from commit 26521faae48c14597877e330911171105ab6c30f)


  Commit: 540e4388bc9d2f3ebcd15aea84b7568d152c3459
      https://github.com/openssl/openssl/commit/540e4388bc9d2f3ebcd15aea84b7568d152c3459
  Author: Dr. David von Oheimb <David.von.Oheimb at siemens.com>
  Date:   2023-02-24 (Fri, 24 Feb 2023)

  Changed paths:
    M crypto/cms/cms_smime.c

  Log Message:
  -----------
  CMS_decrypt_set1_password(): prevent mem leak on any previously set decryption key

Reviewed-by: Tomas Mraz <tomas at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb at siemens.com>
(Merged from https://github.com/openssl/openssl/pull/20209)

(cherry picked from commit c5febe9923cd5465fae5209a4f8e8be4aab43cac)


  Commit: b2df2651dfc3fc78bac403c1cafbca9df4001273
      https://github.com/openssl/openssl/commit/b2df2651dfc3fc78bac403c1cafbca9df4001273
  Author: Dr. David von Oheimb <David.von.Oheimb at siemens.com>
  Date:   2023-02-24 (Fri, 24 Feb 2023)

  Changed paths:
    M crypto/cms/cms_smime.c

  Log Message:
  -----------
  CMS_decrypt_set1_*(): remove misleading error queue entry when recipient mismatch was not the issue

Reviewed-by: Tomas Mraz <tomas at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb at siemens.com>
(Merged from https://github.com/openssl/openssl/pull/20209)

(cherry picked from commit 27d87391e63dcbbc576fe3b5f1e1c9615a1ac5ff)


  Commit: da30fd892e2b4df737ff21101c153aef297b3c50
      https://github.com/openssl/openssl/commit/da30fd892e2b4df737ff21101c153aef297b3c50
  Author: Dr. David von Oheimb <David.von.Oheimb at siemens.com>
  Date:   2023-02-24 (Fri, 24 Feb 2023)

  Changed paths:
    M crypto/cms/cms_smime.c

  Log Message:
  -----------
  CMS_decrypt_set1_*(): fix NULL deref on unsuitable content type

Fixes #19975
for CMS_decrypt_set1_pkey_and_peer() in the obvious way,
and a related potential crash in CMS_decrypt_set1_password().

The point is that the input might have an unexpected content type,
so a guard is needed at both places after `ec` is obtained.

Note that in CMS_decrypt_set1_pkey_and_peer() there was
no such ec != NULL guard for
```
    if (ris != NULL)
        debug = ec->debug;
```
maybe because it is implied here by ris != NULL.

Reviewed-by: Tomas Mraz <tomas at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
Reviewed-by: David von Oheimb <david.von.oheimb at siemens.com>
(Merged from https://github.com/openssl/openssl/pull/20209)

(cherry picked from commit ceb767bea442a3f9d8a40a26df1f760b7142e616)


Compare: https://github.com/openssl/openssl/compare/3c8cee206504...da30fd892e2b