[External] : Re: DHE key exchange fails with the FIPS provider
Thomas Dwyer III
thomas.dwyer at oracle.com
Fri Aug 18 23:50:16 UTC 2023
Following up on this, I have legacy applications that are FIPS-enabled
with OpenSSL 1.0.2+FOM that I'm porting to OpenSSL 3.x and these
applications are failing because of the now-invalid DH parameters that
weren't invalid in 1.0.2. I understand that compliance based on whether
the parameters are validated using 186-4 vs 186-2 is a question for the
FIPS lab rather than the OpenSSL community, but if I wanted to revert
back to 186-2 validation in the FIPS provider and submit that to CMVP
what would be the best way to achieve this? I see this comment in
ffc_params_validate.c directly above the definition of
/* This may be used in FIPS mode to validate deprecated FIPS-186-2
int ossl_ffc_params_FIPS186_2_validate(OSSL_LIB_CTX *libctx,
const FFC_PARAMS *params,
int *res, BN_GENCB *cb)
but I can't find a codepath through the FIPS provider that gets me
there. All of the logic that checks FFC_PARAM_FLAG_VALIDATE_LEGACY is
removed when FIPS_MODULE is defined. Is the comment incorrect?
A heavy-handed approach would be to #undef FIPS_MODULE for the relevant
source files but that would also affect parameter generation, not just
validation. I tried just simply changing all calls to
ossl_ffc_params_FIPS186_4_validate() to instead call
ossl_ffc_params_FIPS186_2_validate() but I must be missing something
else because that didn't work.
By the way, I'm seeing surprising behavior on the command line. With
configuration files that load & activate the base & fips providers
(leaving the default provider inactive) and this command line:
openssl dhparam 2048 | openssl3 dhparam -noout -check -provider
default -propquery fips=yes
(where "openssl" is 1.1.1 and "openssl3" is 3.0.x) I see the following:
DH parameters appear to be ok.
Changing that to "-provider fips" changed the output to:
Error, invalid parameters generated
Why does the first command line succeed? Shouldn't -propquery have
forced dhparam to use the FIPS provider in both cases? Furthermore, when
omitting the -provider option and just specifying "-propquery fips=no" I
would have expected this to fail because nothing in my configuration
activates the default provider, but that's not what happens--it's using
the FIPS provider anyway and failing with "invalid parameters
generated". It appears that -propquery is being ignored here?
Other commands (e.g. dgst) don't appear to have this issue:
$ openssl3 dgst -md5 -provider default -propquery fips=no < /etc/motd
$ openssl3 dgst -md5 -provider default -propquery fips=yes < /etc/motd
dgst: Unknown option or message digest: md5
dgst: Use -help for summary.
default library context, Algorithm (md5 : 102), Properties (fips=yes)
On 8/14/23 23:47, Tomas Mraz wrote:
> FIPS requires that the DH parameters used in case of protocols that
> cannot transfer the q value (which TLS is) are known safe primes.
> Apparently the debian.com server does not use safe prime DH parameters
> with TLS-1.2. Nowadays it is better to just disable DHE ciphersuites
> with FIPS for maximum interoperability.
> Tomas Mraz, OpenSSL
> On Mon, 2023-08-14 at 18:37 -0700, Thomas Dwyer III wrote:
>> I'm having a problem connecting to particular machines via TLSv1.2
>> with the FIPS provider. The handshake fails with:
>> 1022FDB6:error:0A000066:SSL routines:(unknown function):bad dh
>> and I can't figure out what the problem is. The weird thing is the
>> connection always succeeds with the default provider, but with the
>> FIPS provider it works with some servers (e.g. oracle.com) and fails
>> with other servers (e.g. debian.com). I have been able to reproduce
>> the problem with the openssl command line using options that force
>> the same cipher & key exchange that is negotiated by my application
>> openssl s_client -4 -tls1_2 -sigalgs rsa_pkcs1_sha256 -cipher DHE-
>> RSA-AES128-GCM-SHA256 -trace -connect hostname:443
>> When using the FIPS provider and connecting to oracle.com this works
>> fine. The exact same command line fails with debian.com. Connections
>> to both machines work fine with the default provider. Both machines
>> use 4K RSA certificates.
>> What is causing OpenSSL FIPS to fail the DHE key exchange?
>> Incidentally, setting "security-checks = 0" in the configuration
>> file has no obvious effect on the problem.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the openssl-users