[openssl-dev] [openssl.org #4678] Bug: the 'dhtest_rfc5114_2048_224_bad_y' in dhtest.c didn't fail in FIPS mode

[Ziyan Zhou via RT]

Hi,

The test case openssl-1.0.2h/test/dhtest.c failed when running in FIPS 
mode, because the BAD test vector 'dhtest_rfc5114_2048_224_bad_y' didn't 
fail.
I found this issue when I was trying to run regular OpenSSL test code in 
FIPS mode.

OpenSSL version: 1.0.2
OpenSSL fips version: 2.0.12
OS: CentOS release 6.7 (Final)


Before building the dhtest.c, I did some code changes.
[STEP 1]
Calling FIPS_mode_set(1); in dhtest.c

[STEP 2]
Modifying the 'prime_len' of DH_generate_parameters_ex (line 128) to 
1024 bits since the minimal bit for FIPS mode is 1024-bit.

[STEP 3]
# gcc -I /usr/local/ssl/include/ -L /usr/local/ssl/lib/ -lcrypto 
-Wl,-rpath=/usr/local/ssl/lib/ dhtest.c

[STEP 4]

# ./a.out
..+...............
...

RFC5114 parameter test 1 OK
RFC5114 parameter test 2 OK
RFC5114 parameter test 3 OK
Test failed RFC5114 set 4

The expected return value of DH_compute_key(Z1, bady, dhA); is -1, but I 
got 256.

Thanks,
Ziyan



-- 
Ticket here: http://rt.openssl.org/Ticket/Display.html?id=4678
Please log in as guest with password guest if prompted

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3707 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20160918/9227061c/attachment.bin>