[openssl-users] Failed TLSv1.2 handshake

[Nounou Dadoun]

Hi Viktor, thanks very much for taking a look, my plan now is to experiment disabling ciphers until I figure out which one is causing some kind of mismatch in TLS_RSA_WITH_AES_256_GCM_SHA384

Unlikely to be the RSA so I'll poke at disabling the aes-gcm and sha384 respectively.


BTW, the client uses OpenSSL 1.0.1h, in boost it's configured with:
SslContextPtr_t pSslContext = std::make_shared<Net::SslContext>(pIoService, boost::asio::ssl::context::sslv23);

With no other options.
Client-side configure header is:

VERSION=1.0.1h
MAJOR=1
MINOR=0.1
SHLIB_VERSION_NUMBER=1.0.0
SHLIB_VERSION_HISTORY=
SHLIB_MAJOR=1
SHLIB_MINOR=0.0
SHLIB_EXT=
PLATFORM=VC-WIN64A
OPTIONS=enable-rsa enable-md5 enable-sha enable-sha1 enable-dsa enable-des enable-rc4 enable-dh no-bf no-camellia no-cast no-ec_nistp_64_gcc_128 no-gmp no-idea no-jpake no-krb5 no-md2 no-md4 no-mdc2 no-rc2 no-rc5 no-rfc3779 no-ripemd no-sctp no-shared no-store no-zlib no-zlib-dynamic
CONFIGURE_ARGS=no-idea no-rc5 enable-rsa no-md2 no-md4 enable-md5 enable-sha enable-sha1 enable-dsa no-ripemd enable-des enable-rc4 no-rc2 no-bf no-camellia no-cast no-mdc2 enable-dh VC-WIN64A
SHLIB_TARGET=

And as I mentioned earlier, the server uses OpenSSL 1.0.2d which makes for an unusual interop problem!

Thanks again ... N

Nou Dadoun
Senior Firmware Developer, Security Specialist

-----Original Message-----
From: openssl-users [mailto:openssl-users-bounces at openssl.org] On Behalf Of Viktor Dukhovni
Sent: Monday, December 07, 2015 5:49 PM
To: openssl-users at openssl.org
Subject: Re: [openssl-users] Failed TLSv1.2 handshake

On Mon, Dec 07, 2015 at 10:46:26PM +0000, Nounou Dadoun wrote:

> The cipher setting on the server is:
> SSL_CTX_set_cipher_list(pSslContext->GetNativeRef().impl(),  
> "ALL:SEED:!EXPORT:!LOW:!DES:!RC4");

Note, your cipher setting is likely not what you intend it to be, instead try:

	"DEFAULT:!EXPORT:!LOW:!RC4:+SEED"

Unless you know what you're doing in enabling anonymous ciphers.
Also note the difference between ":SEED" and ":+SEED".

You're also using a version 1 server certificate with a public exponent of "3".  This is a really bad idea.  You've not defined any DH or ECDH parameters, so the cipher selected uses RSA key transport, not a good idea, but should work barring bugs on either side.

> Any suggestions?  Is it possible that we've selected a cipher setting which is not compiled in?

No, that gives you plenty of ciphers (more than you need).  Perhaps the client is buggy.  Have you tried OpenSSL 1.0.2e?  What software is the client running?

In any case, there are enough red flags all over the place that make it likely that other mistakes are being made.

-- 
	Viktor.
_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users