[openssl-users] openssl client v1.1.0 can not connect: handshake failure:ssl/record/rec_layer_s3.c:1385:SSL alert number 40

[Matthias Apitz]

Hello,

We run a SSL client/server application, where the server is written in
Java using jdk1.8.0_31 and the client is written in C. We prepare the
update to OpenSSL 1.1.0 for the C client and are facing the problem, that
not even the OpenSSL s_client can now connect to our server. It says:

4146546432:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:1385:SSL alert number 40

More details below.



server:

written in Java, using jdk1.8.0_31


client:

$ openssl version
OpenSSL 1.0.2e-freebsd 3 Dec 2015

$ nohup openssl s_client -tls1_2 -connect 10.23.33.55:58076 
quit


$ cat nohup.out
depth=0 C = de, ST = Germany, L = Munich, O = unixarea.de, OU = gTech, CN = Matthias Apitz
verify error:num=18:self signed certificate
verify return:1
depth=0 C = de, ST = Germany, L = Munich, O = unixarea.de, OU = gTech, CN = Matthias Apitz
verify return:1
CONNECTED(00000004)
---
Certificate chain
 0 s:/C=de/ST=Germany/L=Munich/O=unixarea.de/OU=gTech/CN=Matthias Apitz
   i:/C=de/ST=Germany/L=Munich/O=unixarea.de/OU=gTech/CN=Matthias Apitz
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDPDCCAvmgAwIBAgIEcrMKAzALBgcqhkjOOAQDBQAwbzELMAkGA1UEBhMCZGUx
...
-----END CERTIFICATE-----
subject=/C=de/ST=Germany/L=Munich/O=unixarea.de/OU=gTech/CN=Matthias Apitz
issuer=/C=de/ST=Germany/L=Munich/O=unixarea.de/OU=gTech/CN=Matthias Apitz
---
No client certificate CA names sent
Peer signing digest: SHA1
Server Temp Key: DH, 1024 bits
---
SSL handshake has read 1427 bytes and written 507 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-DSS-AES128-GCM-SHA256
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : DHE-DSS-AES128-GCM-SHA256
    Session-ID: 58A69B4328BCDD246B3C2B1D7B600273AF8ACC16DE91EBB94980B1909D1D17C8
    Session-ID-ctx: 
    Master-Key: 78F0BA616EE9DBFF8BDF4A294DA70494979CBE9761185228A056C07DEC9EEBC8D126D14A27F1FDA55D4AA42DFB29E684
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1487313732
    Timeout   : 7200 (sec)
    Verify return code: 18 (self signed certificate)
---
OK - message received
closed




client:

$ /usr/local/sisis-pap/bin/openssl version
OpenSSL 1.1.0d  26 Jan 2017


$ /usr/local/sisis-pap/bin/openssl s_client -connect 10.23.33.55:58076
CONNECTED(00000003)
4146546432:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:1385:SSL alert number 40
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 176 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1487313886
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

-- 
Matthias Apitz, ✉ guru at unixarea.de, ⌂ http://www.unixarea.de/  ☎ +49-176-38902045