[openssl-users] 1.1.0b fails to negotiate with an old OpenSSL client

Wed Oct 26 22:48:28 UTC 2016

On 26/10/16 21:06, Michael Kocum wrote:
> 1.1.0b fails to negotiate from an old program that uses OpenSSL.
> The same old program can connect to 1.0.2h without any problem.
> 
> Here is the debug log of the server. Maybe someone can point me in the right direction what the problem might be.
> 
> openssl s_server -debug -state -bugs -serverpref -tlsextdebug -accept 25 -cert selfsigned.pem
> Using default temp DH parameters
> ACCEPT
> SSL_accept:before SSL initialization
> read from 0x14fffe0 [0x1509b53] (5 bytes => 5 (0x5))
> 0000 - 80 c8 01 03 01                                    .....
> read from 0x14fffe0 [0x1509b58] (197 bytes => 197 (0xC5))
> 0000 - 00 9f 00 00 00 20 00 c0-14 00 c0 0a 00 00 39 00   ..... ........9.
> 0010 - 00 38 00 c0 0f 00 c0 05-00 00 35 00 00 88 00 00   .8........5.....
> 0020 - 87 00 00 84 00 c0 12 00-c0 08 00 00 16 00 00 13   ................
> 0030 - 00 c0 0d 00 c0 03 00 00-0a 07 00 c0 00 c0 13 00   ................
> 0040 - c0 09 00 00 33 00 00 32-00 c0 0e 00 c0 04 00 00   ....3..2........
> 0050 - 2f 00 00 9a 00 00 99 00-00 45 00 00 44 00 00 96   /........E..D...
> 0060 - 00 00 41 00 00 07 05 00-80 03 00 80 00 c0 11 00   ..A.............
> 0070 - c0 07 00 c0 0c 00 c0 02-00 00 05 00 00 04 01 00   ................
> 0080 - 80 00 00 15 00 00 12 00-00 09 06 00 40 00 00 14   ............ at ...
> 0090 - 00 00 11 00 00 08 00 00-06 04 00 80 00 00 03 02   ................
> 00a0 - 00 80 00 00 ff 30 16 85-97 e0 9f e3 aa b1 07 47   .....0.........G
> 00b0 - 99 a5 7c 38 20 cd 51 39-a1 14 2f 60 50 87 26 62   ..|8 .Q9../`P.&b
> 00c0 - 0e c8 73 53 86                                    ..sS.

The above indicates that the client has sent an SSLv2 Compatible
ClientHello, although has indicated support for TLSv1.0. OpenSSL 1.1.0
doesn't support SSLv2, but it *does* still support the (very old) SSLv2
Compat ClientHello format. Unfortunately an SSLv2 compat hello does
*not* have an extensions section, which is important for communicating
info such as supported curves etc when using EC based ciphersuites.

> SSL_accept:before SSL initialization
> SSL_accept:SSLv3/TLS read client hello
> SSL_accept:SSLv3/TLS write server hello
> SSL_accept:SSLv3/TLS write certificate
> SSL_accept:SSLv3/TLS write key exchange
> write to 0x14fffe0 [0x1512d58] (1281 bytes => 1281 (0x501))
> 0000 - 16 03 01 00 51 02 00 00-4d 03 01 2c 21 40 97 a5   ....Q...M..,!@..
> 0010 - 67 b2 a4 a7 63 cc f0 58-af 24 a4 ca 61 d8 fa bf   g...c..X.$..a...
> 0020 - a8 50 84 29 20 54 70 1e-f5 8e c2 20 bf ad ba d7   .P.) Tp.... ....
> 0030 - fa 23 5b 77 eb 0f 30 a2-49 61 f9 ca 9f 28 3f 14   .#[w..0.Ia...(?.
> 0040 - bb d7 cd cf 5c 1b 69 d8-6b db 0e f7 c0 14 00 00   ....\.i.k.......
> 0050 - 05 ff 01 00 01 00 

This is the ServerHello that the server is sending back to the client.
The "c0 14" near the end of line 0040 indicates that the server has
selected TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA as the ciphersuite.

>                          16 03-01 03 6e 0b 00 03 6a 00   ..........n...j.
> 0060 - 03 67 00 03 64 30 82 03-60 30 82 02 48 02 09 00   .g..d0..`0..H...
...snip...lots of uninteresting lines

This is Certificate message sent from server to client.

>                                   16 03 01 01 2a 0c 00   .8...........*..
> 03d0 - 01 26 03 00 1d 20 4f b4-34 86 a8 a0 0a 45 5b 80   .&... O.4....E[.
> 03e0 - b0 79 9e bf 4b 91 ed ae-2c b7 ee 64 ff 39 78 c8   .y..K...,..d.9x.
> 03f0 - a0 e7 37 e6 a5 13 01 00-1a 9f 48 8e 91 73 55 3e   ..7.......H..sU>
> 0400 - 86 16 04 7e a9 b2 49 16-d6 f6 b3 c2 17 d1 4e 11   ...~..I.......N.
> 0410 - c4 67 7c 81 e6 49 a2 04-d7 bc 42 04 8c 2a 0f da   .g|..I....B..*..
> 0420 - a0 75 7d 80 98 5b 1a 0f-e2 48 55 06 95 38 0d a6   .u}..[...HU..8..
> 0430 - 84 f0 42 37 6b ee ca e9-e5 7e 13 11 d7 f9 3e f4   ..B7k....~....>.
> 0440 - b2 ae f1 01 e6 56 ab 7b-46 3b bd 66 de aa ad d7   .....V.{F;.f....
> 0450 - 41 59 2b 80 2d 76 98 a0-82 c8 d1 00 05 e8 11 da   AY+.-v..........
> 0460 - c3 4b c5 15 23 c0 ba 66-8c 9b fc 80 33 c4 e8 f9   .K..#..f....3...
> 0470 - 1f c7 82 ba b1 58 0c 87-54 42 b4 ce ed 66 4e 4e   .....X..TB...fNN
> 0480 - 3e 51 d4 9f 5f 1e 20 18-b1 5e 6a b9 bb e7 6c b2   >Q.._. ..^j...l.
> 0490 - 2d 27 52 90 70 9f b0 97-cb 6d 23 0b 9d 1c e6 9d   -'R.p....m#.....
> 04a0 - 71 2a ab 9b a9 42 c9 16-ce a1 86 63 96 fe b2 b6   q*...B.....c....
> 04b0 - 49 69 5c 80 7b 9d 3d 40-a8 4a 70 51 0a a1 99 a8   Ii\.{.=@.JpQ....
> 04c0 - b8 72 52 39 6b 8c c6 91-13 36 fb d5 fe 7d 2b db   .rR9k....6...}+.
> 04d0 - 45 3d 73 d9 be de fd 40-19 ed ec 41 84 d5 17 a7   E=s.... at ...A....
> 04e0 - 6e 32 05 51 5b e6 56 44-40 2b e8 54 d9 36 cc bb   n2.Q[.VD at +.T.6..
> 04f0 - 77 17 cd f3 7c e7 00 60-

This is the ServerKeyExchange

The "00 1d" on line 03d0 tells you the curve that the server has
selected. That corresponds to Curve 25519!!!! This is a modern curve
which an old client will not understand. The server has selected it
because it didn't get an extension from the client saying what curves it
supports, so it just picked one.

This is very likely to be your problem. To test the theory, try adding
"-named_curve P-256" onto your s_server line. P-256 is a much more
widely supported curve.

Matt

>                                16 03 01 00 04 0e         w...|..`......
> 0501 - <SPACES/NULS>
> SSL_accept:SSLv3/TLS write server done
> read from 0x14fffe0 [0x1509b53] (5 bytes => 5 (0x5))
> 0000 - 15 03 01 00 02                                    .....
> read from 0x14fffe0 [0x1509b58] (2 bytes => 2 (0x2))
> 0000 - 02 50                                             .P
> SSL3 alert read:fatal:internal error
> SSL_accept:error in SSLv3/TLS write server done
> ERROR
> 0:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:ssl\record\rec_layer_s3.c:1382:SSL alert number 80
> shutting down SSL
> CONNECTION CLOSED
> 
> --
> Michael Kocum [DataEnter]
> michael at dataenter.co.at
> 
> 