[openssl-users] Ubuntu Xenial + Postgresql v9.5 == SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:

Thu Nov 9 13:30:19 UTC 2017

On 09 Nov 2017, at 2:57 PM, Michael Wojcik <Michael.Wojcik at microfocus.com> wrote:

>> This is openssl v1.0.1f (ubuntu xenial) talking to openssl v1.0.1f (ubuntu xenial), although
>> trying openssl as shipped by MacOS Sierra on the client side gives the same result.
> 
> At least prior to 1.1.0, to use ECC in OpenSSL the application has to make some additional calls. (I don't remember offhand how much of this goes away in the 1.1.0 API.) So it's quite possible for two applications using stock OpenSSL 1.0.x to fail to use an ECC suite.
> 
>> I set the ciphers explicitly on the server side to DEFAULT and got the same result (eliminating
>> whatever weird settings postgresql-on-ubuntu might have as a default).
> 
> DEFAULT includes ECC suites. You should try something like DEFAULT:!ECDHE:!ECDH to eliminate the ECC Kx suites.

I just tried that - no change in behaviour, apart from the negotiation of a different cipher before the connection fails (0x9f).

>> When openssl v1.0.2m tries to connect to postgresql running openssl v1.0.1f (ubuntu xenial), I get different behaviour:
>> ...
>> 2017-11-09 11:01:19 UTC [12025-1] [unknown]@[unknown] LOG:  invalid length of startup packet
> 
> Offhand, I don't know what the problem is here.

Does or did openssl server have any known bugs with respect to the length of a ClientHello packet being in excess of 255 bytes?

This is what a tcpdump looks like when psql linked to openssl v1.0.1f attempts to connect to postgresql linked to openssl v1.0.1f, the client side sends 8 bytes, then 1 byte, then 305 bytes in my case:

listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:51:45.113996 IP 172.29.231.43.40454 > 172.29.228.240.5432: Flags [S], seq 3580373978, win 26883, options [mss 8961,sackOK,TS val 12226525 ecr 0,nop,wscale 7], length 0
11:51:45.114035 IP 172.29.228.240.5432 > 172.29.231.43.40454: Flags [S.], seq 3310545722, ack 3580373979, win 26847, options [mss 8961,sackOK,TS val 12327573 ecr 12226525,nop,wscale 7], length 0
11:51:45.114243 IP 172.29.231.43.40454 > 172.29.228.240.5432: Flags [.], ack 1, win 211, options [nop,nop,TS val 12226525 ecr 12327573], length 0
11:51:45.114271 IP 172.29.231.43.40454 > 172.29.228.240.5432: Flags [P.], seq 1:9, ack 1, win 211, options [nop,nop,TS val 12226525 ecr 12327573], length 8
11:51:45.114277 IP 172.29.228.240.5432 > 172.29.231.43.40454: Flags [.], ack 9, win 210, options [nop,nop,TS val 12327573 ecr 12226525], length 0
11:51:45.114934 IP 172.29.228.240.5432 > 172.29.231.43.40454: Flags [P.], seq 1:2, ack 9, win 210, options [nop,nop,TS val 12327574 ecr 12226525], length 1
11:51:45.115132 IP 172.29.231.43.40454 > 172.29.228.240.5432: Flags [.], ack 2, win 211, options [nop,nop,TS val 12226525 ecr 12327574], length 0
11:51:45.117703 IP 172.29.231.43.40454 > 172.29.228.240.5432: Flags [P.], seq 9:314, ack 2, win 211, options [nop,nop,TS val 12226526 ecr 12327574], length 305
11:51:45.119459 IP 172.29.228.240.5432 > 172.29.231.43.40454: Flags [P.], seq 2:3941, ack 314, win 219, options [nop,nop,TS val 12327575 ecr 12226526], length 3939
11:51:45.120234 IP 172.29.231.43.40454 > 172.29.228.240.5432: Flags [P.], seq 314:321, ack 3941, win 350, options [nop,nop,TS val 12226526 ecr 12327575], length 7
11:51:45.120324 IP 172.29.231.43.40454 > 172.29.228.240.5432: Flags [R.], seq 321, ack 3941, win 350, options [nop,nop,TS val 12226526 ecr 12327575], length 0

The openssl v1.0.1f server side responds with a ServerHello, however the client side rejects the ServerHello saying "unknown ca", even though this same set of certificates works fine in Ubuntu Trusty.

In a second test, if I use openssl v1.0.2m compiled from source to connect to the same server, the client side sends 308 bytes in one go:

listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:53:02.032126 IP 172.29.228.7.54912 > 172.29.228.240.5432: Flags [S], seq 1471313029, win 26883, options [mss 8961,sackOK,TS val 645036074 ecr 0,nop,wscale 7], length 0
11:53:02.032165 IP 172.29.228.240.5432 > 172.29.228.7.54912: Flags [S.], seq 126514461, ack 1471313030, win 26847, options [mss 8961,sackOK,TS val 12346803 ecr 645036074,nop,wscale 7], length 0
11:53:02.032490 IP 172.29.228.7.54912 > 172.29.228.240.5432: Flags [.], ack 1, win 211, options [nop,nop,TS val 645036074 ecr 12346803], length 0
11:53:02.039507 IP 172.29.228.7.54912 > 172.29.228.240.5432: Flags [P.], seq 1:309, ack 1, win 211, options [nop,nop,TS val 645036076 ecr 12346803], length 308
11:53:02.039521 IP 172.29.228.240.5432 > 172.29.228.7.54912: Flags [.], ack 309, win 219, options [nop,nop,TS val 12346805 ecr 645036076], length 0
11:53:02.040625 IP 172.29.228.240.5432 > 172.29.228.7.54912: Flags [F.], seq 1, ack 309, win 219, options [nop,nop,TS val 12346805 ecr 645036076], length 0
11:53:02.041682 IP 172.29.228.7.54912 > 172.29.228.240.5432: Flags [.], ack 2, win 211, options [nop,nop,TS val 645036077 ecr 12346805], length 0
11:53:02.049476 IP 172.29.228.7.54912 > 172.29.228.240.5432: Flags [F.], seq 309, ack 2, win 211, options [nop,nop,TS val 645036078 ecr 12346805], length 0
11:53:02.049492 IP 172.29.228.240.5432 > 172.29.228.7.54912: Flags [.], ack 310, win 219, options [nop,nop,TS val 12346807 ecr 645036078], length 0

In this case the postgresql linked to openssl v1.0.1f immediately slams the phone down after the initial ClientHello, leading to the "SSL handshake has read 0 bytes" in the openssl client side.

In my case I have an SNI header with a long DNS name in it, and a very long list of ciphers on the client side that postgresql does not appear to give me control over.

Is this is known problem, does it ring any bells?

Regards,
Graham
—

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3240 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20171109/98e56a09/attachment.bin>