TLS handshake fails ("SSL_accept:error in error") for server->server connection (smtp submit dovecot->postfix) if /etc/pki/tls/openssl.cnf "Options=" includes 'ServerPreference' ?

[Viktor Dukhovni]

On Thu, Sep 24, 2020 at 07:43:04AM -0700, PGNet Dev wrote:

> > I'd be tempted to drop most if not all of those settings, they're not email-friendly.
> 
> PUBLIC email non-friendly, because of still-frequent old cipher/protocol implementations?
> 
> or,
> 
> inherently problematic with TLS in/onr SMTP?

Mostly the former.

> > Also, Postfix has no knowledge of TLS 1.3 cipher suites, Postfix has
> > only cipher configuration knobs only for the TLS <= 1.2 ciphers, so
> > I don't know how that particular string ended up in your logs. 
> 
> a bit too postfix-y for this list, but ...
> 
> I'm then perhaps misreading
> 
> 	http://www.postfix.org/TLS_README.html
> 	http://www.postfix.org/FORWARD_SECRECY_README.html
> 
> In any case, the following should be with defaults.
> 
> > Is there something in your Postfix configuration that resembles that
> > particular blob?  If so, it should not be there...
> 
> yep. now removed ...

That's very likely to have been the cause of the problem.  That setting
was not valid as a TLS <= 1.2 cipherlist.

> simplifying
> 
> 	/etc/pki/tls/openssl.cnf
> 		openssl_conf = default_conf
> 		[default_conf]
> 		ssl_conf = ssl_sect
> 		[ssl_sect]
> 		system_default = system_default_sect
> 		[system_default_sect]
> 		Options = PrioritizeChaCha
> 
> send/receive is successful.
> 
> 	postfix logs
> 
>               Trusted TLS connection established from
>               internal.mx.example.com[10.0.1.17]: TLSv1.3 with
>               cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
>               key-exchange X25519 server-signature ECDSA (P-384)
>               server-digest SHA384 client-signature ECDSA (P-384)
>               client-digest SHA384

So client and server both support TLS 1.3, and use AES-256-GCM by
default.

> changing only
> 
> 	/etc/pki/tls/openssl.cnf
> -		Options = PrioritizeChaCha
> +		Options = ServerPreference,PrioritizeChaCha
> 
> @ re-test submit to dovecot FAILs,
> 
> 	postfix log
> 
> 		Sep 24 05:01:44 mx postfix/submit-from-dovecot-proxy/smtpd[11261]: connect from internal.mx.example.com[10.0.1.17]
> 		Sep 24 05:01:44 mx postfix/submit-from-dovecot-proxy/smtpd[11261]: SSL_accept error from internal.mx.example.com[10.0.1.17]: -1
> 		Sep 24 05:01:44 mx postfix/submit-from-dovecot-proxy/smtpd[11261]: warning: TLS library problem: error:1408F10B:SSL routines:ssl3_get_record:wrong version number:ssl/record/ssl3_record.c:331:
> 		Sep 24 05:01:44 mx postfix/submit-from-dovecot-proxy/smtpd[11261]: lost connection after CONNECT from internal.mx.example.com[10.0.1.17]
> 		Sep 24 05:01:44 mx postfix/submit-from-dovecot-proxy/smtpd[11261]: disconnect from internal.mx.example.com[10.0.1.17] commands=0/0
> 
> again, the _only_ change between the two submissions is the addition of the "ServerPreference" option to the openssl.cnf config.

This looks like the protocol version is no longer TLS 1.3 as a result,
and one side or the other now expects or sent the wrong protocol
version.  For further progress a PCAP file is needed which contains a
full capture of exactly one TCP connection corresponding to this
failure.

You should check for any other non-default Postfix TLS settings that
may have been set to poorly chosen values.

> still not clear to me which piece(s) of that^ are having an issue with it. or why.

Ultimately, the TLS library (OpenSSL) is failing to interoperate between
client and server after this change.  But whether this is a bug in
OpenSSL, or a problem setting in the application is not yet clear.

> for this list, my initial question is -- *IS* it openssl's "fault"?
> or mine, or one of the other apps'?

You need to post A PCAP file that tshark can read with a single
TCP session containing the failed handshake.

What are the exact OpenSSL versons on the client and server?

Anything interesting in openssl.cnf on the client end?

-- 
    Viktor.