TLS handshake fails ("SSL_accept:error in error") for server->server connection (smtp submit dovecot->postfix) if /etc/pki/tls/openssl.cnf "Options=" includes 'ServerPreference' ?

[Viktor Dukhovni]

On Fri, Sep 25, 2020 at 07:36:44AM -0700, PGNet Dev wrote:

> > But that is clearly not the case, because you're testing different server endpoints, with port
> > 60465 for the "working" case, and "465" for the non-working case.
> 
> that's simply not the case
> 
> as stated
> 
>  60465 is the dovecot submission port
>  465 it the postfix submission port

Well, I expected you to post a working and non-workin trace for the
*same* server endpoint, with the good and bad configuration.

Secondly, if dovecot is in fact configured to use implicit TLS when
forwarding to port 465, then sending a cleartext "QUIT" to that port is
rather unexpected behaviour.  That would be a dovecot bug.  But if,
absent the setting in question it actually performs the implicit TLS
handshake, despite that setting having no effect on client TLS, then
that's rather a mystery.

> the mua submits to dovecot at port 60465 dovecot resubmits to postfix
> at port 465

Where's the recording of the successful transmission to port 465 (and
not say 587).

> that same configuration is used in each/every test.

The cleartext "QUIT" sent by the client strongly suggests that's not the
case.  Miracles may happen, but otherwise the only explanation is that
the working connections also differed in additional ways beyond the
ChaCha preference.

> again, the ONLY thing that changed between the 'working' and 'failed'
> cases is the setting in openssl.cnf

How does that cause Dovecot to use cleartext on port 465, and not
send a TLS client HELO?  Or was that recording from a connection
made by hand with telnet or the like?

> I never directly submit to 465
> 
> > It seems likely that you don't have TLS wrapper mode on port 60465.
> 
> port 60465 is, and always has been, configured for implicit SSL -- not starttls usage.

Well, who or what sent QUIT to 465?  This time this carefully enough to
not make any mistakes, making sure that the recording has the correct
data.

There should be two separate recordings, both to port 465, one working
and one not.  Both made by Dovecot, before and after the openssl.cnf
change.  Make sure to record the second of two failure cases, just
in case Dovecot is doing some sort of connection caching, and sends
QUIT in the clear by mistake when dropping a cached connection.

-- 
    Viktor.