How to debug a TLSv1.3 protocol problem?

[Matt Caswell]

Are you able to capture a wireshark trace of the handshake?

Matt

On 18/05/2020 17:59, Claus Assmann wrote:
> I'm stuck and looking for some hints/help.  I have two MTAs (let's
> call them M1 and S8), both built with OpenSSL 1.1.1g. The problem
> is M1 cannot establish a TLSv1.3 connection with S8. Using other
> MTAs/sites/protocols/tools works just fine, e.g., M1 can send mail
> to google using TLSv1.3, and S8 can send mail to M1. Replacing the
> server or client with openssl s_client/s_server also works.
> 
> I've added some TLS callbacks to S8 which I found in s_cb.c, but
> all I get at the end is "SSL_accept:error in TLSv1.3 early data"
> (see "full" trace below for the steps leading to this).
> Unfortunately I cannot find a way to figure out more details or
> what kind of error that is.  Any hints how to determine (and fix?)
> the problem?
> 
> S8 server side:
> info_callback where=0x10, ret=1
> info_callback where=0x2001, ret=1
> SSL_accept:before SSL initialization
> ssl_msg_cb, writep=0, version=0, len=5, ct=256
> ssl_msg_cb, before SSL initialization
> info_callback where=0x2001, ret=1
> SSL_accept:before SSL initialization
> ssl_msg_cb, writep=0, version=772, len=512, ct=22
> ssl_msg_cb, SSLv3/TLS read client hello
> info_callback where=0x2001, ret=1
> SSL_accept:SSLv3/TLS read client hello
> ssl_msg_cb, writep=1, version=0, len=5, ct=256
> ssl_msg_cb, SSLv3/TLS write server hello
> ssl_msg_cb, writep=1, version=772, len=88, ct=22
> ssl_msg_cb, SSLv3/TLS write server hello
> info_callback where=0x2001, ret=1
> SSL_accept:SSLv3/TLS write server hello
> ssl_msg_cb, writep=1, version=0, len=5, ct=256
> ssl_msg_cb, SSLv3/TLS write change cipher spec
> ssl_msg_cb, writep=1, version=772, len=1, ct=20
> ssl_msg_cb, SSLv3/TLS write change cipher spec
> info_callback where=0x2001, ret=1
> SSL_accept:SSLv3/TLS write change cipher spec
> info_callback where=0x2001, ret=1
> SSL_accept:TLSv1.3 early data
> info_callback where=0x2002, ret=-1
> SSL_accept:error in TLSv1.3 early data
> 
> M1 client side:
> apps_ssl_info_cb, where=10, ret=1
> apps_ssl_info_cb, SSL_connect=before SSL initialization
> ssl_msg_cb, writep=1, version=0, len=5, ct=100
> ssl_msg_cb, SSLv3/TLS write client hello
> ssl_msg_cb, writep=1, version=772, len=512, ct=16
> ssl_msg_cb, SSLv3/TLS write client hello
> apps_ssl_info_cb, SSL_connect=SSLv3/TLS write client hello
> ssl_msg_cb, writep=0, version=0, len=5, ct=100
> ssl_msg_cb, SSLv3/TLS write client hello
> apps_ssl_info_cb, SSL_connect=SSLv3/TLS write client hello
> ssl_msg_cb, writep=0, version=772, len=88, ct=16
> ssl_msg_cb, SSLv3/TLS read server hello
> apps_ssl_info_cb, SSL_connect=SSLv3/TLS read server hello
> ssl_msg_cb, writep=1, version=0, len=5, ct=100
> ssl_msg_cb, SSLv3/TLS write change cipher spec
> ssl_msg_cb, writep=1, version=772, len=1, ct=14
> ssl_msg_cb, SSLv3/TLS write change cipher spec
> apps_ssl_info_cb, SSL_connect=SSLv3/TLS write change cipher spec
> ssl_msg_cb, writep=1, version=0, len=5, ct=100
> ssl_msg_cb, SSLv3/TLS write client hello
> ssl_msg_cb, writep=1, version=772, len=512, ct=16
> ssl_msg_cb, SSLv3/TLS write client hello
> apps_ssl_info_cb, SSL_connect=SSLv3/TLS write client hello
> ssl_msg_cb, writep=0, version=0, len=5, ct=100
> ssl_msg_cb, SSLv3/TLS write client hello
> 
> and here it hangs until timeout.
>