Program works with older libssl, but not with newer

[Viktor Dukhovni]

On Tue, Mar 31, 2020 at 04:51:32PM +0200, Christoph Pleger wrote:

> > > I have here a self-written server program and the corresponding
> > > self-written client program. These run well together with libssl 1.1.0l,
> > > but with libssl 1.1.1d, the same programs give errors SSL_ERROR_SYSCALL
> > > in SSL_read(), no matter if I recompile the programs and then run them,
> > > or just replace libssl with the newer version.
> > 
> > OpenSSL 1.1.1 supports TLS 1.3, which OpenSSL 1.1.0 did not.
> > 
> > > So, I want to ask if there are any known incompabilities in the libssl
> > > versions that require me to change the code of the programs, or if there
> > > is
> > > any known bug in libssl1.1.1d that may cause the mentioned errors.
> > 
> > Use of TLS 1.3 changes the communication patterns of the TLS protocol in
> > some non-trivial ways, and, if your application were fragile, it might
> > have gotten by with TLS 1.2, but the latent bugs could show up with TLS
> > 1.3.
> 
> Now, I replaced TLS_server_method() and TLS_client_method() with 
> TLSv1_2_server_method() and TLSv1_2_client_method() respectively, and the same 
> error occurs. 

Well, in that case, you need to provide more detail.  Does the handshake
complete?  If not, at what stage does it fail?

A PCAP file may be needed.  And you need to explain what operation
fails with SSL_ERROR_SYSCALL, and do an "strace" or equivalent to
understand what the relevant socket read calls returned.

-- 
    Viktor.