ssl3_get_record:decryption failed on some machines

[fergtm at hyperion.io]

Sorry to bring this up again but I really don't know how to fix. I already
re-wrote my code to use SSL_read/SSL_write instead of a SSL filter BIO but I
still get the same error.

I can reproduce when the sender is nginx, socat openssl-listen or openssl
s_server. Both the server and client are running in the same machine.

The SSL object is not using a socket BIO instead I use a BIO pair.  I may be
using the BIO pair incorrectly but I haven't found any complete examples on
how to use them.

It works perfectly if I use a debug build of OpenSSL

Thanks

-----Original Message-----
From: openssl-users <openssl-users-bounces at openssl.org> On Behalf Of
Fernando Gutierrez Mendez
Sent: Monday, November 18, 2019 2:34 PM
To: openssl-users at openssl.org
Subject: Re: ssl3_get_record:decryption failed on some machines

The writer is my own code but I can also reproduce the problem when server
is nginx and client is my app.

In my code I do not use OpenSSL socket BIOs instead I do read/writes through
a BIO pair:

  pairBase = BIO_new(BIO_s_bio());
  pairInt  = BIO_new(BIO_s_bio());

  [...]

  BIO_make_bio_pair(pairBase, pairInt);

  [...]

  sslBIO = BIO_new_ssl(ssl_ctx, 1 /* Client */);

  [...]

  BIO_push(sslBIO, pairInt);

After each BIO_read/BIO_write to sslBIO I read/write any available data from
the network to pairBase.

I think I'm handling partial writes correctly:

  SSL_CTX_set_mode(ssl_ctx, SSL_MODE_AUTO_RETRY |
SSL_MODE_ENABLE_PARTIAL_WRITE | SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER);

  [..]

  ret = BIO_write(sslBIO, buf, (int)length);

  if (ret <= 0 && !BIO_should_retry(sslBIO))
  {
      /* Handle error */
      return;
  }

  if (ret > 0)
  {
      buf = ((uint8_t *)buf) + (size_t)ret;
      length -= (size_t)ret;
  }

but again the problem reproduces even if the writer is nginx.

Thanks

On Mon, Nov 18, 2019 at 02:19:30PM -0500, Viktor Dukhovni wrote:
> > On Nov 18, 2019, at 1:44 PM, Fernando Gutierrez Mendez
<fergtm at hyperion.io> wrote:
> > 
> > I use non-blocking IO with a SSL BIO so a call to BIO_read eventually
returns -1, when this happens I call BIO_should_retry to test if this is due
an error or because of the underlying non-blocking transport.
> 
> Is the writer side also non-blocking?  Is it your own code?
> 
> > This code works correctly but after transferring between 1Mb to 5Mb (it
varies every time) BIO_should_rety returns false and SSL_get_error returns
SSL_ERROR_SSL. The error is "139964546914112:error:1408F119:SSL
routines:ssl3_get_record:decryption failed or bad record
mac:../ssl/record/ssl3_record.c:677"
> 
> One way to get decryption integrity failure is for a non-blocking 
> writer to not handle partial writes correctly, if on an incomplete 
> write the writer resends the whole buffer, rather than only what it 
> failed to send last time, the TCP stream ends up stuttering 
> ciphertext, and the reader sees data integrity errors.
> 
> This can be seen by looking for unexpected runs of repeated ciphertext 
> in a PCAP capture of the data.
> 
> Whether the data sent to a particular reader ever ends up blocked at 
> the TCP layer for a given writer can depend on various network-layer 
> issues making some machines more prone to problems than others.
> 
> -- 
> 	Viktor.
>