[openssl-dev] tls_enc() and pad/aad failures (detected behavior change after 1.0.1p)

[Massimiliano Pala]

Hi all,

we are facing a weird problem that we started to notice recently and we 
could not really find the source of the issue. The issue presented 
itself when starting using OpenSSL v1.0.1p (upgrading from v1.0.1m) in 
TLS connections. We started noticing the following error:

     error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert ad record mac

This seems to be related to the SSL_AD_BAD_RECORD_MAC. Since this error 
happens randomly and it is very difficult to reproduce, I tried to see 
the changes in the tls1_enc() function (t1_enc.c) from the ssl, which is 
used in the SSL_ENC_METHOD for enc/dec the records. I noticed the 
following change:

    /* openssl v1.0.1m/ssl/t1_enc.c */:
    812: pad = EVP_CIPHER_CTX_ctrl(ds, EVP_CTRL_AEAD_TLS1_AAD, 13, buf);
    813: if (send) {

While in the updated version:

    /* openssl v1.0.1p/ssl/t1_enc.c */
    812: pad = EVP_CIPHER_CTX_ctrl(ds, EVP_CTRL_AEAD_TLS1_AAD,
    813:                               EVP_AEAD_TLS1_AAD_LEN, buf);
    814:        if (pad <= 0)
    815:            return -1;
    816:        if (send) {

This change seems pretty interesting, as it might be related to the 
errors we have started seeing after the upgrade. Now, my questions are 
(given this might be the source of the error):

  * Has anybody noticed the different behavior (the same code from
    1.0.1p is in the current 1.0.2e - plus some additional checks for
    DTLS) ?
  * If the pad <= 0 is detected (not checked before), what can cause the
    error ? Network errors should be detected at the TCP level, so... it
    might be an application-level error ? Corrupted buffers ? Shooting
    in the dark here...

Any ideas on how to fix the issue and/or to understand if this is an 
OpenSSL issue or an application layer issue that would be really 
appreciated. I am not sure how to proceed at this point.. help ?

Thanks,
Max

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20160114/28c1c18d/attachment-0001.html>