[openssl-users] How to debug SSLV3_ALERT_BAD_RECORD_MAC

[Matt Caswell]

On 25/04/17 22:37, Craig_Weeks at trendmicro.com wrote:
> We have recently upgraded our product to 1.0.2k.  We are getting this
> error on a packet sent to us from our browser-based user interface.  I
> really need some suggestions as to how to debug this problem.  I know it
> is in our code rather than OpenSSL but I have no idea how to dig into
> what is happening.

Is this a reproducible problem? Normally bad_record_mac would only occur
if there was some implementation issue in the SSL/TLS stack itself or if
something is corrupting the records after they have been generated by
the stack.

I'd start by looking at the end-to-end pipe between the client SSL/TLS
stack and the server stack and validating that the records look sane and
unchanged at each step.

If that doesn't pin-point the problem then you may need to dig a little
deeper. bad_record_mac can cover a multitude of sins. You need to figure
out what specific sin you are committing. If it was me I would be
instrumenting the OpenSSL code in this area to see what it thinks it is
barfing on. You might want to start with the tls1_enc() function in
ssl/t1_enc.c. If its a non-AEAD ciphersuite then you may need to look at
tls1_mac() too (also in ssl/t1_enc.c). Possibly parts of
ssl3_get_record() in ssl/s3_pkt.c

Hope that helps,

Matt