[openssl-users] EVP_DecryptUpdate: why is this failing when out == in?

Paul Smith paul at mad-scientist.net
Wed Dec 19 06:56:26 UTC 2018

As I understand it, it's legal to provide the exact same input and
output buffer to EVP_EncryptUpdate and EVP_DecryptUpdate, but it's not
legal to provide pointers into different parts of the same buffer. 
That's a good check.

However, my implementation is getting triggered by this code in

    if (ctx->final_used) {
        /* see comment about PTRDIFF_T comparison above */
=>      if (((PTRDIFF_T)out == (PTRDIFF_T)in)
            || is_partially_overlapping(out, in, b)) {
            return 0;

Can someone explain why, only in this specific situation where we're
decrypting the final block, we require that OUT and IN not be the same
buffer?  Everywhere else we check is_partially_overlapping() only,
without equality.

I read the comment about PTRDIFF_T but I didn't come up with a reason
for the equality check.  This check was added back in 2016 in SHA
5fc77684f1 FWIW.

More information about the openssl-users mailing list