[openssl-dev] why TLSv1 need two tls1_enc to get decrypted data while TLSv1.1/TLSv1.2 need one in OpenSSL1.1.0f?

[Ma chunhui]

Hi,

I met one problem when using OpenSSL1.1.0f with protocol TLSv1.
In brief, when using TLSv1,  after server side received encrypted data, and
after function tls1_enc finished, the decrypted data is not put in result
buffer, after another tls1_enc, the decrypted data is put in result buffer.
While TLSv1.1/TLSv1.2 needs only one tls1_enc.


The way to reproduce it is quite simple:

1.some preparation: openssl req -x509 -newkey rsa:2048 -keyout key.pem -out
cert.pem -days 365 -nodes
2.start server: openssl s_server -key key.pem -cert cert.pem -accept 44330
-www
    it's better to start server with gdb, and set breakpoints at tls1_enc,
then continue to run.
3.openssl s_client -connect localhost:44330 -tls1 -debug

After the client is started,  the server side will stop at breakpoint, do
several "c" to make it continue to run to wait for client's messages
Then at client side, type a simple "hello" message and press Enter. Then
server side will stop at tls1_enc, the input data is same as encrypted data
from client side, but after Evp_Cipher and some pad removing, the decrypted
data length is 0. After another tls1_enc, the decrypted data "hello" is put
in the result buffer.

But if client use -tls11 or -tls12, the decrypted "hello" is put in the
result buffer after the first tls1_enc.

Could anyone explains why the behavior of decryption is different between
TLSv1 and TLSv1.1/TLSv1.2?

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20170927/01942c37/attachment.html>