[openssl-users] Doubt regarding process of invalid [D]TLS record

[Raja ashok]

Hi All,

In dtls1_get_record(), we are calling ssl3_read_n to get 13 bytes of DTLS record header from socket and then based on the length in record header, we again call ssl3_read_n to get record payload from socket. Here we are handling invalid record, like length less 13 bytes or invalid version in record header or invalid epoch etc. If such invalid record comes then we are dropping that record and going to read again from socket, by calling ssl3_read_n again. If a peer is continuously sending invalid DTLS record, then our execution will be just running inside dtls1_get_record function. It wont come out of that function. So if a malicious peer wants to block our DTLS connection thread, then it can do by simply sending invalid DTLS record continuously. This behaviour would be same for both synchronous and asynchronous UDP sockets. This will simply block the application for long time, for the API SSL_connect, SSL_accept or SSL_read.

I feel here we should not block the application call for long time, we should have some mechanism to fail the connection and inform application if we get invalid DTLS record continuously. For example, if we receive around 100 invalid DTLS record continuously, then we should come out of this "goto" in dtls1_get_record and return failure to application. And also we can send alert to peer and close the DTLS connection.

For similar issue in ssl3_get_record(), we are having a max limit(MAX_EMPTY_RECORDS) for the received empty record. I feel similar limit should be there for invalid DTLS record in dtls1_get_record.

Similarly in ssl3_get_message and dtls1_get_message_fragment, if we receive Hello request message continuously, then we are just dropping and continuously going back to read on socket. This also may cause a long time block to application, for the API SSL_connect if malicious peer is continuously sending Hello request msg.

I feel blocking of application call for a long time should be avoided. Please check this and clarify me, if my understanding is wrong.

Note : I am referring openssl-1.0.2k and asking this doubt.

Regards,
Ashok

________________________________
[Company_logo]

Raja Ashok V K
Huawei Technologies
Bangalore, India
http://www.huawei.com
________________________________
本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁
止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中
的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！
This e-mail and its attachments contain confidential information from HUAWEI, which
is intended only for the person or entity whose address is listed above. Any use of the
information contained herein in any way (including, but not limited to, total or partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170207/060ca00c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 6737 bytes
Desc: image001.jpg
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170207/060ca00c/attachment-0001.jpg>