<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks for the detailed investigation. I don't know if we have a BIO callback or modified any BIO code. I'll have to dig into our version of Asterisk to see if I can tell.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The version confusion is mine. We really are running 1.0.1e 58 from Centos6. I got crossed up because when I checked the documentation it only goes back to 1.0.2, so that got into my head.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
If we have to rebuild the code, even just to debug the external cause (I expect there is something funny being fed to openSSL) is it going to be compatible for me to use the 1.0.2 source code or are there a bunch of related packages that I would need to upgrade
in parallel? </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks.</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Matt Caswell <matt@openssl.org><br>
<b>Sent:</b> September 25, 2019 9:49 AM<br>
<b>To:</b> Ian Sinclair <ian.sinclair@emetrotel.com>; openssl-users@openssl.org <openssl-users@openssl.org><br>
<b>Subject:</b> Re: Crash in OpenSSL v1.0.1 from dtls1_do_write OPENSSL_assert(len == (unsigned int)ret);</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText"><br>
<br>
On 24/09/2019 14:11, Ian Sinclair wrote:<br>
> I'm working with Asterisk PBX code which uses openSSL v1.0.2 (from Centos6). On<br>
> one site we're getting a crash from dtls1_do_write and as far as I can tell it's<br>
> from the assertion coded:<br>
> <br>
> /* bad if this assert fails, only part of the handshake<br>
> * message got sent. but why would this happen? */<br>
> OPENSSL_assert(len == (unsigned int)ret);<br>
> <br>
> My question is the same as some previous author - why would this happen?<br>
<br>
I've taken a look at the code in this area to try and figure out a reason.<br>
<br>
I note that the code before this looks like this:<br>
<br>
ret = dtls1_write_bytes(s, type, &s->init_buf->data[s->init_off],<br>
len);<br>
if (ret < 0) {<br>
....omitted...<br>
} else {<br>
<br>
/*<br>
* bad if this assert fails, only part of the handshake message<br>
* got sent. but why would this happen?<br>
*/<br>
OPENSSL_assert(len == (unsigned int)ret);<br>
<br>
Most significantly I notice that this doesn't seem to handle the ret == 0 case -<br>
which is normally used to indicate an error. So looking at dtls1_write_bytes we<br>
should check whether there are any cases where it could return 0. Well that<br>
function doesn't actually do very much except call do_dtls1_write() and return<br>
the result of that.<br>
<br>
There are a few possible points where that function could return a value:<br>
<br>
1) Returns the result of ssl3_write_pending()<br>
2) Returns the result of ssl_dispatch_alert(). Following the logic of that we<br>
find that it ends up returning the result of a recursive do_dtls1_write() call -<br>
so we can ignore this one.<br>
3) If len == 0 and !create_empty_fragment then we return 0. If that is the case<br>
then len == 0 and ret == 0 so the assertion would not be hit, so we can ignore this.<br>
4) if create_empty_fragment is true we return wr->length. But there are no calls<br>
of do_dtls1_write where create_empty_fragment is true anywhere in the code. This<br>
is actually dead code (we should remove this). We can ignore this.<br>
5) On error we return -1, which we can ignore.<br>
<br>
So it seems do_dtls1_write() will only return 0 if ssl3_write_pending() does.<br>
<br>
Looking at that function there are 3 possible points where it can return:<br>
<br>
1) On error it returns -1, so we can ignore that.<br>
2) In some circumstances it can return s->wpend_ret. wpend_return is set in<br>
do_dtls1_write to the value of len, so the only way it could be 0 is if len is,<br>
which would not cause the assertion to fail so we can ignore this.<br>
3) In other circumstances it returns the result of a BIO_write() call<br>
<br>
So the only way I can see for a 0 return to come back is if BIO_write() returns<br>
that. Looking at the implementation of BIO_write() it will return 0 if:<br>
<br>
- the BIO is NULL. But there is a check for this in ssl3_write_pending() so we<br>
know this will never be the case.<br>
- If a BIO callback has been set (via BIO_set_callback()) then it will return<br>
the result from that<br>
- Otherwise the return value is the result of a write call from the underlying<br>
BIO implementation.<br>
<br>
In DTLS the underlying BIO implementation is usually a BIO_s_datagram(). That<br>
write function will return the value from a system call to send or sendto. The<br>
man pages indicate this returns -1 on error, or otherwise the number of bytes<br>
sent - so I don't see how we could ever get 0 here.<br>
<br>
So, the summary of all of that is, I think we could get a situation where the<br>
assertion is triggered if:<br>
<br>
1) You are using BIO_set_callback() and the callback you set returns 0<br>
2) You are using some source/sink BIO other than BIO_s_datagram() and it<br>
sometimes returns 0 from its write call.<br>
3) You are using a filter BIO and that returns 0 during a write call<br>
<br>
Other possibilities that spring to mind are if you are using a custom BIO that<br>
doesn't behave well and returns a positive value from its write call that is not<br>
equal to the number of bytes it was asked to write.<br>
<br>
> <br>
> Is there any meaningful way I can debug this? Some flag I can set that will show<br>
> the DTLS packets to try to find a cause? Some way to get rid of the assertion so<br>
> that the failure doesn't take down the whole system, because currently the<br>
> assertion causes a reboot?<br>
<br>
In later releases I notice that this "hard" assertion has been converted to a<br>
soft assertion - i.e. it only crashes in a debug build. Otherwise it just<br>
returns -1 to indicate an error. So the equivalent of changing the code to look<br>
like this in a production build:<br>
<br>
/*<br>
* bad if this assert fails, only part of the handshake message<br>
* got sent. but why would this happen?<br>
*/<br>
if (len != (unsigned int)ret)<br>
return -1;<br>
<br>
<br>
> It's happening on an end customer site so building a<br>
> debug load isn't particularly viable, but if that's the only option tell me how.<br>
> <br>
> Is this a known problem that is only fixed as a non-security fix in a later<br>
> release?<br>
<br>
It's not a know problem as far as I can remember. But on the other hand there<br>
have been a lot of DTLS bug fixes that have gone in between 1.0.2 and 1.1.1.<br>
<br>
Note that 1.0.2 is approaching EOL (at the end of this year), so is only<br>
receiving security fixes. Since any fix we would apply doesn't sound like a<br>
security fix it wouldn't get backported to 1.0.2.<br>
<br>
> We are current for the release, I believe, with v1.0.1e 58.el6_10.<br>
<br>
I'm slightly confused here you're talking about 1.0.1e but above you said 1.02<br>
above.<br>
<br>
Matt<br>
</div>
</span></font></div>
</body>
</html>