<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hello everyone!</p>
<p>This is my first "post" to the mailing list.</p>
<p>I have been working on an OSSL wrapper for C# for several months,
and am currently implementing the DTLS portion. I've succeeded in
establishing a DTLS session with cookie exchange, sending and
receiving data, and gracefully closing the connection.</p>
<p>I've appended a snippet of the C# for those who may be curious.</p>
<p>Once I got the ugly bits working, I started chipping away at
building a nice API. This is where my problem started.</p>
<p>I've spend hours debugging and I think I've hit the limit of my
abilities. I have 2 unit tests in C# (at the end of this mail).
The first one (BasicDtlsTest) is a strightforward echo of one
client's data. The second (BasicListenerTest) is essentially the
same, only wrapped in a friendlier API.</p>
<p>Specifically, my problem is that the second unit test fails to
reply to a ClientHello <i>with</i> a cookie (it successfully send
the HelloVerifyRequest with the cookie). I have traced the problem
to where I <i>think</i> things go wrong. I followed the call
stack starting from SSL_do_handshake.</p>
<p>SSL_do_handshake -> ossl_statem_accept -> state_machine
-> read_state_machine -> dtls_get_message ->
dtls_get_reassembled_message -> dtls1_read_bytes ->
dtls1_get_record -> ssl3_read_n.</p>
<p>I stepped through both the <i>working</i> unit test and the <i>non-working</i>
one in order to find differences in the result. What I have found
is that, in ssl3_read_n, the call to BIO_read (line 300 in
rec_layer_s3.c) returns -1.</p>
<pre>ret = BIO_read(s->rbio, pkt + len + left, max - left);
</pre>
<p>At this line, pkt is a char[8], len and left = 0 and max = 16717</p>
<p>I'm curious as to why the "data" argument is not a pointer to a
buffer, but rather the result of an addition. Maybe my C isnt
strong enough...</p>
<p>Going even further down the stack, I finally end up at the
bottom: <br>
</p>
<pre style="background-color:#2b2b2b;color:#a9b7c6;font-family:'Bitstream Vera Sans Mono';font-size:9.0pt;"><font size="+1"><span style="color:#cc7832;">static int </span><span style="color:#ffc66d;">mem_read</span>(<span style="color:#b9bcd1;">BIO </span>*b<span style="color:#cc7832;">, char </span>*out<span style="color:#cc7832;">, int </span>outl)</font></pre>
<p>And this is where the -1 that is a thorn in my side originates:</p>
<pre style="background-color:#2b2b2b;color:#a9b7c6;font-family:'Bitstream Vera Sans Mono';font-size:9.0pt;"><font size="+1">ret = (outl >= <span style="color:#6897bb;">0 </span>&& (<span style="color:#b9bcd1;">size_t</span>)outl > bm-><span style="color:#9373a5;">length</span>) ? (<span style="color:#cc7832;">int</span>)bm-><span style="color:#9373a5;">length </span>: outl<span style="color:#cc7832;">;</span></font></pre>
<p>At this line, (size_t)outl is 16717 and bm->length is 0, so
ret = (int)bm->length. Then, a little further:</p>
<pre style="background-color:#2b2b2b;color:#a9b7c6;font-family:'Bitstream Vera Sans Mono';font-size:9.0pt;"><font size="+1"><span style="color:#cc7832;">if </span>((out != <span style="color:#908b25;">NULL</span>) && (ret > <span style="color:#6897bb;">0</span>)) {
memcpy(out<span style="color:#cc7832;">, </span>bm-><span style="color:#9373a5;">data</span><span style="color:#cc7832;">, </span>ret)<span style="color:#cc7832;">;
</span><span style="color:#cc7832;"> </span>bm-><span style="color:#9373a5;">length </span>-= ret<span style="color:#cc7832;">;
</span><span style="color:#cc7832;"> </span>bm-><span style="color:#9373a5;">max </span>-= ret<span style="color:#cc7832;">;
</span><span style="color:#cc7832;"> </span>bm-><span style="color:#9373a5;">data </span>+= ret<span style="color:#cc7832;">;
</span>} <span style="color:#cc7832;">else if </span>(bm-><span style="color:#9373a5;">length </span>== <span style="color:#6897bb;">0</span>) {
ret = b-><span style="color:#9373a5;">num</span><span style="color:#cc7832;">;
</span><span style="color:#cc7832;"> if </span>(ret != <span style="color:#6897bb;">0</span>)
<span style="color:#908b25;">BIO_set_retry_read</span>(b)<span style="color:#cc7832;">;
</span>}
<span style="color:#cc7832;">return </span>ret<span style="color:#cc7832;">;
</span></font><font size="+1"><span style="color:#cc7832;"></span></font>
</pre>
<p><font size="+1">At this point, ret = -1 and that just gets
propagated all the way back up, causing the entire record to be
thrown out and ignored. The client keeps sending ClientHellos
with the cookie and the server keeps ignoring them.</font></p>
<p><font size="+1">I really have no idea what is happening.</font></p>
<p><font size="+1">You can see my entire source code
here:<a class="moz-txt-link-freetext" href="https://gitlab.com/matthew.goulart/openssl.net">https://gitlab.com/matthew.goulart/openssl.net</a></font></p>
<p><font size="+1">I realize it's a lot to ask you to read all of my
code, so if anyone has suggestions as to what I might try next,
I'm open!</font></p>
<p><font size="+1"><br>
</font></p>
<p><font size="+1">Thanks!<br>
</font></p>
<pre style="background-color:#2b2b2b;color:#a9b7c6;font-family:'Bitstream Vera Sans Mono';font-size:9.0pt;"><font size="+1"><span style="color:#cc7832;"></span></font></pre>
<pre style="background-color:#1e1e1e;color:#dcdcdc;font-family:'Bitstream Vera Sans Mono';font-size:9.0pt;">[<span style="color:#4ec9b0;">TestMethod</span>]
<span style="color:#569cd6;">public async </span><span style="color:#4ec9b0;">Task </span>BasicDtlsTest()
{
<span style="color:#569cd6;">var </span>socket = <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">Socket</span>(<span style="color:#b8d7a3;">AddressFamily</span>.InterNetwork, <span style="color:#b8d7a3;">SocketType</span>.Dgram, <span style="color:#b8d7a3;">ProtocolType</span>.Udp);
socket.SetSocketOption(<span style="color:#b8d7a3;">SocketOptionLevel</span>.Socket, <span style="color:#b8d7a3;">SocketOptionName</span>.ReuseAddress, <span style="color:#569cd6;">true</span>);
socket.Bind(<span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">IPEndPoint</span>(<span style="color:#4ec9b0;">IPAddress</span>.Loopback, <span style="color:#b5cea8;">1114</span>));
<span style="color:#569cd6;">var </span>ctx = <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">Context</span>(<span style="color:#4ec9b0;">SslMethod</span>.DTLS);
ctx.UseCertificateFile(<span style="color:#d69d85;">"certs/cert.crt"</span>, <span style="color:#b8d7a3;">CertificateType</span>.PEM);
ctx.UsePrivateKeyFile(<span style="color:#d69d85;">"certs/key.key"</span>, <span style="color:#b8d7a3;">CertificateType</span>.PEM);
ctx.SetVerify(<span style="color:#b8d7a3;">SslVerificationKind</span>.SSL_VERIFY_PEER | <span style="color:#b8d7a3;">SslVerificationKind</span>.SSL_VERIFY_CLIENT_ONCE,
VerifyCert);
ctx.SetGenerateCookieCallback(GenerateCookie);
ctx.SetVerifyCookieCallback(VerifyCookie);
ctx.SetOptions(<span style="color:#b8d7a3;">SslOptions</span>.SSL_OP_COOKIE_EXCHANGE);
<span style="color:#569cd6;">var </span>ssl = <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">Ssl</span>(ctx, <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">MemoryBio</span>(), <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">MemoryBio</span>());
<span style="color:#569cd6;">var </span><span style="color:#a9a9a9;">remoteEp </span>= <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">IPEndPoint</span>(<span style="color:#4ec9b0;">IPAddress</span>.Any, <span style="color:#b5cea8;">0</span>);
<span style="color:#569cd6;">var </span>owner = <span style="color:#4ec9b0;">MemoryPool</span><<span style="color:#569cd6;">byte</span>>.Shared.Rent(<span style="color:#b5cea8;">4096</span>);
<span style="color:#569cd6;">var </span>mem = owner.Memory;
<span style="color:#4ec9b0;">MemoryMarshal</span>.TryGetArray(mem, <span style="color:#569cd6;">out </span><span style="color:#4ec9b0;">ArraySegment</span><<span style="color:#569cd6;">byte</span>> seg);
<span style="color:#569cd6;">var </span>buf = seg.Array;
<span style="color:#4ec9b0;">SocketReceiveFromResult </span>rcv;
<span style="color:#4ec9b0;">BioAddress </span>bioAddr = <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">BioAddress</span>();
rcv = <span style="color:#569cd6;">await </span>socket.ReceiveFromAsync(seg, <span style="color:#b8d7a3;">SocketFlags</span>.None, <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">IPEndPoint</span>(<span style="color:#4ec9b0;">IPAddress</span>.Any, <span style="color:#b5cea8;">0</span>));
ssl.ReadBio.Write(buf, rcv.ReceivedBytes);
<span style="color:#4ec9b0;">Ssl</span>.DtlsListen(ssl, bioAddr);
<span style="color:#569cd6;">var </span>bytesRead = ssl.WriteBio.Read(buf, <span style="color:#4ec9b0;">Ssl</span>.TLS_MAX_RECORD_SIZE);
<span style="color:#569cd6;">await </span>socket.SendToAsync(seg.Slice(<span style="color:#b5cea8;">0</span>, bytesRead), <span style="color:#b8d7a3;">SocketFlags</span>.None, rcv.RemoteEndPoint);
rcv = <span style="color:#569cd6;">await </span>socket.ReceiveFromAsync(seg, <span style="color:#b8d7a3;">SocketFlags</span>.None, <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">IPEndPoint</span>(<span style="color:#4ec9b0;">IPAddress</span>.Any, <span style="color:#b5cea8;">0</span>));
ssl.ReadBio.Write(buf, rcv.ReceivedBytes);
<span style="color:#4ec9b0;">Assert</span>.IsTrue(<span style="color:#4ec9b0;">Ssl</span>.DtlsListen(ssl, bioAddr));
socket.Close();
<span style="color:#569cd6;">var </span>clientSock = <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">Socket</span>(<span style="color:#b8d7a3;">AddressFamily</span>.InterNetwork, <span style="color:#b8d7a3;">SocketType</span>.Dgram, <span style="color:#b8d7a3;">ProtocolType</span>.Udp);
clientSock.SetSocketOption(<span style="color:#b8d7a3;">SocketOptionLevel</span>.Socket, <span style="color:#b8d7a3;">SocketOptionName</span>.ReuseAddress, <span style="color:#569cd6;">true</span>);
clientSock.Bind(<span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">IPEndPoint</span>(<span style="color:#4ec9b0;">IPAddress</span>.Loopback, <span style="color:#b5cea8;">1114</span>));
clientSock.Connect(rcv.RemoteEndPoint);
<span style="color:#569cd6;">while </span>(!ssl.IsInitFinished)
{
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"Current handshake state is {</span>ssl.HandshakeState<span style="color:#d69d85;">}"</span>);
<span style="color:#569cd6;">int </span>bytesRcvd = <span style="color:#569cd6;">await </span>clientSock.ReceiveAsync(mem, <span style="color:#b8d7a3;">SocketFlags</span>.None);
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"{</span>bytesRcvd<span style="color:#d69d85;">} bytes read from socket."</span>);
<span style="color:#569cd6;">int </span>bytesWritten = ssl.ReadBio.Write(buf, bytesRcvd);
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"{</span>bytesWritten<span style="color:#d69d85;">} bytes written to SSL"</span>);
<span style="color:#4ec9b0;">Debug</span>.Assert(bytesRcvd == bytesWritten);
<span style="color:#569cd6;">int </span>acc = ssl.Accept();
<span style="color:#b8d7a3;">SslError </span>result = ssl.GetError(acc);
<span style="color:#569cd6;">if </span>(result == <span style="color:#b8d7a3;">SslError</span>.SSL_ERROR_SYSCALL) <span style="color:#569cd6;">throw new </span><span style="color:#4ec9b0;">SslException</span>(<span style="color:#4ec9b0;">ErrorHandler</span>.GetError());
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"Auth result was: {</span>result<span style="color:#d69d85;">}"</span>);
<span style="color:#569cd6;">if </span>(ssl.WriteBio.BytesPending > <span style="color:#b5cea8;">0</span>)
{
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"{</span>ssl.WriteBio.BytesPending<span style="color:#d69d85;">} bytes pending in write bio"</span>);
bytesRead = ssl.WriteBio.Read(buf, <span style="color:#b5cea8;">4096</span>);
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"{</span>bytesRead<span style="color:#d69d85;">} bytes read from write bio"</span>);
<span style="color:#569cd6;">int </span>bytesSent = <span style="color:#569cd6;">await </span>clientSock.SendAsync(mem.Slice(<span style="color:#b5cea8;">0</span>, bytesRead), <span style="color:#b8d7a3;">SocketFlags</span>.None);
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"{</span>bytesSent<span style="color:#d69d85;">} bytes sent to client"</span>);
}
}
ssl.Shutdown();
<span style="color:#569cd6;">if </span>(ssl.WriteBio.BytesPending > <span style="color:#b5cea8;">0</span>)
{
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"{</span>ssl.WriteBio.BytesPending<span style="color:#d69d85;">} bytes pending in write bio"</span>);
bytesRead = ssl.WriteBio.Read(buf, <span style="color:#b5cea8;">4096</span>);
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"{</span>bytesRead<span style="color:#d69d85;">} bytes read from write bio"</span>);
<span style="color:#569cd6;">int </span>bytesSent = <span style="color:#569cd6;">await </span>clientSock.SendAsync(mem.Slice(<span style="color:#b5cea8;">0</span>, bytesRead), <span style="color:#b8d7a3;">SocketFlags</span>.None);
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">$"{</span>bytesSent<span style="color:#d69d85;">} bytes sent to client"</span>);
}
}
[<span style="color:#4ec9b0;">TestMethod</span>]
<span style="color:#569cd6;">public async </span><span style="color:#4ec9b0;">Task </span>BasicListenerTest()
<span style="background-color:#0e4583;">{</span>
<span style="color:#569cd6;">var </span>listener = <span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">DtlsListener</span>(<span style="color:#569cd6;">new </span><span style="color:#4ec9b0;">IPEndPoint</span>(<span style="color:#4ec9b0;">IPAddress</span>.Loopback, <span style="color:#b5cea8;">1114</span>), <span style="color:#d69d85;">"certs/cert.crt"</span>,
<span style="color:#d69d85;">"certs/key.key"</span>);
listener.Start();
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">"Server => Signalling ready to accept client"</span>);
<span style="color:#569cd6;">var </span>client = <span style="color:#569cd6;">await </span>listener.AcceptClientAsync();
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#a9a9a9;">$</span><span style="color:#d69d85;">"Server => Client session started"</span>);
<span style="color:#569cd6;">await </span>client.AuthenticateAsServerAsync();
<span style="color:#4ec9b0;">Debug</span>.WriteLine(<span style="color:#d69d85;">"Server => Client successfully authenticated"</span>);
<span style="color:#569cd6;">var </span>msg = <span style="color:#569cd6;">await </span>client.ReceiveAsync();
<span style="color:#4ec9b0;">Assert</span>.IsTrue(msg.Length >= <span style="color:#b5cea8;">4 </span>&& msg.Length <= <span style="color:#b5cea8;">5</span>);
<span style="color:#4ec9b0;">Assert</span>.AreEqual(<span style="color:#d69d85;">"test"</span>, <span style="color:#4ec9b0;">Encoding</span>.Default.GetString(msg.Span.Slice(<span style="color:#b5cea8;">0</span>, <span style="color:#b5cea8;">4</span>)));
<span style="color:#569cd6;">await </span>client.SendAsync(<span style="color:#4ec9b0;">MemoryMarshal</span>.AsMemory(msg));
<span style="color:#569cd6;">await </span>listener.Stop();
<span style="background-color:#0e4583;">}</span>
<span style="background-color:#0e4583;">}</span></pre>
</body>
</html>