<div dir="ltr">Hi All,<div><br></div><div>We built openssl code by enabling debug options and found that in TLS_ST_SR_CERT_VRFY state openssl checks for the following conditions</div><div><br></div><div>1) If the datagram type is SCTP : : In this case it is DTLS and set to 1.</div><div><br></div><div>2) Value of s->renegotiate : : in this case the value was set to 2. I guess this is set in post processing function of client hello.</div><div><br></div><div>3) If any sctp message is waiting to be received : : In this case this was true (1). When we debug inside the BIO_dgram_sctp_msg_waiting. We observed that the recvmsg system call returning the size (n = 14 bytes) that is exactly size of the next package (Auth Change Cipher Spec) as shown in the below screenshot. </div><div><br></div><div>So this function return "WORK_MORE_A" to the caller "read_state_machine" which intern returns SUB_STATE_ERROR. And this flow keeps repeating.</div><div>And also we observed that this issue is something to do with the timing of the messages received at SCTP layer. If the post processing message function executed before the next message arrives at SCTP layer, then the SSL negotiation is successful else SSL negotiation hangs at TLS_ST_SR_CERT_VEFY state.</div><div><br></div><div><br></div><div>Can someone who is familiar with the openssl code give us some insights about what is the logic behind the if check in TLS_ST_SR_CERT_VRFY state in post processing function ?</div><div><br></div><div><img src="cid:ii_15b80ab71343a4fe" alt="Inline image 1" width="562" height="468"><br></div><div><br></div><div><img src="cid:ii_15b80b1742b706bb" alt="Inline image 2" width="562" height="315"><br></div><div><br></div><div><br></div><div><br></div><div><img src="cid:ii_15b80b4bc10c3a72" alt="Inline image 3" width="562" height="334"><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Apr 17, 2017 at 4:14 PM, mahesh gs <span dir="ltr"><<a href="mailto:mahesh116@gmail.com" target="_blank">mahesh116@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi All,<div><br></div><div>Adding to this, When we changed the path MTU of the source and destination VM's we could see the SSL connection is successful and application data getting exchanged which confirms that there is issue in segmentation and reassembly of packets.</div><div><br></div><div><br></div><div>Do we have some system configuration that need to be enabled for DTLS ? as i mentioned in my previous mails, TLS negotiation is successful without changing path MTU but not DTLS.</div><div><br></div><div><br></div><div>Thanks,</div><div>Mahesh G S </div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Apr 17, 2017 at 12:36 PM, mahesh gs <span dir="ltr"><<a href="mailto:mahesh116@gmail.com" target="_blank">mahesh116@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi All,<div><br></div><div>Sorry for delayed response, thank you all for responding to my query.<br><div><div><br></div><div>1) We are using non-blocking sockets on RHEL 7.0 machine with the kernel version 3.10.</div><div><br></div><div>2) When we run client and server in the same machine it works fine. Client and server able to exchange the data without any issues. But if we run client and server in different machines, we face this problem.</div><div><br></div><div>Server side logs when we run the server and client in same machine</div><div><br></div><div><img src="cid:ii_15b7a4e5b66eb931" alt="Inline image 1" width="514" height="386"><br></div><div><br></div><div>Client side log when we run the server and client in the same machine.</div><div><br></div><div><img src="cid:ii_15b7a4f422f7f978" alt="Inline image 2" width="460" height="321"><br></div><div><br></div><div><br></div><div>3) as suggested by Matt i disable the client auth when client-server run in different machines and surprisingly the TLS negotiation is successful. That leads me to suspect on segmentation and reassembly.</div><div><br></div><div>Server side logs when the client auth is disabled</div><div><br></div><div><img src="cid:ii_15b7a5a3c00fb766" alt="Inline image 3" width="447" height="562"> </div><div><br></div><div><br></div><div>Client side logs when the client auth is disabled.</div><div><br></div><div><img src="cid:ii_15b7a5ac32d66aac" alt="Inline image 4" width="480" height="556"><br></div><div><br></div></div></div><div><br></div><div>I took a wireshark trace and found that the server is able to receive the client certificate in 3 fragments and responding with the proper sack with right TSN. But server do not respond with its own certificate after that. I have attached the wireshark trace for the same. Path MTU in both the systems set to 1500.</div><div><br></div><div><br></div><div>I took the strace of the calls between server and client but i could not figure out anything with the strace. I have attached the same in the mail. strace summary table is as below.</div><div><br></div><div><div>^C% time seconds usecs/call calls errors syscall</div><div>------ ----------- ----------- --------- --------- ----------------</div><div> 33.71 0.000326 3 99 76 open</div><div> 18.10 0.000175 3 54 mmap</div><div> 15.10 0.000146 4 37 mprotect</div><div> 9.72 0.000094 13 7 recvmsg</div><div> 5.07 0.000049 2 26 read</div><div> 4.03 0.000039 2 24 close</div><div> 3.93 0.000038 2 17 fstat</div><div> 3.83 0.000037 19 2 sendto</div><div> 1.24 0.000012 3 4 3 stat</div><div> 0.93 0.000009 2 5 brk</div><div> 0.62 0.000006 6 1 munmap</div><div> 0.52 0.000005 3 2 futex</div><div> 0.41 0.000004 1 3 rt_sigaction</div><div> 0.41 0.000004 4 1 1 access</div><div> 0.41 0.000004 1 3 socket</div><div> 0.41 0.000004 4 1 execve</div><div> 0.31 0.000003 3 1 set_tid_address</div><div> 0.21 0.000002 1 3 rt_sigprocmask</div><div> 0.21 0.000002 1 2 bind</div><div> 0.21 0.000002 1 3 getsockname</div><div> 0.21 0.000002 2 1 getrlimit</div><div> 0.21 0.000002 2 1 arch_prctl</div><div> 0.21 0.000002 2 1 set_robust_list</div><div> 0.00 0.000000 0 25 write</div><div> 0.00 0.000000 0 1 ioctl</div><div> 0.00 0.000000 0 1 listen</div><div> 0.00 0.000000 0 6 clone</div><div> 0.00 0.000000 0 4 epoll_ctl</div><div> 0.00 0.000000 0 1 timerfd_create</div><div> 0.00 0.000000 0 1 eventfd2</div><div> 0.00 0.000000 0 1 epoll_create1</div><div>------ ----------- ----------- --------- --------- ----------------</div><div>100.00 0.000967 338 80 total</div></div><div><br></div><div><br></div><div>I am new to openssl code could not figure out how to debug the issue at socket layer. Can some one help me to understand how to go forward to resolve this issue ?</div><div><br></div><div><br></div><div>Thanks and Regards,</div><div>Mahesh G S</div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_2383347118826974705h5">On Fri, Apr 14, 2017 at 3:32 AM, Matt Caswell <span dir="ltr"><<a href="mailto:matt@openssl.org" target="_blank">matt@openssl.org</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_2383347118826974705h5"><div class="m_2383347118826974705m_-5657518782184815366HOEnZb"><div class="m_2383347118826974705m_-5657518782184815366h5"><br>
<br>
On 13/04/17 18:26, Martin Brejcha wrote:<br>
><br>
><br>
> Matt Caswell wrote on 04/13/2017 03:45 PM:<br>
>><br>
>><br>
>> On 13/04/17 10:11, mahesh gs wrote:<br>
>>> Hi,<br>
>>><br>
>>> We are running SCTP connections with DTLS enabled in our application. We<br>
>>> have adapted openssl version (openssl-1.1.0e) to achieve the same.<br>
>>><br>
>>> We have generated the self signed root and node certificates for<br>
>>> testing. We have a strange problem with the incomplete DTLS handshake if<br>
>>> we run the DTLS client and DTLS server is different systems.If we run<br>
>>> the DTLS client and server in same system handshake is successful,<br>
>>> handshake is not successful if run client and server in different VM's.<br>
>>><br>
>>> This strange problem happens only for SCTP/DTLS connection. With the<br>
>>> same set of certificates TCP/TLS connection is successful and we are<br>
>>> able to exchange the application data.<br>
>>><br>
>>> I am attaching the code bits for SSL_accept and SSL_connect and also the<br>
>>> wireshark trace of unsuccessful handshake. Please assist me to debug<br>
>>> this problem.<br>
>>><br>
>>> SSL_accept returns SSL_ERROR_WANT_READ(2) infinite times but<br>
>>> SSL_connect is called 4 or 5 times and select system call timeout.<br>
>><br>
>> Your trace shows the following interactions occurring:<br>
>><br>
>> Client Server<br>
>> ------ ------<br>
>><br>
>> ClientHello --------><br>
>> <-------- ServerHello<br>
>> <-------- Certificate<br>
>> <-------- CertificateRequest<br>
>> <-------- ServerDone<br>
>> Certificate ---------><br>
>> ClientKeyExchange ---------><br>
>> CertificateVerify ---------><br>
>> CCS ---------><br>
>> [Encrypted Finished]<br>
>><br>
>> We would expect the server to continue with its own CCS and Encrypted<br>
>> Finished to complete the handshake. It seems that, for some reason, the<br>
>> server is not receiving (or acting upon) the client's second flight of<br>
>> messages.<br>
>><br>
>> Normally in DTLS this sort of thing can happen due to lost messages etc<br>
>> but, obviously, with SCTP, this is not the case. Something else must be<br>
>> happening.<br>
>><br>
><br>
> There are some SCTP segmented messages during handshake.<br>
> May be some issue in reassembling could lead to strange behavior.<br>
> Can be observed these segmented messages also when the handshake is successful?<br>
<br>
</div></div>That's an interesting question. The segmented messages are for the<br>
Certificate messages. Obviously the client is able to read them just<br>
fine (because it responds with its own Certificate message), but there<br>
could plausibly be an issue on the server side. It would be interesting<br>
to see what happens if you temporarily disable client auth so that the<br>
client does not send this large Certficate message.<br>
<span class="m_2383347118826974705m_-5657518782184815366HOEnZb"><font color="#888888"><br>
Matt<br>
<br>
</font></span><br></div></div><span>--<br>
openssl-users mailing list<br>
To unsubscribe: <a href="https://mta.openssl.org/mailman/listinfo/openssl-users" rel="noreferrer" target="_blank">https://mta.openssl.org/mailma<wbr>n/listinfo/openssl-users</a><br>
<br></span></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>