[openssl-users] Query regarding DTLS handshake

[Martin Brejcha]

Matt Caswell wrote on 04/13/2017 03:45 PM:
> 
> 
> On 13/04/17 10:11, mahesh gs wrote:
>> Hi,
>>
>> We are running SCTP connections with DTLS enabled in our application. We
>> have adapted openssl version (openssl-1.1.0e) to achieve the same.
>>
>> We have generated the self signed root and node certificates for
>> testing. We have a strange problem with the incomplete DTLS handshake if
>> we run the DTLS client and DTLS server is different systems.If we run
>> the DTLS client and server in same system handshake is successful,
>> handshake is not successful if run client and server in different VM's.
>>
>> This strange problem happens only for SCTP/DTLS connection. With the
>> same set of certificates TCP/TLS connection is successful and we are
>> able to exchange the application data.
>>
>> I am attaching the code bits for SSL_accept and SSL_connect and also the
>> wireshark trace of unsuccessful handshake. Please assist me to debug
>> this problem.
>>
>> SSL_accept returns  SSL_ERROR_WANT_READ(2) infinite times but
>> SSL_connect is called 4 or 5 times and select system call timeout.
> 
> Your trace shows the following interactions occurring:
> 
> Client                         Server
> ------                         ------
> 
> ClientHello          -------->
>                      <-------- ServerHello
>                      <-------- Certificate
>                      <-------- CertificateRequest
>                      <-------- ServerDone
> Certificate          --------->
> ClientKeyExchange    --------->
> CertificateVerify    --------->
> CCS                  --------->
> [Encrypted Finished]
> 
> We would expect the server to continue with its own CCS and Encrypted
> Finished to complete the handshake. It seems that, for some reason, the
> server is not receiving (or acting upon) the client's second flight of
> messages.
> 
> Normally in DTLS this sort of thing can happen due to lost messages etc
> but, obviously, with SCTP, this is not the case. Something else must be
> happening.
> 

There are some SCTP segmented messages during handshake.
May be some issue in reassembling could lead to strange behavior.
Can be observed these segmented messages also when the handshake is successful?

M.


> In your description you say SSL_accept() gets called repeatedly and
> always gives SSL_ERROR_WANT_READ. Looking at your code it looks like you
> are calling pollSocketForEvents() after each accept. I am assuming that
> this is returning true each time (otherwise you would break out of the
> loop). This suggests that the "select" call thinks there is something to
> read from the underlying socket. Am I correct? The question is why
> doesn't OpenSSL then read that data out of the socket?
> 
> Are you able to build a debug version of OpenSSL (run "config" with -d),
> and step through to figure out where it gets stuck. Is it attempting to
> read the data and failing, or does it not get as far attempting to read it?
> 
> Another question: does this fail every time or does it sometimes work
> and sometimes not (which might suggest some race condition)?
> 
> Matt
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0xB42AB632.asc
Type: application/pgp-keys
Size: 3086 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170413/6c67b1e0/attachment.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170413/6c67b1e0/attachment.sig>