[openssl-users] Query regarding DTLS handshake

[Matt Caswell]

On 13/04/17 18:26, Martin Brejcha wrote:
> 
> 
> Matt Caswell wrote on 04/13/2017 03:45 PM:
>>
>>
>> On 13/04/17 10:11, mahesh gs wrote:
>>> Hi,
>>>
>>> We are running SCTP connections with DTLS enabled in our application. We
>>> have adapted openssl version (openssl-1.1.0e) to achieve the same.
>>>
>>> We have generated the self signed root and node certificates for
>>> testing. We have a strange problem with the incomplete DTLS handshake if
>>> we run the DTLS client and DTLS server is different systems.If we run
>>> the DTLS client and server in same system handshake is successful,
>>> handshake is not successful if run client and server in different VM's.
>>>
>>> This strange problem happens only for SCTP/DTLS connection. With the
>>> same set of certificates TCP/TLS connection is successful and we are
>>> able to exchange the application data.
>>>
>>> I am attaching the code bits for SSL_accept and SSL_connect and also the
>>> wireshark trace of unsuccessful handshake. Please assist me to debug
>>> this problem.
>>>
>>> SSL_accept returns  SSL_ERROR_WANT_READ(2) infinite times but
>>> SSL_connect is called 4 or 5 times and select system call timeout.
>>
>> Your trace shows the following interactions occurring:
>>
>> Client                         Server
>> ------                         ------
>>
>> ClientHello          -------->
>>                      <-------- ServerHello
>>                      <-------- Certificate
>>                      <-------- CertificateRequest
>>                      <-------- ServerDone
>> Certificate          --------->
>> ClientKeyExchange    --------->
>> CertificateVerify    --------->
>> CCS                  --------->
>> [Encrypted Finished]
>>
>> We would expect the server to continue with its own CCS and Encrypted
>> Finished to complete the handshake. It seems that, for some reason, the
>> server is not receiving (or acting upon) the client's second flight of
>> messages.
>>
>> Normally in DTLS this sort of thing can happen due to lost messages etc
>> but, obviously, with SCTP, this is not the case. Something else must be
>> happening.
>>
> 
> There are some SCTP segmented messages during handshake.
> May be some issue in reassembling could lead to strange behavior.
> Can be observed these segmented messages also when the handshake is successful?

That's an interesting question. The segmented messages are for the
Certificate messages. Obviously the client is able to read them just
fine (because it responds with its own Certificate message), but there
could plausibly be an issue on the server side. It would be interesting
to see what happens if you temporarily disable client auth so that the
client does not send this large Certficate message.

Matt

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 480 bytes
Desc: OpenPGP digital signature
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170413/62f90cb0/attachment.sig>