[openssl-users] Query regarding DTLS handshake

[mahesh gs]

Hi,

We are using RHEL 7.0

Linux insvm02 3.10.0-229.el7.x86_64 #1 SMP Thu Jan 29 18:37:38 EST 2015
x86_64 x86_64 x86_64 GNU/Linux


We built openssl code by enabling debug options and found that in
TLS_ST_SR_CERT_VRFY state openssl checks  for the following conditions

1) If the datagram type is SCTP : : In this case it is DTLS and set to 1.

2) Value of s->renegotiate  : : in this case the value was set to 2. I
guess this is set in post processing function of client hello.

3) If any sctp message is waiting to be received : : In this case this was
true (1). When we debug inside the BIO_dgram_sctp_msg_waiting. We observed
that the recvmsg system call returning the size (n = 14 bytes) that is
exactly size of the next  package (Auth Change Cipher Spec) as shown in the
below screenshot.

So this function return "WORK_MORE_A" to the caller "read_state_machine"
which intern returns SUB_STATE_ERROR. And this flow keeps repeating.
And also we observed that this issue is something to do with the timing of
the messages received at SCTP layer. If the post processing message
function executed before the next message arrives at SCTP layer, then the
SSL negotiation is successful else SSL negotiation hangs at
TLS_ST_SR_CERT_VEFY state.


Can someone who is familiar with the openssl code give us some insights
about what is the logic behind the if check in TLS_ST_SR_CERT_VRFY state in
post processing function ?

[image: Inline image 1]

[image: Inline image 2]



[image: Inline image 3]


Thanks,
Mahesh G S



On Wed, Apr 19, 2017 at 1:47 AM, Michael Tuexen <
Michael.Tuexen at lurchi.franken.de> wrote:

> > On 13. Apr 2017, at 11:11, mahesh gs <mahesh116 at gmail.com> wrote:
> >
> > Hi,
> >
> > We are running SCTP connections with DTLS enabled in our application. We
> have adapted openssl version (openssl-1.1.0e) to achieve the same.
> >
> > We have generated the self signed root and node certificates for
> testing. We have a strange problem with the incomplete DTLS handshake if we
> run the DTLS client and DTLS server is different systems.If we run the DTLS
> client and server in same system handshake is successful, handshake is not
> successful if run client and server in different VM's.
> >
> > This strange problem happens only for SCTP/DTLS connection. With the
> same set of certificates TCP/TLS connection is successful and we are able
> to exchange the application data.
> >
> > I am attaching the code bits for SSL_accept and SSL_connect and also the
> wireshark trace of unsuccessful handshake. Please assist me to debug this
> problem.
> >
> > SSL_accept returns  SSL_ERROR_WANT_READ(2) infinite times but
> SSL_connect is called 4 or 5 times and select system call timeout.
> Which OS are you using? With a test program I could reproduce SSL_accept()
> returning SSL_ERROR_WANT_READ under FreeBSD,
> but not under Linux. Haven't figured out what the problem is. So if you
> are using FreeBSD we might experience the same problem...
>
> Best regards
> Michael
> >
> > Thanks,
> > Mahesh G S
> >
> >
> > <testcode.txt><proxy.cap>--
> > openssl-users mailing list
> > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
>
> --
> openssl-users mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170419/8bc3ae20/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 59412 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170419/8bc3ae20/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 30459 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170419/8bc3ae20/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 120261 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20170419/8bc3ae20/attachment-0005.png>