Fwd: Proper API usage with DTLS over custom net transport

[Matt Caswell]

On 20/10/2022 20:33, Павел Балашов wrote:
> So now the questions:
> (1) If we receive some dtls data at the line above with '????' what 
> should we do in terms of OpenSSL API calls ?  I assume this dtls data 
> could be a client's retransmission due to server's last flight was lost 
> or this could be client receiving server's last flight duplicated 
> (theoretically could happen as long as lower layer protocol is UDP) or 
> this could be DTLS-encrypted real-app data or this could be 
> DTLS-renegotiation, this also could be a DTLS shutdown alert and 
> anything else DTSL-related. What is the supposed way of inferring and 
> reacting to those different events with API ?

Call SSL_read(). If it is app data it will be returned. If it is 
something else it will be handled.

> (2) Is the whole usage of OpenSSL even right for this scenario - maybe 
> the structure and sequence of API calls should be rearranged somehow ?

Seems approximately right except that BIO_s_mem() is not a good choice 
for DTLS. It does not respect packet semantics, so if you receive 2 
packets and they both get pushed into the BIO_s_mem(), the reader will 
just get both packets together.

Unfortunately there is no equivalent of BIO_s_mem() that respects 
packets in 3.0/1.1.1. There is in the master branch (what will become 
3.2), where we have BIO_s_dgram_mem().

Also note that the only BIO that provides the various ctrls for querying 
the underlying MTU is BIO_s_dgram(). If you use something else that 
doesn't have those ctrls it will fallback to some worse case MTU.

It might be possible to write a custom BIO equivalent of BIO_s_mem() 
that does the right thing with respect to packets and has the right MTU 
ctrls.

On the server side you might want to consider whether DTLSv1_listen() 
should be used in your scenario, and whether you should set the cookie 
callback.

You should check the return value from SSL_do_handshake() and use 
SSL_get_error() to interpret any failure return codes.

> (3) There is an option to pass custom info_callback 
> with SSL_CTX_set_info_callback(). Would there be a proper usage of 
> this kind of callback in this scenario ?

Not sure what you're asking here. There is a man page for this function 
here:

https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_info_callback.html

Matt


> 
> Any other input, links to any kind of relevant supplemental material is 
> really appreciated.
> 
> Thanks a lot for reading, very special thanks to authors and maintainers 
> for all the hard work on this project.
> ---
> Regards,
> Pavel.
> 
>