[openssl-users] Multi client DTLS server on OpenSSL 1.1.x broken?

[Richard Weinberger]

Hi!

I have a hard time figuring how to write a DTLS UDP server that supports multiple
clients. My dummy single user server works fine.

To support multiple clients I tried two approaches:
1. singled threaded async IO, preferred since I have to deal with many clients
2. multi threaded, one thread per client

Both approaches seem to be doomed for the very same reason, namely that
DTLSv1_listen() does peek into the kernel queue and does not consume
the client hello from the UDP socket.

Both loop around DTLSv1_listen() and as soon the function returns > 0 a new
socket for the client is created using bind/connect and the client address
as returned by DTLSv1_listen().

This client socket is then passed to a new thread or feed into the event loop.
In both cases the client hello is still in the queue of the server socket
and the program will over and over create new client sockets.

After searching the web for examples I've found this thread[0], where the approaches
I tried are advertised.
In [1] the demo server at [3] is suggested as good example.

dtls_udp_echo.c from [3] does exactly what I did in my 2nd approach, and it fails in
the same way.
As soon one client connects, it creates over and over new sockets until it dies due
to too many open files.

After digging a bit into the source it looks to me like since commit [3],
DTLSv1_listen() assumes that you re-use the same socket for the new client.
Which makes supporting multiple clients impossible.

Given that I'm not an OpenSSL DTLS expert I still hope I miss something.
Can you please help me to figure what the correct approach for multiple clients is?

Thanks,
//richard

P.s: I'm on Linux, OpenSSL 1.1.0h, but tried as OpenSSL git as of today.

[0] https://mta.openssl.org/pipermail/openssl-users/2018-April/007861.html
[1] https://mta.openssl.org/pipermail/openssl-users/2018-April/007864.html
[2] https://web.archive.org/web/20150806185102/http://sctp.fh-muenster.de:80/dtls/dtls_udp_echo.c
[3] https://github.com/openssl/openssl/commit/e3d0dae7cf8363ca462ac425b72c7bb31c3b4b7a