[openssl/openssl] 564a8d: Fix an assertion in the DTLS server code

[bernd-edlinger]

  Branch: refs/heads/OpenSSL_1_1_1-stable
  Home:   https://github.openssl.org/openssl/openssl
  Commit: 564a8d442cbd8ce68d452ff2e8a58c0aea6b0632
      https://github.openssl.org/openssl/openssl/commit/564a8d442cbd8ce68d452ff2e8a58c0aea6b0632
  Author: Bernd Edlinger <bernd.edlinger at hotmail.de>
  Date:   2022-04-14 (Thu, 14 Apr 2022)

  Changed paths:
    M ssl/statem/statem_dtls.c
    M test/dtls_mtu_test.c

  Log Message:
  -----------
  Fix an assertion in the DTLS server code

This fixes an internal error alert from the server and
an unexpected connection failure in the release version,
but a failed assertion and a server crash in the
debug version.

Reproduce this issue with a DTLS server/client like that:

./openssl s_server -dtls -mtu 1500
./openssl s_client -dtls -maxfraglen 512

In the debug version a crash happens in the Server now:

./openssl s_server -dtls -mtu 1500
Using default temp DH parameters
ACCEPT
ssl/statem/statem_dtls.c:269: OpenSSL internal error: Assertion failed: len == written
Aborted (core dumped)

While in the release version the handshake exceeds the
negotiated max fragment size, and fails because of this:

$ ./openssl s_server -dtls -mtu 1500
Using default temp DH parameters
ACCEPT
ERROR
4057152ADA7F0000:error:0A0000C2:SSL routines:do_dtls1_write:exceeds max fragment size:ssl/record/rec_layer_d1.c:826:
shutting down SSL
CONNECTION CLOSED

>From the client's point of view the connection fails
with an Internal Error Alert:

$ ./openssl s_client -dtls -maxfraglen 512
Connecting to ::1
CONNECTED(00000003)
40B76343377F0000:error:0A000438:SSL routines:dtls1_read_bytes:tlsv1 alert internal error:ssl/record/rec_layer_d1.c:613:SSL alert number 80

and now the connection attempt fails unexpectedly.

Reviewed-by: Tomas Mraz <tomas at openssl.org>
Reviewed-by: Matt Caswell <matt at openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18093)

(cherry picked from commit e915c3f5381cd38ebdc1824c3ba9896ea7160103)