[openssl-users] Query regarding MSG_NOSIGNAL with SSL_Write

[Matt Caswell]

On 27/04/17 15:53, Viktor Dukhovni wrote:
> On Thu, Apr 27, 2017 at 12:32:42PM +0000, Salz, Rich via openssl-users wrote:
> 
>>> Does openssl  provide any way to set MSG_NOSIGNAL on sendmsg (Underlying TCP/IP socket layer) ?
>>
>> No.  You will have to modify the code yourself.
> 
> Actually, it is possible to do the I/O in application code, using
> any "write some data down a socket" API of the application's choice.
> 
>     https://www.openssl.org/docs/man1.0.2/crypto/BIO_s_bio.html
> 
> In particular, the OP could use sendmsg() to move data between the
> SSL layer and the network.
> 
> For a complete example, see network_biopair_interop() function in
> Postfix 2.3 (recent Postfix releases no longer use this approach).
> 
>     https://github.com/vdukhovni/postfix/blob/postfix-2.3/postfix/src/tls/tls_bio_ops.c
> 

The OP is using SCTP (which uses DTLS). The above approach is
problematic in DTLS. The DTLS code assumes that the BIO will provide a
set of datagram related ctrls (which are of course available if you use
a straight BIO_s_datagram()). BIO pairs don't support those ctrls.
Additionally they don't respect datagram boundaries.

You could use a custom filter BIO for a similar effect which can pass on
the ctrls down to the final source/sink BIO - and just use it to
intercept the "write" calls and plug in your own custom call of
sendmsg(). That would probably work with straight DTLS over UDP.

Unfortunately the libssl SCTP code is even more restrictive than normal
DTLS. It tests whether you are using SCTP by calling BIO_dgram_is_sctp()
on the read or write BIO:

int BIO_dgram_is_sctp(BIO *bio)
{
    return (BIO_method_type(bio) == BIO_TYPE_DGRAM_SCTP);
}

If you plug in your own custom BIO it fails to detect SCTP :-(

The code also calls a number of other BIO specific functions such as
BIO_dgram_sctp_wait_for_dry() and BIO_dgram_sctp_msg_waiting().

In other words the libssl SCTP code is tightly coupled to the SCTP BIO
implementation - which effectively rules out custom BIOs.

The code could do with an overhaul, but not that many people use SCTP so
it hasn't really been a priority :-(

Matt