[EXT] Re: KTLS with openssl 3.0 fail with error ENOTCONN(Transport endpoint is not connected)

[Gaurav Jain]

Hi

> -----Original Message-----
> From: John Baldwin <jhb at FreeBSD.org>
> Sent: Thursday, January 6, 2022 12:26 AM
> To: Gaurav Jain <gaurav.jain at nxp.com>; borisp at mellanox.com; openssl-
> users at openssl.org
> Cc: Varun Sethi <V.Sethi at nxp.com>; Pankaj Gupta <pankaj.gupta at nxp.com>
> Subject: [EXT] Re: KTLS with openssl 3.0 fail with error ENOTCONN(Transport
> endpoint is not connected)
> 
> Caution: EXT Email
> 
> On 1/4/22 11:49 PM, Gaurav Jain wrote:
> > Hello Boris/John
> >
> > I am from NXP and currently working on enabling KTLS on NXP platforms via
> openssl.
> > I see that you enabled KTLS support in openssl
> 3.0(https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
> .openssl.org%2Fnews%2Fchangelog.html%23openssl-
> 30&data=04%7C01%7Cgaurav.jain%40nxp.com%7Ce87da43a5488475b2aa
> d08d9d07d05b0%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C63777
> 0057654781203%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=vZa0aCu
> D%2FzrXB0vv23DZiOWSVichep42YLqA4a1JeXY%3D&reserved=0).
> >
> > when I configure openssl 3.0 or 3.1.0 with enable-ktls and and try to run the
> s_server, s_client application.
> > I observe that connection is successfully established - but it didn't use KTLS.
> >
> > Then I added additional log in kernel(file net/tls/tls_main.c) and see
> > that kernel is returning error -ENOTCONN when (sk->sk_state !=
> > TCP_ESTABLISHED) in function static int tls_init(struct sock *sk)
> 
> To be clear, I have worked on KTLS support for FreeBSD, not for Linux.
> 
> However, I think the error you are seeing is a red herring.  I think you are seeing
> the setsockopt() call from ktls_enable() fail because it is invoked on the listen
> socket since ktls_enable() is called when sockets are created by libssl.
> 
> For KTLS to work on the server side on Linux what you need to find out is when
> ktls_enable() is invoked on the socket returned by accept() and why that is failing.
> 
> --
> John Baldwin

Thanks John for your input.
Ktls_enable() after accept() is successful on server side.
I added debug logs, ktls_start() is failing with error Invalid argument.

Logs:
openssl s_server -ktls -key rsa.key -cert server.pem -accept 443
Using default temp DH parameters
ACCEPT
ktls_enable setsockopt success, ret = 0
ktls_enable(4) = 1

fd = 4, is_tx = 0, tls_crypto_info_len = 1872610871009456445
ktls_start setsockopt failed, 22, Invalid argument
fd = 4, is_tx = 2, tls_crypto_info_len = 8329596950154514032
ktls_start setsockopt failed, 22, Invalid argument
print_connection_info
-----BEGIN SSL SESSION PARAMETERS-----
MF8CAQECAgMDBALAMAQABDCU12qWDAhzfFI9tbKjWZnN8PrRZgrd3Cge3b5YSeiA
DRWol3d1kQU85QU7C7ZOoA2hBgIEYdbmOaIEAgIcIKQGBAQBAAAArQMCAQGzAwIB
HQ==
-----END SSL SESSION PARAMETERS-----
Shared ciphers:ECDHE-RSA-AES256-GCM-SHA384
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512
Shared Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512
Supported Elliptic Curve Point Formats: uncompressed:ansiX962_compressed_prime:ansiX962_compressed_char2
Supported groups: x25519:secp256r1:x448:secp521r1:secp384r1
Shared groups: x25519:secp256r1:x448:secp521r1:secp384r1
CIPHER is ECDHE-RSA-AES256-GCM-SHA384
Secure Renegotiation IS supported
Using Kernel TLS for sending fail
Using Kernel TLS for receiving fail

Regards
Gaurav Jain