Why can't I force a specific cipher with the openssl app with TLS 1.3?

[Matt Caswell]

On 14/11/2019 17:46, Phil Neumiller wrote:
> Here is my server script is:
> 
> PSK=63ef2024b1
> openssl s_server -accept 4433 -tls1_3  -nocert -psk $PSK -ciphersuites
> TLS_AES_256_GCM_SHA384
> 
> Here is the client:
> 
> PSK=63ef2024b1
> openssl s_client -tls1_3 -psk $PSK -connect :4433  -ciphersuites
> TLS_AES_256_GCM_SHA384
> 
> And here is the error:
> 
> Using default temp DH parameters
> ACCEPT
> ERROR
> C0:65:9F:08:01:00:00:00:error:SSL routines::no suitable signature
> algorithm:ssl/t1_lib.c:2810:
> shutting down SSL
> CONNECTION CLOSED
> 
> So why can't I force the usage of this cipher?  Why does it complain about
> signature algorithms when I didn't specify any?

All TLSv1.3 PSKs must have an associated hash algorithm. The "-psk"
arguument is the old TLSv1.2 way of doing PSKs. This works perfectly
fine in TLSv1.3 too but uses a default hash. Which is, as defined by the
TLSv1.3 spec, SHA-256. Because your chosen ciphersuite uses SHA-384 it
is not compatible with a SHA-256 PSK and therefore the PSK attempt fails.

Given that the attempt to use a PSK has failed OpenSSL then tries to
fallback to a full handshake. One of the first steps in that is to
select which certificate to use. It compares the set of signature
algorithms supplied by the client to see which ones are compatible with
the set of certificates configured for use on the server. In your case
you have specified "-nocert" so, because there are no certificates,
there are no suitable signature algorithms that match them.

In order to get this to work you need to do one of two things:

1) Change your ciphersuite to one that uses SHA-256

or

2) Use the "-psksession" argument instead of "-psk". That will require
you to have an SSL_SESSION object serialised into a file to read. That
SSL_SESSION needs to be set up as described on this page:

https://www.openssl.org/docs/man1.1.1/man3/SSL_set_psk_use_session_callback.html


Matt