[openssl-users] Unexpected behaviors in TLS handshake

[Matt Caswell]

On 20/06/18 14:51, Devang Kubavat wrote:
> Hi Matt, 
> Thanks for reply. 
> 
> I also used both functions SSL_CTX_set1_sigalgs_list() 
> SSL_CTX_set1_client_sigalgs_list() 
> but same thing happens. 
> I set client side “RSA+SHA512” using SSL_CTX_set1_sigalgs_list() but still it is accepting sever certificate which has signature algorithm SHA256withRSAencryption.

RFC5246 (TLSv1.2) says this about sigalgs:

   The client uses the "signature_algorithms" extension to indicate to
   the server which signature/hash algorithm pairs may be used in
   digital signatures.  The "extension_data" field of this extension
   contains a "supported_signature_algorithms" value.

There are actually 2 places in TLS where the server sends digital
signatures to the client: in the ServerKeyExchange message, and in the
server's Certificate.

An OpenSSL server will use the client's sigalgs to guide which of its
certificates are appropriate to use for the client - so if it has an
ECDSA cert but the client hasn't offered any ECDSA sig algs then it
won't use that cert.

The server will also sign the ServerKeyExchange with one of the sig algs
offered by the client.

Note though that an OpenSSL server does *not* (by default) check the
signature in its certs for compliance with the sig algs. So, for
example, if it has an ECDSA cert signed by the CA using RSA+SHA256, but
the client only offers ECDSA sig algs then it will still use that
certificate. You can configure the server to operate in "strict" mode
(e.g. using the "-strict" command line arg to s_server). In that mode it
will also check its certificate chain for compliance with the client's
sig algs.

An OpenSSL client will enforce that the ServerKeyExchange signature is
consistent with the sig algs that it sent. It does *not* enforce that
the server's certificate signatures are consistent with those sig algs.
I don't think there is any equivalent of the server's "-strict" to
switch this checking on.

Note that in TLSv1.3 there are actually *two* sig algs extensions, i.e.
"signature_algorithms" and "signature_algorithms_cert". The latter
enables you to specify acceptable signature algorithms in a certificate
chain separately from signatures algorithms in the TLSv1.3
CertificateVerify message.

Hope that helps,

Matt



> 
> Best Regards, 
> Devang
> 
> Sent from my iPhone
> 
>> On 20-Jun-2018, at 2:25 PM, Matt Caswell <matt at openssl.org> wrote:
>>
>>
>>
>>> On 20/06/18 09:44, Devang Kubavat wrote:
>>> Hi all,
>>>
>>> I set the signature algorithm using in client,
>>>
>>> /* signature algorithm list */
>>>
>>> (void)SSL_CTX_set1_client_sigalgs_list(ctx, “RSA+SHA512”);
>>>
>>>  
>>>
>>> Expected behavior: client only accepts server certificate which has
>>> signature algorithm SHA512withRSAencryption during TLS handshake.
>>>
>>>  
>>>
>>> But, here even I set “RSA+SHA512” signature algorithm, still client is
>>> accepting the server certificate which has signature algorithm
>>> SHA256withRSAencryption. Why?
>>
>> As I said in reply to your other post:
>>
>> "The function "SSL_CTX_set1_client_sigalgs_list()" is for setting
>> signature algorithms related to *client authentication*. This is not the
>> same as the sig algs sent in the ClientHello. For that you need to use
>> SSL_CTX_set1_sigalgs_list()."
>>
>> Matt
>> -- 
>> openssl-users mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users