Handling signature_algorithm extension on TLS1.3 server

[Raja Ashok]

Thanks for the detailed explanation.

So rsaEncryption cert can do both RSASSA-PKCS-v1_5 and RSASSA-PSS type
signature. And also the digital signature present on the cert can be of
type RSASSA-PKCS-v1_5 or RSASSA-PSS.

Currently in 1.1.1c's has_usable_cert() function, digital signature
(Issuer's signature) present on rsaEncryption cert type is not checked. So
if TLS1.3 client sends rsa_pss_rsae_xxx in "signature_algorithm" extension
and if the server's rsaEncrypted cert has digital signature of type
RSASSA-PKCS-v1_5, then it should not use that certificate but it is using
currently.

And also the hash algorithm used on rsaEncryption cert's digital signature
is not checked with "signature_algorithm" extension received from client.
If TLS1.3 client sends rsa_pss_rsae_sha384 and rsa_pss_rsae_sha512 in
"signature_algorithm" extension, and if server has a rsaEncrypted cert with
signature of type rsa_pss_rsae_sha256 then it should not use that
certificate, but it is using currently.


On Thu, Jun 6, 2019 at 9:11 PM Matt Caswell <matt at openssl.org> wrote:

>
>
> On 06/06/2019 16:15, Raja Ashok wrote:
> > Hi,
> >
> > Currently has_usable_cert() function is called on tls_choose_sigalg() to
> find
> > out the suitable certificate available. But currently rsa_pkcs1_xxx and
> > rsa_pss_rsae_xxx certs are stored on same index SSL_PKEY_RSA. Because of
> this it
> > may ends in choosing rsa_pkcs1_xxx cert for rsa_pss_rsae_xxx extension.
> Is this
> > behaviour correct ?
>
> There are two things to consider:
>
> 1) The OID in the RSA cert, which can be one of rsaEncryption or
> RSASSA-PSS. The
> former is for "traditional" RSA certs, whilst the latter *only* allows use
> of
> the key for signing (it cannot be used for encryption).
>
> 2) The type of signing in use, e.g. RSASSA-PKCS-v1_5 or RSASSA-PSS.
>
> rsaEncryption certs are capable of doing *either* form of signing, whilst
> RSASSA-PSS certs can only do PSS signing.
>
> We store rsaEncryption certs under the SSL_PKEY_RSA index, and RSASSA-PSS
> certs
> under the SSL_PKEY_RSA_PSS_SIGN index.
>
> TLSv1.2 and below signs handshake messages using PKCS v1.5. which
> corresponds to
> these signature algorithms:
>
>           rsa_pkcs1_sha256(0x0401)
>           rsa_pkcs1_sha384(0x0501)
>           rsa_pkcs1_sha512(0x0601)
>
> These sig algs cannot be used in TLSv1.3 for signing handshake messages,
> although they may appear in a ClientHello for backwards compatibility with
> TLSv1.2. You can only use these sig algs with "traditional" RSA certs (not
> PSS
> RSA certs).
>
> TLSv1.3 signs handshake messages using PSS which corresponds to these
> signature
> algorithms for "traditional" (rsaEncryption) certs:
>
>           rsa_pss_rsae_sha256(0x0804)
>           rsa_pss_rsae_sha384(0x0805)
>           rsa_pss_rsae_sha512(0x0806)
>
> Or these signature algorithms for PSS certs:
>
>           rsa_pss_pss_sha256(0x0809)
>           rsa_pss_pss_sha384(0x080a)
>           rsa_pss_pss_sha512(0x080b)
>
> Therefore it is perfectly correct behaviour that a cert stored under the
> SSL_PKEY_RSA index could be used for signing handshake message using either
> rsa_pkcs1_xxx or for rsa_pss_rsae_xxx. The former is used in TLSv1.2 and
> the
> latter is used in TLSv1.3.
>
> Matt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20190607/2f9483ad/attachment.html>