Nerd-sniped on rsa_pkcs1_sha1 Server Key Exchange signature algorithm.

[Ralph Corderoy]

Hi,

An nginx serving TLS on both www.example.com and plain example.com
uses different signature algorithms in the TLSv1.2 server key exchange.
I'd really like to understand why.

It's the one nginx server configured with multiple ‘server {...}’
blocks.  Both blocks have the same TLS configuration by including the
same file with the ssl_...  statements.  The one Let's Encrypt
certificate chain is used by both.  When retrieved with

    openssl s_client -connect     example.com:https -showcerts </dev/null
    openssl s_client -connect www.example.com:https -showcerts </dev/null

the same chain is received.  The chain's first certificate has the plain
example.com as its CN with the www domain in the X509v3 SAN.

        Subject: CN = example.com
    ...
            X509v3 Subject Alternative Name:
                DNS:example.com, DNS:www.example.com

Those openssl commands above say the plain uses SHA256 whilst the www
uses SHA1.

     No client certificate CA names sent
    -Peer signing digest: SHA256
    +Peer signing digest: SHA1
     Peer signature type: RSA
     Server Temp Key: ECDH, P-256, 256 bits

Using tshark(1), I can see more detail when comparing other TLS
connections to the two domains.  Left is plain, right is www.

      TLSv1.2 Record Layer: Handshake Protocol: Server Key Exchange
          Content Type: Handshake (22)
          Version: TLS 1.2 (0x0303)
          Length: 333
          Handshake Protocol: Server Key Exchange
              Handshake Type: Server Key Exchange (12)
              Length: 329
              EC Diffie-Hellman Server Params
                  Curve Type: named_curve (0x03)
                  Named Curve: secp256r1 (0x0017)
                  Pubkey Length: 65
                  Pubkey: 04547a26…                                |   Pubkey: 04f37c7c…
                  Signature Algorithm: rsa_pkcs1_sha256 (0x0401)   |   Signature Algorithm: rsa_pkcs1_sha1 (0x0201)
                      Signature Hash Algorithm Hash: SHA256 (4)    |       Signature Hash Algorithm Hash: SHA1 (2)
                      Signature Hash Algorithm Signature: RSA (1)

Both connections have the same cipher suites in the Client Hello.

    TLS_AES_256_GCM_SHA384 (0x1302)                               
    TLS_CHACHA20_POLY1305_SHA256 (0x1303)                         
    TLS_AES_128_GCM_SHA256 (0x1301)                               
    TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)              
    TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)                
    TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f)                  
    TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)        
    TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)          
    TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xccaa)            
    TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)              
    TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)                
    TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)                  
    TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)              
    TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)                
    TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x006b)                  
    TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)              
    TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)                
    TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x0067)                  
    TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)                 
    TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)                   
    TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)                     
    TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)                 
    TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)                   
    TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)                     
    TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)                      
    TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)                      
    TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)                      
    TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)                      
    TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)                         
    TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)                         
    TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff)                    

And the same signature hash algorithms in the Client Hello's
signature_algorithms extension.

    ecdsa_secp256r1_sha256 (0x0403)                
    ecdsa_secp384r1_sha384 (0x0503)                
    ecdsa_secp521r1_sha512 (0x0603)                
    ed25519 (0x0807)                               
    ed448 (0x0808)                                 
    rsa_pss_pss_sha256 (0x0809)                    
    rsa_pss_pss_sha384 (0x080a)                    
    rsa_pss_pss_sha512 (0x080b)                    
    rsa_pss_rsae_sha256 (0x0804)                   
    rsa_pss_rsae_sha384 (0x0805)                   
    rsa_pss_rsae_sha512 (0x0806)                   
    rsa_pkcs1_sha256 (0x0401)                      
    rsa_pkcs1_sha384 (0x0501)                      
    rsa_pkcs1_sha512 (0x0601)                      
    SHA224 ECDSA (0x0303)                          
    SHA224 RSA (0x0301)                            
    SHA224 DSA (0x0302)                            
    SHA256 DSA (0x0402)                            
    SHA384 DSA (0x0502)                            
    SHA512 DSA (0x0602)                            

This is an ancient Ubuntu with an old openssl-1.0.1f.  I realise the fix
is ‘upgrade’ but I want to understand what's happening and learn from
it...

I think ssl3_send_server_key_exchange() is getting ‘md’
with

    pkey=ssl_get_sign_pkey(s,s->s3->tmp.new_cipher,&md)

which is later used to

    tls12_get_sigandhash(p, pkey, md)

That function searches through table tls12_md[].  If I'm right so far,
I'm puzzled why it searches for NID_sha256 for the plain domain but
NID_sha1 for the www.

Odds are, I'm wrong and that's why I'm a bit stuck.

-- 
Cheers, Ralph.