SSL_CTX_set_verify uses the "wrong" certificate chain (cross signed certificate )

[Viktor Dukhovni]

On Sat, Oct 02, 2021 at 05:24:26PM +0200, Alex Robuchon wrote:

> The project has a callback function registered with SSL_CTX_set_verify
> and failed to verify DST Root CA X3 since it expired.

That happens when either:

    * The configured trust store (perhaps not the default system
      trust store) does not contain the ISRG root CA certificate

    * The version of OpenSSL used (perhaps indirectly via some
      library that is linked with an older OpenSSL) is 1.0.x
      rather than 1.1.0 or later.

> From what I understood about the let's encrypt certificate chain, R3 is
> cross signed and two chained could be built:
> 
>    - leaf <- R3 <- ISRG Root X1 <- DST Root CA X3 (self-signed)
>    - leaf <- R3 <- ISRG Root X1 (self-signed)

Yes.  To make things even more complex, a few sites also have an older
version of R3 that is directly signed by the DST root:

     - leaf <- R3 <- DST Root CA X3 (self-signed)

but that's far from common at this point.

> The servers by default return the first chain but from what I
> understand depending on the openssl version/flags it should  use the
> second path if ISRG Root X1 is in the store

Yes, see above.

> My config :
> openssl version : OpenSSL 1.1.1f  31 Mar 2020

Are you sure there are no other versions of OpenSSL also installed on
the system, and/or no other trust stores that the application in
question might be using instead of the default system store?

An "strace" of the process should show which shared objects it is
loading, and which cert store files it is opening...  Best to not
post the complete strace output here, it may contain sensitive
data.  Try to interpret it on your end and post your findings.

> ISRG Root X1 self signed is in my cert store.

This presumes a *singular* "trust store", but the application could
easily be looking elsewhere.

> So from what I understand, trusted first is default in this version of
> openssl and the second path should be taken.

Yes, per the pre-requisite conditions.

> For the record s_client can valide the second path on my machine :
> $ echo | openssl s_client -connect retouche-pro.ch:443 -name retouche-pro.ch -servername retouche-pro.ch
> CONNECTED(00000003)
> depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
> verify return:1
> depth=1 C = US, O = Let's Encrypt, CN = R3
> verify return:1
> depth=0 CN = retouche-pro.ch
> verify return:1

As expected for 1.1.1f and a trust store that contains the ISRG X1 root.

> So I suppose the problem relies on the use of openssl in the ruby
> libs.

Are the ruby libraries linked with OpenSSL 1.1.1?  Are they overriding
the default system trust store?

> However what I'm a bit surprised with is the fact that
> SSL_CTX_set_verify is called with the "wrong" certificate chain.

I think you mean the callback specified via that function is called
with unexpected chain elements, not the function itself...

> The documentation says it's going to be called for each certificate in
> the chain but do not specify if it's the chain specified by the server
> or the one built by openssl with trusted first algorithm.

Except when various earlier errors take place during chain construction,
those callbacks take place as part of signature verification on the
already constructed chain, so the callback normally sees the "trusted"
chain, not the server provided certificate "heap" (which may not be
linearly ordered).

> Is it normal to have the callback defined in SSL_CTX_set_verify to be
> called with the wrong chain ? Or do you think something is not configured
> correctly on these gems ?

See above, the callbacks report the constructed chain, except when some
error happens during chain construction.

-- 
    Viktor.