X509_verify_cert() rejects all trusted certs with "default" X509_VERIFY_PARAM

[Tomas Mraz]

Hello,

is this a regression when comparing with OpenSSL-1.1.1?

If so, it might be a good idea to report this as an issue to the
project in GitHub.

Tomas

On Fri, 2021-05-28 at 13:30 +0200, Graham Leggett via openssl-users
wrote:
> Hi all,
> 
> While running code that calls X509_verify_cert(), the trusted root
> certificates (“BEGIN TRUSTED CERTIFICATE”) loaded into the
> verification are failing verification with “certificate rejected”:
> 
> 2: CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE: verify
> failed: certificate rejected
> 
> The code path we’re following looks like this:
> 
>   * frame #0: 0x000000010060b808 libcrypto.3.dylib`obj_trust(id=910,
> x=0x000000010096da70, flags=8) at x509_trs.c:271:17
>     frame #1: 0x000000010060b672
> libcrypto.3.dylib`X509_check_trust(x=0x000000010096da70, id=0,
> flags=0) at x509_trs.c:72:16
>     frame #2: 0x000000010061207b
> libcrypto.3.dylib`check_trust(ctx=0x00000001009fe5b0,
> num_untrusted=2) at x509_vfy.c:776:17
>     frame #3: 0x0000000100610e7e
> libcrypto.3.dylib`build_chain(ctx=0x00000001009fe5b0) at
> x509_vfy.c:3124:37
>     frame #4: 0x000000010060d655
> libcrypto.3.dylib`verify_chain(ctx=0x00000001009fe5b0) at
> x509_vfy.c:216:15
>     frame #5: 0x000000010060d27b
> libcrypto.3.dylib`X509_verify_cert(ctx=0x00000001009fe5b0) at
> x509_vfy.c:295:15
> 
> In X509_check_trust() we get to this line of code which appears to
> ask “trust roots with NID_anyExtendedKeyUsage":
> 
> https://github.com/openssl/openssl/blob/master/crypto/x509/x509_trs.c#L72
> 
> int X509_check_trust(X509 *x, int id, int flags)
> {
>     X509_TRUST *pt;
>     int idx;
> 
>     /* We get this as a default value */
>     if (id == X509_TRUST_DEFAULT)
>         return obj_trust(NID_anyExtendedKeyUsage, x,
>                          flags | X509_TRUST_DO_SS_COMPAT);
> 
> This leads us to this code here:
> 
> https://github.com/openssl/openssl/blob/master/crypto/x509/x509_trs.c#L268
> 
> 
>         for (i = 0; i < sk_ASN1_OBJECT_num(ax->trust); i++) {
>             ASN1_OBJECT *obj = sk_ASN1_OBJECT_value(ax->trust, i);
>             int nid = OBJ_obj2nid(obj);
> 
>             if (nid == id || (nid == NID_anyExtendedKeyUsage &&
>                 (flags & X509_TRUST_OK_ANY_EKU)))
>                 return X509_TRUST_TRUSTED;
>         }
> 
> We iterate through the above loop twice for our root certificate,
> once with a nid of:
> 
> (lldb) print OBJ_nid2sn(nid)
> (const char *) $2 = 0x000000010067b13d “emailProtection"
> 
> and a second time with a nid of:
> 
> (lldb) print OBJ_nid2sn(nid)
> (const char *) $3 = 0x000000010067b0d2 “serverAuth"
> 
> Neither “emailProtection” nor “serverAuth” are equal to
> “anyExtendedKeyUsage”, and so we drop to this line which triggers the
> rejection of our root certificate:
> 
>         return X509_TRUST_REJECTED;
> 
> I am lost - I can fully understand what the code is doing, but I
> can’t see why openssl only trusts certs with “anyExtendedKeyUsage”.
> 
> Can anyone explain why openssl would reject this certificate?
> 
> I am using the “default” X509_VERIFY_PARAM.
> 
> Alas the source code apps/verify.c makes no attempt to set the trust
> parameter, and the docs for X509_VERIFY_PARAM_set_trust() say "sets
> the trust setting in param to trust” but doesn’t explain what
> possible values there are for “trust” or their effect.
> 
> Regards,
> Graham
> —
> 
> 
-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
                                              Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]