X509_verify_cert() rejects all trusted certs with "default" X509_VERIFY_PARAM

[Graham Leggett]

Hi all,

While running code that calls X509_verify_cert(), the trusted root certificates (“BEGIN TRUSTED CERTIFICATE”) loaded into the verification are failing verification with “certificate rejected”:

2: CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE: verify failed: certificate rejected

The code path we’re following looks like this:

  * frame #0: 0x000000010060b808 libcrypto.3.dylib`obj_trust(id=910, x=0x000000010096da70, flags=8) at x509_trs.c:271:17
    frame #1: 0x000000010060b672 libcrypto.3.dylib`X509_check_trust(x=0x000000010096da70, id=0, flags=0) at x509_trs.c:72:16
    frame #2: 0x000000010061207b libcrypto.3.dylib`check_trust(ctx=0x00000001009fe5b0, num_untrusted=2) at x509_vfy.c:776:17
    frame #3: 0x0000000100610e7e libcrypto.3.dylib`build_chain(ctx=0x00000001009fe5b0) at x509_vfy.c:3124:37
    frame #4: 0x000000010060d655 libcrypto.3.dylib`verify_chain(ctx=0x00000001009fe5b0) at x509_vfy.c:216:15
    frame #5: 0x000000010060d27b libcrypto.3.dylib`X509_verify_cert(ctx=0x00000001009fe5b0) at x509_vfy.c:295:15

In X509_check_trust() we get to this line of code which appears to ask “trust roots with NID_anyExtendedKeyUsage":

https://github.com/openssl/openssl/blob/master/crypto/x509/x509_trs.c#L72

int X509_check_trust(X509 *x, int id, int flags)
{
    X509_TRUST *pt;
    int idx;

    /* We get this as a default value */
    if (id == X509_TRUST_DEFAULT)
        return obj_trust(NID_anyExtendedKeyUsage, x,
                         flags | X509_TRUST_DO_SS_COMPAT);

This leads us to this code here:

https://github.com/openssl/openssl/blob/master/crypto/x509/x509_trs.c#L268


        for (i = 0; i < sk_ASN1_OBJECT_num(ax->trust); i++) {
            ASN1_OBJECT *obj = sk_ASN1_OBJECT_value(ax->trust, i);
            int nid = OBJ_obj2nid(obj);

            if (nid == id || (nid == NID_anyExtendedKeyUsage &&
                (flags & X509_TRUST_OK_ANY_EKU)))
                return X509_TRUST_TRUSTED;
        }

We iterate through the above loop twice for our root certificate, once with a nid of:

(lldb) print OBJ_nid2sn(nid)
(const char *) $2 = 0x000000010067b13d “emailProtection"

and a second time with a nid of:

(lldb) print OBJ_nid2sn(nid)
(const char *) $3 = 0x000000010067b0d2 “serverAuth"

Neither “emailProtection” nor “serverAuth” are equal to “anyExtendedKeyUsage”, and so we drop to this line which triggers the rejection of our root certificate:

        return X509_TRUST_REJECTED;

I am lost - I can fully understand what the code is doing, but I can’t see why openssl only trusts certs with “anyExtendedKeyUsage”.

Can anyone explain why openssl would reject this certificate?

I am using the “default” X509_VERIFY_PARAM.

Alas the source code apps/verify.c makes no attempt to set the trust parameter, and the docs for X509_VERIFY_PARAM_set_trust() say "sets the trust setting in param to trust” but doesn’t explain what possible values there are for “trust” or their effect.

Regards,
Graham
—