X509_verify_cert() rejects all trusted certs with "default" X509_VERIFY_PARAM

[Viktor Dukhovni]

On Fri, May 28, 2021 at 01:30:14PM +0200, Graham Leggett via openssl-users wrote:

> While running code that calls X509_verify_cert(), the trusted root
> certificates (“BEGIN TRUSTED CERTIFICATE”) loaded into the
> verification are failing verification with “certificate rejected”:

Typically, certififcates in the OpenSSL trust stores used by most users
aren't wrapped up as "TRUSTED CERTIFICATES" that are annotated with
explicit trust EKUs.  What sort of trust store are you using that has
these annotations?

Can you be more explicit about the "default" X509_VERIFY_PARAM?  Are you
referring to the default "purpose"?

If your root is CA is tagged with a restricted set of trust EKUs,
verification will only succeed for a purpose that matches one of
those trust EKUs.

>         for (i = 0; i < sk_ASN1_OBJECT_num(ax->trust); i++) {
>             ASN1_OBJECT *obj = sk_ASN1_OBJECT_value(ax->trust, i);
>             int nid = OBJ_obj2nid(obj);
> 
>             if (nid == id || (nid == NID_anyExtendedKeyUsage &&
>                 (flags & X509_TRUST_OK_ANY_EKU)))
>                 return X509_TRUST_TRUSTED;
>         }
> 
> We iterate through the above loop twice for our root certificate, once with a nid of:
> 
> (lldb) print OBJ_nid2sn(nid)
> (const char *) $2 = 0x000000010067b13d “emailProtection"
> 
> and a second time with a nid of:
> 
> (lldb) print OBJ_nid2sn(nid)
> (const char *) $3 = 0x000000010067b0d2 “serverAuth"

Looks like your CA cert is annotated with "emailProtection" and
"serverAuth", and so can only be used to verify TLS server and
SMIME certficates, which don't match the "default" (unrestricted)
purpose.

> Neither “emailProtection” nor “serverAuth” are equal to
> “anyExtendedKeyUsage”, and so we drop to this line which triggers the
> rejection of our root certificate:

Specify a matching purpose, or use a root CA that is not annotated
with a limited set of trust EKUs.

> Can anyone explain why openssl would reject this certificate?
> 
> I am using the “default” X509_VERIFY_PARAM.

If you're using verify(1), you can set the purpose via the
"-purpose" option to one of:

    - any
    - crlsign
    - nssslserver
    - ocsphelper
    - smimeencrypt
    - smimesign
    - sslclient
    - sslserver
    - timestampsign

> Alas the source code apps/verify.c makes no attempt to set the trust
> parameter,

This is set indirectly via the "-purpose" option.

-- 
    Viktor.