Openssl v1.1.1d: pubkey_cb() returning SUCCESS for x509_pubkey_decode() Failure

[Matt Caswell]

On 28/03/2022 13:11, Brahmaji K wrote:
> Hi Team,
> 
> I'm trying to store the invalid EC certificate as a negative test for my 
> application. My application calls the X509_STORE_load_locations() to 
> load the certificate from a specific path. For invalid EC certificate it 
> is expected to FAIL but it is returning the SUCCESS.
> 
> I have done some debugging and found the following:
> 
> pubkey_cb() calls x509_pubkey_decode(), the x509_pubkey_decode() has the 
> check only for -1 as shown below:
> 
>   46         if (x509_pubkey_decode(&pubkey->pkey, pubkey) == -1)
>   47             return 0;
> 
> But x509_pubkey_decode() can return zero ("0") also in the failure case. 
> Is there any intention to have the above check? or is it a known issue?

The fuller context around these lines gives the answer:

         /*
          * Opportunistically decode the key but remove any non fatal errors
          * from the queue. Subsequent explicit attempts to decode/use 
the key
          * will return an appropriate error.
          */
         ERR_set_mark();
         if (x509_pubkey_decode(&pubkey->pkey, pubkey) == -1)
             return 0;
         ERR_pop_to_mark();

The -1 return from x509_pubkey_decode() indicates a fatal error (e.g. a 
malloc failure). A 0 error return is considered non-fatal (e.g. an 
unrecognised key) and the code is deliberately written to continue in 
the case of non-fatal errors.

Matt



> 
> Call trace for more information:
> #0  x509_pubkey_decode (ppkey=ppkey at entry=0x123d5ffd0,
>      key=key at entry=0x123d5ffc0) at crypto/x509/x_pubkey.c:125
> #1  0x00000001201f5888 in pubkey_cb (operation=operation at entry=5,
>      pval=pval at entry=0x123d5fe40, it=it at entry=0x1206026c8,
>      exarg=exarg at entry=0x0) at crypto/x509/x_pubkey.c:46
> #2  0x0000000120152bac in asn1_item_embed_d2i (pval=pval at entry=0x123d5fe40,
>      in=in at entry=0xffff654288, len=0, it=0x1206026c8, tag=<optimized out>,
>      tag at entry=-1, aclass=<optimized out>, aclass at entry=0,
>      opt=<optimized out>, ctx=ctx at entry=0xffff6546a0, depth=<optimized out>,
>      depth at entry=2) at crypto/asn1/tasn_dec.c:413
> #3  0x0000000120153660 in asn1_template_noexp_d2i (val=0x123d5fe40,
>      in=0xffff6543a0, len=322, tt=0x120618ad8, opt=<optimized out>,
>      ctx=0xffff6546a0, depth=<optimized out>) at crypto/asn1/tasn_dec.c:624
> #4  0x0000000120153968 in asn1_template_ex_d2i (val=0x123d5fe40,
>      in=in at entry=0xffff6543a0, inlen=<optimized out>, 
> tt=tt at entry=0x120618ad8,
>      opt=<optimized out>, ctx=ctx at entry=0xffff6546a0, depth=depth at entry=2)
>      at crypto/asn1/tasn_dec.c:499
> #5  0x0000000120153064 in asn1_item_embed_d2i (pval=pval at entry=0xffff654490,
>      in=in at entry=0xffff654488, len=322, it=0x1206027f8, tag=<optimized out>,
>      tag at entry=-1, aclass=<optimized out>, aclass at entry=0,
>      opt=<optimized out>, ctx=ctx at entry=0xffff6546a0, depth=2, 
> depth at entry=1)
>      at crypto/asn1/tasn_dec.c:363
> #6  0x0000000120153660 in asn1_template_noexp_d2i (val=0xffff654490,
>      in=0xffff6545a0, len=507, tt=0x120618970, opt=<optimized out>,
>      ctx=0xffff6546a0, depth=<optimized out>) at crypto/asn1/tasn_dec.c:624
> #7  0x0000000120153968 in asn1_template_ex_d2i (val=0x123d5fdf0,
>      in=in at entry=0xffff6545a0, inlen=<optimized out>, 
> tt=tt at entry=0x120618970,
>      opt=<optimized out>, ctx=ctx at entry=0xffff6546a0, depth=depth at entry=1)
>      at crypto/asn1/tasn_dec.c:499
> #8  0x0000000120153064 in asn1_item_embed_d2i (pval=pval at entry=0x123d5ef40,
>      in=0xffff654710, len=507, it=it at entry=0x1206027c0, tag=<optimized out>,
>      aclass=<optimized out>, opt=<optimized out>, ctx=0xffff6546a0, depth=1,
>      depth at entry=0) at crypto/asn1/tasn_dec.c:363
> #9  0x0000000120153ac8 in ASN1_item_ex_d2i (pval=0x123d5ef40,
>      in=<optimized out>, len=<optimized out>, it=0x1206027c0,
>      tag=<optimized out>, aclass=<optimized out>, opt=<optimized out>,
>      ctx=<optimized out>) at crypto/asn1/tasn_dec.c:124
> #10 0x0000000120153b60 in ASN1_item_d2i (pval=0x123d5ef40,
>      in=<optimized out>, len=<optimized out>, it=<optimized out>)
>      at crypto/asn1/tasn_dec.c:114
> #11 0x00000001202cd744 in PEM_X509_INFO_read_bio (bp=0x123d5ee50, sk=0x0,
>      cb=0, u=0x1204ca0c0) at crypto/pem/pem_info.c:195
> #12 0x0000000120285fe8 in X509_load_cert_crl_file (file=<optimized out>,
>      ctx=0x123d5f3c0, type=<optimized out>) at crypto/x509/by_file.c:202
> #13 X509_load_cert_crl_file (ctx=0x123d5f3c0, file=<optimized out>,
>      type=<optimized out>) at crypto/x509/by_file.c:188
> #14 0x00000001202861c8 in by_file_ctrl (ctx=<optimized out>,
>      cmd=<optimized out>, argp=<optimized out>, argl=<optimized out>,
>      ret=<optimized out>) at crypto/x509/by_file.c:64
> #15 0x00000001201e60ec in X509_STORE_load_locations (ctx=0x123d5f2c0,
>      file=0xffff654868 "/certs/test.crt", path=0x0)
>      at crypto/x509/x509_d2.c:44