issue with X509_issuer_and_serial_hash returning different values under OpenSSL 3

[Matt Caswell]

On 08/03/2023 11:18, adv2011 at rustichelli.net wrote:
> 
> UPDATE: I now compile a version of the code that replaces all of the 
> pointers, but still I don't get the result as from OpenSSL 1.
> This is the current exceprt of interest... from here, I'm stuck:
> 
>      // cannot do this under OpenSSL 3: f = 
> X509_NAME_oneline(a->cert_info.issuer, NULL, 0);
>          f = X509_NAME_oneline(X509_get_issuer_name(a), NULL, 0);
>      [...]
>          if (!EVP_DigestUpdate
>          // cannot do this under OpenSSL 3: (ctx, (unsigned char 
> *)a->cert_info.serialNumber.data,
>                  (ctx, ASN1_STRING_data(X509_get_serialNumber(a)),
>          // OpenSSL 1: (unsigned long)a->cert_info.serialNumber.length))
>                  (unsigned 
> long)ASN1_STRING_length(X509_get_serialNumber(a))))
> 
> What am I doing wrong?

IIRC, I think the format of the output from X509_NAME_oneline may have 
changed subtly from 1.0.2 to 3.0 (although I don't think it did between 
1.1.1 and 3.0??). I don't remember the details. Anyway I'd start 
investigating further there. Compare the output from that function that 
you are seeing for the same certificate on the two different OpenSSL 
versions.

Matt


> 
> On 3/8/23 10:55, adv2011 at rustichelli.net wrote:
>> (reposted with the right subject, sorry)
>>
>> Hi all, I am starting to port some code to OpenSSL 3 (it's my first 
>> taste of it), and I'm stuck with a problem. I'm working under Ubuntu 22.
>>
>> I saw that the function X509_issuer_and_serial_hash doesn't return the 
>> same value it did before (though not for an obvious reason), and since 
>> that value is used by my software to identify some certificates 
>> against a DB, I need to replicate the old behaviour.
>>
>> To do so, I'm first trying to change the old function (from OpenSSL 
>> 1.1) so that it compiles under OpenSSL 3.
>>
>> Here, a is of type X509, I always accessed most data from pointers. 
>> Now that they are gone, how do I read the following information to 
>> obtain exactly the same data?
>>
>> - a->cert_info.issuer ...is it X509_get_issuer_name(a) exactly the same?
>>
>> - a->cert_info.serialNumber.data ?
>>
>> - a->cert_info.serialNumber.length ?
>>
>> For completeness, my first, very raw code follows, where you can see 
>> how I'd use the values.
>>
>> Thank you very much - Ubi
>>
>>
>> #if OPENSSL_VERSION_NUMBER >= 0x30000000L
>> #warning "I WILL HAVE MY LOCAL X509_issuer_and_serial_hash, UNDER 
>> OPENSSL 3"
>>
>> unsigned long custom_X509_issuer_and_serial_hash(X509 *a)
>> {
>>     unsigned long ret = 0;
>>     EVP_MD_CTX *ctx = EVP_MD_CTX_new();
>>     unsigned char md[16];
>>     char *f = NULL;
>>
>>     if (ctx == NULL)
>>         goto err;
>>         // cannot do this under OpenSSL 3 (code from v 1.1): f = 
>> X509_NAME_oneline(a->cert_info.issuer, NULL, 0);
>>         f = X509_NAME_oneline(X509_get_issuer_name(a), NULL, 0);
>>     if (f == NULL)
>>         goto err;
>>     if (!EVP_DigestInit_ex(ctx, EVP_md5(), NULL))
>>         goto err;
>>     if (!EVP_DigestUpdate(ctx, (unsigned char *)f, strlen(f)))
>>         goto err;
>>     if (!EVP_DigestUpdate
>>         // cannot do this under OpenSSL 3 (code from v 1.1): (ctx, 
>> (unsigned char *)a->cert_info.serialNumber.data,
>>         // ...but how do I get the data from here?
>>         (ctx, X509_get_serialNumber(a),
>>         // ...same problem here: how do I get the data length?
>>         (unsigned long)a->cert_info.serialNumber.length))
>>         goto err;
>>     if (!EVP_DigestFinal_ex(ctx, &(md[0]), NULL))
>>         goto err;
>>     ret = (((unsigned long)md[0]) | ((unsigned long)md[1] << 8L) |
>>            ((unsigned long)md[2] << 16L) | ((unsigned long)md[3] << 24L)
>>         ) & 0xffffffffL;
>>  err:
>>     OPENSSL_free(f);
>>     EVP_MD_CTX_free(ctx);
>>     return ret;
>> }
>>
>> #endif
>>
>>
> 
>