openssl 3.0 - id2_x509() now fails

[Tomas Mraz]

On Mon, 2021-08-09 at 09:48 -0400, Ken Goldman wrote:
> On 8/9/2021 3:50 AM, Tomas Mraz wrote:
> > On Fri, 2021-08-06 at 18:06 -0400, Ken Goldman wrote:
> > > On 8/6/2021 1:11 PM, Ken Goldman wrote:
> > > > I have an application where I have to create a partial x509
> > > > certificate.  It gets sent to an HSM, which fills in the public
> > > > key
> > > > and signs it.
> > > > 
> > > > I was calling
> > > > 
> > > >       X509_new
> > > >       X509_set_version
> > > >       X509_set_issuer_name
> > > >       X509_get_notBefore
> > > >       X509_get_notAfter
> > > >       X509_set_subject_name
> > > >       X509_EXTENSION_create_by_OBJ
> > > > 
> > > > and then
> > > >       i2d_x509
> > > > to send the serialized partial certificate to the HSM.
> > > > 
> > > > This worked in 1.0.1, 1.0.2, 1.1.1, but fails in 3.0.0.
> > > > 
> > > > In debugging, even this fails.
> > > > 
> > > >       X509_new
> > > >       i2d_x509
> > > > 
> > > > Suggestions?
> > > 
> > > Following up, I found that just omitting the signature from the
> > > X509 structure causes i2d_x509 to fail.
> > > 
> > > I tried i2d_re_X509_tbs(), but it also failed.
> > 
> > I am afraid with the current 3.0 codebase there are not many
> > options
> > how to workaround apart from either signing the certificate with a
> > bogus key - if the HSM is able to re-sign such certificate.
> 
> My hope is that the maintainers will revert this change.  Perhaps
> they can write a new variant of i2d_x509 that requires the full
> certificate rather than change the existing API.
> 
> The i2d__re_x509_tbs() API seems promising (tbs is 'to be signed'),
> but it apparently is strict on what data must be there.
> 
> The HSM (TPM, ISO 11889) cannot change.  It expects a
> partial certificate.  It's API is already defined.
> 
> > Another (more complicated) option would be to define your own ASN.1
> > X509 structure where the signature would be optional and thus the
> > stricter encoder that is now in 3.0 codebase would allow encoding
> > the
> > incomplete certificate.
> 
> If you can post some hints on how to do this, I'll try it.
> 
> My alternative is to write the asn1 code from scratch, but I know
> how fragile that will be.

Why would you write asn1 code from scratch when OpenSSL has all the
APIs needed to create any ASN.1 structure.

Look at the crypto/x509/x_x509.c and make the signature optional. Of
course you would not be able to use the X509_set/get functions and
would have to also copy the X509_CINF definition. Which makes the
workaround quite complicated anyway.

-- 
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
                                              Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]