[openssl-users] RSA_method_set_sign

[Dr. Stephen Henson]

On Sat, Feb 25, 2017, Melvyn Sopacua wrote:

> Hi Stephen,
> 
> thank you for taking the time to answer.
> 
> On Saturday 25 February 2017 04:18:01 Dr. Stephen Henson wrote:
> > On Sat, Jan 14, 2017, Melvyn Sopacua wrote:
> > > This is caused by the code in [3].
> > > That file also shows the problem: OpenSSL itself has access to
> > > X509_SIG (and friends) internals as demonstrated in encode_pkcs1().
> > > But, I don't see any way to setup the same context(s) from outside
> > > OpenSSL. There's no X509_*_set_<anything> to setup the algorithm
> > > and parameters.
> > > 
> > > Am I missing something or is it simply no longer possible to
> > > implement these callbacks properly?
> > 
> > Can you give a pointer to the part that is causing problems?
> 
> The method I'm trying to port is this:
> https://github.com/melvyn-sopacua/qca/blob/openssl11-compat/plugins/qca-ossl/qca-ossl.cpp#L2745
> 
> > The rsa_sign interface is used where the only interface available is
> > passed the digest algorithm and the raw digest and it performs its
> > own formatting using DigestInfo etc.
> > 
> > If you don't want to do that then the rsa_priv_enc method is more
> > appropriate: it gets passed the block to encrypt (sign) and all the
> > DigestInfo formatting is handled by OpenSSL itself.
> 
> This may be a better approach in the long run. Thank you.
> 

Yes it looks like if you implement rsa_priv_enc instead the function becomes
much simpler and you just need the last part at around line 2814. You also have
to check the padding mode, all the X509_SIG stuff is then handle by OpenSSL
itself.

> > If you really need to it should be possible to set up or examine an
> > X509_SIG structure using the available APIs. For example to retieve
> > its fields you use X509_SIG_get0 and to set them X509_SIG_getm.
> 
> Well, that explains why I couldn't find it. I was looking for something 
> X509_SIG_*set* and never thought X509_SIG_getm() would be what I needed.
> 
> So:
> sig.algor= &algor;
> sig.digest = &digest;
> becomes
> X509_SIG_getm(sig, palg, pdigest);
> *palg = algor;
> *pdigest = digest;
> 
> And I'm guessing I have to free the structures retrieved by getm() if 
> they're not NULL. I may just wrap this in a X509_SIG_setup() function so 
> the freeing isn't forgotten.
> 

You can set the values in place using something like this:

unsigned char *tmps = NULL;
int tmpslen;
X509_SIG *sig = X509_SIG_new();
X509_ALGOR *alg;
ASN1_OCTET_STRING *digest;
X509_SIG_getm(sig, &alg, &digest);
X509_ALGOR_set0(alg, OBJ_nid2obj(type), V_ASN1_NULL, NULL);
ASN1_STRING_set(digest, m, m_len);
/* Allocate and encode */
tmpslen = i2d_X509_SIG(&sig, &tmps);
X509_SIG_free(sig);

Then the encoded structure is "tmpslen" bytes in the buffer "tmps" which you
have to free up after use with OPENSSL_free().

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org