How to plug an external encryption to CMS_SignerInfo signing?

[Francesco Pretto]

Hi Dmitry,

thank you for the prompt answer. Are you able to provide me with a
link to an example of creating such engines that will fit this use
case? On my searches I was able to find staff like EVP_PKEY_METHOD[1]
but I wasn't able to use them for my purpose. Not assuming how stuff
works today, ideally for me such engine should work like this:
- I prepare an engine that is able to deal with private keys only, and
just for encryption;
- I create a fake EVP_PKEY* that will use this engine;
- I supply this private key to CMS_add1_signer(), together with the
usual/regular X509 certificate, parsed for example from DER/PEM.

Even if CMS_add1_signer()[2] is currently performing a validation of
the EVP_PKEY passed, it shouldn't be hard to patch to avoid doing such
a check, for example by supplying a newly created flag that specifies
to not do it.

If engines can't be operated this way, then I'm afraid they will still
require more boilerplate code than really necessary.

Cheers,
Francesco

[1] https://www.openssl.org/docs/man1.1.1/man3/EVP_PKEY_METHOD.html
[2] https://github.com/openssl/openssl/blob/d1fb6b481b1d70932a1435f83eae10cc68edbe36/crypto/cms/cms_sd.c#L269


On Sat, 24 Oct 2020 at 12:12, Dmitry Belyavsky <beldmit at gmail.com> wrote:
>
> Dear Francesco,
>
> On Sat, Oct 24, 2020 at 1:06 PM Francesco Pretto <ceztko at gmail.com> wrote:
>>
>> Hello,
>>
>> I'm trying to create a CMS context for subsequent export using
>> CMS_sign(). I add a signer using CMS_add1_signer() that allows me to
>> specify a X509 certificate and a hash function. I would like the CMS
>> context to perform hash computation and ANS1 structure filling, but I
>> want to delegate encryption to an external service, for example an
>> hardware encryption token (I'm assuming this is a very common use
>> case). At this point I'm in a stalemate since CMS_add1_signer() asks
>> me for a private EVP_PKEY that is compatible with the public key
>> present in the X509 certificate. No other function seems to exist to
>> create a CMS_SignerInfo by providing an external mechanism for
>> encryption.
>>
>> My hacky solution was to add a signer CMS_add1_signer() supplying the
>> public key stored in the X509 certificate in the place of the private
>> one. This passes internal checks of the function and allows me to
>> subsequently handle (manually) all the ANS1 structure filling and hash
>> computations. This is barely doable with public openssl API and still
>> requires a big rip-off of private openssl code (attached as a
>> standalone C++ class, if it can be useful for someone).
>>
>> My question is: is there an easier mechanism to plug a separate
>> encryption method when creating the CMS_SignerInfo structure and have
>> openssl do all the other dirty work for me? If so, is it possible to
>> do with openssl 1.1.0/1.1.1?
>
>
> Engines allow operating by private keys in such a manner.
> You have to reimplement all the callbacks dealing with private keys. Also, it's possible you have to write some wrappers for the functions dealing with public keys.
>
> For 3.0, the providers should do the same trick, I think.
>
> --
> SY, Dmitry Belyavsky