Sign without having the private key

[Timo Lange]

Hi all,

thanks for the input so far.
What I have been playing with is something like this:
RSA_METHOD *meth = RSA_meth_dup(RSA_get_default_method());
RSA_meth_set1_name(meth, "myrsa");
RSA_meth_set_sign(meth, sign);
RSA_meth_set_verify(meth, verify);
RSA_set_default_method(meth);

with sign and verify being functions that I implemented.
But non of these two is ever invoked.
If I also set the init and final callbacks, they are properly executed.

Any ideas?

Br, Timo

________________________________
Von: openssl-users <openssl-users-bounces at openssl.org> im Auftrag von Dmitry Belyavsky <beldmit at gmail.com>
Gesendet: Montag, 11. Januar 2021 19:00
An: openssl-users at openssl.org <openssl-users at openssl.org>
Betreff: Re: Sign without having the private key

Dear Timo,

For 1.0* versions it was possible to provide custom RSA_METHOD and EC_METHOD and implement an IPC callback.
I think it still should work for 1.1.1

It may be also useful to take a look at the async API.

On Mon, Jan 11, 2021 at 6:56 PM Timo Lange <tiolangit at outlook.com<mailto:tiolangit at outlook.com>> wrote:
Hey all,

I have a question similar to http://openssl.6102.n7.nabble.com/private-key-not-available-for-client-cert-cb-td79369.html, that I am actively following, but though it differs in detail.

What I want to achieve is the following:
My client applications runs inside a container and needs to establish a mutual TLS connection to a server.
The client certificate is available in the container.
The root certificate, as well as the client private key is not available inside the container, but stored in a HSM.
For sure the private key may never leave the HSM and also the root certificate should not.

The application cannot directly interfere with the HSM through standardized mechanisms as it is not accessible from inside the container.
For doing so a proprietary interprocess-communication is required.

I now want something like a "verify callback" and a "sign callback".

The "verify callback" would be needed in order to verify the server certificate against the root certificate. It seems to be easy using: https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_cert_verify_callback.html

I need the same, something like a "sign callback" also for the private key, when a signature is required during handshake. Such that requests from openSSL to sign something can be forwarded through the inter-process-communication into the HSM. So that the actual signing happens there.
This would only be required during handshake. For the actual encryption symmetric keys can be used, such that the encryption takes place in the openSSL library, not in the HSM.

I assume I need to write a custom ENGINE, but failed with all my approaches.

Can someone give me brief hint on where to start and which API to look at first?

Thanks a lot!
Timo



--
SY, Dmitry Belyavsky
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20210112/f8d772e7/attachment.html>