Custom Sign Callback for ECC TLS Handshake

[Matt Caswell]

On 16/06/2023 10:19, nocommercials at t-online.de wrote:
> Hey there,
> I currently create a program that does a TLS handshake using ECC 
> certificates.
> The private key is not accessible to that program so that I could load 
> it into openssl and hence the sign step during handshake has to be 
> offloaded to another component in the system.
> For that purpose I do not find a callback or something.
> 
> What would be the correct way to achive this?
> The way to have the data signed is non-standard, so I cannot use some 
> standard such as PKCS11 or something.
> I am glad to get just pointed to where to start with, rest I can figure out.

I don't think there is an easy way to achieve this.

It could be done by writing a custom provider - but it would be quite a 
bit of work to get it right. There's some provider documentation here:

https://www.openssl.org/docs/man3.1/man7/provider.html

You'd need to implement signature support:

https://www.openssl.org/docs/man3.1/man7/provider-signature.html

and an associated key manager:

https://www.openssl.org/docs/man3.1/man7/provider-keymgmt.html

There's a "toy" provider here:

https://github.com/provider-corner/vigenere


Matt