Is there a simple implementation of hooking external private key with openssl-3.0 API?
Matt Caswell
matt at openssl.org
Mon Apr 11 16:31:00 UTC 2022
On 11/04/2022 16:53, Alon Bar-Lev wrote:
> On Mon, Apr 11, 2022 at 11:52 AM Matt Caswell <matt at openssl.org> wrote:
>>
>>
>>
>> On 10/04/2022 19:18, Alon Bar-Lev wrote:
>>> Hello,
>>>
>>> I am trying to migrate to openssl-3.0 API, it seems to be very
>>> complicated to hook primitive private key usage to a custom function.
>>> This is required, for example, to use private keys that reside on
>>> hardware devices or when an application wishes to externalize private
>>> key operations to other programs using IPC.
>>>
>>> I hope I am wrong but it seems like an entirely new provider must be
>>> implemented with complete reimplementation of the default providers,
>>> including serialization, padding etc... While in openssl-0/1 it was
>>> quite easy.
>>>
>>
>> You will need to implement a provider, and include a key manager plus an
>> implementation of whatever operations you want to support, e.g.
>> signature or asymcipher. Typically where a private key resides on a
>> hardware device then you don't need to support
>> serialization/deserialization because the keys can't be
>> serialized/deserialized anyway. If you do want to support that then the
>> key manager just needs to be able to import or export keys using the
>> standard parameters for the algorithm and it will automatically be able
>> to use the default provider's encoders and decoders. Support for key
>> generation is also probably optional. You would need to support any
>> padding that you need - that's considered part of the low level
>> algorithm implementation.
>>
>
> Thank you Matt,
> I am aware I can implement new three providers from scratch.
> However, I was hopping you will show me a way to cascade the existing
> providers just like we have done in the past with the RSA_METHOD.
It is actually technically possible to cascade providers. There is an
example of this happening in the test code. We have a filter provider
which "wraps" the default provider and modifies its behaviour. See:
https://github.com/openssl/openssl/blob/master/test/filterprov.c
However, I'm not sure that it helps you in this particular instance.
We also have instances of "composite" algorithms, e.g. where a high
level algorithm makes use of some other low level algorithm (consider a
DigestSign operation which uses RSA for signing, but also uses some
digest algorithm as part of its work).
> I would like the exact behavior of the existing providers while
> overriding the low level RSA operations.
The problem is that we don't currently expose the low level RSA
operations as provider operations. Feasibly we could have two RSA
implementations - a "low" level one and a "high" level one, where the
high level one does all padding etc, and the "low" level one does the
actual raw RSA work. The high level one could then call the low level
one. If we had that then it would probably be feasible for you to hook
in an alternative implementation for just the low level pieces.
That would require some refactoring though.
> Are you opened for a discussion for improving this?
Absolutely. I'd suggest you raise an issue on github to discuss it.
I'm sure someone else raised a similar issue some while ago and we had
some similar discussions, but I've gone searching for it and couldn't
find it. I did come across this issue though which touches on some
similar topics:
https://github.com/openssl/openssl/issues/17187
Matt
>
> [1] https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/xkey_provider.c
> [2] https://github.com/OpenVPN/openvpn/blob/master/src/openvpn/xkey_helper.c
>
>>
>>> I wrote a testcase program using openssl-1 APIs[1] which also works
>>> using openssl-3, in this testcase I prepare a new RSA method based on
>>> the default method, hook the private operations and then hook the RSA
>>> object to use the custom method.
>>>
>>> I am looking for a way to implement the __hook_evp_pkey function in
>>> openssl-3 api, so that when a private key operation is executed on the
>>> EVP_PKEY or EVP_PKEY_CTX a custom callback will be executed while
>>> public key operation continue to be executed normally.
>>>
>>> While looking into the existing RSA providers I can see that the
>>> providers continue to use the deprecated RSA_* functions with the
>>> following comment:
>>>
>>> /*
>>> * RSA low level APIs are deprecated for public use, but still ok for
>>> * internal use.
>>> */
>>>
>>> This is exactly what I need... :) To have the RSA low level API be
>>> redirected back to the application so that I can enjoy the default
>>> implementation of signature/rsa_sig.c padding etc while being able to
>>> override the private encrypt. But these low level functions are hidden
>>> from the user.
>>
>> As the comment says, RSA low level APIs are deprecated. Deprecated does
>> *not* mean removed. So you can still use them for now, although expect
>> them to be removed from some future version of OpenSSL.
>
> Marking deprecated APIs is announcing your intentions and gives enough
> time for everyone to workout the gaps (if any).
> I believe there is a gap which will introduce a great burden for
> developers in the existing design, I would like to work with you to
> reach similar solution we had in prior openssl versions by leveraging
> the current provider approach and create a reference implementation
> similar to what I've provided.
> If I understand the design correctly the missing bits are the ability
> to cascade a provider and access low level primitives, maybe as its
> own provider.
>
>> Matt
>>
>
> Thanks,
> Alon
>
>>>
>>> Can anyone help us to create a testcase of openssl-3? This will help
>>> many applications such as opensc/libp11 opensc/pkcs11-helper openvpn
>>> and probably more.
>>>
>>> For your convenience, you may find the program here[1].
>>>
>>> Regards,
>>> Alon Bar-Lev
>>>
>>> [1] https://github.com/alonbl/openssl-external/blob/master/example.c
>>>
>>> ---
>>>
>>> #include <openssl/err.h>
>>> #include <openssl/evp.h>
>>> #include <openssl/pem.h>
>>> #include <openssl/rsa.h>
>>> #include <string.h>
>>> #include <stdio.h>
>>>
>>> static RSA_METHOD *__example_rsa_method;
>>> static int __example_rsa_index;
>>>
>>> static int __example_rsa_priv_enc(int flen, const unsigned char *from,
>>> unsigned char *to, RSA *rsa, int padding) {
>>> const RSA_METHOD *rsa_method = NULL;
>>> int ret = -1;
>>>
>>> if ((rsa_method = RSA_get_method(rsa)) == NULL) {
>>> goto cleanup;
>>> }
>>>
>>> /*
>>> * Do it.
>>> */
>>> printf("ENCRYPT\n");
>>> memset(to, 0, flen);
>>> ret = 1;
>>>
>>> cleanup:
>>>
>>> return ret;
>>> }
>>>
>>> static int __example_rsa_priv_dec(int flen, const unsigned char *from,
>>> unsigned char *to, RSA *rsa, int padding) {
>>> const RSA_METHOD *rsa_method = NULL;
>>> int ret = -1;
>>>
>>> if ((rsa_method = RSA_get_method(rsa)) == NULL) {
>>> goto cleanup;
>>> }
>>>
>>> /*
>>> * Do it.
>>> */
>>> printf("DECRYPT\n");
>>> memset(to, 0, flen);
>>> ret = 1;
>>>
>>> cleanup:
>>>
>>> return ret;
>>> }
>>>
>>>
>>> static int __prepare_method(void) {
>>> int ret = 0;
>>>
>>> if ((__example_rsa_method =
>>> RSA_meth_dup(RSA_get_default_method())) == NULL) {
>>> goto cleanup;
>>> }
>>>
>>> if (!RSA_meth_set1_name(__example_rsa_method, "example")) {
>>> goto cleanup;
>>> }
>>>
>>> if (!RSA_meth_set_priv_dec(__example_rsa_method, __example_rsa_priv_dec)) {
>>> goto cleanup;
>>> }
>>>
>>> if (!RSA_meth_set_priv_enc(__example_rsa_method, __example_rsa_priv_enc)) {
>>> goto cleanup;
>>> }
>>>
>>> if ((__example_rsa_index = RSA_get_ex_new_index(0, "example",
>>> NULL, NULL, NULL)) == -1) {
>>> goto cleanup;
>>> }
>>>
>>> ret = 1;
>>>
>>> cleanup:
>>>
>>> return ret;
>>> }
>>>
>>> static int __free_method(void) {
>>> RSA_meth_free(__example_rsa_method);
>>> }
>>>
>>> static int __hook_evp_pkey(EVP_PKEY *evp_pkey) {
>>>
>>> RSA *rsa = NULL;
>>> int ret = 0;
>>>
>>> /*
>>> * Hook private key methods
>>> */
>>>
>>> if (EVP_PKEY_id(evp_pkey) != EVP_PKEY_RSA) {
>>> goto cleanup;
>>> }
>>>
>>> if ((rsa = EVP_PKEY_get1_RSA(evp_pkey)) == NULL) {
>>> goto cleanup;
>>> }
>>>
>>> if (!RSA_set_method(rsa, __example_rsa_method)) {
>>> goto cleanup;
>>> }
>>>
>>> if (!RSA_set_ex_data(rsa, __example_rsa_index, "mystate")) {
>>> goto cleanup;
>>> }
>>>
>>> if (EVP_PKEY_set1_RSA(evp_pkey, rsa) != 1) {
>>> goto cleanup;
>>> }
>>>
>>> ret = 1;
>>>
>>> cleanup:
>>>
>>> RSA_free(rsa);
>>>
>>> return ret;
>>> }
>>>
>>> const static char *pem = (
>>> "-----BEGIN CERTIFICATE-----\n"
>>> "MIIFMDCCBBigAwIBAgISA6sbShb1HQ3TpSVvhSPOS4JJMA0GCSqGSIb3DQEBCwUA\n"
>>> "MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD\n"
>>> "EwJSMzAeFw0yMjAzMTAxNzQ4MDdaFw0yMjA2MDgxNzQ4MDZaMBoxGDAWBgNVBAMT\n"
>>> "D210YS5vcGVuc3NsLm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB\n"
>>> "AMZvA0BbvdyVc+06j5e5k6dUr8gqL0KZw0w4xJ0QD6jD/o+czNEMz13YDxuZ5utL\n"
>>> "YGq8uohlK8l2DWqvDfGfm1T4VYQhD2z0Ky0JDTsxDIb5i6kKA+o2j2VPAivfMkBp\n"
>>> "f47rLITa4vqZ8/aro3E0ZVWfbpOOGASteM/g9mLEpRLJQA2/o4uu9xLCsyJkLG8F\n"
>>> "8eTCHUJ8388ZO/3fv8LnN1+/WwciSYcZcZNN44OsrgLNoLh6dzSY+oNZyVGdqxUy\n"
>>> "ZSO2dURx4/28w26RLzXFnGOZinupE6KoVhCHHM0Wqx7YkfudymzwBCPP3+X4Hkab\n"
>>> "1gkZZO9wTpRKrhuW3XtaBMkCAwEAAaOCAlYwggJSMA4GA1UdDwEB/wQEAwIFoDAd\n"
>>> "BgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAdBgNV\n"
>>> "HQ4EFgQUW/ht3YVQnVmfAWGArMLkgIyUFNYwHwYDVR0jBBgwFoAUFC6zF7dYVsuu\n"
>>> "UAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsGAQUFBzABhhVodHRwOi8v\n"
>>> "cjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6Ly9yMy5pLmxlbmNyLm9y\n"
>>> "Zy8wJwYDVR0RBCAwHoIPbXRhLm9wZW5zc2wub3JnggtvcGVuc3NsLm9yZzBMBgNV\n"
>>> "HSAERTBDMAgGBmeBDAECATA3BgsrBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIBFhpo\n"
>>> "dHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCCAQMGCisGAQQB1nkCBAIEgfQEgfEA\n"
>>> "7wB2AEHIyrHfIkZKEMahOglCh15OMYsbA+vrS8do8JBilgb2AAABf3Uo37wAAAQD\n"
>>> "AEcwRQIhAMDDz1KXMWXblh9maYNLF6vlZOcSXNlp3RgxJhRBYhACAiBB8mU+mqDa\n"
>>> "8RNog7zLQq3426vcfH4r1wufDnQ0su3GyQB1ACl5vvCeOTkh8FZzn2Old+W+V32c\n"
>>> "YAr4+U1dJlwlXceEAAABf3Uo4ZoAAAQDAEYwRAIgVD5+n6KMePTQF2GN4ZKIE8Oz\n"
>>> "lzZPeY90EPY5APu3ZrECIE4HWJ/ZQ/qZ3/7x4Vo+1a1gPoPBM4rsh3d3ormsrkiW\n"
>>> "MA0GCSqGSIb3DQEBCwUAA4IBAQA+TYBjasfMBLlXbwNdYGaVtfbBKyPPhHFHOqi2\n"
>>> "iJfdRnx2Z/KS0gmBisD6SS62dKAjHrUy4wSfRTSpAHAOvo3n7BuYSE+3HIYwyFpB\n"
>>> "P54tJTiEYiAHJvWsPRl8rEqxzYnaR+u0zdKL7Wauk9gJMwGX6fdwhhAgS5WmBe05\n"
>>> "O4mf8jdWgtLQYxS/kvQYrNDTTBA6J+UoNM/JIxXENMh2/6zcFgy0D2ewr0NjAYWU\n"
>>> "Ylf5jVgHjxleRSGnbt19v8dwZcHyBhq+vdndQt0sDQl7aoNEKiCXU2/y0KAtDjGF\n"
>>> "tsFic9a3WMzENWlAUcfACBaGx8Qm9161M9BO396tgHavQLQ8\n"
>>> "-----END CERTIFICATE-----\n"
>>> );
>>>
>>> int main(void) {
>>> BIO *bio = NULL;
>>> X509 *x509 = NULL;
>>> EVP_PKEY *evp_pkey = NULL;
>>> EVP_PKEY_CTX *evp_pkey_ctx = NULL;
>>> int ret = 1;
>>>
>>> if (__prepare_method() < 1) {
>>> goto cleanup;
>>> }
>>>
>>> if ((bio = BIO_new_mem_buf(pem, strlen(pem))) == NULL) {
>>> goto cleanup;
>>> }
>>>
>>> if ((x509 = PEM_read_bio_X509(bio, NULL, NULL, NULL)) == NULL) {
>>> goto cleanup;
>>> }
>>>
>>> if ((evp_pkey = X509_get_pubkey(x509)) == NULL) {
>>> goto cleanup;
>>> }
>>>
>>> if (__hook_evp_pkey(evp_pkey) < 1) {
>>> goto cleanup;
>>> }
>>>
>>> #if OPENSSL_VERSION_NUMBER < 0x30000000
>>> if ((evp_pkey_ctx = EVP_PKEY_CTX_new(evp_pkey, NULL)) == NULL) {
>>> goto cleanup;
>>> }
>>> #else
>>> if ((evp_pkey_ctx = EVP_PKEY_CTX_new_from_pkey(NULL, evp_pkey,
>>> NULL)) == NULL) {
>>> goto cleanup;
>>> }
>>> #endif
>>>
>>> if (EVP_PKEY_sign_init(evp_pkey_ctx) < 1) {
>>> goto cleanup;
>>> }
>>>
>>> {
>>> char buf[1024];
>>> size_t len = sizeof(buf);
>>> if (EVP_PKEY_sign(evp_pkey_ctx, buf, &len, "Test", 4) < 1) {
>>> goto cleanup;
>>> }
>>> }
>>>
>>> ret = 0;
>>>
>>> cleanup:
>>>
>>> ERR_print_errors_fp(stdout);
>>>
>>> EVP_PKEY_CTX_free(evp_pkey_ctx);
>>> EVP_PKEY_free(evp_pkey);
>>> X509_free(x509);
>>> BIO_free(bio);
>>>
>>> __free_method();
>>>
>>> return ret;
>>> }
>>>
>
More information about the openssl-users
mailing list