ODP: CSR creation using pkcs11 dynamic engine

[Piotr Lobacz]

Hi Selva,
i have found the cause of the problem. It is the HasNoPrivateKey function which is in dotnet OpenSsl ./src/Native/Unix/System.Security.Cryptography.Native/pal_rsa.c implementation. The crux of the problem is in these lines:

    // The method has descibed itself as having the private key external to the structure.
    // That doesn't mean it's actually present, but we can't tell.
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wcast-qual"
    if (RSA_meth_get_flags((RSA_METHOD*)meth) & RSA_FLAG_EXT_PKEY)
#pragma clang diagnostic pop
    {
        return 0;
    }

For some reason i suspect that i do not have this RSA_FLAG_EXT_PKEY flag set. Btw. when i am calling ENGINE_load_private_key method, the arguments i'm passing are the engine and the key. Both ui_method and callback_data are passed as NULLs. I talked with guys from dotnet in here https://github.com/dotnet/runtime/issues/53345 and i need to check RSA flags.But the thing that puzzles me is that shouldn't that flag be inserted by the engine during the call of ENGINE_load_private_key?

BR
Piotr
________________________________
Od: Selva Nair <selva.nair at gmail.com>
Wysłane: piątek, 28 maja 2021 20:47
Do: Piotr Lobacz <piotr.lobacz at softgent.com>
DW: openssl-users at openssl.org <openssl-users at openssl.org>
Temat: Re: CSR creation using pkcs11 dynamic engine

Hi,

On Fri, May 28, 2021 at 1:44 PM Piotr Lobacz <piotr.lobacz at softgent.com> wrote:
>
> Ok, i have found out that dotnet OpenSsl library has it's own code for verification is key private. For this it needs the whole data of private key from which this method:
>
> static int HasNoPrivateKey(RSA* rsa)
>
>  which is in ./src/Native/Unix/System.Security.Cryptography.Native/pal_rsa.c of dotnet verifies occurence of all private parameters. Unfortunately from what i know private keys are not extractable from tokens because of CKA_EXTRACTABLE=false parameter.
>
> Correct me if i'm wrong but from what i know about openssl, when i'm switching to a closed engine the whole cryptography is being made by the engine module. I think that there should be some other method verifing if key is private. Maybe somebody could give me a hint?

Its not verifying, but signing operation that is failing. This sounds
like something wrong in the way you are using the dotnet interface or
possibly a bug in (or limitation of) that implementation itself. Like
its not meant to be used when keys are "external".  I have no idea
having never used C#.

But you are right, when the private key is loaded through the pkcs11
engine the key is external (can stay non-extractable), and the signing
operation gets delegated to the engine. Are you sure that the pkey
returned by the ENGINE_get_private_key() and rsa handle generated from
that pkey are valid? I see no error checks in your code unless dotnet
will automatically trigger exceptions on error.

You may get more relevant help in the dotnet community.


Selva
[https://softgent.com/wp-content/uploads/2020/01/Zasob-14.png]<https://www.softgent.com>

Softgent Sp. z o.o., Budowlanych 31d, 80-298 Gdansk, POLAND

KRS: 0000674406, NIP: 9581679801, REGON: 367090912

www.softgent.com

Sąd Rejonowy Gdańsk-Północ w Gdańsku, VII Wydział Gospodarczy Krajowego Rejestru Sądowego

KRS 0000674406, Kapitał zakładowy: 25 000,00 zł wpłacony w całości.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20210528/eecaaacd/attachment.html>