provider not loaded on MacOS - Re: How to access keys on HW tokens via PKCS11 Provider?

[Tomas Mraz]

On Wed, 2023-09-06 at 18:32 +0200, David von Oheimb wrote:
> On 06.09.23 09:02, Tomas Mraz wrote:
>  
> > On Tue, 2023-09-05 at 23:53 +0200, David von Oheimb wrote:
> >  
> > > Yet with such a config file, as recommended by
> > > https://github.com/latchset/pkcs11-provider/blob/main/HOWTO.md,
> > > there are many pitfalls.
> > > 
> > > [...]
> > > 
> > > Moreover, looks like OpenSSL does not automatically load all
> > > providers listed in 
> > > 
> > > [provider_sect]
> > > default = default_sect
> > > pkcs11 = pkcs11_sect
> > > 
> > > but only "predefined" ones. At least, I still need to explicitly
> > > reference it on the command line, e.g.:
> > > 
> > > openssl req  -new -subj "/CN=x" -provider pkcs11 -key 
> > > "pkcs11:object=...;type=private" 
> > That is not intended. There should be no such distinction between
> > default and pkcs11 provider assuming both are activated in the
> > configuration.
> I see.
>  
> > It is either some misconfiguration or a bug in OpenSSL. Do you have
> > activate=1 in pkcs11_sect?
>  Yes, as recommended by that HOWTO.md.
>  
>  
> > Also, is the pkcs11 module configured correctly for the pkcs11
> > provider?
>  I believe so.
>  I've meanwhile tested with Linux, using (modulo the respective lib
> file path names) the same config file contents, and there it works.
>  
>  
> > If you try to strace the command without -provider pkcs11 do you
> > see any attempt to load the provider shared module?
> Good thought to try 'strace' instead of the hard-to-use and in this
> case not very useful OPENSSL_TRACE=CONF.
>  Yet I cannot reproduce the problem on Linux, while on MacOS, strace
> is not available.
>  Would be really good if the OpenSSL config module loader provided
> better tracing.
>  I had manually added some printfs to crypto/provider_core.c to find
> out that, for some reason,
>  provider_activate() and provider_init() only get called for
> "default", but not for "pkcs11":
> OPENSSL_TRACE=CONF apps/openssl x509 -noout -text -in
> "pkcs11:object=...;type=cert"
> 
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Configuration in section
> openssl_init
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Adding config module
> 'alg_section'
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Adding config module 'providers'
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Adding config module 'random'
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Loading providers module:
> section provider_sect
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Configuring provider pkcs11
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider command: module =
> /usr/local/lib/ossl-modules/pkcs11.dylib
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider command: pkcs11-module-
> path = /Library/OpenSC/lib/onepin-opensc-pkcs11.so
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: start section
> pkcs11_sect
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: module =
> /usr/local/lib/ossl-modules/pkcs11.dylib
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: pkcs11-module-
> path = /Library/OpenSC/lib/onepin-opensc-pkcs11.so
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: finish section
> pkcs11_sect
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Configuring provider default
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider command: activate = 1
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: start section
> default_sect
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: activate = 1
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Provider params: finish section
> default_sect
> provider_activate name = default
> provider_init name = default
> TRACE[80:A0:6D:F0:01:00:00:00]:CONF: Running module providers
> (provider_sect) returned 1
> Yet when I add  -provider pkcs11  to the command line, this output
> gets extended by:
> provider_activate name = pkcs11
> provider_init name = pkcs11
> module_path = (null)
> merged_path = /Users/david/openssl/providers/pkcs11.dylib
> and the provider loading works, making use, e.g, of 
> pkcs11-module-path = /Library/OpenSC/lib/onepin-opensc-pkcs11.so
> Maybe this related to the annoying fact that LD_LIBRARY_PATH does not
> work with MacOS, while DYLD_LIBRARY_PATH is a kind of replacement.
>     David

Not sure how LD_LIBRARY_PATH is related. It is not used when loading
the provider modules. 

Is the provider module path correct in the TRACE above? Could you try
this tracing on Linux to compare?

It is suspicious that there is no provider command activate=1 trace
line for the pkcs11 provider.

-- 
Tomáš Mráz, OpenSSL