[openssl-users] pkeyutl does not invoke hash?

[Blumenthal, Uri - 0553 - MITLL]

I’m not sure whether this is a bug (as I suspect – hence sending to
openssl-dev), or a poorly-documented “feature” (so copying to
openssl-users).

I am trying to use “openssl pkeyutl” to digitally sign (and verify) a file.

When the file size matches the size of the specified digest (32 bytes for
SHA256, 48 bytes for SHA384, etc), everything seems fine:
$ openssl pkeyutl -engine pkcs11 -keyform engine -sign -inkey
"pkcs11:object=SIGN%20key;object-type=private" -pkeyopt digest:sha384 -out
t384.ecdsa.sig -in t384.dat

engine "pkcs11" set.

PKCS#11 token PIN: 

$ openssl pkeyutl -engine pkcs11 -keyform engine -verify -pubin -inkey
"pkcs11:object=SIGN%20pubkey;object-type=public" -sigfile t384.ecdsa.sig -in
t384.dat

engine "pkcs11" set.

Signature Verified Successfully

$


However, if I try to feed it a larger file, it complains regardless of
whether I specify a hash-function to digest the file:
$ openssl pkeyutl -engine pkcs11 -keyform engine -sign -inkey
"pkcs11:object=SIGN%20key;object-type=private" -pkeyopt digest:sha1 -out
t2570.ecdsa.sig -in t2570.dat

engine "pkcs11" set.

PKCS#11 token PIN: 

Public Key operation error

140735120921424:error:80029001:PKCS11 library:func(41):Cancel:p11_ops.c:61:

$ openssl pkeyutl -engine pkcs11 -keyform engine -sign -inkey
"pkcs11:object=SIGN%20key;object-type=private" -pkeyopt digest:sha384 -out
t2570.ecdsa.sig -in t2570.dat

engine "pkcs11" set.

PKCS#11 token PIN: 

Public Key operation error

140735120921424:error:80029001:PKCS11 library:func(41):Cancel:p11_ops.c:61:

$ openssl pkeyutl -engine pkcs11 -keyform engine -sign -inkey
"pkcs11:object=SIGN%20key;object-type=private" -out t2570.ecdsa.sig -in
t2570.dat

engine "pkcs11" set.

PKCS#11 token PIN: 

Public Key operation error

140735120921424:error:80029001:PKCS11 library:func(41):Cancel:p11_ops.c:61:

$


If the input to "pkeyutl –sign” is supposed to be digest output only – then
what’s the point of having command line arguments specifying the digest to
use? And if the input can be an arbitrary file (like for “dgst”), then why
it doesn’t seem to work?

I’d appreciate comments, guidance, etc.

Thanks!
-- 
Regards,
Uri Blumenthal


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20160113/9a8eef24/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4308 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20160113/9a8eef24/attachment-0001.bin>