[openssl-dev] Access to ECDSA_METHOD do_verify function from engine

[Johannes Bauer]

On 21.07.2017 15:08, Douglas E Engert wrote:

> I don't see your problem with OpenSSL-1.1.0f. I don't recall seeing it with
> earlier version either. p11_ec.c does:
> 
> 
> 647         static EC_KEY_METHOD *ops = NULL;
> 648         int (*orig_sign)(int, const unsigned char *, int, unsigned
> char *,
> 649                 unsigned int *, const BIGNUM *, const BIGNUM *,
> EC_KEY *) = NULL;
> 
> 653                 ops = EC_KEY_METHOD_new((EC_KEY_METHOD
> *)EC_KEY_OpenSSL());
> 654                 EC_KEY_METHOD_get_sign(ops, &orig_sign, NULL, NULL);
> 655                 EC_KEY_METHOD_set_sign(ops, orig_sign, NULL,
> pkcs11_ecdsa_sign_sig);

Ah, interesting! You call EC_KEY_METHOD_get_sign on the (inherited) copy
of the EC_KEY_METHOD. I didn't, but called it on the original source
(otherwise, very similar code):

int (*openssl_sign)(int type, const unsigned char *dgst, int dlen,
unsigned char *sig, unsigned int *siglen, const BIGNUM *kinv, const
BIGNUM *r, EC_KEY *eckey) = NULL;
int (*openssl_sign_setup)(EC_KEY *eckey, BN_CTX *ctx_in, BIGNUM **kinvp,
BIGNUM **rp) = NULL;
EC_KEY_METHOD_get_sign((EC_KEY_METHOD*)EC_KEY_OpenSSL(), &openssl_sign,
&openssl_sign_setup, NULL);

The case of EC_KEY_OpenSSL() from const EC_KEY_METHOD* to EC_KEY_METHOD*
gives a -Wqual-cast diagnostic:

usockeng.c:245:25: warning: cast discards ‘const’ qualifier from pointer
target type [-Wcast-qual]
  EC_KEY_METHOD_get_sign((EC_KEY_METHOD*)EC_KEY_OpenSSL(),
&openssl_sign, &openssl_sign_setup, NULL);

I've changed my code now to also use the (mutable) new EC_KEY_METHOD*,
which doesn't give a diagnostic. Regardless, I believe that the first
parameter of EC_KEY_METHOD_get_sign should be const EC_KEY_METHOD*, not
EC_KEY_METHOD*.

Cheers,
Johannes