[openssl-users] Problems with deriving EC public key from private

[Billy Brumley]

On Tue, Dec 18, 2018 at 12:07 AM Mike Blaguszewski <mikeb at preveil.com> wrote:
>
> Some code of mine reads a NIST P256 private key from bytes and derives the public key from it, and this derived public key is incorrect about 0.4% of the time. I’ve attached a sample program that does the following.
>
> 1. Generate a key-pair of type NID_X9_62_prime256v1
> 2. Write the public and private components to memory
> 3. Read the private key back from memory, derive the public key, and write that back out.
> 4. Compare this “round-tripped” public key to the public key generated in step 2.
>
> The public key from step 2 almost always matches the public key from step 3, but about 0.4% of the time they will differ. (The sample program runs a loop to determine this.) Further experiments suggest it’s the private_key_from_binary() function that is the problem, where I derive the public key using EC_POINT_mul(). The sample program omits error checking, but in the production code no errors are reported.
>
> Does anyone see a flaw in my logic, especially in how I’m deriving the public key from the private key? Also let me know if this would be better submitted as a GitHub issue, or even if it needs to be handled as a paid support request.

The sample code just segfaults for me in the first iteration, before
really generating a key, so it's hard to test:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7a7e3e0 in pkey_set_type (pkey=0x380, type=408, str=0x0,
len=-1) at crypto/evp/p_lib.c:181
(gdb) bt
#0  0x00007ffff7a7e3e0 in pkey_set_type (pkey=0x380, type=408,
str=0x0, len=-1) at crypto/evp/p_lib.c:181
#1  0x00007ffff7a7e546 in EVP_PKEY_set_type (pkey=0x380, type=408) at
crypto/evp/p_lib.c:221
#2  0x00007ffff7a7e663 in EVP_PKEY_assign (pkey=0x380, type=408,
key=0x5555557587c0) at crypto/evp/p_lib.c:249
#3  0x00007ffff7a248fb in pkey_ec_keygen (ctx=0x555555758760,
pkey=0x380) at crypto/ec/ec_pmeth.c:416
#4  0x00007ffff7a80912 in EVP_PKEY_keygen (ctx=0x555555758760,
ppkey=0x7fffffffdd18) at crypto/evp/pmeth_gn.c:107
#5  0x0000555555555046 in generate_ec_key () at foo.c:18
#6  0x0000555555555256 in main () at foo.c:73

But 0.4% is suspiciously close to 1/256, so I'm willing to bet your
problem surrounds your size assumptions in various functions. Check
the manpage of e.g. EC_POINT_point2oct and grep for usage in the
library, but the idea is to pass NULL first, then malloc, then pass
that pointer. BN_bn2bin is different. Probably the size won't be fixed
(e.g., there is a 1/256 chance you'll have one byte less, i.e. leading
zero).

So all the static 32 and 33 byte assumptions aren't holding.

Also BN_bn2bin and EC_KEY_oct2priv are not inverses of each other
IIRC. (The former is raw bytes, latter ASN1 encoding.)

BBB