[openssl-dev] [openssl.org #3607] nistz256 is broken.

[Adam Langley via RT]

On Fri, Dec 5, 2014 at 6:33 AM, Andy Polyakov via RT <rt at openssl.org> wrote:
> Attached. A little bit worse performance on some CPUs. I also took
> opportunity to harmonize ecp_nistz256_from_mont by applying same pattern
> for reduction. The patch is cumulative, i.e. is not incremental to
> previously posted one[s], and addresses both problems, originally
> reported one and discovered in the course. Patch to ecp_nistz256.c
> referred above doesn't matter.

When applying just that patch, the original test case fails. Specially
this test code (C++):


  BIGNUM *n = nullptr, *X = nullptr, *Y = nullptr, *Z = nullptr;
  BIGNUM *x = BN_new();
  BIGNUM *y = BN_new();

  ASSERT_NE(BN_hex2bn(&n,
"2269520AFB46450398DE95AE59DDBDC1D42B8B7030F81BCFEF12D819C1D678DD"),
0);
  ASSERT_NE(BN_hex2bn(&X,
"C4EB2994C09557B400FF6A543CFB257F945E86FE3DF1D32A8128F32927666A8F"),
0);
  ASSERT_NE(BN_hex2bn(&Y,
"3D5283F8F10F559AE5310005005F321B28D2D699F3E01F179F91AC6660013328"),
0);
  ASSERT_NE(BN_hex2bn(&Z,
"F97FD7E6757991A2C7E0C2488FF3C54E58030BCACF3FB95954FD3EF211C24631"),
0);

  EC_GROUP *group = EC_GROUP_new_by_curve_name(NID_X9_62_prime256v1);
  EC_POINT *p = EC_POINT_new(group);
  BN_CTX *ctx = BN_CTX_new();
  ASSERT_EQ(1, EC_POINT_set_Jprojective_coordinates_GFp(group, p, X,
Y, Z, ctx));

  EC_POINT *r = EC_POINT_new(group);
  // Set r = 𝕡×n.
  ASSERT_EQ(1, EC_POINT_mul(group, r, NULL, p, n, ctx));

  ASSERT_EQ(1, EC_POINT_get_affine_coordinates_GFp(group, r, x, y, ctx));
  char *x_out = BN_bn2hex(x);
  char *y_out = BN_bn2hex(y);
  EXPECT_STREQ("C2910AA0216D12DE30C5573CCFC4116546E3091DC1E9EC8604F634185CE40863",
x_out);
  EXPECT_STREQ("C9071E13D688C305CE179C6168DD9066657BC6CDC1639A44B68DF7F1E0A40EDF",
y_out);

  free(x_out);
  free(y_out);
  BN_free(x);
  BN_free(y);
  BN_free(X);
  BN_free(Y);
  BN_free(Z);
  BN_free(n);
  EC_POINT_free(r);
  EC_POINT_free(p);
  BN_CTX_free(ctx);
  EC_GROUP_free(group);


Just to check that I'm not doing anything stupid (which is always a
distinct possibility), here are the .pl[1] and resulting .s[2] file
that I ended up with.

[1] https://drive.google.com/file/d/0B_OzbbAp1CG5OVdVc196QmV3bG8/view?usp=sharing
[2] https://drive.google.com/file/d/0B_OzbbAp1CG5Z3NoZzBqU09scFE/view?usp=sharing


Cheers

AGL