[openssl-users] Certificate verification fails with latest commits (ECDSA)

[jan.weil at ptb.de]

Hi,

we have noticed that with the latest Debian wheezy-security update of the 
libssl1.0.0 package sudenly the verification of some of our ECDSA-signed 
certificates failed.

I've looked into this and I've traced it down to the following patch

https://github.com/openssl/openssl/commit/684400ce192dac51df3d3e92b61830a6ef90be3e

which introduces a new statement to check the length of the DER-coded 
signature in ecs_vrf.c:

- if (d2i_ECDSA_SIG(&s, &sigbuf, sig_len) == NULL) goto err;
+ if (d2i_ECDSA_SIG(&s, &p, sig_len) == NULL) goto err;
+ /* Ensure signature uses DER and doesn't have trailing garbage */
+ derlen = i2d_ECDSA_SIG(s, &der);
+ if (derlen != sig_len || memcmp(sigbuf, der, derlen))
+ goto err;

This check fails for some of our certificates and the reason is that 
openssl adds a padding byte for BIGNUMs in crypto/asn1/x_bignum.c if the 
MSB is set. Our encoding does not contain these padding bytes and, 
consequently, the re-encoded version of our certificate signature is two 
bytes longer than before which results in an error.

RFC3279 defines

   Ecdsa-Sig-Value  ::=  SEQUENCE  {
           r     INTEGER,
           s     INTEGER  }

I've looked up the DER encoding rules for INTEGER here

http://www.itu.int/rec/T-REC-X.690-200811-I

and I can't find any evidence that this padding byte is mandatory. See 
below for the relevant paragraph.

So my question is: Would you agree that this is an openssl bug or is the 
padding byte indeed mandatory and we have to adapt the encoding of our 
certificates?

I am attaching one of the certificates for which the verification fails 
along with the root ca's certificate. 

Cheers,

Jan


X.690 INTEGER:

8.3 Encoding of an integer value
8.3.1 The encoding of an integer value shall be primitive. The contents 
octets shall consist of one or more octets.
8.3.2 If the contents octets of an integer value encoding consist of more 
than one octet, then the bits of the first
octet and bit 8 of the second octet:
a) shall not all be ones; and
b) shall not all be zero.
NOTE ? These rules ensure that an integer value is always encoded in the 
smallest possible number of octets.
8.3.3 The contents octets shall be a two's complement binary number equal 
to the integer value, and consisting of
bits 8 to 1 of the first octet, followed by bits 8 to 1 of the second 
octet, followed by bits 8 to 1 of each octet in turn up to
and including the last octet of the contents octets.
NOTE ? The value of a two's complement binary number is derived by 
numbering the bits in the contents octets, starting with bit
1 of the last octet as bit zero and ending the numbering with bit 8 of the 
first octet. Each bit is assigned a numerical value of 2N,
where N is its position in the above numbering sequence. The value of the 
two's complement binary number is obtained by
summing the numerical values assigned to each bit for those bits which are 
set to one, excluding bit 8 of the first octet, and then
reducing this value by the numerical value assigned to bit 8 of the first 
octet if that bit is set to one.





Jan Weil
Physikalisch-Technische Bundesanstalt
Arbeitsgruppe 8.52 Datenkommunikation und -sicherheit
Abbestr. 2 - 12
10587 Berlin
Telefon: (+49 30) 34 81 - 77 64
Fax: (+49 30) 34 81 - 69 77 64
Email: jan.weil at ptb.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: GTS001.pem
Type: application/octet-stream
Size: 631 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20150203/60eee7d0/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OSM_ROOT.pem
Type: application/octet-stream
Size: 652 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20150203/60eee7d0/attachment-0001.obj>