[openssl-dev] Certificate torture test

[David Woodhouse]

On Fri, 2016-09-02 at 20:20 +0000, Salz, Rich wrote:
> > 
> > I've started collecting a certificate torture test suite at
> > http://git.infradead.org/users/dwmw2/openconnect.git/blob/HEAD:/tests/
> > Makefile.am
> 
> I think this is cool, and splitting it off is a good idea.  I think
> some IETF folks would be interested, too.

Any suggestions on where to discuss this would be most welcome. I'd
quite like to solicit input *before* splitting it out into its own
project and presenting it as a fait accompli.

For me, this is part of a larger crusade around SSL client
certificates. For users, *all* software really sucks for this.

Applications need to do better, and crypto libraries need to stop
making it *hard* for applications to do better.

I've *tried* to do a good job in OpenConnect, and it still sucks.

Users report "my cert didn't work" bugs, but for some reason they're
reluctant to provide a copy of the file and its passphrase, for me to
reproduce the issue. :)

I was actually trying to fix curl when I came up with the idea of the
torture test... and quickly came to the realisation that I had to get
my own house in order first. Which first involved fixing GnuTLS and
OpenSSL because for some cases they weren't just making it *hard* for
the application to do the right thing; they were making it
*impossible*.

So now I have a collection of RSA/DSA/EC certificates and keys in
various esoteric file formats, and PKCS#11 objects where the key
doesn't expose its public component or the certificate is marked
private, etc.

But there's more to it than just splitting out my own test suite into a
separate project in its own right — I actually want to set an
expectation everywhere that "well behaved" applications *will* pass it,
so that users can actually come to depend on consistent behaviour, and
file bugs that get taken seriously if there's a failure.

As things stand, users experience a whole *bunch* of crap that they
just shouldn't, including:

 "My distribution builds this application against GnuTLS but this PEM 
  file is in the non-standard OpenSSL encrypted form so it doesn't 
  work."

 "My token doesn't expose the EC public key so I can't authenticate to
  the VPN... unless I rebuild OpenConnect to use GnuTLS instead of 
  OpenSSL. But I'm on CentOS where GnuTLS is too old to support Cisco's
  broken pre-standard version of DTLS."

 "My distribution builds cURL against NSS so the standard form of the
  URI doesn't work and I have to use a NSS-specific way of specify the
  cert instead."

 "Hm, my token is installed correctly and works everywhere. I can see 
  my cert in 'seahorse' and 'p11tool' and use it from OpenConnect… but
  why doesn't it work in Firefox? Oh, because NSS doesn't actually 
  load the tokens listed in the system configuration."

 "OK, so I fixed that... but now why doesn't it show up in Chrome. That
  uses NSS too, right? Oh, I have to add it to a *different*
  configuration file, ffs."

 "Now openvpn doesn't support the standard URI form either, it uses
  libpkcs11-helper which has an entirely different format that I need
  to use to specify the certificate."

 "Ick, when wpa_supplicant is built against OpenSSL it needs all this 
  explicit "engine" configuration nonsense in *addition* to needing
  the cert to be specified in a non-standard form. Unlike when built
  with GnuTLS, when it does just accept a RFC7512 PKCS#11 URI in its
  'certificate' option."

 "Argh, FFS. The OpenSSL engine doesn't load the tokens listed in the 
  system configuration *either*, and needs to be explicitly told which
  provider module to load. And my application doesn't even give me the 
  hook to do that."

 "Oh, 'openvpn --cert' takes PEM files according to its documentation…
  but wait, *this* is a PEM file and it doesn't seem to work. Why?"

 "Hm, all this *used* to work but when I just renewed my certificate,
  it magically stopped working... oh, because the new cert was issued
  with OpenSSL 1.1 and that uses a SHA256 HMAC by default which this
  version of GnuTLS doesn't cope with.

 "Oh, my cert is marked with CKA_PRIVATE but libp11 doesn't return it
  from PKCS11_enumerate_certs() even *after* I log in, so I can't use 
  it."

 "Hm, I need to configure my VPN to use the cert in my token but the
  GUI configuration dialog only lets me choose from a file."

Now, I've actually *fixed* most of the above; they are historical
examples — even for the 'NSS doesn't take RFC7512 URIs' one I had a
GSoC student this year work on it, and patches are filed in bugzilla.
But there are *plenty* more where they came from.

I have attempted to address expectations through the Fedora packaging
guidelines — which already state that applications which accept
certificate filenames SHOULD also accept a PKCS#11 URI according to
RFC7512, and Just Work.

I'm sure I can make a case for expanding that to cover a more
comprehensive set of file and PKCS#11 tests. And that *does* have a
knock-on effect in the wider ecosystem, as we fix upstream projects to
comply.

But this kind of thing shouldn't be distribution-specific. That just
*happens* to the policy handle that I can easily influence.

I haven't *yet* had an upstream push back and say that they don't want
to do this "just for Fedora", but it wouldn't surprise me to see it
happen.

I'd *really* like to achieve a wider consensus, and I'd very much
welcome suggestions for where and how to do that.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse at intel.com                              Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5760 bytes
Desc: not available
URL: <http://mta.openssl.org/pipermail/openssl-dev/attachments/20160908/13191d25/attachment.bin>