[openssl-users] [openssl-dev] Failed TLSv1.2 handshake with error 67702888--bad signature

[Dr. Stephen Henson]

On Sat, Feb 27, 2016, Nounou Dadoun wrote:

> Thanks for the response,
> 
> I'm not sure what you're saying here other than TLS 1.2 client cert auth
> processing is different from TLS x (where x<1.2); I would assume that the
> range of mechanisms would expand to include more robust algorithms as time
> goes on.  However, here something is breaking backward compatibility with a
> client certificate that is still valid and otherwise correct as far as I can
> tell.  Our (many) deployed clients support TLSv1.2 and this certificate is
> widely distributed - we are trying to upgrade the server side from TLSv1 to
> TLSv1.2 and this appears to be a blocker.
> 
> Any recommendations?  I'm still not clear what it is about this certificate
> that fails in TLSv1.2; I would define a server callback for the certificate
> verification (I might experiment with that anyway) but it's not clear to me
> that it would call the callback if the signature is failing.
> 

Well which version of OpenSSL is being used by the server and what software is
the client using?

When client auth is enabled the client presents its certificate chain to the
server. It also signs some data using the private key corresponsing to the
public key in the client certificate and the server verifies that signature.

It looks like it is this latter signature which is causing the problems and
nothing to do with the certificates (though they have some odd fields in there
OpenSSL wouldn't reject them). As I mentioned the type of signature a client
generates during client auth changed in TLS 1.2. It is possible that the
client is sending an invalid signature which the server is rejecting while
the different form used for TLS 1.1 and earlier is fine.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org