[openssl-users] Bug in OpenSSL nameConstraints validation

[Ben Humpert]

Hi,

Based on https://tools.ietf.org/pdf/draft-wilson-wpkops-browser-processing-02.pdf
section 3.3.1.2. I ran my own tests. I wrote an email
(https://mta.openssl.org/pipermail/openssl-users/2015-May/001387.html)
with the results (attachments in
https://mta.openssl.org/pipermail/openssl-users/2015-May/001388.html).

I observed that OpenSSL s_client always throws "Verify return code: 51
(unsupported name constraint type)" when the name IP is present in the
subjectAltName extension. It does not do so when the name DNS is used
or when no subjectAltName extension is present at all.
In some situations it throws "Verify return code: 47 (permitted
subtree violation)" while there is no violation.

It was also clear that s_client does not check for nameConstraints
violation in CN at all.

However, OpenSSL itself behaves differently. I tried to EAP-TLS with
Android 4.4.4 and always got "certificate unknown" in the RADIUS log.
I did so too with eapol_test and noticed that OpenSSL does not send
the correct error to the server (This is another bug). The client
reported "EAP: Status notification: local TLS alert (param=certificate
unknown)" but additionally "EAP: Status notification: remote
certificate verification (param=permitted subtree violation)" and
"OpenSSL: openssl_handshake - SSL_connect error:14090086:SSL
routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed" but
the server side always just received "certificate unknown" which is
like "unknown error" on windows.

With EAP-TLS I observed that (Ubuntu 14.04.2 server with OpenSSL
1.0.1f as well as Android 4.4.4) actually does check for
nameConstraints violation in CN. The used nameConstraints are

permitted;DNS.0=.lan
permitted;DNS.1=.local
permitted;DNS.3=.de
permitted;email.0=.de
permitted;IP.0=10.0.0.0/255.0.0.0
permitted;IP.1=172.16.0.0/255.240.0.0
permitted;IP.2=192.168.0.0/255.255.0.0

and the RADIUS server certificate subject is
"/C=DE/ST=DE-BY/L=Munich/O=Example Company/CN=RADIUS
Server/emailAddress=root at an3k.de" with
subjectAltName="DNS:radius.home.lan,IP:10.11.12.13"

The subtree violation that OpenSSL thinks has happened is not in
emailAddress or in subjectAltName. It is the CN field because "RADIUS
Server" is not ending on .lan .local or .de BUT s_client never
validated the CN field so why does OpenSSL so? Additionally "RADIUS
Server" is neither an IP address nor a DNS name, thus there is
actually no violation. And last but not least the subjectAltName
attribute is present, thus CN should not be validated at all!

This behavior is validated by testing a new server certificate/key but
without subjectAltName and another new cert/key without subjectAltName
but CN set to "radius.home.lan".

Windows XP, Vista, 7, 8 and 10 as well as MacOS and iOS accepted the
initial RADIUS Server certificate. It is just OpenSSL doing some weird
stuff with nameConstraints validation.

Thank you very much in advance!


Best regards,

Ben