[openssl-users] x509_config nameConstraints

[Ben Humpert]

Ok, after plenty of testing and some googling: the name constraints
extension is ... improvable. I ran plenty of tests but it looks like
that the extension is not very well implemented in todays browsers.

I have attached three txt files (DOS format) with the settings and
results of each test run. Between each test the browsers cache, etc.
was completely cleared and the browser got restarted. I validated the
used leaf certificates using serial number / hash and the signing CA
hash between each test run.

I used "certificate warning" if an error is shown but the user is
allowed to continue browsing and "certificate error" if the user is
NOT allowed to continue.

Results:
- Internet Explorer 11 does not understand the name IP in the
subjectAltName extension. However it understands the name DNS.
- Internet Explorer 11 just knows one certificate warning "This
website's address doesn't match the address in the security
certificate" regardless of why the certificate is invalid and no error
at all.

- (Test Run A.txt) nameConstraints extension NOT present
  - everything is fine

- (Test Run B.txt) nameConstraints extension present with
permitted;DNS and permitted;IP
  - OpenSSL s_client throws "Verify return code: 51 (unsupported name
constraint type)" whenever the name IP is present in the
subjectAltName extension. It does not do so when the name DNS is used
or when no subjectAltName extension is present at all. See Test B1,
B5, B8 and compare with Test B2, B3, B7
  - OpenSSL s_client throws "Verify return code: 47 (permitted subtree
violation)" while there is no violation. See Test B2
  - OpenSSL s_client does not check for nameConstraints violation in
CN at all. See Test B7, B10
  - Firefox does NOT check for nameConstraints violation in CN if
subjectAltName is present. See Test B5
  - Firefox just throws a warning "ssl_error_bad_cert_domain" instead
of an error when the certificate is used on a domain / ip address
which is not specified in the certificate. See Test B3, B4
  - Chrome throws an error "Server's certificate is invalid" when
there is no subjectAltName present but the ip address matches the
certificate CN. See Test B4

- (Test Run C.txt) nameConstraints extension present with
permitted;DNS and permitted;IP and permitted;dirName
  - Firefox throws an error "sec_error_cert_not_in_name_space" even
when the domain is specified in subjectAltName and no nameConstraints
violation exists. It's by the way the first time Internet Explorer
acted correctly ;). See Test C2
  - OpenSSL s_client throws "Verify return code: 47 (permitted subtree
violation)" while there is no violation. See Test C2
  - Chrome, Firefox (and for sure Internet Explorer) throwed an error
while there is no nameConstraints violation. Only OpenSSL s_client
acted correctly (but only because it doesn't check CN). See Test C4