configuring callbacks (or not) and SNI vs not... no shared cipher from server end

[Viktor Dukhovni]

On Wed, May 08, 2019 at 02:15:43PM -0400, Michael Richardson wrote:

> Diversionary issue:
>      https://www.openssl.org/docs/manmaster/man3/SSL_set_tlsext_host_name.html
> and: https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_client_hello_cb.html
> 
> are pretty vague.  I think that SSL_set_tlsext_host_name() is probably
> intended to be used on the client to set the SNI, but I'm not sure.

Yes, e.g. in the Postfix TLS client:

    https://github.com/vdukhovni/postfix/blob/master/postfix/src/tls/tls_client.c#L1035-L1045

> The legacy cb function returns int, but it's values are not documented.

On the server side I'm using SSL_CTX_set_tlsext_servername_callback():

    https://github.com/vdukhovni/postfix/blob/2399e9e179ee025d03155fa3637cccab0a23ddce/postfix/src/tls/tls_misc.c#L1040-L1043
    https://github.com/vdukhovni/postfix/blob/master/postfix/src/tls/tls_misc.c#L668

> I guess the point is that CB can set the server certificate to something
> appropriate, or I think, it could just decide to ignore the SNI value
> completely and force the certificate regardless.

Yes.

> What is the SNI functionality otherwise on the server?

You can interpose a secondary "virtual-host-specific" SSL_CTX for
for the rest of the handshake.  This carries the server certificate,
but also the trust store settings for validating client certificates,
the settings to request (or not) client certificates, the verification
callbacks, ...  It is a rather heavyweight object, best cached and
re-used for multiple connections.

In Postfix, it is configured with the same settings as the initial
SSL_CTX, *but* no server certificates.  During the SNI callback I
interpose the certificate-less context, and then set the certificate
chain on the connection handle (SSL *) instead.

> Is there any support for picking a certificate based upon the SNI name?

The application does the "picking"...  The application sets one or
more certificate chains (one per supported public key algorithm)
that best match the SNI name, and then OpenSSL chooses one of these
based on the client's advertised supported signature algorithms,
...

-- 
	Viktor.