Cert hot-reloading

[Karl Denninger]

On 8/30/2020 20:19, Jordan Brown wrote:
> Well, I can restate the problem that I encountered.
>
> We deliver an integrated storage system.  Under the covers it is a 
> modified Solaris running a usual collection of proprietary and 
> open-source components.  We supply an administrative user interface 
> that, among many other things, lets you manage a list of "trusted" 
> certificates - typically CA certificates that a program would use to 
> authenticate its peers.  That is, it's the equivalent of Firefox's 
> Tools / Options / Privacy & Security / Certificates / View 
> Certificates, and the "Servers" and "Authorities" tabs there, with the 
> additional tidbit that for each certificate you can control which 
> services (e.g. LDAP, et cetera) that certificate is trusted for.
>
> When an administrator makes a change to the trusted-certificates list, 
> we want that change to take effect, system-wide.
>
> The problem is that that means that some number of processes with 
> active OpenSSL contexts need to drop those contexts and recreate them, 
> and we don't know which processes those are.  Client operations are 
> typically driven through a library, not a separate daemon, and so 
> there's no centralized way to know which processes might be TLS 
> clients.  In addition, there's the question of how to *tell* the 
> process to recreate the context.  Simply restarting them may involve 
> disruption of various sorts.
>
> What we'd like would be for OpenSSL to, on every authentication, stat 
> the file or directory involved, and if it's changed then wipe the 
> in-memory cache.
>
> Yes, aspects of this are system-specific, but that's true of many 
> things.  There could easily be an internal API that captures a 
> current-stage object, and another that answers "is this still the 
> same".  The default implementation could always say "yes".

I'm trying to figure out why you want to replace the context in an 
*existing* connection that is currently passing data rather than for new 
ones.

For new ones, as I've noted, it already works as you'd likely expect it 
to work, at least in my use case, including in multiple threads where 
the context is picked up and used for connections in more than one 
place.  I've had no trouble with this and a perusal of the documentation 
(but not the code in depth) suggested it would be safe due to how 
OpenSSL does reference counts.

While some of the client connections to the back end in my use case are 
"controlled" (an app on a phone, for example, so I could have control 
over what happens on the other end and could, for example, send down a 
sequence demanding the client close and reconnect) there is also a 
general web-style interface so the connecting party could be on any of 
the commodity web browsers, over which I have no code control.

Example meta-code:

get_lock(mutex)

if (web_context) { /* If there is an existing context then free it up */
     SSL_CTX_free(web_context);
     www_context = NULL;    /* It is not ok to attempt to use SSL */
}

www_context = SSL_CTX_new(server_method);    /* Now get a new context */
.... (set options, callbacks, verification requirement on certs 
presented, DH and ECDH preferences, cert and key, etc)
if NOT (happy with the previous sequence of options, setting key and 
cert, etc) {
     SSL_CTX_free(web_context);
     web_context = NULL;
}

unlock(mutex)

Then in the code that actually accepts a new connection:

get_lock(mutex)

if (web_context) {
     ssl_socket = starttls(inbound_socket, www_context, &error);
     .... check non-null to know it's ok, if it is, store and use it
}

unlock(mutex)

("starttls" does an SSL_new on the context passed, does the SSL_set_fd 
and SSL_accept, etc, handles any errors generated from that and if 
everything is ok returns the SSL structure)

I've had no trouble with this for a good long time; if there are 
existing connections they continue to run on the previous www_context 
until they close.  New connections come off the new one.  You just have 
to run a mutex to make sure that you don't try to create a new 
connection while the "re-keying" is "in process".

-- 
Karl Denninger
karl at denninger.net <mailto:karl at denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20200831/f6564aa1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4897 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://mta.openssl.org/pipermail/openssl-users/attachments/20200831/f6564aa1/attachment.bin>