[openssl-users] ENGINE API and a compromised client or server

[chris.gray at kiffer.be]

Hi all,

On a project I am working on we are wrestling with the problem of
preventing discovery of the private key, in a consumer device. I have been
advocating the use of the ENGINE API as a general model which can be used
to provide the best isolation possible on the customer's chosen hardware:
 * at a minimum no direct accessibility of the private key file (or
dedicated flash region) from the internet-facing process;
 * better still if internet-facing process runs in an LXC (esp. if
unprivileged);
 * ideally of course true SHM ($$$);
without needing to maintain multiple codebases.

Some colleagues have expressed concern that API methods such as
RSA_(public,private]_[en,de]crypt could be used by a compromised
client/server to launch a chosen-plaintext attack. (For the record we are
also using ECDSA, so this question is not purely RSA-specific). I am not
unduly worried about the _public_ functions because we should be able to
insist that a decent padding algorithm is used and then we are pretty
safe, right? I am more nervous about RSA_public_encrypt, which I think is
needed by TLS and which only seems to support RSA_PKCS1_PADDING (not
good). Is this a feasible attack vector?

Note that we are not only talking about servers here, rather we currently
have only one internet-facing server (HTTPS) and a growing number of
XXX-over-TLS clients, so if anything these are a greater source of
concern.

Any pointers are very welcome!

Chris Gray