[openssl-users] Using FIPS mode and modifying apps

[Tom Francis]

> On Jan 26, 2015, at 6:21 PM, jonetsu at teksavvy.com wrote:
> 
> On Fri, 16 Jan 2015 10:16:48 -0500
> Steve Marquess <marquess at openssl.com> wrote:
> 
>> On 01/15/2015 05:52 AM, Marcus Meissner wrote:
> 
>>> On Linux usually triggered by /proc/sys/crypto/fips_enabled
>>> containing "1" or the environment variable
>>> OPENSSL_FORCE_FIPS_MODE=1 (at least for the certs done by SUSE and
>>> Redhat, which do not use the container blob).
> 
>> That is (presumably) true for the proprietary RH and SUSE distros,
>> not so for the open source based OpenSSL FIPS Object Module or other
>> Linux distros.
> 
> I'm afraid it does not come across clear to me.  So, maybe the
> following pondering is relevant - or not.  Basically, I'm looking at
> how to integrate a FIPS-enabled OpenSSL that will be used by some
> common Open Sources applications, as well as a 3rd party application
> (with source code provided).

This is a bad idea.  It can generally be done, and it’s probably not even too hard (for some uses, anyway).  But it’s a bad idea.  Here’s why:

1) Applications that don’t know they’re operating in FIPS mode may attempt to use algorithms that are disallowed in FIPS mode.  Because the application wasn’t designed for this situation, if the algorithm is attempted through the proper APIs, use of the algorithm will fail, and the application’s error report will be incorrect (assuming it even handles the error coming from OpenSSL, and doesn’t fail much later!).
2) Applications that don’t know they’re operating in FIPS mode may attempt to use algorithms that are disallowed in FIPS mode, but using an API that will actually succeed.  This will cause issues for the USER, since the user believes the application is operating properly in FIPS mode, when it just did something that’s disallowed!
3) Applications that don’t know they’re operating in FIPS mode may attempt to use SHA-1 in operations that are disallowed in FIPS mode.  OpenSSL will allow most uses of SHA-1 in FIPS mode, relying on the application to only use it appropriately.  Again, this can cause problems for the user who thinks the application is operating in FIPS mode, but the application has done things that are disallowed!
4) If POST fails or some other problem is detected, where FIPS mode fails to be enabled, an application that doesn’t know anything about FIPS mode will fail in a manner unexpected by the application.  Even if you’ve found a way to notify the user (ha! how you gonna do that not knowing if the application is graphical or not, has access to stderr, syslog, or something else entirely?), the application’s failure may not release resources appropriately, or may cause other problems, especially for the end user who keeps trying it over and over again. :)

Someone who controls all the applications on a system, however, that one could deal with all of those issues and more.  It’d be expensive, and one would probably be unwilling to give that work away whenever one doesn’t have to. :)

I’ll let Steve answer the stuff I snipped. :)
<snip>

TOM