[openssl-users] Build of 1.0.1g fails

[Jason Schultz]

Greetings. I'm having problems building OpenSSL, starting with 1.0.1g. The scenario is as follows.

I'm not sure when the problem was introduced; however, with the compiling-out of SSLv2 *by default* in -1.0.2g, that change has exacerbated this problem.  (That is, instead of affecting only those who selected "no-ssl2", it now affects everyone *except* those that explicitly select "ssl2".)

First, the existing package runs a self-test during the package build process.  One of those tests verifies SSL (ssl/ssltest.c), and another verifies SSL usage when FIPS is active (test/testfipsssl).  The code in ssl/ssltest.c has a section that detects if the requested encryption mechanism has been disabled at build time ("compiled out").  If this situation is detected, an "OK" status is returned so that the test driver can determine what to do.  When FIPS is compiled, configured, and enabled, calling the SSL verification from test/testfipsssl to verify SSLv2 or SSLv3 support should result in a "Fail" status since neither SSLv2 nor SSLv3 is supported with FIPS.  However, when the "no-sslv2" and/or "no-sslv3" build options are selected, neither mechanism gets compiled in, so the SSL verification test detects this and immediately returns "OK" status.  Since FIPS is compiled, configured, and enabled, a "Fail" status is expected by test/testfipsssl instead, so the "OK" status that is received because the ciphers are not present is handled as a test failure thereby aborting the build.

To make the package build correctly with "no-sslv2" or "no-sslv3" specified, I had to add the following:

Index: ssl/ssltest.c
===================================================================
--- ssl/ssltest.c (revision 4068)
+++ ssl/ssltest.c (working copy)
@@ -1203,8 +1203,20 @@
     if (no_protocol) {
         fprintf(stderr, "Testing was requested for a disabled protocol. "
                 "Skipping tests.\n");
+#ifdef OPENSSL_FIPS
+        /*
+         * If FIPS is enabled, then neither SSLv2 nor SSLv3 are permitted anyway.
+         * In this case, the fact that one or both are compiled-out is a good thing,
+         * so we continue onward to return the expected error status instead.
+         */
+        if (!fips_mode || !FIPS_mode_set(1) || !(ssl2 || ssl3)) {
+            ret = 0;
+            goto end;
+        }
+#else
         ret = 0;
         goto end;
+#endif
     }

     if (!ssl2 && !ssl3 && !tls1 && !dtls1 && !dtls12 && number > 1 && !reuse && !force) {

Is this a known problem? Is there a solution available?

Thanks in advance.