[openssl-users] FIPS 140-2, a game of chance

[Steve Marquess]

If you don't know or care what FIPS 140-2 is, trash this message quickly
before it harshes your mellow.

The "RE" validation, an "Alternative Scenario 1A" clone of the #1747
validation, was approved today
(http://csrc.nist.gov/groups/STM/cmvp/documents/140-1/140val-all.htm#2473).

It was submitted along with its identical twin "RE" validation on April
17. The two sets of paperwork differed in only one trivial aspect, "RE"
in the module name for one versus "SE" for the other. Same module, same
test lab, same paperwork, submitted together at the same time. We could
not have devised a more perfect controlled study if we'd tried.

The "SE" validation was approved on June 25 (#2398), after a little more
than two months (69 calendar days, 48 working days).

The "RE" validation was not approved for almost seven months (210
calendar days, 145 working days). That's three times as long for the
exact same submission. I've had the experience for years now of doing
very similar validation submissions and noting very different outcomes,
but this is the most striking example yet of CMVP capriciousness.

Why the wild disparity? Well, probably because the two identical
submissions were farmed out to two different reviewers. The review
process is notoriously subjective, and in fact we received "comments"
(requirements for changes) for the "RE" validation whereas the "SE" one
was approved as-is. As a result the two Security Policy documents are no
longer identical. That doesn't explain the time discrepancy, though, as
those "comments" weren't received until long after "SE" had been approved.

The moral here is that FIPS 140-2 validations are a crapshoot; it's
impossible to make any reliable predictions on how long any validation
action will take or how it will be received. If you have really deep
pockets you can submit the same validation multiple times to hedge your
bets (we've actually done that[1]), but for most of us it's an open
ended gamble: submit, hope, wait, ...

-Steve M.

[1] See http://veridicalsystems.com/blog/the-fickleness-of-fips/; note
that dual submission did pay off for that client.

-- 
Steve Marquess
OpenSSL Software Foundation
1829 Mount Ephraim Road
Adamstown, MD  21710
USA
+1 877 673 6775 s/b
+1 301 874 2571 direct
marquess at openssl.com
gpg/pgp key: http://openssl.com/docs/0x6D1892F5.asc