[openssl-dev] Where is the sample-comprehensive CAVS test vectors' set with all 259 test vectors

[Steve Marquess]

On 04/14/2016 07:20 AM, cyriac wrote:
> Thanx! That link works now. Infact, we had some samples from there already.
>  We understand now that the test vectors do change over time and there is
> nothing like a "final" set.
> 
> And yes, we are working with an accredited test lab already. The intention
> behind getting hold of a complete set in advance was to have a trial run of
> the tests in advance till we wait for official test vectors from the lab.

IMHO the algorithm testing process is tedious enough as it is; since in
general you cannot get a "complete set" in advance because the format
changes so frequently, you're just asking for unnecessary grief and
frustration. You'll encounter enough of that in the normal course of
events without seeking it out :-)

Your lab should have told you that...

> And as I understand, officially, these have to be verified with the CAVS
> tool which can be done only by the lab.

Correct.

> However, the perl script fipsalgtest.pl is capable to verify the .rsp files
> against the .fax files (provided along with the vectors) and to provide a
> test summary report. (With the exception of some key gen vectors which could
> be verified only by CAVS tool)
> We have done this for a set of vectors and it passes too.
> 
> *Only one clarification sought for.. If fipsalgtest.pl tells me that my
> vectors are verified without errors, should I still be skeptical until the
> lab confirms it ?*

Yes, for several reasons:

1) That check only compares the results from a presumed known good
platform against the target response files.

2) The test vector set you're using is probably obsolete, and so is no
good for your intended outcome even if "correct".

3) Even of "current", with "correct" response files relative to the
request files, the request files may be wrong (as in not what is
required by the CAVP). Those files are generated from the CAVS tool via
a labor intensive manual process, and the CAVS tool is updated
frequently and sometimes has bugs. Errors in one or both (manual process
or tool) are not at all uncommon; I'd say the error rate is in the 10%
range. So you can find that the test vectors you processed without
apparent error, and even that the test lab confirmed, can still turn out
to be unsuitable. Usually you don't have to reprocess them all, though I
usually do given that it's easier to use fipsalgtest.pl on a full test
vector set than to manually manipulate individual request files. Note I
like to hang on to the test device until the CMVP formally approves the
related validation action, as on occasion we've have to re-do testing
that was first done months ago.

-Steve M.

-- 
Steve Marquess
OpenSSL Validation Services, Inc.
1829 Mount Ephraim Road
Adamstown, MD  21710
USA
+1 877 673 6775 s/b
+1 301 874 2571 direct
marquess at openssl.com
gpg/pgp key: http://openssl.com/docs/0x6D1892F5.asc