[openssl-dev] Test framework improvements

[Paul Dale]

A number of improvements to the output of the C portion of the test framework have been made.
Specifically, a number of functions have been added to provide uniform reporting of test case failures.

You access this functionality by including "testutil.h"

There are two unconditional functions: TEST_info and TEST_error which print informative and error messages respectively.  They have no return value and accept a printf format string plus arguments.

All of the remaining functions are conditional tests.  They return 1 if the condition is true and 0 if false.  They output a uniform diagnostic message in the latter case and nothing in the former.  The majority of these are of the form TEST_type_cond, where _type_ is the type being tested and _cond_ is the relation being tested.  The currently available types are:

        type                C type
        int                 int             
        uint                unsigned int    
        char                char            
        uchar               unsigned char   
        long                long            
        ulong               unsigned long   
        size_t              size_t          
        ptr                 void *          
        str                 char *          
        mem                 void *, size_t  


For the integral types, cond can be:

        cond                C comparison
        eq                  ==
        ne                  !=
        gt                  >
        ge                  >=
        lt                  <
        le                  <=


For the pointer types, cond can only be _eq_ or _ne_.  In the case of _str_ and _mem_, the memory pointed to is compared.  For _ptr_ just the pointer themselves are compared.  The _mem_ comparisons take a pair of pointers + sizes as arguments (i.e. ptr1, size1, ptr2, size2).


There are two additional short hand calls for ptr:

        TEST_ptr            ptr != NULL
        TEST_ptr_null       ptr == NULL


Finally, there are two calls to check Boolean values:

        TEST_true           checks for != 0
        TEST_false          checks for == 0


In all cases, it is up to the test executable to process the return codes and to indicate success or failure to the PERL test framework.  This would usually be done using:

        if (!TEST_cond(args))
            return 0;


See the brief notes at the end of test/README and look at some of the test cases that have been converted already:

        test/asn1_internal_test.c
        test/cipherlist_test.c
        test/crltest.c
        test/lhash_test.c
        test/mdc2_internal_test.c
        test/pkey_meth_test.c
        test/poly1305_internal_test.c
        test/ssl_test.c
        test/ssl_test_ctx_test.c
        test/stack_test.c
        test/tls13encryptiontest.c
        test/tls13secretstest.c
        test/x509_internal_test.c
        test/x509_time_test.c


To see some examples output of failing tests, test the test_test case:

        make test TESTS=test_test VERBOSE=1

To see examples of all of the new test functions have a look in test/test_test.c  This features both passing and failing calls.  However, the actual error handling is not normal for a test executable because it treats desired failures as passes.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia