[openssl] master update

[Dr. Paul Dale]

The branch master has been updated
       via  c678f68a19638c1e2bbfee6a7a1d8d728976ce66 (commit)
       via  a21a1c23c97163d341d1caf07c2160fcb68506b0 (commit)
      from  2f06c34b0eb658e80d82be8f7f0b54305a9cf68b (commit)


- Log -----------------------------------------------------------------
commit c678f68a19638c1e2bbfee6a7a1d8d728976ce66
Author: Pauli <paul.dale at oracle.com>
Date:   Sat Dec 12 21:38:17 2020 +1000

    test: document the random test ordering env variable
    
    Reviewed-by: Tomas Mraz <tmraz at fedoraproject.org>
    (Merged from https://github.com/openssl/openssl/pull/13672)

commit a21a1c23c97163d341d1caf07c2160fcb68506b0
Author: Pauli <paul.dale at oracle.com>
Date:   Sat Dec 12 21:25:40 2020 +1000

    test: print OPENSSL_TEST_RAND_ORDER=x when a randomised test fails.
    
    The previous message "random seed x" is a lot less descriptive.
    
    Reviewed-by: Tomas Mraz <tmraz at fedoraproject.org>
    (Merged from https://github.com/openssl/openssl/pull/13672)

-----------------------------------------------------------------------

Summary of changes:
 test/README.md         | 20 ++++++++++++++++++++
 test/testutil/driver.c |  2 +-
 2 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/test/README.md b/test/README.md
index 43f8471120..9a5c91953d 100644
--- a/test/README.md
+++ b/test/README.md
@@ -148,3 +148,23 @@ for `TAP::Harness` to know more.
 To run up to four tests in parallel at any given time:
 
     $ make HARNESS_JOBS=4 test
+
+Randomisation of Test Ordering
+------------------------------
+
+By default, the test harness will execute tests in the order they were added.
+By setting the `OPENSSL_TEST_RAND_ORDER` environment variable to zero, the
+test ordering will be randomised.  If a randomly ordered test fails, the
+seed value used will be reported.  Setting the `OPENSSL_TEST_RAND_ORDER`
+environment variable to this value will rerun the tests in the same
+order.  This assures repeatability of randomly ordered test runs.
+This repeatability is independent of the operating system, processor or
+platform used.
+
+To randomise the test ordering:
+
+    $ make OPENSSL_TEST_RAND_ORDER=0 test
+
+To run the tests using the order defined by the random seed `42`:
+
+    $ make OPENSSL_TEST_RAND_ORDER=42 test
diff --git a/test/testutil/driver.c b/test/testutil/driver.c
index 9343dfb016..0b4332b492 100644
--- a/test/testutil/driver.c
+++ b/test/testutil/driver.c
@@ -256,7 +256,7 @@ PRINTF_FORMAT(2, 3) static void test_verdict(int verdict,
     test_flush_stderr();
 
     if (verdict == 0 && seed != 0)
-        test_printf_tapout("# random seed: %d\n", seed);
+        test_printf_tapout("# OPENSSL_TEST_RAND_ORDER=%d\n", seed);
     test_printf_tapout("%s ", verdict != 0 ? "ok" : "not ok");
     va_start(ap, description);
     test_vprintf_tapout(description, ap);