Travis in solid red mode again

[Matt Caswell]

On 31/01/2020 17:07, Matthias St. Pierre wrote:
> 
> After a much to short green period, Travis has turned to solid red mode
> again:
> 
> https://travis-ci.org/openssl/openssl/builds?utm_medium=notification&utm_source=github_status
> 
> 
> This fact causes a lot of pain for reviewers, because every failing job
> needs to be checked manually.
> I would be happy to fix some of the failures if I could, but in most
> cases I have no clue what the problem is,
> because I can't reproduce the issue locally. Could we try to fix them in
> a joint effort?  For your convenience,
> I added a summary of all failing jobs of Build #31837 below.

Trial fix for at least some of these issues in #10989. We'll see what
travis makes of it.

> 
> Since this is not the first time that this happens, I think it is
> overdue that the OTC discusses those constant
> build failures and proposes some measures to raise the priority for
> fixing CI errors. One drastic possibility
> would be to allow only CI fixing pull requests to be merged as long as
> master and the stable branches are red.
> But I am sure there is a golden middle way between such an extreme
> sanction and our current indifference.
> 

Part of the issue is that the master branch travis failures are not
visible in the PRs because the master branch runs the full tests
(including the extended tests), whereas the PRs don't run the extended
tests by default. However running the extended tests takes too long.

Could we add a CI check for PRs that confirms that the target branch is
green in travis?

Matt


> 
> Regards,
> 
> Matthias
> 
> 
> 
> ----
> 
> 
> Last Successful Build: #31583 (10 days ago!)
> https://travis-ci.org/openssl/openssl/builds/639939072?utm_medium=notification&utm_source=github_status
> 
> 
> Recent Build: #31837
> https://travis-ci.org/openssl/openssl/builds/644238908?utm_medium=notification&utm_source=github_status
> 
> 
> 
> SSL TESTS
> =========
> 
> *
> https://travis-ci.org/openssl/openssl/jobs/644238919?utm_medium=notification&utm_source=github_status
> 
> *
> https://travis-ci.org/openssl/openssl/jobs/644238926?utm_medium=notification&utm_source=github_status
> 
> *
> https://travis-ci.org/openssl/openssl/jobs/644238929?utm_medium=notification&utm_source=github_status
> 
> 
>>  Test Summary Report
>> -------------------
>>  80-test_ssl_new.t                (Wstat: 256 Tests: 30 Failed: 1)
>>    Failed test:  4
>>    Non-zero exit status: 1
>>  80-test_ssl_old.t                (Wstat: 256 Tests: 6 Failed: 1)
>>    Failed test:  2
>>    Non-zero exit status: 1
> 
> 
> 
> BORINGSSL AND KRB5
> ==================
> 
> *
> https://travis-ci.org/openssl/openssl/jobs/644238927?utm_medium=notification&utm_source=github_status
> 
> 
>> Test Summary Report
>> -------------------
> 
>> 95-test_external_boringssl.t (Wstat: 256 Tests: 1 Failed: 1)
>>   Failed test:  1
>>   Non-zero exit status: 1
>> 95-test_external_krb5.t     (Wstat: 256 Tests: 1 Failed: 1)
>>   Failed test:  1
>>   Non-zero exit status: 1
> 
> 
> 
> 95-test_external_boringssl.t:
> -----------------------
> 
>     grep 'go.*FAILED' tffff
>     go test: FAILED (SSL3-Client-ClientAuth-RSA)
>     go test: FAILED (SSL3-Server-ClientAuth-RSA)
>     go test: FAILED (ClientAuth-Sign-RSA-SSL3)
>     go test: FAILED (ClientAuth-InvalidSignature-RSA-SSL3)
>     go test: FAILED (CertificateVerificationSucceed-Server-SSL3-Sync)
>     go test: FAILED
> (CertificateVerificationSucceed-Server-SSL3-Sync-SplitHandshakeRecords)
>     go test: FAILED
> (CertificateVerificationSucceed-Server-SSL3-Sync-PackHandshakeFlight)
>     go test: FAILED (CertificateVerificationSucceed-Server-SSL3-Async)
>     go test: FAILED
> (CertificateVerificationSucceed-Server-SSL3-Async-SplitHandshakeRecords)
>     go test: FAILED
> (CertificateVerificationSucceed-Server-SSL3-Async-PackHandshakeFlight)
> 
> 
> 
> 95-test_external_krb5.t:
> -----------------------
> 
>     LD_LIBRARY_PATH=`echo -L../../../lib | sed -e "s/-L//g" -e "s/
> /:/g"` KRB5_CONFIG=../../../config-files/krb5.conf LC_ALL=C ./t_nfold
>     N-fold
>         Input:    "basch"
>         192-Fold:     1aab6b42 964b98b2 1f8cde2d 2448ba34 55d7862c 9731643f
>         Input:    "eichin"
>         192-Fold:     65696368 696e4b73 2b4b1b43 da1a5b99 5a58d2c6 d0d2dcca
>         Input:    "sommerfeld"
>         192-Fold:     2f7a9855 7c6ee4ab adf4e711 92dd442b d4ff5325 a5def75c
>         Input:    "MASSACHVSETTS INSTITVTE OF TECHNOLOGY"
>         192-Fold:     db3b0d8f 0b061e60 3282b308 a5084122 9ad798fa b9540c1b
>     RFC tests:
> 
>     ...
> 
>     Generating random keyblock: . . . OK
>     Creating opaque key from keyblock: . . . OK
>     Encrypting (c): . . . Failed: Cannot allocate memory <===ALLOC
> FAILURE==
>     Aborted (core
> dumped)                                                 <===CORE DUMPED==
>     Makefile:684: recipe for target 'check-unix' failed
>     make[5]: *** [check-unix] Error 134
>     make[5]: Leaving directory
> '/home/travis/build/openssl/openssl/krb5/src/lib/crypto/crypto_tests'
>     Makefile:1107: recipe for target 'check-recurse' failed
>     make[4]: *** [check-recurse] Error 1
>     make[4]: Leaving directory
> '/home/travis/build/openssl/openssl/krb5/src/lib/crypto'
>     Makefile:992: recipe for target 'check-recurse' failed
>     make[3]: *** [check-recurse] Error 1
>     make[3]: Leaving directory
> '/home/travis/build/openssl/openssl/krb5/src/lib'
>     Makefile:1533: recipe for target 'check-recurse' failed
>     make[2]: *** [check-recurse] Error 1
>     make[2]: Leaving directory
> '/home/travis/build/openssl/openssl/krb5/src'
>     ../../util/shlib_wrap.sh
> ../recipes/95-test_external_krb5_data/krb5.sh => 2
>     not ok 1 - running krb5 tests
> 
> 
> 
>