[openssl-dev] Travis [extended tests] tag

Andy Polyakov appro at openssl.org
Sun Feb 26 22:23:42 UTC 2017


In order to improve CI turn-around times Travis config in master branch
was tweaked to minimize the time it takes to process pull requests. This
is done by "short-circuiting" most expensive tests: sanitizers,
coverage, wine-based tests. Thing to keep in mind is that
"short-circuited" test come out as passed/green. Rationale is that if
minimum tests pass, the build should still be marked green on github.
Even though it gives somewhat deceiving picture, in sense that you get
green check mark for test that might have failed otherwise. Expensive
tests are marked with "EXTENDED_TEST=yes" on the build page, and one can
easily see if it was skipped by looking at time it took to skip it, it
should be ~1 minute.

At the same time it would be inappropriate to deny the mere possibility
to exercise complete test set even on per-pull-request basis. [Note that
complete tests are always executed for each repo-push.] For this reason
possibility to "opt-in" for expensive tests was arranged by adding
"[extended tests]" tag to *last* commit. If forgotten (in case you
reckoned that request is "worthy" extended tests), or claimed desired
afterwards, it's possible to simply amend the last commit, add the tag
and force push. In such case minimal tests would be effectively wasted
(because they will be executed twice), but overall it should still be
resource saving, since majority of pull requests won't require extended
testing.

And in the context it's worth keeping in mind that it's possible to skip
CI tests altogether by tagging commit with "[skip ci]". This option is
appropriate for commentary or documentation typo fixes, readme updates,
non-x86 code updates...


More information about the openssl-dev mailing list