[tools] master update
Dr. Paul Dale
pauli at openssl.org
Thu Sep 2 22:57:38 UTC 2021
The branch master has been updated
via f6070cac86caad71cde3b62cd3fd0e35c724eae3 (commit)
from 35717050e0abb88170873e8403c369127fb877b1 (commit)
- Log -----------------------------------------------------------------
commit f6070cac86caad71cde3b62cd3fd0e35c724eae3
Author: Tomas Mraz <tomas at openssl.org>
Date: Thu Sep 2 14:04:36 2021 +0200
Remove run-checker completely as it is not used anymore
Reviewed-by: Richard Levitte <levitte at openssl.org>
Reviewed-by: Paul Dale <pauli at openssl.org>
(Merged from https://github.com/openssl/tools/pull/93)
-----------------------------------------------------------------------
Summary of changes:
run-checker/README | 88 -----------
run-checker/build-gost.sh | 41 ------
run-checker/run-checker-autohooks/README | 50 -------
run-checker/run-checker-autohooks/hook-end | 100 -------------
run-checker/run-checker-autohooks/hook-prepare | 49 -------
run-checker/run-checker-autohooks/hook-start | 81 -----------
run-checker/run-checker-autohooks/hook-takedown | 7 -
run-checker/run-checker-cleanup.sh | 18 ---
run-checker/run-checker.sh | 186 ------------------------
9 files changed, 620 deletions(-)
delete mode 100644 run-checker/README
delete mode 100755 run-checker/build-gost.sh
delete mode 100644 run-checker/run-checker-autohooks/README
delete mode 100755 run-checker/run-checker-autohooks/hook-end
delete mode 100755 run-checker/run-checker-autohooks/hook-prepare
delete mode 100755 run-checker/run-checker-autohooks/hook-start
delete mode 100755 run-checker/run-checker-autohooks/hook-takedown
delete mode 100755 run-checker/run-checker-cleanup.sh
delete mode 100755 run-checker/run-checker.sh
diff --git a/run-checker/README b/run-checker/README
deleted file mode 100644
index b9540a0..0000000
--- a/run-checker/README
+++ /dev/null
@@ -1,88 +0,0 @@
-This tool is used to build various predefined config options of
-openssl and generate okay/fail reports. The array of config options
-is found in run-checker.sh, assigned to 'opts'.
-
-Quick manual run
-----------------
-
-To run a check on the master branch:
-
- git clone git://git.openssl.org/openssl.git openssl
- /path/to/run-checker.sh
-
-To run a check on a release branch:
-
- git clone -b OpenSSL_1_1_0-stable git://git.openssl.org/openssl.git openssl
- /path/to/run-checker.sh
-
-
-Hooks
------
-
-run-checker supports a few hooks, in form of scripts that are
-executed:
-
-hook-prepare This script is run once, when run-checker is
- starting. If it exits with a status other
- than zero, run-checker.sh will stop. It gets
- no arguments.
-
-hook-start This script is run before each option build.
- If it exits with a status other than zero,
- run-checker.sh will skip the current build.
- It gets the following arguments:
-
- $1 The build directory.
- $2 The option being checked.
- $3... Configure options and arguments.
-
-hook-end This script is run after each option build.
- If gets the following arguments:
-
- $1 The build directory.
- $2 "pass" or "fail", depending on the
- build result.
-
-hook-takedown This script is run once, just before
- run-checker terminates. It gets no arguments.
-
-
-The hooks and documentation in run-checker-autohooks are an advanced
-example, and what the OpenSSL Team runs daily (automatically).
-
-
-Example hooks 1
----------------
-
-The run-checker script uses disk space by leaving every build tree
-behind! It may be that you want to clear the build tree after each
-build. This little hook can help:
-
-hook-end:
-
- #! /bin/sh
- builddir="$1"
- mv "$builddir"/build.log "$builddir".log && rm -rf "$builddir"
-
-Example hook 2
---------------
-
-This is a variant of Example hook 1 that saves away the build dir into
-a tarball:
-
-hook-end:
-
- #! /bin/sh
- builddir="$1"
- tar --remove-files -cJf "$builddir.tar.xz" "./$builddir"
-
-Example hook 3
---------------
-
-You might want to avoid some builds, based on the options. For
-example, all the fuzz builds may require installations that you're not
-willing to do. hook-start is the perfect place for this:
-
- #! /bin/sh
- if [ echo "$2" | grep -E '.*fuzz.*' ]; then exit 1; fi
- exit 0
diff --git a/run-checker/build-gost.sh b/run-checker/build-gost.sh
deleted file mode 100755
index a6f3468..0000000
--- a/run-checker/build-gost.sh
+++ /dev/null
@@ -1,41 +0,0 @@
-#! /bin/bash
-#
-# Run in a directory for a gost engine build.
-# Two subdirectories will be created:
-#
-# gost-engine a checkout of https://github.com/gost-engine/engine.git
-# openssl a checkout of https://github.com/openssl/openssl.git
-#
-# Required ubuntu packages to run this script:
-#
-# build-essential
-# cmake
-# perl
-# git
-
-if [ -d openssl ]; then
- (cd openssl; git pull --rebase)
-else
- git clone -b OpenSSL_1_1_0-stable --depth 1 --single-branch \
- https://github.com/openssl/openssl.git openssl
-fi
-
-if [ -d gost-engine ]; then
- (cd gost-engine; git pull --rebase)
-else
- git clone https://github.com/gost-engine/engine.git gost-engine
-fi
-
-OPENSSL_PREFIX=$(pwd)/openssl/_install
-(
- cd openssl
- ./config --prefix=$OPENSSL_PREFIX \
- && make -j8 build_libs \
- && make install_dev
-) && (
- cd gost-engine
- cmake -DOPENSSL_ROOT_DIR=$OPENSSL_PREFIX \
- -DCMAKE_MODULE_LINKER_FLAGS='-Wl,--enable-new-dtags' \
- .
- make
-)
diff --git a/run-checker/run-checker-autohooks/README b/run-checker/run-checker-autohooks/README
deleted file mode 100644
index 87e4be4..0000000
--- a/run-checker/run-checker-autohooks/README
+++ /dev/null
@@ -1,50 +0,0 @@
-run-checker autohooks
-=====================
-
-The purpose of these run-checker hooks is to provide a foundation for
-automatic rebuilds. Setup is easy, just create an empty directory,
-copy or symlink run-checker.sh and these hooks into it, clone an
-openssl repo into the directory "openssl", then run run-checker.sh
-repeatedly, for example as a cron job.
-
-Example setup:
-
- $ git clone git://git.openssl.org/tools.git tools
- $ mkdir ~/run-checker
- $ cd ~/run-checker
- $ ln -s ../tools/run-checker/run-checker.sh \
- ../tools/run-checker/run-checker-autohooks/hook-{prepare,start,end,takedown} \
- .
- $ git clone openssl-git at git.openssl.org:openssl.git openssl
-
-A cronjob would look like this:
-
- 2 */2 * * * cd $HOME/run-checker && bash ./run-checker.sh
-
-
-hook-config
------------
-
-In addition to the hooks, one can also have a shell script called
-`hook-config` with configuration variables. Currently understood are:
-
-- REPORT_FROM
-
- The email address used in the report From: header. This MUST be
- assigned a value or reports will not be posted.
-
-- REPORT_RECIPIENT
-
- The email address reports will get sent to. This MUST be assigned a
- value or reports will not be posted.
-
-- SKIP_OPTS
-
- MUST be a bash array, containing run-checker options that should be
- skipped. Defaults to the empty array.
-
-- ONLY_OPTS
-
- MUST be a bash array. If there are any elements, the elements are
- the only run-checker options that will be built. Defaults to the
- empty array.
diff --git a/run-checker/run-checker-autohooks/hook-end b/run-checker/run-checker-autohooks/hook-end
deleted file mode 100755
index c95051b..0000000
--- a/run-checker/run-checker-autohooks/hook-end
+++ /dev/null
@@ -1,100 +0,0 @@
-#! /bin/bash
-
-here=$(cd $(dirname $0); pwd)
-rcd=$here/.run-checker-data
-
-REPORT_RECIPIENT=
-REPORT_FROM=
-if [ -f $rcd/new/hook-config ]; then
- . $rcd/new/hook-config
-fi
-
-builddir=$1
-newstatus=$2
-
-# Created by hook-start
-optdir="$rcd/builds/$builddir"
-curoptdir="$optdir/cur"
-newoptdir="$optdir/new"
-
-if (
- set -e
-
- echo $newstatus > "$newoptdir/status"
- cp "./$builddir/build.log" "$newoptdir/build.log"
-
- # Compress the hell out of the build
- tar --remove-files -cJf "./$builddir.tar.xz" "./$builddir"
-
- curstatus=$(if [ -f "$curoptdir/status" ]; then
- cat "$curoptdir/status"
- else
- # If it was never built previously, pretend the previous
- # build passed
- echo pass
- fi)
- gitbranch=$(cat $rcd/new/branch)
- expandedopts=$(cat "$newoptdir/options")
-
- # If the build failed or the status changed since last time, report
- if [ "$newstatus" == "fail" -o "$newstatus" != "$curstatus" ]; then
- if [ -n "$REPORT_FROM" -a -n "$REPORT_RECIPIENT" ]; then
- (
- statusword=FAILED
- if [ "$newstatus" == "pass" ]; then
- statusword=SUCCESSFUL
- elif [ "$curstatus" == "fail" ]; then
- statusword="Still FAILED"
- fi
- echo "From: $REPORT_FROM"
- echo "To: $REPORT_RECIPIENT"
- echo "Subject: $statusword build of OpenSSL branch $gitbranch with options $expandedopts"
- cat <<EOF
-
-Platform and configuration command:
-
-EOF
- echo "\$ uname -a"
- uname -a
- head -1 "$newoptdir/build.log"
- if [ -f $rcd/new/rc-force-build ]; then
- cat <<EOF
-
-Forced build, latest commit is:
-
-EOF
- else
- cat <<EOF
-
-Commit log since last time:
-
-EOF
- fi
- cat "$newoptdir/log"
- if [ "$newstatus" == "fail" ]; then
- cat <<EOF
-
-Build log ended with (last 100 lines):
-
-EOF
- tail -100 "$newoptdir/build.log"
- fi
- ) | /usr/lib/sendmail -it
- fi
- fi
-
- if [ -d "$curoptdir" ]; then
- mv "$curoptdir" "$curoptdir.old.$$"
- fi
- mv "$newoptdir" "$curoptdir"
- if [ -d "$curoptdir.old.$$" ]; then
- rm -rf "$curoptdir.old.$$"
- fi
-); then
- exit 0
-fi
-
-# Something went wrong. Clear away $newoptdir and let the next run do
-# this again
-rm -rf "$newoptdir"
-exit 1
diff --git a/run-checker/run-checker-autohooks/hook-prepare b/run-checker/run-checker-autohooks/hook-prepare
deleted file mode 100755
index 25ce667..0000000
--- a/run-checker/run-checker-autohooks/hook-prepare
+++ /dev/null
@@ -1,49 +0,0 @@
-#! /bin/bash
-
-here=$(cd $(dirname $0); pwd)
-rcd=$here/.run-checker-data
-
-# The top directory has this layout:
-#
-# $rcd/new directory with new stuff... it mostly works as a lock
-#
-mkdir -p $rcd
-
-if ! mkdir $rcd/new; then
- echo >&2 "There's already a run-checker running"
- exit 1
-fi
-
-# Copy things that are of interest to the rest of the hooks. This allows
-# the original files to be added/changed without disturbing *this* run.
-if [ -f $here/hook-config ]; then
- cp $here/hook-config $rcd/new
-fi
-
-if [ -f $here/rc-force-build ]; then
- mv $here/rc-force-build $rcd/new
-fi
-
-# Every build is expected to have its own cur and new layout. For build
-# of option 'no-foo', the expected layout is:
-#
-# $rcd/builds/no-foo/new New stuff
-# $rcd/builds/no-foo/cur Stuff from the previous build
-#
-mkdir -p $rcd/builds
-
-echo Starting on $(LANG=C date)
-if (
- set -e
-
- cd openssl
- git pull --rebase
- git rev-parse HEAD > $rcd/new/head
- git rev-parse --abbrev-ref HEAD > $rcd/new/branch
-); then
- exit 0
-fi
-
-# Something went wrong. Clear away $rcd/new so it doesn't stop a later run
-rm -rf $rcd/new
-exit 1
diff --git a/run-checker/run-checker-autohooks/hook-start b/run-checker/run-checker-autohooks/hook-start
deleted file mode 100755
index cbfa64d..0000000
--- a/run-checker/run-checker-autohooks/hook-start
+++ /dev/null
@@ -1,81 +0,0 @@
-#! /bin/bash
-
-here=$(cd $(dirname $0); pwd)
-rcd=$here/.run-checker-data
-
-SKIP_OPTS=()
-ONLY_OPTS=()
-if [ -f $rcd/new/hook-config ]; then
- . $rcd/new/hook-config
-fi
-
-rcd=$here/.run-checker-data
-builddir="$1"; shift
-opt="$1"; shift
-
-if [ ${#SKIP_OPTS[@]} -gt 0 ]; then
- for x in "${SKIP_OPTS[@]}"; do
- if [ "$opt" == "$x" ]; then
- echo "Skipping building with $opt (because in SKIP_OPTS)"
- exit 1
- fi
- done
-fi
-
-if [ ${#ONLY_OPTS[@]} -gt 0 ]; then
- bail_out=/bin/true
- for x in "${ONLY_OPTS[@]}"; do
- if [ "$opt" == "$x" ]; then
- bail_out=/bin/false
- fi
- done
- if $bail_out; then
- echo "Skipping building with $opt (because not in ONLY_OPTS)"
- exit 1
- fi
-fi
-
-optdir="$rcd/builds/$builddir"
-curoptdir="$optdir/cur"
-newoptdir="$optdir/new"
-
-mkdir -p "$optdir"
-if (
- set -e
-
- mkdir "$newoptdir"
-
- echo "$@" > "$newoptdir/options"
- cp $rcd/new/head "$newoptdir/head"
- newhead=$(cat "$newoptdir/head")
-
- # The expression in each condition branch will leave behind an exit code
- # that is returned back to the calling process
- if [ -f $rcd/new/rc-force-build ]; then
- (
- cd openssl
- git log --pretty=oneline --abbrev-commit ${newhead}^..${newhead}
- ) > "$newoptdir/log"
- elif [ ! -d "$curoptdir" ]; then
- # This is an entirely new build
- (
- cd openssl
- git log --pretty=oneline --abbrev-commit ${newhead}~20..${newhead}
- echo ...
- ) > "$newoptdir/log"
- else
- curhead=$(cat "$curoptdir/head")
- (
- cd openssl
- git log --pretty=oneline --abbrev-commit ${curhead}..${newhead}
- ) > "$newoptdir/log" && [ -n "$(cat "$newoptdir/log")" ]
- fi
-); then
- exit 0
-fi
-
-# Something went wrong, or there is nothing to do. Clear away $newoptdir so
-# it doesn't stop a later run
-rm -rf "$newoptdir"
-exit 1
-
diff --git a/run-checker/run-checker-autohooks/hook-takedown b/run-checker/run-checker-autohooks/hook-takedown
deleted file mode 100755
index 8017810..0000000
--- a/run-checker/run-checker-autohooks/hook-takedown
+++ /dev/null
@@ -1,7 +0,0 @@
-#! /bin/bash
-
-here=$(cd $(dirname $0); pwd)
-rcd=$here/.run-checker-data
-
-rm -rf $rcd/new
-echo Finished on $(LANG=C date)
diff --git a/run-checker/run-checker-cleanup.sh b/run-checker/run-checker-cleanup.sh
deleted file mode 100755
index 6e8b383..0000000
--- a/run-checker/run-checker-cleanup.sh
+++ /dev/null
@@ -1,18 +0,0 @@
-#!/bin/bash
-
-#Script for emergency cleanup of the run-checker.sh work directory
-#Place this script in the same directory as run-checker.sh
-
-#This really just runs the takedown hook, so unless there are any
-#hooks present, nothing at all will happen.
-
-here=$(cd $(dirname $0); pwd)
-
-run-hook () {
- local hookname=$1; shift
- if [ -x $here/hook-$hookname ]; then
- (cd $here; ./hook-$hookname "$@")
- fi
-}
-
-run-hook takedown
diff --git a/run-checker/run-checker.sh b/run-checker/run-checker.sh
deleted file mode 100755
index b3be342..0000000
--- a/run-checker/run-checker.sh
+++ /dev/null
@@ -1,186 +0,0 @@
-#!/bin/bash
-
-#Script to check all available no- options
-#Place this script in an empty directory (apart from a few hook scripts,
-#read on).
-#In the same directory clone openssl into an openssl subdir.
-#Then run the script.
-
-#Some hook scripts can be placed in the same directory and are execute if
-#present. They are:
-#
-# hook-prepare - called before anything starts
-# hook-start - called before each option is getting built
-# Takes one argument:
-# $1 the option being built, which is also the build dir
-# $2.. the expanded options
-# hook-end - called after each option has been built
-# Takes two arguments:
-# $1 the option being built, which is also the build dir
-# $2 "pass" or "fail"
-# hook-takedown - called at the very end
-
-here=$(cd $(dirname $0); pwd)
-opts=( '' )
-
-run-hook () {
- local hookname=$1; shift
- if [ -x $here/hook-$hookname ]; then
- (cd $here; ./hook-$hookname "$@")
- fi
-}
-
-log-eval () {
- echo \$ "$@"
- eval "$@"
-}
-
-log-exec () {
- echo \$ "$@"
- exec "$@"
-}
-
-rkill () {
- local signal=$1; shift
- local pid=$1; shift
- local notpid=$1; shift
-
- if children="$(pgrep -P "$pid")"; then
- for child in $children; do
- rkill "$signal" "$child" "$notpid"
- done
- fi
- if [ "$pid" != "$notpid" ]; then
- kill -s "$signal" "$pid"
- fi
-}
-
-if [ ! -d openssl/.git ]; then
- echo >&2 "Missing openssl checkout in openssl/"
- exit 1
-fi
-
-if run-hook prepare; then
- for req_binary in clang afl-clang-fast; do
- which $req_binary >/dev/null 2>&1
- if [ "$?" != "0" ]; then
- echo "Warning: $req_binary does not appear to be in PATH"
- fi
- done
- for opt in "${opts[@]}";
- do
- expandedopts="$opt"
- warnopts="--strict-warnings"
- optcc="clang"
- ldcmd=""
- gost_engine="$OPENSSL_GOST_ENGINE_SO"
-
- if [ "$opt" == "enable-asan" ]; then
- # A documented requirement for enable-asan is no-shared
- expandedopts="enable-asan no-shared no-asm -DOPENSSL_SMALL_FOOTPRINT"
- elif [ "$opt" == "enable-ubsan" ]; then
- # We've seen it on Travis already, ubsan requires -DPEDANTIC and
- # -fno-sanitize=alignment, or crypto/modes will fail to build in
- # some circumstances. Running on a VM seems to be one of them.
- expandedopts="enable-ubsan no-asm -DPEDANTIC -DOPENSSL_SMALL_FOOTPRINT -fno-sanitize=alignment"
- elif [ "$opt" == "enable-fuzz-afl" ]; then
- warnopts=""
- optcc=afl-clang-fast
- expandedopts="enable-fuzz-afl no-shared no-module"
- elif [ "$opt" == "enable-fuzz-libfuzzer" ]; then
- warnopts=""
- ldcmd=clang++
- expandedopts="enable-fuzz-libfuzzer --with-fuzzer-include=../../Fuzzer --with-fuzzer-lib=../../Fuzzer/libFuzzer -DPEDANTIC enable-asan enable-ubsan no-shared"
- elif [ "$opt" == "no-static-engine" ]; then
- expandedopts="no-static-engine no-shared"
- elif [ "$opt" == "no-deprecated" ]; then
- #The gost engine uses some deprecated symbols so we don't use it
- #in a no-deprecated build
- gost_engine=""
- elif [ "$opt" == "no-cached-fetch" ]; then
- expandedopts="no-cached-fetch enable-asan enable-ubsan"
- fi
-
- if [ -z "$opt" ]; then
- builddir=default
- else
- builddir="$(echo $opt | sed -e 's|[ /]|_|g')"
- fi
- if run-hook start "$builddir" "$opt" $warnopts $expandedopts; then
- if (
- set -e
-
- if [ ! -d "./$builddir" ]; then
- mkdir "./$builddir"
- fi
- cd "./$builddir"
-
- echo "Building with '$opt'"
- log-eval \
- CC=$optcc ../openssl/config $warnopts $expandedopts \
- >build.log 2>&1 || \
- exit $?
-
- echo " make clean"
- log-eval make clean >>build.log 2>&1 || exit $?
-
- echo " make depend"
- log-eval make depend >>build.log 2>&1 || exit $?
-
- echo " make -j4"
- log-eval LDCMD=$ldcmd make -j4 >>build.log 2>&1 || exit $?
-
- # Because 'make test' may hang under certain circumstances,
- # we have a timeout mechanism around it.
- (
- testpid=$BASHPID
-
- # Number of seconds to wait for command completion.
- # (3600 = one hour)
- timeout=3600
- # Interval between checks if the process is still alive.
- interval=5
- # Delay between posting the SIGTERM signal and destroying
- # the process by SIGKILL.
- delay=1
-
- # kill -0 pid
- # Exit code indicates if a signal may be sent to $testpid
- # process.
- (
- ((t = timeout))
-
- while ((t > 0)); do
- sleep $interval
- kill -0 $testpid || exit 0
- ((t -= interval))
- done
-
- # Be nice, post SIGTERM first.
- # The 'exit 0' below will be executed if any preceeding
- # command fails.
- rkill SIGTERM $testpid $BASHPID && kill -0 $testpid \
- || exit 0
- sleep $delay
- rkill SIGKILL $testpid $BASHPID
- ) 2> /dev/null &
-
- # If not set to another value, default to 4 test jobs
- echo " make test"
- HARNESS_JOBS=${HARNESS_JOBS:-4} OPENSSL_GOST_ENGINE_SO="$gost_engine" log-exec make test >>build.log 2>&1
- )
- ); then
- echo " PASS"
- run-hook end "$builddir" pass
- else
- echo " FAILED"
- run-hook end "$builddir" fail
- if [ "$opt" = "" ]; then
- break
- fi
- fi
- fi
- done
-
- run-hook takedown
-fi
More information about the openssl-commits
mailing list