2017-09-10 16:44:28 +02:00
|
|
|
#!/bin/sh
|
|
|
|
#
|
|
|
|
# Print output of failing tests
|
|
|
|
#
|
|
|
|
|
2019-01-28 00:26:50 +01:00
|
|
|
. ${0%/*}/lib.sh
|
2017-09-10 16:44:28 +02:00
|
|
|
|
2017-12-27 17:36:00 +01:00
|
|
|
# Tracing executed commands would produce too much noise in the loop below.
|
|
|
|
set +x
|
|
|
|
|
travis-ci: include the trash directories of failed tests in the trace log
The trash directory of a failed test might contain invaluable
information about the cause of the failure, but we have no access to
the trash directories of Travis CI build jobs. The only feedback we
get from there is the build job's trace log, so...
Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the
trash directory of each failed test, encode that archive with base64,
and print the resulting block of ASCII text, so it gets embedded in
the trace log. Furthermore, run tests with '--immediate' to
faithfully preserve the failed state.
Extracting the trash directories from the trace log turned out to be a
bit of a hassle, partly because of the size of these logs (usually
resulting in several hundreds or even thousands of lines of
base64-encoded text), and partly because these logs have CRLF, CRCRLF
and occasionally even CRCRCRLF line endings, which cause 'base64 -d'
from coreutils to complain about "invalid input". For convenience add
a small script 'ci/util/extract-trash-dirs.sh', which will extract and
unpack all base64-encoded trash directories embedded in the log fed to
its standard input, and include an example command to be copy-pasted
into a terminal to do it all at the end of the failure report.
A few of our tests create sizeable trash directories, so limit the
size of each included base64-encoded block, let's say, to 1MB. And
just in case something fundamental gets broken and a lot of tests fail
at once, don't include trash directories when the combined size of the
included base64-encoded blocks would exceed 1MB.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
|
|
|
cd t/
|
|
|
|
|
|
|
|
if ! ls test-results/*.exit >/dev/null 2>/dev/null
|
2017-12-27 17:36:03 +01:00
|
|
|
then
|
|
|
|
echo "Build job failed before the tests could have been run"
|
|
|
|
exit
|
|
|
|
fi
|
|
|
|
|
travis-ci: include the trash directories of failed tests in the trace log
The trash directory of a failed test might contain invaluable
information about the cause of the failure, but we have no access to
the trash directories of Travis CI build jobs. The only feedback we
get from there is the build job's trace log, so...
Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the
trash directory of each failed test, encode that archive with base64,
and print the resulting block of ASCII text, so it gets embedded in
the trace log. Furthermore, run tests with '--immediate' to
faithfully preserve the failed state.
Extracting the trash directories from the trace log turned out to be a
bit of a hassle, partly because of the size of these logs (usually
resulting in several hundreds or even thousands of lines of
base64-encoded text), and partly because these logs have CRLF, CRCRLF
and occasionally even CRCRCRLF line endings, which cause 'base64 -d'
from coreutils to complain about "invalid input". For convenience add
a small script 'ci/util/extract-trash-dirs.sh', which will extract and
unpack all base64-encoded trash directories embedded in the log fed to
its standard input, and include an example command to be copy-pasted
into a terminal to do it all at the end of the failure report.
A few of our tests create sizeable trash directories, so limit the
size of each included base64-encoded block, let's say, to 1MB. And
just in case something fundamental gets broken and a lot of tests fail
at once, don't include trash directories when the combined size of the
included base64-encoded blocks would exceed 1MB.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
|
|
|
case "$jobname" in
|
|
|
|
osx-clang|osx-gcc)
|
|
|
|
# base64 in OSX doesn't wrap its output at 76 columns by
|
|
|
|
# default, but prints a single, very long line.
|
|
|
|
base64_opts="-b 76"
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
|
|
|
|
combined_trash_size=0
|
|
|
|
for TEST_EXIT in test-results/*.exit
|
2017-09-11 03:18:29 +02:00
|
|
|
do
|
|
|
|
if [ "$(cat "$TEST_EXIT")" != "0" ]
|
|
|
|
then
|
|
|
|
TEST_OUT="${TEST_EXIT%exit}out"
|
|
|
|
echo "------------------------------------------------------------------------"
|
|
|
|
echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
|
|
|
|
echo "------------------------------------------------------------------------"
|
|
|
|
cat "${TEST_OUT}"
|
travis-ci: include the trash directories of failed tests in the trace log
The trash directory of a failed test might contain invaluable
information about the cause of the failure, but we have no access to
the trash directories of Travis CI build jobs. The only feedback we
get from there is the build job's trace log, so...
Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the
trash directory of each failed test, encode that archive with base64,
and print the resulting block of ASCII text, so it gets embedded in
the trace log. Furthermore, run tests with '--immediate' to
faithfully preserve the failed state.
Extracting the trash directories from the trace log turned out to be a
bit of a hassle, partly because of the size of these logs (usually
resulting in several hundreds or even thousands of lines of
base64-encoded text), and partly because these logs have CRLF, CRCRLF
and occasionally even CRCRCRLF line endings, which cause 'base64 -d'
from coreutils to complain about "invalid input". For convenience add
a small script 'ci/util/extract-trash-dirs.sh', which will extract and
unpack all base64-encoded trash directories embedded in the log fed to
its standard input, and include an example command to be copy-pasted
into a terminal to do it all at the end of the failure report.
A few of our tests create sizeable trash directories, so limit the
size of each included base64-encoded block, let's say, to 1MB. And
just in case something fundamental gets broken and a lot of tests fail
at once, don't include trash directories when the combined size of the
included base64-encoded blocks would exceed 1MB.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
|
|
|
|
|
|
|
test_name="${TEST_EXIT%.exit}"
|
|
|
|
test_name="${test_name##*/}"
|
|
|
|
trash_dir="trash directory.$test_name"
|
ci/lib.sh: encapsulate Travis-specific things
The upcoming patches will allow building git.git via Azure Pipelines
(i.e. Azure DevOps' Continuous Integration), where variable names and
URLs look a bit different than in Travis CI.
Also, the configurations of the available agents are different. For
example, Travis' and Azure Pipelines' macOS agents are set up
differently, so that on Travis, we have to install the git-lfs and
gettext Homebrew packages, and on Azure Pipelines we do not need to.
Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor
installed.
Finally, on Azure Pipelines the natural way is not to base64-encode tar
files of the trash directories of failed tests, but to publish build
artifacts instead. Therefore, that code to log those base64-encoded tar
files is guarded to be Travis-specific.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 00:26:51 +01:00
|
|
|
case "$CI_TYPE" in
|
|
|
|
travis)
|
|
|
|
;;
|
2019-01-29 15:19:28 +01:00
|
|
|
azure-pipelines)
|
|
|
|
mkdir -p failed-test-artifacts
|
|
|
|
mv "$trash_dir" failed-test-artifacts
|
|
|
|
continue
|
|
|
|
;;
|
2020-04-10 19:18:14 +02:00
|
|
|
github-actions)
|
|
|
|
mkdir -p failed-test-artifacts
|
|
|
|
echo "::set-env name=FAILED_TEST_ARTIFACTS::t/failed-test-artifacts"
|
|
|
|
cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
|
|
|
|
tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
|
|
|
|
continue
|
|
|
|
;;
|
ci/lib.sh: encapsulate Travis-specific things
The upcoming patches will allow building git.git via Azure Pipelines
(i.e. Azure DevOps' Continuous Integration), where variable names and
URLs look a bit different than in Travis CI.
Also, the configurations of the available agents are different. For
example, Travis' and Azure Pipelines' macOS agents are set up
differently, so that on Travis, we have to install the git-lfs and
gettext Homebrew packages, and on Azure Pipelines we do not need to.
Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor
installed.
Finally, on Azure Pipelines the natural way is not to base64-encode tar
files of the trash directories of failed tests, but to publish build
artifacts instead. Therefore, that code to log those base64-encoded tar
files is guarded to be Travis-specific.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 00:26:51 +01:00
|
|
|
*)
|
|
|
|
echo "Unhandled CI type: $CI_TYPE" >&2
|
|
|
|
exit 1
|
|
|
|
;;
|
|
|
|
esac
|
travis-ci: include the trash directories of failed tests in the trace log
The trash directory of a failed test might contain invaluable
information about the cause of the failure, but we have no access to
the trash directories of Travis CI build jobs. The only feedback we
get from there is the build job's trace log, so...
Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the
trash directory of each failed test, encode that archive with base64,
and print the resulting block of ASCII text, so it gets embedded in
the trace log. Furthermore, run tests with '--immediate' to
faithfully preserve the failed state.
Extracting the trash directories from the trace log turned out to be a
bit of a hassle, partly because of the size of these logs (usually
resulting in several hundreds or even thousands of lines of
base64-encoded text), and partly because these logs have CRLF, CRCRLF
and occasionally even CRCRCRLF line endings, which cause 'base64 -d'
from coreutils to complain about "invalid input". For convenience add
a small script 'ci/util/extract-trash-dirs.sh', which will extract and
unpack all base64-encoded trash directories embedded in the log fed to
its standard input, and include an example command to be copy-pasted
into a terminal to do it all at the end of the failure report.
A few of our tests create sizeable trash directories, so limit the
size of each included base64-encoded block, let's say, to 1MB. And
just in case something fundamental gets broken and a lot of tests fail
at once, don't include trash directories when the combined size of the
included base64-encoded blocks would exceed 1MB.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
|
|
|
trash_tgz_b64="trash.$test_name.base64"
|
|
|
|
if [ -d "$trash_dir" ]
|
|
|
|
then
|
|
|
|
tar czp "$trash_dir" |base64 $base64_opts >"$trash_tgz_b64"
|
|
|
|
|
|
|
|
trash_size=$(wc -c <"$trash_tgz_b64")
|
|
|
|
if [ $trash_size -gt 1048576 ]
|
|
|
|
then
|
|
|
|
# larger than 1MB
|
|
|
|
echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, it's too big$(tput sgr0)"
|
|
|
|
continue
|
|
|
|
fi
|
|
|
|
|
|
|
|
new_combined_trash_size=$(($combined_trash_size + $trash_size))
|
|
|
|
if [ $new_combined_trash_size -gt 1048576 ]
|
|
|
|
then
|
|
|
|
echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, there is plenty of trash in there already.$(tput sgr0)"
|
|
|
|
continue
|
|
|
|
fi
|
|
|
|
combined_trash_size=$new_combined_trash_size
|
|
|
|
|
|
|
|
# DO NOT modify these two 'echo'-ed strings below
|
|
|
|
# without updating 'ci/util/extract-trash-dirs.sh'
|
|
|
|
# as well.
|
|
|
|
echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)"
|
|
|
|
cat "$trash_tgz_b64"
|
|
|
|
echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)"
|
|
|
|
fi
|
2017-09-11 03:18:29 +02:00
|
|
|
fi
|
2017-09-10 16:44:28 +02:00
|
|
|
done
|
travis-ci: include the trash directories of failed tests in the trace log
The trash directory of a failed test might contain invaluable
information about the cause of the failure, but we have no access to
the trash directories of Travis CI build jobs. The only feedback we
get from there is the build job's trace log, so...
Modify 'ci/print-test-failures.sh' to create a tar.gz archive of the
trash directory of each failed test, encode that archive with base64,
and print the resulting block of ASCII text, so it gets embedded in
the trace log. Furthermore, run tests with '--immediate' to
faithfully preserve the failed state.
Extracting the trash directories from the trace log turned out to be a
bit of a hassle, partly because of the size of these logs (usually
resulting in several hundreds or even thousands of lines of
base64-encoded text), and partly because these logs have CRLF, CRCRLF
and occasionally even CRCRCRLF line endings, which cause 'base64 -d'
from coreutils to complain about "invalid input". For convenience add
a small script 'ci/util/extract-trash-dirs.sh', which will extract and
unpack all base64-encoded trash directories embedded in the log fed to
its standard input, and include an example command to be copy-pasted
into a terminal to do it all at the end of the failure report.
A few of our tests create sizeable trash directories, so limit the
size of each included base64-encoded block, let's say, to 1MB. And
just in case something fundamental gets broken and a lot of tests fail
at once, don't include trash directories when the combined size of the
included base64-encoded blocks would exceed 1MB.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 00:56:12 +02:00
|
|
|
|
|
|
|
if [ $combined_trash_size -gt 0 ]
|
|
|
|
then
|
|
|
|
echo "------------------------------------------------------------------------"
|
|
|
|
echo "Trash directories embedded in this log can be extracted by running:"
|
|
|
|
echo
|
|
|
|
echo " curl https://api.travis-ci.org/v3/job/$TRAVIS_JOB_ID/log.txt |./ci/util/extract-trash-dirs.sh"
|
|
|
|
fi
|