Add the capability to send smoke reports from the Git test suite.
Currently we only notice bugs in the test suite when it's run
manually. Bugs in Git that only occur on obscure platforms or setups
that the core developers aren't using can thus go unnoticed.
This series aims to change that. With it, anyone that's interested in
avoiding bitrot in Git can volunteer to run a smoke tester. A smoke
tester periodically compiles the latest version of Git, runs the test
suite, and submits a report to a central server indicating how the
test run went.
A smoke tester might run something like this in cron:
#!/bin/sh
cd ~/g/git
git fetch
for branch in maint master next pu; do
git checkout origin/$i &&
make clean all &&
cd t &&
make smoke_report
done
The smoker might want to compile git with non-default flags, include
bisecting functionality or run the tests under valgrind. Doing that is
outside the scope of this patch, this just adds a report submission
mechanism. But including a canonical smoke runner is something we'll
want to include eventually.
What this does now is add smoke and smoke_report targets to t/Makefile
(this example only uses a few tests for demonstration):
$ make clean smoke
rm -f -r 'trash directory'.* test-results
rm -f t????/cvsroot/CVSROOT/?*
rm -f -r valgrind/bin
rm -f .prove
perl ./harness --git-version="1.7.2.1.173.gc9b40" \
--no-verbose \
--archive="test-results/git-smoke.tar.gz" \
t0000-basic.sh t0001-init.sh t0002-gitfile.sh t0003-attributes.sh t0004-unwritable.sh t0005-signals.sh t0006-date.sh
t0000-basic.sh ....... ok
t0001-init.sh ........ ok
t0002-gitfile.sh ..... ok
t0003-attributes.sh .. ok
t0004-unwritable.sh .. ok
t0005-signals.sh ..... ok
t0006-date.sh ........ ok
All tests successful.
Test Summary Report
-------------------
t0000-basic.sh (Wstat: 0 Tests: 46 Failed: 0)
TODO passed: 5
Files=7, Tests=134, 3 wallclock secs ( 0.06 usr 0.05 sys + 0.23 cusr 1.33 csys = 1.67 CPU)
Result: PASS
TAP Archive created at /home/avar/g/git/t/test-results/git-smoke.tar.gz
The smoke target uses TAP::Harness::Archive to aggregate the test
results into a tarball. The tarball contains two things, the output of
every test file that was run, and a metadata file:
Tarball contents:
$ tar xzvf git-smoke.tar.gz
t0004-unwritable.sh
t0001-init.sh
t0002-gitfile.sh
t0005-signals.sh
t0000-basic.sh
t0003-attributes.sh
t0006-date.sh
meta.yml
A test report:
$ cat t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
A metadata file:
---
extra_properties:
file_attributes:
-
description: t0000-basic.sh
end_time: 1280437324.61398
start_time: 1280437324.22186
-
description: t0001-init.sh
end_time: 1280437325.12346
start_time: 1280437324.62393
-
description: t0002-gitfile.sh
end_time: 1280437325.29428
start_time: 1280437325.13646
-
description: t0003-attributes.sh
end_time: 1280437325.59678
start_time: 1280437325.30565
-
description: t0004-unwritable.sh
end_time: 1280437325.77376
start_time: 1280437325.61003
-
description: t0005-signals.sh
end_time: 1280437325.85426
start_time: 1280437325.78727
-
description: t0006-date.sh
end_time: 1280437326.2362
start_time: 1280437325.86768
file_order:
- t0000-basic.sh
- t0001-init.sh
- t0002-gitfile.sh
- t0003-attributes.sh
- t0004-unwritable.sh
- t0005-signals.sh
- t0006-date.sh
start_time: 1280437324
stop_time: 1280437326
The "extra_properties" hash is where we'll stick Git-specific info,
like whether Git was compiled with gettext or the fallback regex
engine, and what branch we're compiling. Currently no metadata like
this is included.
The entire tarball is then submitted to a central smokebox at
smoke.git.nix.is. This is done with curl(1) via the "smoke_report"
target:
$ make smoke_report
curl \
-H "Expect: " \
-F project=Git \
-F architecture=x86_64 \
-F platform=Linux \
-F revision="1.7.2.1.173.gc9b40" \
-F report_file=@test-results/git-smoke.tar.gz \
http://smoke.git.nix.is/app/projects/process_add_report/1 \
| grep -v ^Redirecting
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 117k 100 63 100 117k 3 6430 0:00:21 0:00:18 0:00:03 0
Reported #8 added.
Reports are then made available on the smokebox via a web interface:
http://smoke.git.nix.is/app/projects/smoke_reports/1
The smoke reports are also mirrored to a Git repository hosted on
GitHub:
http://github.com/gitsmoke/smoke-reports
The Smolder SQLite database that contains metadata about the reports
is also made available:
http://github.com/gitsmoke/smoke-database
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
////////////////////////////////////////////////////////////////
GIT - the stupid content tracker
////////////////////////////////////////////////////////////////
"git" can mean anything, depending on your mood.
- random three-letter combination that is pronounceable, and not
actually used by any common UNIX command. The fact that it is a
mispronunciation of "get" may or may not be relevant.
- stupid. contemptible and despicable. simple. Take your pick from the
dictionary of slang.
- "global information tracker": you're in a good mood, and it actually
works for you. Angels sing, and a light suddenly fills the room.
- "goddamn idiotic truckload of sh*t": when it breaks
Git is a fast, scalable, distributed revision control system with an
unusually rich command set that provides both high-level operations
and full access to internals.
Git is an Open Source project covered by the GNU General Public License.
It was originally written by Linus Torvalds with help of a group of
hackers around the net. It is currently maintained by Junio C Hamano.
Please read the file INSTALL for installation instructions.
See Documentation/gittutorial.txt to get started, then see
Documentation/everyday.txt for a useful minimum set of commands, and
Documentation/git-commandname.txt for documentation of each command.
If git has been correctly installed, then the tutorial can also be
read with "man gittutorial" or "git help tutorial", and the
documentation of each command with "man git-commandname" or "git help
commandname".
CVS users may also want to read Documentation/gitcvs-migration.txt
("man gitcvs-migration" or "git help cvs-migration" if git is
installed).
Many Git online resources are accessible from http://git-scm.com/
including full documentation and Git related tools.
The user discussion and development of Git take place on the Git
mailing list -- everyone is welcome to post bug reports, feature
requests, comments and patches to git@vger.kernel.org. To subscribe
to the list, send an email with just "subscribe git" in the body to
majordomo@vger.kernel.org. The mailing list archives are available at
http://marc.theaimsgroup.com/?l=git and other archival sites.
The messages titled "A note from the maintainer", "What's in
git.git (stable)" and "What's cooking in git.git (topics)" and
the discussion following them on the mailing list give a good
reference for project status, development direction and
remaining tasks.
b6b84d1b74