With the new --valgrind-only=<pattern> option, one can enable
--valgrind at a per-test granularity, exactly analogous to
--verbose-only from the previous commit.
The options are wired such that --valgrind implies --verbose (as
before), but --valgrind-only=<pattern> implies
--verbose-only=<pattern> unless --verbose is also in effect.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the new --verbose-only=<pattern> option, one can enable --verbose
at a per-test granularity. The pattern is matched against the test
number, e.g.
./t0000-basic.sh --verbose-only='2[0-2]'
to see only the full output of test 20-22, while showing the rest in the
one-liner format.
As suggested by Jeff King, this takes care to wrap the entire
test_expect_* block, but nothing else, in the verbose toggling. We
can use the test_start/end functions from the previous commit for the
purpose.
This is arguably not *too* useful on its own, but makes the next patch
easier to follow.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t0000 contains some light self-tests of test-lib.sh, but --verbose was
not covered. Add a test.
The only catch is that the presence of a test harness influences the
output (specifically, the presence of some empty lines). So we need
to unset TEST_HARNESS or set it to a known value. Leaving it unset
leads to spurious test failures in the final summary, which come from
the subtest. So we always set it.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This moves
* the early setup part from test_skip to a new function test_start_
* the final common parts of test_expect_* to a new function
test_finish_
to make the next commit more obvious.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's already used twice, and we will have more of the same kind of
matching in a minute.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
1b3185f (MALLOC_CHECK: various clean-ups, 2012-09-14) moved around the
MALLOC_CHECK_ and MALLOC_PERTURB_ assignments, intending to limit
their effect to only the test runs. However, they were actually
enabled only during test cleanup. Call setup/teardown_malloc_check
also around the evaluation of the actual test snippet.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When TEST_OUTPUT_DIRECTORY setting is used, it was handled somewhat
inconsistently between the test framework and t/Makefile, and logic
to summarize the results looked at a wrong place.
* jk/test-output:
t/Makefile: don't define TEST_RESULTS_DIRECTORY recursively
test output: respect $TEST_OUTPUT_DIRECTORY
t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY
Most test results go in $TEST_OUTPUT_DIRECTORY, but the output files for
tests run with --tee or --valgrind just use bare "test-results".
Changes these so that they do respect $TEST_OUTPUT_DIRECTORY.
As a result of this, the valgrind/analyze.sh script may no longer
inspect the correct files so it is also updated to respect
$TEST_OUTPUT_DIRECTORY by adding it to GIT-BUILD-OPTIONS. This may be a
regression for people who have TEST_OUTPUT_DIRECTORY in their config.mak
but want to override it in the environment, but this change merely
brings it into line with GIT_TEST_OPTS which already cannot be
overridden if it is specified in config.mak.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix longstanding issues with the test harness when used with --root=<there>
option.
* jk/test-trash:
t/test-lib.sh: drop "$test" variable
t/test-lib.sh: fix TRASH_DIRECTORY handling
The $test variable is used as an interim buffer for
constructing $TRASH_DIRECTORY, and is almost compatible with
it (the exception being that $test has not been converted to
an absolute path). Let's get rid of it entirely so that
later code does not accidentally use it, thinking the two
are interchangeable.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After the location of $TRASH_DIRECTORY is adjusted by
$TEST_OUTPUT_DIRECTORY, we go on to use the $test variable to make the
trash directory and cd into it. This means that when
$TEST_OUTPUT_DIRECTORY is not "." and an absolute --root has not been
specified, we do not remove the trash directory once the tests are
complete (remove_trash is set to $TRASH_DIRECTORY).
Fix this by always referring to the trash directory as $TRASH_DIRECTORY.
Signed-off-by: John Keeping <john@keeping.me.uk>
Acked-by: Jeff King <peff@peff.net>
Acked-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Typo fix: replacing it's -> its
t: make PIPE a standard test prerequisite
archive: clarify explanation of --worktree-attributes
t/README: --immediate skips cleanup commands for failed tests
The 'PIPE' test prerequisite was already defined identically by t9010
and t9300, therefore it makes sense to make it a predefined
prerequisite.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git archive" reports a failure when asked to create an archive out
of an empty tree. It would be more intuitive to give an empty
archive back in such a case.
* jk/empty-archive:
archive: handle commits with an empty tree
test-lib: factor out $GIT_UNZIP setup
Running tests under helgrind and DRD recently proved useful in
tracking down thread interaction issues. This can unfortunately not
be done through GIT_VALGRIND_OPTIONS because any tool other than
memcheck would complain about unknown options.
Let --valgrind take an optional parameter that describes the valgrind
tool to invoke. The default mode is to run memcheck as before.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git archive" reports a failure when asked to create an archive out
of an empty tree. It would be more intuitive to give an empty
archive back in such a case.
* jk/empty-archive:
archive: handle commits with an empty tree
test-lib: factor out $GIT_UNZIP setup
We set up the $GIT_UNZIP variable and lazy prereq in
multiple places (and the next patch is about to add another
one). Let's factor it out to avoid repeating ourselves.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Native windows binaries do not understand posix-like
path mapping offered by cygwin. Convert paths to native
using "cygpath --windows" before presenting them to p4d.
This is done using the AltRoots mechanism of p4. Both the
posix and windows forms are put in the client specification,
allowing p4 to find its location by native path even though
the environment reports a different PWD.
Shell operations in tests will use the normal form of $cli,
which will look like a posix path in cygwin, while p4 will
use AltRoots to match against the windows form of the working
directory.
This mechanism also handles the symlink issue that was fixed in
23bd0c9 (git p4 test: use real_path to resolve p4 client
symlinks, 2012-06-27). Now that every p4 client view has
an AltRoots with the real_path in it, explicitly calculating
the real_path elsewhere is not necessary.
Thanks-to: Sebastian Schuberth <sschuberth@gmail.com>
Thanks-to: Johannes Sixt <j6t@kdbg.org>
fixup! git p4 test: translate windows paths for cygwin
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* rs/zip-tests:
t5003: check if unzip supports symlinks
t5000, t5003: move ZIP tests into their own script
t0024, t5000: use test_lazy_prereq for UNZIP
t0024, t5000: clear variable UNZIP, use GIT_UNZIP instead
These variables are user parameters to control how to run the perf
tests. Allow users to do so.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update zip tests to skip some that cannot be handled on platform
unzip.
* rs/zip-tests:
t5003: check if unzip supports symlinks
t5000, t5003: move ZIP tests into their own script
t0024, t5000: use test_lazy_prereq for UNZIP
t0024, t5000: clear variable UNZIP, use GIT_UNZIP instead
InfoZIP's unzip takes default parameters from the environment variable
UNZIP. Unset it in the test library and use GIT_UNZIP for specifying
alternate versions of the unzip command instead.
t0024 wasn't even using variable for the actual extraction. t5000
was, but when setting it to InfoZIP's unzip it would try to extract
from itself (because it treats the contents of $UNZIP as parameters),
which failed of course.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Output from the tests is coloured using "green is okay, yellow is
questionable, red is bad and blue is informative" scheme.
* as/test-tweaks:
tests: paint unexpectedly fixed known breakages in bold red
tests: test the test framework more thoroughly
tests: refactor mechanics of testing in a sub test-lib
tests: change info messages from yellow/brown to cyan
tests: paint skipped tests in blue
tests: paint known breakages in yellow
tests: test number comes first in 'not ok $count - $message'
Change color of unexpectedly fixed known breakages to bold red. An
unexpectedly passing test indicates that the test code is somehow
broken or out of sync with the code it is testing. Either way this is
an error which is potentially as bad as a failing test, and as such is
no longer portrayed as a pass in the output.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that we've adopted a "traffic lights" coloring scheme, yellow is
used for warning messages, so we need to re-color info messages to
something less alarmist. Blue is a universal color for informational
messages; however we are using that for skipped tests in order to
align with the color schemes of other test suites. Therefore we use
cyan which is also blue-ish, but visually distinct from blue.
This was suggested on the list a while ago and no-one raised any
objections:
http://thread.gmane.org/gmane.comp.version-control.git/205675/focus=205966
An earlier iteration of this patch used bold cyan, but the point of
this change is to make them less alarming; let's drop the boldness.
Also paint the message to report skipping the whole thing via
GIT_SKIP_TESTS mechanism in the same color as the "info" color
that is used on the final summary line for the entire script.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Skipped tests indicate incomplete test coverage. Whilst this is not a
test failure or other error, it's still not a complete success.
Other testsuite related software like automake, autotest and prove
seem to use blue for skipped tests, so let's follow suit.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Yellow seems a more appropriate color than bold green when
considering the universal traffic lights coloring scheme, where
green conveys the impression that everything's OK, and amber that
something's not quite right.
Likewise, change the color of the summarized total number of known
breakages from bold red to the same yellow to be less alarmist and
more consistent with the above.
An earlier version of this patch used bold yellow but because these
are all long-known failures, reminding them to developers in bold
over and over does not help encouraging them to take a look at them
very much. This iteration paints them in plain yellow instead to
make them less distracting.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The old output to say "not ok - 1 messsage" was working by accident
only because the test numbers are optional in TAP.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
New remote helper for hg.
* fc/remote-hg: (22 commits)
remote-hg: fix for older versions of python
remote-hg: fix for files with spaces
remote-hg: avoid bad refs
remote-hg: try the 'tip' if no checkout present
remote-hg: fix compatibility with older versions of hg
remote-hg: add missing config for basic tests
remote-hg: the author email can be null
remote-hg: add option to not track branches
remote-hg: add extra author test
remote-hg: add tests to compare with hg-git
remote-hg: add bidirectional tests
test-lib: avoid full path to store test results
remote-hg: add basic tests
remote-hg: fake bookmark when there's none
remote-hg: add compat for hg-git author fixes
remote-hg: add support for hg-git compat mode
remote-hg: match hg merge behavior
remote-hg: make sure the encoding is correct
remote-hg: add support to push URLs
remote-hg: add support for remote pushing
...
The "say" function in the test scaffolding incorrectly allowed
"echo" to interpret "\a" as if it were a C-string asking for a BEL
output.
* jc/test-say-color-avoid-echo-escape:
test-lib: Fix say_color () not to interpret \a\b\c in the message
t7502 checks the behavior of commit when we can and cannot
determine a valid committer ident. Let's move that into
test-lib as a lazy prerequisite so other scripts can use it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
No reason to use the full path in case this is used externally.
Otherwise we might get errors such as:
./test-lib.sh: line 394: /home/bob/dev/git/t/test-results//home/bob/dev/git/contrib/remote-hg/test-2894.counts: No such file or directory
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Recent nd/wildmatch series was the first to reveal this ancient bug
in the test scaffolding.
* jc/test-say-color-avoid-echo-escape:
test-lib: Fix say_color () not to interpret \a\b\c in the message
When running with color disabled (e.g. under prove to produce TAP
output), say_color() helper function is defined to use echo to show
the message. With a message that ends with "\c", echo is allowed to
interpret it as "Do not end the line with LF".
Use printf "%s\n" to emit the message literally.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The malloc checks in tests are currently disabled. Actually evaluate
the variable for turning them off and enable them if it's unset.
Also use this opportunity to give it the more descriptive and
consistent name TEST_NO_MALLOC_CHECK.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Only the first test t0000 in the test suite made sure we have built
Git to be tested; move the check to test-lib so that it applies to
all tests equally.
* rr/test-make-sure-we-have-git:
t/test-lib: make sure Git has already been built
The codepath for handling "--tee" ends up relaunching the test
script under a shell, and that one has to be a Bourne. But we
incorrectly used $SHELL, which could be a non-Bourne (e.g. zsh or
csh); we have the Makefile variable $SHELL_PATH for exactly that,
so use it instead.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When tests were run without building git, they stopped with:
.: 54: Can't open /path/to/git/source/t/../GIT-BUILD-OPTIONS
Move the check that makes sure that git has already been built from
t0000 to test-lib, so that any test will do so before it runs.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The most important in this change is to avoid affecting anything
when test-lib is used from perf-lib. It also limits the effect of
the MALLOC_CHECK only to what is run inside the actual test, and
uses a fixed MALLOC_PERTURB_ in order to avoid hurting repeatability
of the tests.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Recent versions of Linux libc (later than 5.4.23) and glibc (2.x)
include a malloc() implementation which is tunable via environment
variables. When MALLOC_CHECK_ is set, a special (less efficient)
implementation is used which is designed to be tolerant against
simple errors, such as double calls of free() with the same argument,
or overruns of a single byte (off-by-one bugs). When MALLOC_CHECK_
is set to 3, a diagnostic message is printed on stderr
and the program is aborted.
Setting the MALLOC_PERTURB_ environment variable causes the malloc
functions in libc to return memory which has been wiped and clear
memory when it is returned.
Of course this does not affect calloc which always does clear the memory.
The reason for this exercise is, of course, to find code which uses
memory returned by malloc without initializing it and code which uses
code after it is freed. valgrind can do this but it's costly to run.
The MALLOC_PERTURB_ exchanges the ability to detect problems in 100%
of the cases with speed.
The byte value used to initialize values returned by malloc is the byte
value of the environment value. The value used to clear memory is the
bitwise inverse. Setting MALLOC_PERTURB_ to zero disables the feature.
This technique can find hard to detect bugs.
It is therefore suggested to always use this flag (at least temporarily)
when testing out code or a new distribution.
But the test suite can use also valgrind(memcheck) via 'make valgrind'
or 'make GIT_TEST_OPTS="--valgrind"'.
Memcheck wraps client calls to malloc(), and puts a "red zone" on
each end of each block in order to detect access overruns.
Memcheck already detects double free() (up to the limit of the buffer
which remembers pending free()). Thus memcheck subsumes all the
documented coverage of MALLOC_CHECK_.
If MALLOC_CHECK_ is set non-zero when running memcheck, then the
overruns that might be detected by MALLOC_CHECK_ would be overruns
on the wrapped blocks which include the red zones. Thus MALLOC_CHECK_
would be checking memcheck, and not the client. This is not useful,
and actually is wasteful. The only possible [documented] advantage
of using MALLOC_CHECK_ and memcheck together, would be if MALLOC_CHECK_
detected duplicate free() in more cases than memcheck because memcheck's
buffer is too small.
Therefore we don't use MALLOC_CHECK_ and valgrind(memcheck) at the
same time.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* rj/tap-fix:
test-lib.sh: Suppress the "passed all ..." message if no tests run
test-lib.sh: Add check for invalid use of 'skip_all' facility
test-lib.sh: Fix some shell coding style violations
t4016-*.sh: Skip all tests rather than each test
t3902-*.sh: Skip all tests rather than each test
t3300-*.sh: Fix a TAP parse error
If a test script issues a test_done without executing any tests, for
example when using the 'skip_all' facility, the output looks something
like this:
$ ./t9159-git-svn-no-parent-mergeinfo.sh
# passed all 0 test(s)
1..0 # SKIP skipping git svn tests, svn not found
$
The "passed all 0 test(s)" comment line, while correct, looks a little
strange. Add a check to suppress this message if no tests have actually
been run.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'skip_all' facility cannot be used after one or more tests
have been executed using (for example) 'test_expect_success'.
To do so results in invalid TAP output, which leads to 'prove'
complaining of "Parse errors: No plan found in TAP output".
Add a check for such invalid usage and abort the test with an
error message to alert the test author.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
UTF8 behaviour of the filesystem (conversion from nfd to nfc) plays a
role in several tests and is tested in several tests. Therefore, move
the test from t0050 into the test lib and use the prerequisite in t0050.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Case insensitivity plays a role in several tests and is tested in several
tests. Therefore, move the test from t003 into the test lib and use the
prerequisite in t0003.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test prerequisite mechanism is a useful way to allow some tests
in a test script to be skipped in environments that do not support
certain features (e.g. it is pointless to attempt checking how well
symbolic links are handled by Git on filesystems that do not support
them). It is OK for commonly used prerequisites to be always tested
during start-up of a test script by having a codeblock that tests a
feature and calls test_set_prereq, but for an uncommon feature,
forcing 90% of scripts to pay the same probing overhead for
prerequisite they do not care about is wasteful.
Introduce a mechanism to probe the prerequiste lazily. Changes are:
- test_lazy_prereq () function, which takes the name of the
prerequisite it probes and the script to probe for it, is
added. This only registers the name of the prerequiste that can
be lazily probed and the script to eval (without running).
- test_have_prereq() function (which is used by test_expect_success
and also can be called directly by test scripts) learns to look
at the list of prerequisites that can be lazily probed, and the
prerequisites that have already been probed that way. When asked
for a prerequiste that can be but haven't been probed, the script
registered with an earlier call to test_lazy_prereq is evaluated
and the prerequisite is set.
- test_run_lazy_prereq_() function is a helper to run the probe
script with the same kind of sandbox as regular tests, helped by
Jeff King.
Update the codeblock to probe and set SYMLINKS prerequisite using
the new mechanism as an example.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reorders t/test-lib.sh so that we dot-source GIT-BUILD-OPTIONS that
records the shell and Perl the user told us to use with Git a lot
early, so that test-lib.sh script itself can use "$PERL_PATH" in
one of its early operations.
* jc/test-lib-source-build-options-early:
test-lib: reorder and include GIT-BUILD-OPTIONS a lot earlier
Finishing touches to the XDG support (new feature for 1.7.12) and
tests.
* mm/config-xdg:
t1306: check that XDG_CONFIG_HOME works
ignore: make sure we have an xdg path before using it
attr: make sure we have an xdg path before using it
test-lib.sh: unset XDG_CONFIG_HOME
Now that git respects XDG_CONFIG_HOME for some lookups, we
must be sure to cleanse the test environment. Otherwise, the
user's XDG_CONFIG_HOME could influence the test results.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This dot-sources GIT-BUILD-OPTIONS a lot earlier in test-lib.sh so
that its use of "perl" can use "$PERL_PATH" to choose the version of
Perl the user told us is suitable for our use.
This is iffy; I didn't check it very carefully, and I would not be
surprised if there are subtle breakages.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most notably, t4031 creates a small shell script that invokes perl
and we want to use "$PERL_PATH" to name the version of Perl suitable
for our use, read from GIT-BUILD-OPTS. The test would fail when it
is directly run in t/ directory from the shell or "make" is run in t/
directory.
This problem was hidden from "make test" run in the top-level
directory, because its Makefile exports PERL_PATH.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A broken shell may not let us set an environment value to an arbitrary
value, interfering with some of the tests. Introduce a test prerequisite
so that we can skip some tests on such a platform.
By Zbigniew Jędrzejewski-Szmek
* zj/mksh-columns-breakage:
test-lib: skip test with COLUMNS=1 under mksh
mksh does not allow $COLUMNS to be set below 12. mksh(1) says that
$COLUMNS is "always set, defaults to 80, unless the value as reported
by stty(1) is non-zero and sane enough". This applies also to setting
it directly for one command:
$ COLUMNS=10 python -c 'import os; print os.environ["COLUMNS"]'
98
Add a test prerequisite by checking if we can set COLUMNS=1, to allow
us to skip tests that needs it.
Signed-off-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
$COLUMNS must be unset to not interfere with the tests. The tests
already ignore the terminal size because output is redirected to a
file, but COLUMNS overrides terminal size detection and changes the
test output away from the standard 80.
Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Update draft release notes to 1.7.9.3 for the last time
http.proxy: also mention https_proxy and all_proxy
t0300: work around bug in dash 0.5.6
t5512 (ls-remote): modernize style
tests: fix spurious error when run directly with Solaris /usr/xpg4/bin/sh
If any test script is run directly with Solaris 10 /usr/xpg4/bin/sh or
/bin/ksh, it fails spuriously with a message like:
t0000-basic.sh[31]: unset: bad argument count
This happens because those shells bail out when encountering a call to
"unset" with no arguments, and such unset call could take place in
'test-lib.sh'. Fix that issue, and add a proper comment to ensure we
don't regress in this respect.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This introduces a performance testing framework under t/perf/. It
tries to be as close to the test-lib.sh infrastructure as possible,
and thus should be easy to get used to for git developers.
The following points were considered for the implementation:
1. You usually want to compare arbitrary revisions/build trees against
each other. They may not have the performance test under
consideration, or even the perf-lib.sh infrastructure.
To cope with this, the 'run' script lets you specify arbitrary
build dirs and revisions. It even automatically builds the revisions
if it doesn't have them at hand yet.
2. Usually you would not want to run all tests. It would take too
long anyway. The 'run' script lets you specify which tests to run;
or you can also do it manually. There is a Makefile for
discoverability and 'make clean', but it is not meant for
real-world use.
3. Creating test repos from scratch in every test is extremely
time-consuming, and shipping or downloading such large/weird repos
is out of the question.
We leave this decision to the user. Two different sizes of test
repos can be configured, and the scripts just copy one or more of
those (using hardlinks for the object store). By default it tries
to use the build tree's git.git repository.
This is fairly fast and versatile. Using a copy instead of a clone
preserves many properties that the user may want to test for, such
as lots of loose objects, unpacked refs, etc.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This just moves all the user-facing functions to a separate file and
sources that instead.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Many of the scripts in the test suite write small helper
shell scripts to disk. It's best if these shell scripts
start with "#!$SHELL_PATH" rather than "#!/bin/sh", because
/bin/sh on some platforms is too buggy to be used.
However, it can be cumbersome to expand $SHELL_PATH, because
the usual recipe for writing a script is:
cat >foo.sh <<-\EOF
#!/bin/sh
echo my arguments are "$@"
EOF
To expand $SHELL_PATH, you have to either interpolate the
here-doc (which would require quoting "\$@"), or split the
creation into two commands (interpolating the $SHELL_PATH
line, but not the rest of the script). Let's provide a
helper function that makes that less syntactically painful.
While we're at it, this helper can also take care of the
"chmod +x" that typically comes after the creation of such a
script, saving the caller a line.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Traditionally, a cleanly resolved merge was committed by "git merge" using
the auto-generated merge commit log message without invoking the editor.
After 5 years of use in the field, it turns out that people perform too
many unjustified merges of the upstream history into their topic branches.
These merges are not just useless, but they are often not explained well,
and making the end result unreadable when it gets time for merging their
history back to their upstream.
Earlier we added the "--edit" option to the command, so that people can
edit the log message to explain and justify their merge commits. Let's
take it one step further and spawn the editor by default when we are in an
interactive session (i.e. the standard input and the standard output are
pointing at the same tty device).
There may be existing scripts that leave the standard input and the
standard output of the "git merge" connected to whatever environment the
scripts were started, and such invocation might trigger the above
"interactive session" heuristics. GIT_MERGE_AUTOEDIT environment variable
can be set to "no" at the beginning of such scripts to use the historical
behaviour while the script runs.
Note that this backward compatibility is meant only for scripts, and we
deliberately do *not* support "merge.edit = yes/no/auto" configuration
option to allow people to keep the historical behaviour.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 781f76b15 (test-lib: redirect stdin of tests) you can't simply put a
"bash &&" into a test for debugging purposes anymore. Instead you'll have
to use "bash <&6 >&3 2>&4".
As that invocation is not that easy to remember add the test_pause
convenience function. It invokes "$SHELL_PATH" to provide a sane shell
for the user.
This function also checks if the -v flag is given and will error out if
that is not the case instead of letting the test hang until ^D is pressed.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We want to run tests in a predictable, sterile environment
so we can get repeatable results. They should take as
little input as possible from the environment outside the
test script. We already sanitize environment variables, but
leave stdin untouched. This means that scripts can
accidentally be impacted by content on stdin, or whether
stdin isatty().
Furthermore, scripts reading from stdin can be annoying to
outer loops which care about their stdin offset, like:
while read sha1; do
make test
done
A test which accidentally reads stdin would soak up all of
the rest of the input intended for the outer shell loop.
Let's redirect stdin from /dev/null, which solves both
of these problems. It won't detect tests accidentally
reading from stdin, but since doing so now gives a
deterministic result, we don't need to consider that an
error.
We'll also leave file descriptor 6 as a link to the original
stdin. Tests shouldn't need to look at this, but it can be
convenient for inserting interactive commands while
debugging tests (e.g., you could insert "bash <&6 >&3 2>&4"
to run interactive commands in the environment of the test
script).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The point of test_config is to simultaneously set a config
variable and register its cleanup handler, like:
test_config core.foo bar
However, it stupidly assumes that $1 contained the name of
the variable, which means it won't work for:
test_config --global core.foo bar
We could try to parse the command-line ourselves and figure
out which parts need to be fed to test_unconfig. But since
this is likely the most common variant, it's much simpler
and less error-prone to simply add a new function.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the skeleton implementation of i18n in Git to one that can show
localized strings to users for our C, Shell and Perl programs using
either GNU libintl or the Solaris gettext implementation.
This new internationalization support is enabled by default. If
gettext isn't available, or if Git is compiled with
NO_GETTEXT=YesPlease, Git falls back on its current behavior of
showing interface messages in English. When using the autoconf script
we'll auto-detect if the gettext libraries are installed and act
appropriately.
This change is somewhat large because as well as adding a C, Shell and
Perl i18n interface we're adding a lot of tests for them, and for
those tests to work we need a skeleton PO file to actually test
translations. A minimal Icelandic translation is included for this
purpose. Icelandic includes multi-byte characters which makes it easy
to test various edge cases, and it's a language I happen to
understand.
The rest of the commit message goes into detail about various
sub-parts of this commit.
= Installation
Gettext .mo files will be installed and looked for in the standard
$(prefix)/share/locale path. GIT_TEXTDOMAINDIR can also be set to
override that, but that's only intended to be used to test Git itself.
= Perl
Perl code that's to be localized should use the new Git::I18n
module. It imports a __ function into the caller's package by default.
Instead of using the high level Locale::TextDomain interface I've
opted to use the low-level (equivalent to the C interface)
Locale::Messages module, which Locale::TextDomain itself uses.
Locale::TextDomain does a lot of redundant work we don't need, and
some of it would potentially introduce bugs. It tries to set the
$TEXTDOMAIN based on package of the caller, and has its own
hardcoded paths where it'll search for messages.
I found it easier just to completely avoid it rather than try to
circumvent its behavior. In any case, this is an issue wholly
internal Git::I18N. Its guts can be changed later if that's deemed
necessary.
See <AANLkTilYD_NyIZMyj9dHtVk-ylVBfvyxpCC7982LWnVd@mail.gmail.com> for
a further elaboration on this topic.
= Shell
Shell code that's to be localized should use the git-sh-i18n
library. It's basically just a wrapper for the system's gettext.sh.
If gettext.sh isn't available we'll fall back on gettext(1) if it's
available. The latter is available without the former on Solaris,
which has its own non-GNU gettext implementation. We also need to
emulate eval_gettext() there.
If neither are present we'll use a dumb printf(1) fall-through
wrapper.
= About libcharset.h and langinfo.h
We use libcharset to query the character set of the current locale if
it's available. I.e. we'll use it instead of nl_langinfo if
HAVE_LIBCHARSET_H is set.
The GNU gettext manual recommends using langinfo.h's
nl_langinfo(CODESET) to acquire the current character set, but on
systems that have libcharset.h's locale_charset() using the latter is
either saner, or the only option on those systems.
GNU and Solaris have a nl_langinfo(CODESET), FreeBSD can use either,
but MinGW and some others need to use libcharset.h's locale_charset()
instead.
=Credits
This patch is based on work by Jeff Epler <jepler@unpythonic.net> who
did the initial Makefile / C work, and a lot of comments from the Git
mailing list, including Jonathan Nieder, Jakub Narebski, Johannes
Sixt, Erik Faye-Lund, Peter Krefting, Junio C Hamano, Thomas Rast and
others.
[jc: squashed a small Makefile fix from Ramsay]
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since bc7a96a (mergetool--lib: Refactor tools into separate files,
2011-08-18) the mergetools and difftools related tests fail under
--valgrind because the mergetools/* scriptlets are not in the exec
path.
For now, symlink the mergetools subdir into the t/valgrind/bin
directory as a whole, since it does not contain anything of interest
to the valgrind wrappers.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jk/color-and-pager:
want_color: automatically fallback to color.ui
diff: don't load color config in plumbing
config: refactor get_colorbool function
color: delay auto-color decision until point of use
git_config_colorbool: refactor stdout_is_tty handling
diff: refactor COLOR_DIFF from a flag into an int
setup_pager: set GIT_PAGER_IN_USE
t7006: use test_config helpers
test-lib: add helper functions for config
t7006: modernize calls to unset
Conflicts:
builtin/commit.c
parse-options.c
There are a few common tasks when working with configuration
variables in tests; this patch aims to make them a little
easier to write and less error-prone.
When setting a variable, you should typically make sure to
clean it up after the test is finished, so as not to pollute
other tests. Like:
test_when_finished 'git config --unset foo.bar' &&
git config foo.bar baz
This patch lets you just write:
test_config foo.bar baz
When clearing a variable that does not exist, git-config
will report a specific non-zero error code. Meaning that
tests which call "git config --unset" often either rely on
the prior tests having actually set it, or must use
test_might_fail. With this patch, the previous:
test_might_fail git config --unset foo.bar
becomes:
test_unconfig foo.bar
Not only is this easier to type, but it is more robust; it
will correctly detect errors from git-config besides "key
was not set".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As we seem to need this variable that holds a single LF character
in many places, define it in test-lib.sh and let the test scripts
use it.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In olden times, tests would quietly exit the script when they failed
at an inconvenient moment, which was a little disconcerting.
Therefore v0.99.5~24^2~4 (Trapping exit in tests, using return for
errors, 2005-08-10) switched to an idiom of using "return" instead,
wrapping evaluation of test code in a function to make that safe:
test_run_ () {
eval >&3 2>&4 "$1"
eval_ret="$?"
return 0
}
Years later, the implementation of test_when_finished (v1.7.1.1~95,
2010-05-02) and v1.7.2-rc2~1^2~13 (test-lib: output a newline before
"ok" under a TAP harness, 2010-06-24) took advantage of test_run_ as a
place to put code shared by all test assertion functions, without
paying attention to the function's former purpose:
test_run_ () {
...
eval >&3 2>&4 "$1"
eval_ret=$?
if should run cleanup
then
eval >&3 2>&4 "$test_cleanup"
fi
if TAP format requires a newline here
then
echo
fi
return 0
}
That means cleanup commands and the newline to put TAP output at
column 0 are skipped when tests use "return" to fail early. Fix it by
introducing a test_eval_ function to catch the "return", with a
comment explaining the new function's purpose for the next person who
might touch this code.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As v0.99.5~24^2~4 (Trapping exit in tests, using return for errors,
2005-08-10) explains, callers to test_run_ (such as test_expect_code)
used to check the result from eval and the return value separately so
tests that fail early could be distinguished from tests that completed
normally with successful (nonzero) status. Eventually tests that
succeed with nonzero status were phased out (see v1.7.4-rc0~65^2~19,
2010-10-03 and especially v1.5.5-rc0~271, 2008-02-01) but the weird
two-return-value calling convention lives on.
Let's get rid of it. The new rule: test_run_ succeeds (returns 0)
if and only if the test succeeded.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests try to be too careful about cleaning themselves up and
do
test_expect_success description '
set-up some test refs and/or configuration &&
test_when_finished "revert the above changes" &&
the real test
'
Which is nice to make sure that a potential failure would not have
unexpected interaction with the next test. This however interferes when
"the real test" fails and we want to see what is going on, by running the
test with --immediate mode and descending into its trash directory after
the test stops. The precondition to run the real test and cause it to fail
is all gone after the clean-up procedure defined by test_when_finished is
done.
Update test_run_ which is the workhorse of running a test script
called from test_expect_success and test_expect_failure, so that we do not
run clean-up script defined with test_when_finished when a test that is
expected to succeed fails under the --immediate mode.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Jeff King <peff@peff.net>
When we run tests under valgrind, we symlink anything
executable that starts with git-* or test-* into a special
valgrind bin directory, and then make that our
GIT_EXEC_PATH.
However, shell libraries like git-sh-setup do not have the
executable bit marked, and did not get symlinked. This
means that any test looking for shell libraries in our
exec-path would fail to find them, even though that is a
fine thing to do when testing against a regular git build
(or in a git install, for that matter).
t2300 demonstrated this problem. The fix is to symlink these
shell libraries directly into the valgrind directory.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ab/i18n-fixup: (24 commits)
i18n: use test_i18n{cmp,grep} in t7600, t7607, t7611 and t7811
i18n: use test_i18n{grep,cmp} in t7508
i18n: use test_i18ngrep in t7506
i18n: use test_i18ngrep and test_i18ncmp in t7502
i18n: use test_i18ngrep in t7501
i18n: use test_i18ncmp in t7500
i18n: use test_i18ngrep in t7201
i18n: use test_i18ncmp and test_i18ngrep in t7102 and t7110
i18n: use test_i18ncmp and test_i18ngrep in t5541, t6040, t6120, t7004, t7012 and t7060
i18n: use test_i18ncmp and test_i18ngrep in t3700, t4001 and t4014
i18n: use test_i18ncmp and test_i18ngrep in t3203, t3501 and t3507
i18n: use test_i18ngrep in t2020, t2204, t3030, and t3200
i18n: use test_i18ngrep in lib-httpd and t2019
i18n: do not overuse C_LOCALE_OUTPUT (grep)
i18n: use test_i18ncmp in t1200 and t2200
i18n: .git file is not a human readable message (t5601)
i18n: do not overuse C_LOCALE_OUTPUT
i18n: mark init-db messages for translation
i18n: mark checkout plural warning for translation
i18n: mark checkout --detach messages for translation
...
* mk/grep-pcre:
git-grep: Fix problems with recently added tests
git-grep: Update tests (mainly for -P)
Makefile: Pass USE_LIBPCRE down in GIT-BUILD-OPTIONS
git-grep: update tests now regexp type is "last one wins"
git-grep: do not die upon -F/-P when grep.extendedRegexp is set.
git-grep: Bail out when -P is used with -F or -E
grep: Add basic tests
configure: Check for libpcre
git-grep: Learn PCRE
grep: Extract compile_regexp_failed() from compile_regexp()
grep: Fix a typo in a comment
grep: Put calls to fixmatch() and regmatch() into patmatch()
contrib/completion: --line-number to git grep
Documentation: Add --line-number to git-grep synopsis
* jn/userdiff-perl-updates:
userdiff/perl: tighten BEGIN/END block pattern to reject here-doc delimiters
tests: make test_expect_code quieter on success
userdiff/perl: catch sub with brace on second line
userdiff/perl: match full line of POD headers
userdiff/perl: anchor "sub" and "package" patterns on the left
t4018 (funcname patterns): minor cleanups
t4018 (funcname patterns): make configuration easier to track
t4018 (funcname patterns): make .gitattributes state easier to track
A command exiting with the expected status is not particularly
notable.
While the indication of progress might be useful when tracking down
where in a test a failure has happened, the same applies to most other
test helpers, which are quiet about success, so this single helper's
output stands out in an unpleasant way. An alternative method for
showing progress information might to invent a --progress option that
runs tests with "set -x", or until that is available, to run tests
using commands like
prove -v -j2 --shuffle --exec='sh -x' t2202-add-addremove.sh
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/fix-diff-files-unmerged:
diff-files: show unmerged entries correctly
diff: remove often unused parameters from diff_unmerge()
diff.c: return filepair from diff_unmerge()
test: use $_z40 from test-lib
This modest patch adds simple tests for git grep -P/--perl-regexp and
its interoperation with -i and -w.
Tests are only enabled when prerequisite LIBPCRE is defined (it's
automatically set based on USE_LIBPCRE in test-lib.sh).
Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/fix-diff-files-unmerged:
diff-files: show unmerged entries correctly
diff: remove often unused parameters from diff_unmerge()
diff.c: return filepair from diff_unmerge()
test: use $_z40 from test-lib
* ab/i18n-fixup: (24 commits)
i18n: use test_i18n{cmp,grep} in t7600, t7607, t7611 and t7811
i18n: use test_i18n{grep,cmp} in t7508
i18n: use test_i18ngrep in t7506
i18n: use test_i18ngrep and test_i18ncmp in t7502
i18n: use test_i18ngrep in t7501
i18n: use test_i18ncmp in t7500
i18n: use test_i18ngrep in t7201
i18n: use test_i18ncmp and test_i18ngrep in t7102 and t7110
i18n: use test_i18ncmp and test_i18ngrep in t5541, t6040, t6120, t7004, t7012 and t7060
i18n: use test_i18ncmp and test_i18ngrep in t3700, t4001 and t4014
i18n: use test_i18ncmp and test_i18ngrep in t3203, t3501 and t3507
i18n: use test_i18ngrep in t2020, t2204, t3030, and t3200
i18n: use test_i18ngrep in lib-httpd and t2019
i18n: do not overuse C_LOCALE_OUTPUT (grep)
i18n: use test_i18ncmp in t1200 and t2200
i18n: .git file is not a human readable message (t5601)
i18n: do not overuse C_LOCALE_OUTPUT
i18n: mark init-db messages for translation
i18n: mark checkout plural warning for translation
i18n: mark checkout --detach messages for translation
...
Apply parameter expansion. Also use here document to save
test results instead of appending each line with ">>".
Signed-off-by: Mathias Lafeldt <misfire@debugon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There is no need to duplicate the definition of $_z40 and $_x40 that
test-lib.sh supplies the test scripts.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of skipping the whole test, introduce test_i18ngrep wrapper that
pretends a successful result under GETTEXT_POISON build.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is too coarse-grained way that led to artificial splitting of a
logically single test case into "do" and "check only without poison".
As the majority of check is done by comparing expected and actual output
stored in a file with test_cmp anyway, just introduce test_i18ncmp that
pretends the actual output matched the expected one when gettext-poison
is in effect.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Otherwise the created test repositories will be affected by users ~/.gitconfig.
For example, setting core.logAllrefupdates in users config will make all
calls to "git config --unset core.logAllrefupdates" fail which will break
the first test which uses the statement and expects it to succeed.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In commit 95a1d12e9b ("tests: scrub environment of GIT_* variables") all
environment variables starting with "GIT_" were unset for the tests using
a perl script rather than unsetting them one by one. Only three exceptions
were made to make them work as before: "GIT_TRACE*", "GIT_DEBUG*" and
"GIT_USE_LOOKUP".
Unfortunately some environment variables used by the test framework itself
were not added to the exceptions and thus stopped working when given
before the make command instead of after it. Those are:
- GIT_NOTES_TIMING_TESTS
- GIT_PATCHID_TIMING_TESTS
- GIT_PROVE_OPTS
- GIT_REMOTE_SVN_TEST_BIG_FILES
- GIT_SKIP_TESTS
- GIT_TEST*
- GIT_VALGRIND_OPTIONS
I noticed that when skipping a test the way I was used to suddenly failed:
GIT_SKIP_TESTS='t1234' GIT_TEST_OPTS='--root=/dev/shm' make -j10 test
This should work according to t/README, but didn't anymore, so let's fix
that by adding them to the exception list. And to avoid having a long
regexp put the exceptions in a separate variable using nicer formatting.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Thanks-to: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/test-sanitize-git-env:
tests: scrub environment of GIT_* variables
config: drop support for GIT_CONFIG_NOGLOBAL
gitattributes: drop support for GIT_ATTR_NOGLOBAL
tests: suppress system gitattributes
tests: stop worrying about obsolete environment variables
