In 8bf4bec (add "ok=sigpipe" to test_must_fail and use it to
fix flaky tests, 2015-11-27), test_must_fail learned to
recognize "141" as a sigpipe failure. However, testing for
a signal is more complicated than that; we should use
test_match_signal to implement more portable checking.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In POSIX shells, a program which exits due to a signal
generally has an exit code of 128 plus the signal number.
However, ksh uses 256 plus the signal number. We've
accounted for that in t0005, but not in other tests. Let's
pull out the logic so we can use it elsewhere.
It would be nice for debugging if this additionally printed
errors to stderr, like our other test_* helpers. But we're
going to need to use it in other places besides the innards
of a test_expect block. So let's leave it as generic as
possible.
Note that we also leave the magic "3" for Windows out of the
generic helper. This is an artifact of the way we use
raise() to kill ourselves in test-sigchain.c, and will not
necessarily apply to all programs. So it's better to keep it
out of the helper, to reduce the chance of confusing it with
a real call to exit(3).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The one-shot environment variable syntax:
FOO=BAR some-program
is unportable when some-program is actually a shell
function, like test_must_fail (on some shells FOO remains
set after the function returns, and on others it does not).
We sometimes get around this by using env, like:
test_must_fail env FOO=BAR some-program
But that only works because test_must_fail's arguments are
themselves a command which can be run. You can't run:
env FOO=BAR test_must_fail some-program
because env does not know about our shell functions. So
there is no equivalent for test_commit, for example, and one
must resort to:
(
FOO=BAR
export FOO
test_commit
)
which is a bit verbose. Let's add a version of "env" that
works _inside_ the shell, by creating a subshell, exporting
variables from its argument list, and running the command.
Its use is demonstrated on a currently-unportable case in
t4014.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rewrite the 'seq' imitation using only commands and features that
are typically found built into modern POSIX shells, instead of
relying on Perl to run a single-liner script.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We never used the "letters" form since we came up with "test_seq" to
replace use of non-portable "seq" in our test script, which we
introduced it at d17cf5f3 (tests: Introduce test_seq, 2012-08-04).
We use this helper to either iterate for N times (i.e. the values on
the lines do not even matter), or just to get N distinct strings
(i.e. the values on the lines themselves do not really matter, but
we care that they are different from each other and reproducible).
Stop promising that we may allow using "letters"; this would open an
easier reimplementation that does not rely on $PERL, if somebody
later wants to.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a command is marked as test_must_fail but dies with a
signal, we consider that a problem and report the error to
stderr. However, we don't say _which_ signal; knowing that
can make debugging easier. Let's share as much as we know.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5516 "75 - deny fetch unreachable SHA1, allowtipsha1inwant=true" is
flaky in the following case:
1. remote upload-pack finds out "not our ref"
2. remote sends a response and closes the pipe
3. fetch-pack still tries to write commands to the remote upload-pack
4. write call in wrapper.c dies with SIGPIPE
The test is flaky because the sending fetch-pack may or may
not have finished writing its output by step (3). If it did,
then we see a closed pipe on the next read() call. If it
didn't, then we get the SIGPIPE from step (4) above. Both
are fine, but the latter fools test_must_fail.
t5504 "9 - push with transfer.fsckobjects" is flaky, too, and returns
SIGPIPE once in a while. I had to remove the final "To dst..." output
check because there is no output if the process dies with SIGPIPE.
Accept such a death-with-sigpipe also as OK when we are expecting a
failure.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Add an (optional) first parameter "ok=<special case>" to test_must_fail
and return success for "<special case>". Add "success" as
"<special case>" and use it to implement "test_might_fail". This removes
redundancies in test-lib-function.sh.
You can pass multiple <special case> arguments divided by comma (e.g.
"test_must_fail ok=success,something")
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Jeff King <peff@peff.net>
When prefixing a Git call in the test suite with 'debug ', it will
now be run with GDB, allowing the developer to debug test failures
more conveniently.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
test_when_finished does nothing in a subshell because the change to
test_cleanup does not affect the parent.
There is no POSIX way to detect that we are in a subshell ($$ and $PPID
are specified to remain unchanged), but we can detect it on Bash and
fall back to ignoring the bug on other shells.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If used in a subshell, test_config cannot unset variables at the end of
a test. This is a problem when testing submodules because we do not
want to "cd" at to top level of a test script in order to run the
command inside the submodule.
Add a "-C" option to test_config (and test_unconfig) so that test_config
can be kept outside subshells and still affect subrepositories.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Help us to find broken test script that splits the body part of the
test by mistaken use of wrong kind of quotes.
* jc/test-prereq-validate:
test: validate prerequistes syntax
Brian Carson noticed that a test piece in t5601 had a pair of single
quotes in the body, which made it into 4 parameter call to
test_expect_success, as if its test title were a prerequisite.
As the prerequisites have a specific syntax (i.e. comma separated
tokens spelled in capital letters, possibly prefixed with ! for
negation), validate them to catch such a mistake in the future.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The second argument to test_path_is_file and test_path_is_dir
must be $2 and not $*, which instead would repeat the file
name in the error message.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Test clean-up.
* jc/diff-test-updates:
test_ln_s_add: refresh stat info of fake symbolic links
t4008: modernise style
t/diff-lib: check exact object names in compare_diff_raw
tests: do not borrow from COPYING and README from the real source
t4010: correct expected object names
t9300: correct expected object names
t4008: correct stale comments
Test clean-up.
* jc/diff-test-updates:
test_ln_s_add: refresh stat info of fake symbolic links
t4008: modernise style
t/diff-lib: check exact object names in compare_diff_raw
tests: do not borrow from COPYING and README from the real source
t4010: correct expected object names
t9300: correct expected object names
t4008: correct stale comments
We have a helper function test_ln_s_add that inserts a symbolic link
into the index even if the file system does not support symbolic links.
There is a small flaw in the emulation path: the added entry does not
pick up stat information of the fake symbolic link from the file system,
as a consequence, the index is not exactly the same as for the "regular"
path (where symbolic links are available). To fix this, just call
git update-index again.
This flaw was revealed by the earlier change that tightened
compare_diff_raw(), because a test case in t4008 depends on the
correctly updated index.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The clean-up of this test script was long overdue and is a very
welcome change.
* da/mergetool-tests:
test-lib-functions: adjust style to match CodingGuidelines
t7610-mergetool: use test_config to isolate tests
t7610-mergetool: add missing && and remove commented-out code
t7610-mergetool: use tabs instead of a mix of tabs and spaces
Prefer "test" over "[ ]" for conditionals.
Prefer "$()" over backticks for command substitutions.
Avoid control structures on a single line with semicolons.
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For small outputs, we sometimes use:
test "$(some_cmd)" = "something we expect"
instead of a full test_cmp. The downside of this is that
when it fails, there is no output at all from the script.
Let's introduce a small helper to make tests easier to
debug.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For the upcoming submodule test framework we often need to assert that an
empty directory exists in the work tree. Add the test_dir_is_empty()
function which asserts that the given argument is an empty directory.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* 'mt/patch-id-stable' (early part):
patch-id-test: test stable and unstable behaviour
patch-id: make it stable against hunk reordering
test doc: test_write_lines does not split its arguments
test: add test_write_lines helper
The construct is error-prone; "test" being built-in in most modern
shells, the reason to avoid "test <cond> && test <cond>" spawning
one extra process by using a single "test <cond> -a <cond>" no
longer exists.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
test_cmp() is primarily meant to compare text files (and display the
difference for debug purposes).
Raw "cmp" is better suited to compare binary files (tar, zip, etc.).
On MinGW, test_cmp is a shell function mingw_test_cmp that tries to
read both files into environment, stripping CR characters (introduced
in commit 4d715ac0).
This function usually speeds things up, as fork is extremly slow on
Windows. But no wonder that this function is extremely slow and
sometimes even crashes when comparing large tar or zip files.
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* tg/index-v4-format:
read-cache: add index.version config variable
test-lib: allow setting the index format version
introduce GIT_INDEX_VERSION environment variable
Allow adding a TEST_GIT_INDEX_VERSION variable to config.mak to set the
index version with which the test suite should be run.
If it isn't set, the default version given in the source code is
used (currently version 3).
To avoid breakages with index versions other than [23], also set the
index version under which t2104 is run to 3. This test only tests
functionality specific to version 2 and 3 of the index file and would
fail if the test suite is run with any other version.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We do not run the httpd nor git-daemon tests by default, as
they are rather heavyweight and require network access
(albeit over localhost). However, it would be nice if more
pepole ran them, for two reasons:
1. We would get more test coverage on more systems.
2. The point of the test suite is to find regressions. It
is very easy to change some of the underlying code and
break the httpd code without realizing you are even
affecting it. Running the httpd tests helps find these
problems sooner (ideally before the patches even hit
the list).
We still want to leave an "out", though, for people who really do
not want to run them. For that reason, the GIT_TEST_HTTPD and
GIT_TEST_GIT_DAEMON variables are now tri-state booleans
(true/false/auto), so you can say GIT_TEST_HTTPD=false to turn the
tests back off. To support those who want a stable single way to
disable these tests across versions of Git before and after this
change, an empty string explicitly set to these variables is also
taken as "false", so the behaviour changes only for those who:
a. did not express any preference by leaving these variables
unset. They did not test these features before, but now they
do; or
b. did express that they want to test these features by setting
GIT_TEST_FEATURE=false (or any equivalent other ways to tell
"false" to Git, e.g. "0"), which has been a valid but funny way
to say that they do want to test the feature only because we
used to interpret any non-empty string to mean "yes please
test". They no longer test that feature.
In addition, we are forgiving of common setup failures (e.g., you do
not have apache installed, or have an old version) when the
tri-state is "auto" (or unset), but report an error when it is
"true". This makes "auto" a sane default, as we should not cause
failures on setups where the tests cannot run. But it allows people
who use "true" to catch regressions in their system (e.g., they
uninstalled apache, but were expecting their automated test runs to
test git-httpd, and would want to be notified).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A #! line in these files is misleading, since these scriptlets are
meant to be sourced with '.' (using whatever shell sources them)
instead of run directly using the interpreter named on the #! line.
Removing the #! line shouldn't hurt syntax highlighting since
these files have filenames ending with '.sh'. For documentation,
add a brief description of how the files are meant to be used in
place of the shebang line.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As of the last commit, we can use "perl" instead of
"$PERL_PATH" when running tests, as the former is now a
function which uses the latter. As the shorter "perl" is
easier on the eyes, let's switch to using it everywhere.
This is not quite a mechanical s/$PERL_PATH/perl/
replacement, though. There are some places where we invoke
perl from a script we generate on the fly, and those scripts
do not have access to our internal shell functions. The
result can be double-checked by running:
ln -s /bin/false bin-wrappers/perl
make test
which continues to pass even after this patch.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Once upon a time, we assumed that calling a bare "perl" in
the test scripts was OK, because we would find the perl from
the user's PATH, and we were only asking that perl to do
basic operations that work even on old versions of perl.
Later, we found that some systems really prefer to use
$PERL_PATH even for these basic cases, because the system
perl misbehaves in some way (e.g., by handling line endings
differently). We then switched "perl" invocations to
"$PERL_PATH" to respect the user's choice.
Having to use "$PERL_PATH" is ugly and cumbersome, though.
Instead, let's provide a perl() shell function that tests
can use, which will transparently do the right thing.
Unfortunately, test writers still have to use $PERL_PATH in
certain situations, so we still need to keep the advice in
the README.
Note that this may fix test failures in t5004, t5503, t6002,
t6003, t6300, t8001, and t8002, depending on your system's
perl setup. All of these can be detected by running:
ln -s /bin/false bin-wrappers/perl
make test
which fails before this patch, and passes after.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a number of tests, output that was produced by a shell script is
compared to expected output using test_cmp. Unfortunately, the MSYS bash--
when invoked via git, such as in hooks--converts LF to CRLF on output
(as produced by echo and printf), which leads to many false positives.
Implements a diff tool that undoes the converted CRLF. To avoid that
sub-processes are spawned (which is very slow on Windows), the tool is
implemented as a shell function. Diff is invoked as usual only when a
difference is detected by the shell code.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allows N instances of tests run in parallel, each running 1/N parts
of the test suite under Valgrind, to speed things up.
* tr/test-v-and-v-subtest-only:
perf-lib: fix start/stop of perf tests
test-lib: support running tests under valgrind in parallel
test-lib: allow prefixing a custom string before "ok N" etc.
test-lib: valgrind for only tests matching a pattern
test-lib: verbose mode for only tests matching a pattern
test-lib: self-test that --verbose works
test-lib: rearrange start/end of test_expect_* and test_skip
test-lib: refactor $GIT_SKIP_TESTS matching
test-lib: enable MALLOC_* for the actual tests
This moves
* the early setup part from test_skip to a new function test_start_
* the final common parts of test_expect_* to a new function
test_finish_
to make the next commit more obvious.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are quite a lot places where an output file is expected to be
empty, and we fail the test when it is not. The output from running
the test script with -i -v can be helped if we showed the unexpected
contents at that point.
We could of course do
>expected.empty && test_cmp expected.empty actual
but this is commmon enough to be done with a dedicated helper.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a new function that creates a symbolic link and adds it to the index
to be used in cases where a symbolic link is not required on the file
system. We will use it to remove many SYMLINKS prerequisites from test
cases.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Let us use not just memgrind but other *grind debuggers.
* tr/valgrind:
tests: notice valgrind error in test_must_fail
tests --valgrind: provide a mode without --track-origins
tests: parameterize --valgrind option
t/README: --valgrind already implies -v
"git apply --whitespace=fix" was not prepared to see a line getting
longer after fixing whitespaces (e.g. tab-in-indent aka Python).
* jc/apply-ws-fix-tab-in-indent:
test: resurrect q_to_tab
apply --whitespace=fix: avoid running over the postimage buffer
Consolidate codepaths that inspect log-message-to-be and decide to
add a new Signed-off-by line in various commands.
* bc/append-signed-off-by:
git-commit: populate the edit buffer with 2 blank lines before s-o-b
Unify appending signoff in format-patch, commit and sequencer
format-patch: update append_signoff prototype
t4014: more tests about appending s-o-b lines
sequencer.c: teach append_signoff to avoid adding a duplicate newline
sequencer.c: teach append_signoff how to detect duplicate s-o-b
sequencer.c: always separate "(cherry picked from" from commit body
sequencer.c: require a conforming footer to be preceded by a blank line
sequencer.c: recognize "(cherry picked from ..." as part of s-o-b footer
t/t3511: add some tests of 'cherry-pick -s' functionality
t/test-lib-functions.sh: allow to specify the tag name to test_commit
commit, cherry-pick -s: remove broken support for multiline rfc2822 fields
sequencer.c: rework search for start of footer to improve clarity
We tell valgrind to return 126 if it notices that something is wrong,
but we did not actually handle this in test_must_fail, leading to
false negatives. Catch and report it.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
