Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
We have various behavior that's shared across our Makefiles, or that
really should be (e.g. via defined templates). Let's create a
top-level "shared.mak" to house those sorts of things, and start by
adding the ".DELETE_ON_ERROR" flag to it.
See my own 7b76d6bf221 (Makefile: add and use the ".DELETE_ON_ERROR"
flag, 2021-06-29) and db10fc6c09f (doc: simplify Makefile using
.DELETE_ON_ERROR, 2021-05-21) for the addition and use of the
".DELETE_ON_ERROR" flag.
I.e. this changes the behavior of existing rules in the altered
Makefiles (except "Makefile" & "Documentation/Makefile"). I'm
confident that this is safe having read the relevant rules in those
Makfiles, and as the GNU make manual notes that it isn't the default
behavior is out of an abundance of backwards compatibility
caution. From edition 0.75 of its manual, covering GNU make 4.3:
[Enabling '.DELETE_ON_ERROR' is] almost always what you want
'make' to do, but it is not historical practice; so for
compatibility, you must explicitly request it.
This doesn't introduce a bug by e.g. having this
".DELETE_ON_ERROR" flag only apply to this new shared.mak, Makefiles
have no such scoping semantics.
It does increase the danger that any Makefile without an explicit "The
default target of this Makefile is..." snippet to define the default
target as "all" could have its default rule changed if our new
shared.mak ever defines a "real" rule. In subsequent commits we'll be
careful not to do that, and such breakage would be obvious e.g. in the
case of "make -C t".
We might want to make that less fragile still (e.g. by using
".DEFAULT_GOAL" as noted in the preceding commit), but for now let's
simply include "shared.mak" without adding that boilerplate to all the
Makefiles that don't have it already. Most of those are already
exposed to that potential caveat e.g. due to including "config.mak*".
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 17:04:13 +01:00
|
|
|
# Import tree-wide shared Makefile behavior and libraries
|
|
|
|
|
include ../shared.mak
|
|
|
|
|
|
2005-05-14 07:50:32 +02:00
|
|
|
# Run tests
|
|
|
|
|
#
|
|
|
|
|
# Copyright (c) 2005 Junio C Hamano
|
|
|
|
|
#
|
2005-05-14 17:58:22 +02:00
|
|
|
|
2010-05-14 11:31:38 +02:00
|
|
|
-include ../config.mak.autogen
|
2009-08-09 10:50:37 +02:00
|
|
|
-include ../config.mak
|
|
|
|
|
|
2012-12-09 11:36:17 +01:00
|
|
|
#GIT_TEST_OPTS = --verbose --debug
|
2005-09-24 21:50:29 +02:00
|
|
|
SHELL_PATH ?= $(SHELL)
|
t/Makefile: introduce TEST_SHELL_PATH
You may want to run the test suite with a different shell
than you use to build Git. For instance, you may build with
SHELL_PATH=/bin/sh (because it's faster, or it's what you
expect to exist on systems where the build will be used) but
want to run the test suite with bash (e.g., since that
allows using "-x" reliably across the whole test suite).
There's currently no good way to do this.
You might think that doing two separate make invocations,
like:
make &&
make -C t SHELL_PATH=/bin/bash
would work. And it _almost_ does. The second make will see
our bash SHELL_PATH, and we'll use that to run the
individual test scripts (or tell prove to use it to do so).
So far so good.
But this breaks down when "--tee" or "--verbose-log" is
used. Those options cause the test script to actually
re-exec itself using $SHELL_PATH. But wait, wouldn't our
second make invocation have set SHELL_PATH correctly in the
environment?
Yes, but test-lib.sh sources GIT-BUILD-OPTIONS, which we
built during the first "make". And that overrides the
environment, giving us the original SHELL_PATH again.
Let's introduce a new variable that lets you specify a
specific shell to be run for the test scripts. Note that we
have to touch both the main and t/ Makefiles, since we have
to record it in GIT-BUILD-OPTIONS in one, and use it in the
latter.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08 11:47:22 +01:00
|
|
|
TEST_SHELL_PATH ?= $(SHELL_PATH)
|
2010-08-08 16:49:26 +02:00
|
|
|
PERL_PATH ?= /usr/bin/perl
|
2005-09-30 22:31:16 +02:00
|
|
|
TAR ?= $(TAR)
|
2007-07-14 19:51:44 +02:00
|
|
|
RM ?= rm -f
|
2010-10-14 10:53:36 +02:00
|
|
|
PROVE ?= prove
|
|
|
|
|
DEFAULT_TEST_TARGET ?= test
|
2014-07-09 21:34:42 +02:00
|
|
|
TEST_LINT ?= test-lint
|
2005-05-14 07:50:32 +02:00
|
|
|
|
2013-04-26 20:55:52 +02:00
|
|
|
ifdef TEST_OUTPUT_DIRECTORY
|
2013-05-06 14:35:46 +02:00
|
|
|
TEST_RESULTS_DIRECTORY = $(TEST_OUTPUT_DIRECTORY)/test-results
|
2018-07-11 08:46:34 +02:00
|
|
|
CHAINLINTTMP = $(TEST_OUTPUT_DIRECTORY)/chainlinttmp
|
2013-04-26 20:55:52 +02:00
|
|
|
else
|
|
|
|
|
TEST_RESULTS_DIRECTORY = test-results
|
2018-07-11 08:46:34 +02:00
|
|
|
CHAINLINTTMP = chainlinttmp
|
2013-04-26 20:55:52 +02:00
|
|
|
endif
|
|
|
|
|
|
2005-10-10 22:50:01 +02:00
|
|
|
# Shell quote;
|
2006-02-18 12:40:22 +01:00
|
|
|
SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
|
t/Makefile: introduce TEST_SHELL_PATH
You may want to run the test suite with a different shell
than you use to build Git. For instance, you may build with
SHELL_PATH=/bin/sh (because it's faster, or it's what you
expect to exist on systems where the build will be used) but
want to run the test suite with bash (e.g., since that
allows using "-x" reliably across the whole test suite).
There's currently no good way to do this.
You might think that doing two separate make invocations,
like:
make &&
make -C t SHELL_PATH=/bin/bash
would work. And it _almost_ does. The second make will see
our bash SHELL_PATH, and we'll use that to run the
individual test scripts (or tell prove to use it to do so).
So far so good.
But this breaks down when "--tee" or "--verbose-log" is
used. Those options cause the test script to actually
re-exec itself using $SHELL_PATH. But wait, wouldn't our
second make invocation have set SHELL_PATH correctly in the
environment?
Yes, but test-lib.sh sources GIT-BUILD-OPTIONS, which we
built during the first "make". And that overrides the
environment, giving us the original SHELL_PATH again.
Let's introduce a new variable that lets you specify a
specific shell to be run for the test scripts. Note that we
have to touch both the main and t/ Makefiles, since we have
to record it in GIT-BUILD-OPTIONS in one, and use it in the
latter.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08 11:47:22 +01:00
|
|
|
TEST_SHELL_PATH_SQ = $(subst ','\'',$(TEST_SHELL_PATH))
|
2013-01-03 00:20:19 +01:00
|
|
|
PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
|
2013-04-26 20:55:52 +02:00
|
|
|
TEST_RESULTS_DIRECTORY_SQ = $(subst ','\'',$(TEST_RESULTS_DIRECTORY))
|
2018-07-11 08:46:34 +02:00
|
|
|
CHAINLINTTMP_SQ = $(subst ','\'',$(CHAINLINTTMP))
|
2005-10-10 22:50:01 +02:00
|
|
|
|
t/Makefile: Use $(sort ...) explicitly where needed
Starting from GNU Make 3.82 $(wildcard ...) no longer sorts the result
(from NEWS):
* WARNING: Backward-incompatibility!
Wildcards were not documented as returning sorted values, but the results
have been sorted up until this release.. If your makefiles require sorted
results from wildcard expansions, use the $(sort ...) function to request
it explicitly.
http://repo.or.cz/w/make.git/commitdiff/2a59dc32aaf0681dec569f32a9d7ab88a379d34f
I usually watch test progress visually, and if tests are sorted, even
with make -j4 they go more or less incrementally by their t number. On
the other side, without sorting, tests are executed in seemingly random
order even for -j1. Let's please maintain sane tests order for perceived
prettyness.
Another note is that in GNU Make sort also works as uniq, so after sort
being removed, we might expect e.g. $(wildcard *.sh a.*) to produce
duplicates for e.g. "a.sh". From this point of view, adding sort could
be seen as hardening t/Makefile from accidentally introduced dups.
It turned out that prevous releases of GNU Make did not perform full
sort in $(wildcard), only sorting results for each pattern, that's why
explicit sort-as-uniq is relevant even for older makes.
Signed-off-by: Kirill Smelkov <kirr@navytux.spb.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-03 22:41:21 +02:00
|
|
|
T = $(sort $(wildcard t[0-9][0-9][0-9][0-9]-*.sh))
|
2014-07-09 21:34:12 +02:00
|
|
|
THELPERS = $(sort $(filter-out $(T),$(wildcard *.sh)))
|
2022-09-01 02:29:55 +02:00
|
|
|
TLIBS = $(sort $(wildcard lib-*.sh)) annotate-tests.sh
|
2020-10-20 15:41:02 +02:00
|
|
|
TPERF = $(sort $(wildcard perf/p[0-9][0-9][0-9][0-9]-*.sh))
|
2022-09-01 02:29:55 +02:00
|
|
|
TINTEROP = $(sort $(wildcard interop/i[0-9][0-9][0-9][0-9]-*.sh))
|
2018-07-11 08:46:34 +02:00
|
|
|
CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.test)))
|
t/Makefile: apply chainlint.pl to existing self-tests
Now that chainlint.pl is functional, take advantage of the existing
chainlint self-tests to validate its operation. (While at it, stop
validating chainlint.sed against the self-tests since it will soon be
retired.)
Due to chainlint.sed implementation limitations leaking into the
self-test "expect" files, a few of them require minor adjustment to make
them compatible with chainlint.pl which does not share those
limitations.
First, because `sed` does not provide any sort of real recursion,
chainlint.sed only emulates recursion into subshells, and each level of
recursion leads to a multiplicative increase in complexity of the `sed`
rules. To avoid substantial complexity, chainlint.sed, therefore, only
emulates subshell recursion one level deep. Any subshell deeper than
that is passed through as-is, which means that &&-chains are not checked
in deeper subshells. chainlint.pl, on the other hand, employs a proper
recursive descent parser, thus checks subshells to any depth and
correctly flags broken &&-chains in deep subshells.
Second, due to sed's line-oriented nature, chainlint.sed, by necessity,
folds multi-line quoted strings into a single line. chainlint.pl, on the
other hand, employs a proper lexical analyzer which preserves quoted
strings as-is, including embedded newlines.
Furthermore, the output of chainlint.sed and chainlint.pl do not match
precisely in terms of whitespace. However, since the purpose of the
self-checks is to verify that the ?!AMP?! annotations are being
correctly added, minor whitespace differences are immaterial. For this
reason, rather than adjusting whitespace in all existing self-test
"expect" files to match the new linter's output, the `check-chainlint`
target ignores whitespace differences. Since `diff -w` is not POSIX,
`check-chainlint` attempts to employ `git diff -w`, and only falls back
to non-POSIX `diff -w` (and `-u`) if `git diff` is not available.
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-01 02:29:46 +02:00
|
|
|
CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl
|
2005-05-14 07:50:32 +02:00
|
|
|
|
2022-09-01 02:29:55 +02:00
|
|
|
# `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`)
|
|
|
|
|
# checks all tests in all scripts via a single invocation, so tell individual
|
tests: run internal chain-linter under "make test"
Since 69b9924b875 (t/Makefile: teach `make test` and `make prove` to run
chainlint.pl, 2022-09-01), we run a single chainlint.pl process for all
scripts, and then instruct each individual script to run with the
equivalent of --no-chain-lint, which tells them not to redundantly run
the chainlint script themselves.
However, this also disables the internal linter run within the shell by
eval-ing "(exit 117) && $1" and confirming we get code 117. In theory
the external linter produces a superset of complaints, and we don't need
the internal one anymore. However, we know there is at least one case
where they differ. A test like:
test_expect_success 'should fail linter' '
false &&
sleep 2 &
pid=$! &&
kill $pid
'
is buggy (it ignores the failure from "false", because it is
backgrounded along with the sleep). The internal linter catches this,
but the external one doesn't (and teaching it to do so is complicated[1]).
So not only does "make test" miss this problem, but it's doubly
confusing because running the script standalone does complain.
Let's teach the suppression in the Makefile to only turn off the
external linter (which we know is redundant, as it was already run) and
leave the internal one intact.
I've used a new environment variable to do this here, and intentionally
did not add a "--no-ext-chain-lint" option. This is an internal
optimization used by the Makefile, and not something that ordinary users
would need to tweak.
[1] For discussion of chainlint.pl and this case, see:
https://lore.kernel.org/git/CAPig+cQtLFX4PgXyyK_AAkCvg4Aw2RAC5MmLbib-aHHgTBcDuw@mail.gmail.com/
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-30 21:27:38 +02:00
|
|
|
# scripts not to run the external "chainlint.pl" script themselves
|
|
|
|
|
CHAINLINTSUPPRESS = GIT_TEST_EXT_CHAIN_LINT=0 && export GIT_TEST_EXT_CHAIN_LINT &&
|
2022-09-01 02:29:55 +02:00
|
|
|
|
2010-10-14 10:53:36 +02:00
|
|
|
all: $(DEFAULT_TEST_TARGET)
|
|
|
|
|
|
2018-07-11 08:46:34 +02:00
|
|
|
test: pre-clean check-chainlint $(TEST_LINT)
|
2022-09-01 02:29:55 +02:00
|
|
|
$(CHAINLINTSUPPRESS) $(MAKE) aggregate-results-and-cleanup
|
2005-11-08 10:51:10 +01:00
|
|
|
|
t/Makefile: add a rule to re-run previously-failed tests
This patch automates the process of determining which tests failed
previously and re-running them.
While developing patch series, it is a good practice to run the test
suite from time to time, just to make sure that obvious bugs are caught
early. With complex patch series, it is common to run `make -j15 -k
test`, i.e. run the tests in parallel and *not* stop at the first
failing test but continue. This has the advantage of identifying
possibly multiple problems in one big test run.
It is particularly important to reduce the turn-around time thusly on
Windows, where the test suite spends 45 minutes on the computer on which
this patch was developed.
It is the most convenient way to determine which tests failed after
running the entire test suite, in parallel, to look for left-over "trash
directory.t*" subdirectories in the t/ subdirectory. However, those
directories might live outside t/ when overridden using the
--root=<directory> option, to which the Makefile has no access. The next
best method is to grep explicitly for failed tests in the test-results/
directory, which the Makefile *can* access.
Please note that the often-recommended `prove` tool requires Perl, and
that opens a whole new can of worms on Windows. As no native Windows Perl
comes with Subversion bindings, we have to use a Perl in Git for Windows
that uses the POSIX emulation layer named MSYS2 (which is a portable
version of Cygwin). When using this emulation layer under stress, e.g.
when running massively-parallel tests, unexplicable crashes occur quite
frequently, and instead of having a solution to the original problem, the
developer now has an additional, quite huge problem. For that reason, this
developer rejected `prove` as a solution and went with this patch instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-27 18:21:30 +01:00
|
|
|
failed:
|
|
|
|
|
@failed=$$(cd '$(TEST_RESULTS_DIRECTORY_SQ)' && \
|
|
|
|
|
grep -l '^failed [1-9]' *.counts | \
|
|
|
|
|
sed -n 's/\.counts$$/.sh/p') && \
|
|
|
|
|
test -z "$$failed" || $(MAKE) $$failed
|
|
|
|
|
|
2018-07-11 08:46:34 +02:00
|
|
|
prove: pre-clean check-chainlint $(TEST_LINT)
|
2022-09-01 02:29:55 +02:00
|
|
|
@echo "*** prove ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec '$(TEST_SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS)
|
2012-05-02 17:31:52 +02:00
|
|
|
$(MAKE) clean-except-prove-cache
|
2010-10-14 10:53:36 +02:00
|
|
|
|
2005-11-08 10:51:10 +01:00
|
|
|
$(T):
|
t/Makefile: introduce TEST_SHELL_PATH
You may want to run the test suite with a different shell
than you use to build Git. For instance, you may build with
SHELL_PATH=/bin/sh (because it's faster, or it's what you
expect to exist on systems where the build will be used) but
want to run the test suite with bash (e.g., since that
allows using "-x" reliably across the whole test suite).
There's currently no good way to do this.
You might think that doing two separate make invocations,
like:
make &&
make -C t SHELL_PATH=/bin/bash
would work. And it _almost_ does. The second make will see
our bash SHELL_PATH, and we'll use that to run the
individual test scripts (or tell prove to use it to do so).
So far so good.
But this breaks down when "--tee" or "--verbose-log" is
used. Those options cause the test script to actually
re-exec itself using $SHELL_PATH. But wait, wouldn't our
second make invocation have set SHELL_PATH correctly in the
environment?
Yes, but test-lib.sh sources GIT-BUILD-OPTIONS, which we
built during the first "make". And that overrides the
environment, giving us the original SHELL_PATH again.
Let's introduce a new variable that lets you specify a
specific shell to be run for the test scripts. Note that we
have to touch both the main and t/ Makefiles, since we have
to record it in GIT-BUILD-OPTIONS in one, and use it in the
latter.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08 11:47:22 +01:00
|
|
|
@echo "*** $@ ***"; '$(TEST_SHELL_PATH_SQ)' $@ $(GIT_TEST_OPTS)
|
2005-05-14 07:50:32 +02:00
|
|
|
|
2008-06-08 16:04:35 +02:00
|
|
|
pre-clean:
|
2013-04-26 20:55:52 +02:00
|
|
|
$(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)'
|
2008-06-08 16:04:35 +02:00
|
|
|
|
2018-07-11 08:46:34 +02:00
|
|
|
clean-except-prove-cache: clean-chainlint
|
t/Makefile: don't remove test-results in "clean-except-prove-cache"
When "make test" is run with the default of "DEFAULT_TEST_TARGET=test"
we'll leave the "test-results" directory in-place, but don't do so for
the "prove" target.
The reason for this is that when 28d836c8158 (test: allow running the
tests under "prove", 2010-10-14) allowed for running the tests under
"prove" there was no point in leaving the "test-results" in place.
The "prove" target provides its own summary, so we don't need to run
"aggregate-results", which is the reason we have "test-results" in the
first place. See 2d84e9fb6d2 (Modify test-lib.sh to output stats to
t/test-results/*, 2008-06-08).
But in a subsequent commit test-lib.sh will start emitting reports of
memory leaks in test-results/*, and it will be useful to analyze these
after the fact.
This wouldn't be a problem as failing tests will halt the removal of
the files (we'll never reach "clean-except-prove-cache" from the
"prove" target), but will be subsequently as we'll want to report a
successful run, but might still have e.g. logs of known memory leaks
in test-results/*.
So let's stop removing this, it's sufficient that "make clean" removes
it, and that "pre-clean" (which both "test" and "prove" depend on)
will remove it, i.e. we'll never have a stale "test-results" because
of this change.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-28 01:13:35 +02:00
|
|
|
$(RM) -r 'trash directory'.*
|
2010-03-13 21:41:20 +01:00
|
|
|
$(RM) -r valgrind/bin
|
2012-05-02 17:31:52 +02:00
|
|
|
|
|
|
|
|
clean: clean-except-prove-cache
|
t/Makefile: remove 'test-results' on 'make clean'
The 't/test-results' directory and its contents are by-products of the
test process, so 'make clean' should remove them, but, alas, this has
been broken since fee65b194d (t/Makefile: don't remove test-results in
"clean-except-prove-cache", 2022-07-28).
The 'clean' target in 't/Makefile' was not directly responsible for
removing the 'test-results' directory, but relied on its dependency
'clean-except-prove-cache' to do that [1]. ee65b194d broke this,
because it only removed the 'rm -r test-results' command from the
'clean-except-prove-cache' target instead of moving it to the 'clean'
target, resulting in stray 't/test-results' directories.
Add that missing cleanup command to 't/Makefile', and to all
sub-Makefiles touched by that commit as well.
[1] 60f26f6348 (t/Makefile: retain cache t/.prove across prove runs,
2012-05-02)
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-20 22:16:19 +02:00
|
|
|
$(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)'
|
2010-07-24 00:58:44 +02:00
|
|
|
$(RM) .prove
|
2005-11-08 10:51:10 +01:00
|
|
|
|
2018-07-11 08:46:34 +02:00
|
|
|
clean-chainlint:
|
|
|
|
|
$(RM) -r '$(CHAINLINTTMP_SQ)'
|
|
|
|
|
|
|
|
|
|
check-chainlint:
|
|
|
|
|
@mkdir -p '$(CHAINLINTTMP_SQ)' && \
|
t/Makefile: apply chainlint.pl to existing self-tests
Now that chainlint.pl is functional, take advantage of the existing
chainlint self-tests to validate its operation. (While at it, stop
validating chainlint.sed against the self-tests since it will soon be
retired.)
Due to chainlint.sed implementation limitations leaking into the
self-test "expect" files, a few of them require minor adjustment to make
them compatible with chainlint.pl which does not share those
limitations.
First, because `sed` does not provide any sort of real recursion,
chainlint.sed only emulates recursion into subshells, and each level of
recursion leads to a multiplicative increase in complexity of the `sed`
rules. To avoid substantial complexity, chainlint.sed, therefore, only
emulates subshell recursion one level deep. Any subshell deeper than
that is passed through as-is, which means that &&-chains are not checked
in deeper subshells. chainlint.pl, on the other hand, employs a proper
recursive descent parser, thus checks subshells to any depth and
correctly flags broken &&-chains in deep subshells.
Second, due to sed's line-oriented nature, chainlint.sed, by necessity,
folds multi-line quoted strings into a single line. chainlint.pl, on the
other hand, employs a proper lexical analyzer which preserves quoted
strings as-is, including embedded newlines.
Furthermore, the output of chainlint.sed and chainlint.pl do not match
precisely in terms of whitespace. However, since the purpose of the
self-checks is to verify that the ?!AMP?! annotations are being
correctly added, minor whitespace differences are immaterial. For this
reason, rather than adjusting whitespace in all existing self-test
"expect" files to match the new linter's output, the `check-chainlint`
target ignores whitespace differences. Since `diff -w` is not POSIX,
`check-chainlint` attempts to employ `git diff -w`, and only falls back
to non-POSIX `diff -w` (and `-u`) if `git diff` is not available.
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-01 02:29:46 +02:00
|
|
|
for i in $(CHAINLINTTESTS); do \
|
|
|
|
|
echo "test_expect_success '$$i' '" && \
|
|
|
|
|
sed -e '/^# LINT: /d' chainlint/$$i.test && \
|
|
|
|
|
echo "'"; \
|
|
|
|
|
done >'$(CHAINLINTTMP_SQ)'/tests && \
|
|
|
|
|
{ \
|
|
|
|
|
echo "# chainlint: $(CHAINLINTTMP_SQ)/tests" && \
|
|
|
|
|
for i in $(CHAINLINTTESTS); do \
|
|
|
|
|
echo "# chainlint: $$i" && \
|
|
|
|
|
sed -e '/^[ ]*$$/d' chainlint/$$i.expect; \
|
|
|
|
|
done \
|
|
|
|
|
} >'$(CHAINLINTTMP_SQ)'/expect && \
|
|
|
|
|
$(CHAINLINT) --emit-all '$(CHAINLINTTMP_SQ)'/tests | \
|
2022-11-11 08:34:54 +01:00
|
|
|
sed -e 's/^[1-9][0-9]* //;/^[ ]*$$/d' >'$(CHAINLINTTMP_SQ)'/actual && \
|
t/Makefile: apply chainlint.pl to existing self-tests
Now that chainlint.pl is functional, take advantage of the existing
chainlint self-tests to validate its operation. (While at it, stop
validating chainlint.sed against the self-tests since it will soon be
retired.)
Due to chainlint.sed implementation limitations leaking into the
self-test "expect" files, a few of them require minor adjustment to make
them compatible with chainlint.pl which does not share those
limitations.
First, because `sed` does not provide any sort of real recursion,
chainlint.sed only emulates recursion into subshells, and each level of
recursion leads to a multiplicative increase in complexity of the `sed`
rules. To avoid substantial complexity, chainlint.sed, therefore, only
emulates subshell recursion one level deep. Any subshell deeper than
that is passed through as-is, which means that &&-chains are not checked
in deeper subshells. chainlint.pl, on the other hand, employs a proper
recursive descent parser, thus checks subshells to any depth and
correctly flags broken &&-chains in deep subshells.
Second, due to sed's line-oriented nature, chainlint.sed, by necessity,
folds multi-line quoted strings into a single line. chainlint.pl, on the
other hand, employs a proper lexical analyzer which preserves quoted
strings as-is, including embedded newlines.
Furthermore, the output of chainlint.sed and chainlint.pl do not match
precisely in terms of whitespace. However, since the purpose of the
self-checks is to verify that the ?!AMP?! annotations are being
correctly added, minor whitespace differences are immaterial. For this
reason, rather than adjusting whitespace in all existing self-test
"expect" files to match the new linter's output, the `check-chainlint`
target ignores whitespace differences. Since `diff -w` is not POSIX,
`check-chainlint` attempts to employ `git diff -w`, and only falls back
to non-POSIX `diff -w` (and `-u`) if `git diff` is not available.
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-01 02:29:46 +02:00
|
|
|
if test -f ../GIT-BUILD-OPTIONS; then \
|
|
|
|
|
. ../GIT-BUILD-OPTIONS; \
|
|
|
|
|
fi && \
|
|
|
|
|
if test -x ../git$$X; then \
|
|
|
|
|
DIFFW="../git$$X --no-pager diff -w --no-index"; \
|
|
|
|
|
else \
|
|
|
|
|
DIFFW="diff -w -u"; \
|
|
|
|
|
fi && \
|
|
|
|
|
$$DIFFW '$(CHAINLINTTMP_SQ)'/expect '$(CHAINLINTTMP_SQ)'/actual
|
2018-07-11 08:46:34 +02:00
|
|
|
|
t/Makefile: ensure that paths are valid on platforms we care
Some pathnames that are okay on ext4 and on HFS+ cannot be checked
out on Windows. Tests that want to see operations on such paths on
filesystems that support them must do so behind appropriate test
prerequisites, and must not include them in the source tree (instead
they should create them when they run). Otherwise, the source tree
cannot even be checked out.
Make sure that double-quotes, asterisk, colon, greater/less-than,
question-mark, backslash, tab, vertical-bar, as well as any non-ASCII
characters never appear in the pathnames with a new test-lint-* target
as part of a `make test`. To that end, we call `git ls-files` (ensuring
that the paths are quoted properly), relying on the fact that paths
containing non-ASCII characters are quoted within double-quotes.
In case that the source code does not actually live in a Git
repository (e.g. when extracted from a .zip file), or that the `git`
executable cannot be executed, we simply ignore the error for now; In
that case, our trusty Continuous Integration will be the last line of
defense and catch any problematic file name.
Noticed when a topic wanted to add a pathname with '>' in it. A
check like this will prevent a similar problems from happening in the
future.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-16 17:13:25 +02:00
|
|
|
test-lint: test-lint-duplicates test-lint-executable test-lint-shell-syntax \
|
|
|
|
|
test-lint-filenames
|
2022-09-01 02:29:55 +02:00
|
|
|
ifneq ($(GIT_TEST_CHAIN_LINT),0)
|
|
|
|
|
test-lint: test-chainlint
|
|
|
|
|
endif
|
2010-12-13 18:22:38 +01:00
|
|
|
|
|
|
|
|
test-lint-duplicates:
|
2020-10-20 15:41:02 +02:00
|
|
|
@dups=`echo $(T) $(TPERF) | tr ' ' '\n' | sed 's/-.*//' | sort | uniq -d` && \
|
2010-12-13 18:22:38 +01:00
|
|
|
test -z "$$dups" || { \
|
|
|
|
|
echo >&2 "duplicate test numbers:" $$dups; exit 1; }
|
|
|
|
|
|
|
|
|
|
test-lint-executable:
|
2020-10-20 15:41:02 +02:00
|
|
|
@bad=`for i in $(T) $(TPERF); do test -x "$$i" || echo $$i; done` && \
|
2010-12-13 18:22:38 +01:00
|
|
|
test -z "$$bad" || { \
|
|
|
|
|
echo >&2 "non-executable tests:" $$bad; exit 1; }
|
|
|
|
|
|
2013-01-03 00:20:19 +01:00
|
|
|
test-lint-shell-syntax:
|
2020-10-20 15:41:02 +02:00
|
|
|
@'$(PERL_PATH_SQ)' check-non-portable-shell.pl $(T) $(THELPERS) $(TPERF)
|
2013-01-03 00:20:19 +01:00
|
|
|
|
t/Makefile: ensure that paths are valid on platforms we care
Some pathnames that are okay on ext4 and on HFS+ cannot be checked
out on Windows. Tests that want to see operations on such paths on
filesystems that support them must do so behind appropriate test
prerequisites, and must not include them in the source tree (instead
they should create them when they run). Otherwise, the source tree
cannot even be checked out.
Make sure that double-quotes, asterisk, colon, greater/less-than,
question-mark, backslash, tab, vertical-bar, as well as any non-ASCII
characters never appear in the pathnames with a new test-lint-* target
as part of a `make test`. To that end, we call `git ls-files` (ensuring
that the paths are quoted properly), relying on the fact that paths
containing non-ASCII characters are quoted within double-quotes.
In case that the source code does not actually live in a Git
repository (e.g. when extracted from a .zip file), or that the `git`
executable cannot be executed, we simply ignore the error for now; In
that case, our trusty Continuous Integration will be the last line of
defense and catch any problematic file name.
Noticed when a topic wanted to add a pathname with '>' in it. A
check like this will prevent a similar problems from happening in the
future.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-16 17:13:25 +02:00
|
|
|
test-lint-filenames:
|
|
|
|
|
@# We do *not* pass a glob to ls-files but use grep instead, to catch
|
|
|
|
|
@# non-ASCII characters (which are quoted within double-quotes)
|
|
|
|
|
@bad="$$(git -c core.quotepath=true ls-files 2>/dev/null | \
|
|
|
|
|
grep '["*:<>?\\|]')"; \
|
|
|
|
|
test -z "$$bad" || { \
|
|
|
|
|
echo >&2 "non-portable file name(s): $$bad"; exit 1; }
|
|
|
|
|
|
2022-09-01 02:29:55 +02:00
|
|
|
test-chainlint:
|
|
|
|
|
@$(CHAINLINT) $(T) $(TLIBS) $(TPERF) $(TINTEROP)
|
|
|
|
|
|
tests: Clarify dependencies between tests, 'aggregate-results' and 'clean'
The Makefile targets 'aggregate-results' and 'clean' pretended to be
independent. This is not true, of course, since aggregate-results
needs the results _before_ they are removed.
Likewise, the tests should have been run already when the results are
to be aggregated.
However, as it is legitimate to run only a few tests, and then aggregate
just those results, so another target is introduced, that depends on all
tests, then aggregates the results, and only then removes the results.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-08 07:59:18 +02:00
|
|
|
aggregate-results-and-cleanup: $(T)
|
|
|
|
|
$(MAKE) aggregate-results
|
|
|
|
|
$(MAKE) clean
|
|
|
|
|
|
2008-06-08 16:04:35 +02:00
|
|
|
aggregate-results:
|
2023-03-09 18:30:32 +01:00
|
|
|
@'$(SHELL_PATH_SQ)' ./aggregate-results.sh '$(TEST_RESULTS_DIRECTORY_SQ)'
|
2008-06-08 16:04:35 +02:00
|
|
|
|
2009-02-04 00:26:18 +01:00
|
|
|
valgrind:
|
2011-06-17 10:29:57 +02:00
|
|
|
$(MAKE) GIT_TEST_OPTS="$(GIT_TEST_OPTS) --valgrind"
|
2009-02-04 00:26:18 +01:00
|
|
|
|
2012-02-17 11:25:09 +01:00
|
|
|
perf:
|
|
|
|
|
$(MAKE) -C perf/ all
|
|
|
|
|
|
2022-09-01 02:29:55 +02:00
|
|
|
.PHONY: pre-clean $(T) aggregate-results clean valgrind perf \
|
|
|
|
|
check-chainlint clean-chainlint test-chainlint
|