Commit Graph

104 Commits

Author SHA1 Message Date
Junio C Hamano
381e8e9de1 Merge branch 'dl/test-must-fail-fixes'
Test clean-up.

* dl/test-must-fail-fixes:
  t1507: inline full_name()
  t1507: run commands within test_expect_success
  t1507: stop losing return codes of git commands
  t1501: remove use of `test_might_fail cp`
  t1409: use test_path_is_missing()
  t1409: let sed open its own input file
  t1307: reorder `nongit test_must_fail`
  t1306: convert `test_might_fail rm` to `rm -f`
  t0020: use ! check_packed_refs_marked
  t0020: don't use `test_must_fail has_cr`
  t0003: don't use `test_must_fail attr_check`
  t0003: use test_must_be_empty()
  t0003: use named parameters in attr_check()
  t0000: replace test_must_fail with run_sub_test_lib_test_err()
  t/lib-git-p4: use test_path_is_missing()
2020-02-05 14:34:57 -08:00
Denton Liu
7717242014 t0000: replace test_must_fail with run_sub_test_lib_test_err()
The test_must_fail function should only be used for git commands since
we should assume that external commands work sanely. We use
test_must_fail to test run_sub_test_lib_test() but that function does
not invoke any git commands internally. Even better, we have a function
that's exactly meant to be used when we expect to have a failing test
suite: run_sub_test_lib_test_err()!

Replace `test_must_fail run_sub_test_lib_test` with
`run_sub_test_lib_test_err`.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-20 11:30:44 -08:00
Junio C Hamano
e0f9ec9027 Merge branch 'sg/test-bool-env'
Recently we have declared that GIT_TEST_* variables take the
usual boolean values (it used to be that some used "non-empty
means true" and taking GIT_TEST_VAR=YesPlease as true); make
sure we notice and fail when non-bool strings are given to
these variables.

* sg/test-bool-env:
  t5608-clone-2gb.sh: turn GIT_TEST_CLONE_2GB into a bool
  tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values
2019-12-05 12:52:48 -08:00
Junio C Hamano
8feb47e882 Merge branch 'dl/t5520-cleanup'
Test cleanup.

* dl/t5520-cleanup:
  t5520: replace `! git` with `test_must_fail git`
  t5520: remove redundant lines in test cases
  t5520: replace $(cat ...) comparison with test_cmp
  t5520: don't put git in upstream of pipe
  t5520: test single-line files by git with test_cmp
  t5520: use test_cmp_rev where possible
  t5520: replace test -{n,z} with test-lib functions
  t5520: use test_line_count where possible
  t5520: remove spaces after redirect operator
  t5520: replace test -f with test-lib functions
  t5520: let sed open its own input
  t5520: use sq for test case names
  t5520: improve test style
  t: teach test_cmp_rev to accept ! for not-equals
  t0000: test multiple local assignment
2019-12-05 12:52:47 -08:00
SZEDER Gábor
43a2afee82 tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values
Since 3b072c577b (tests: replace test_tristate with "git env--helper",
2019-06-21) we get the normalized bool values of various GIT_TEST_*
environment variables via 'git env--helper'.  Now, while the 'git
env--helper' command itself does catch invalid values in the
environment variable or in the given --default and exits with error
(exit code 128 or 129, respectively), it's invoked in conditions like
'if ! git env--helper ...', which means that all invalid bool values
are interpreted the same as the ordinary 'false' (exit code 1).  This
has led to inadvertently skipped httpd tests in our CI builds for a
couple of weeks, see 3960290675 (ci: restore running httpd tests,
2019-09-06).

Let's be more careful about what the test suite accepts as bool values
in GIT_TEST_* environment variables, and error out loud and clear on
invalid values instead of simply skipping tests.  Add the
'test_bool_env' helper function to encapsulate the invocation of 'git
env--helper' and the verification of its exit code, and replace all
invocations of that command in our test framework and test suite with
a call to this new helper (except in 't0017-env-helper.sh', of
course).

  $ GIT_TEST_GIT_DAEMON=YesPlease ./t5570-git-daemon.sh
  fatal: bad numeric config value 'YesPlease' for 'GIT_TEST_GIT_DAEMON': invalid unit
  error: test_bool_env requires bool values both for $GIT_TEST_GIT_DAEMON and for the default fallback

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-23 11:16:08 +09:00
Denton Liu
8cb7980382 t0000: test multiple local assignment
According to POSIX enhancement request '0000767: Add built-in
"local"'[1],

	dash only allows one variable in a local definition; it permits
	assignment though it doesn't document that clearly.

however, this isn't true since t0000 still passes with this patch
applied on dash 0.5.10.2. Needless to say, since `local` isn't POSIX
standardized, it is not exactly clear what `local` entails on different
versions of different shells.

We currently already have many instances of multiple local assignments
in our codebase. Ensure that this is actually supported by explicitly
testing that it is sane.

[1]: http://austingroupbugs.net/bug_view_page.php?bug_id=767

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-21 09:40:08 +09:00
Elijah Newren
7a40cf1553 Fix spelling errors in comments of testcases
Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10 16:00:54 +09:00
Junio C Hamano
4e8371ec26 Merge branch 'dl/t0000-skip-test-test'
test update.

* dl/t0000-skip-test-test:
  t0000: cover GIT_SKIP_TESTS blindspots
2019-10-15 13:48:02 +09:00
Denton Liu
b05b40930e t0000: cover GIT_SKIP_TESTS blindspots
Currently, the tests for GIT_SKIP_TESTS do not cover the situation where
we skip an entire test suite. The tests also do not cover the situation
where we have GIT_SKIP_TESTS defined but the test suite does not match.

Add two test cases so we cover this blindspot.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-09 10:47:08 +09:00
Junio C Hamano
8ae7a46c4d Merge branch 'sg/show-failed-test-names'
The first line of verbose output from each test piece now carries
the test name and number to help scanning with eyeballs.

* sg/show-failed-test-names:
  tests: show the test name and number at the start of verbose output
  t0000-basic: use realistic test script names in the verbose tests
2019-08-22 12:34:11 -07:00
Junio C Hamano
fcdd9cc86d Merge branch 'bc/hash-independent-tests-part-4'
Test fix.

* bc/hash-independent-tests-part-4:
  t0000: reword comments for "local" test
  t: decrease nesting in test_oid_to_path
2019-08-09 10:13:14 -07:00
Jeff King
7f0b590875 t0000: reword comments for "local" test
Commit 01d3a526ad (t0000: check whether the shell supports the "local"
keyword, 2017-10-26) added a test to gather data on whether people run
the test suite with shells that don't support "local".

After almost two years, nobody has complained, and several other uses
have cropped up in test-lib-functions.sh. Let's declare it acceptable to
use.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-08 10:22:55 -07:00
SZEDER Gábor
ffe1afe67c tests: show the test name and number at the start of verbose output
The verbose output of every test looks something like this:

  expecting success:
          echo content >file &&
          git add file &&
          git commit -m "add file"

  [master (root-commit) d1fbfbd] add file
   Author: A U Thor <author@example.com>
   1 file changed, 1 insertion(+)
   create mode 100644 file
  ok 1 - commit works

i.e. first an "expecting success" (or "checking known breakage") line
followed by the commands to be executed, then the output of those
comamnds, and finally an "ok"/"not ok" line containing the test name.
Note that the test's name is only shown at the very end.

With '-x' tracing enabled and/or in longer tests the verbose output
might be several screenfulls long, making it harder than necessary to
find where the output of the test with a given name starts (especially
when the outputs to different file descriptors are racing, and the
"expecting success"/command block arrives earlier than the "ok" line
of the previous test).

Print the test name at the start of the test's verbose output, i.e. at
the end of the "expecting success" and "checking known breakage"
lines, to make the start of a particular test a bit easier to
recognize.  Also print the test script and test case numbers, to help
those poor souls who regularly have to scan through the combined
verbose output of several test scripts.

So the dummy test above would start like this:

  expecting success of 9999.1 'commit works':
          echo content >file &&
  [...]

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-05 15:21:33 -07:00
SZEDER Gábor
96f3ccc2ab t0000-basic: use realistic test script names in the verbose tests
Our test scripts are named something like 't1234-command.sh', but the
script names used in 't0000-basic.sh' don't follow this naming
convention.  Normally this doesn't matter, because the test scripts
themselves don't care how they are called.  However, the next patch
will start to include the test number in the test's verbose output, so
the test script's name will matter in the two tests checking the
verbose output.

Update the tests 'test --verbose' and 'test --verbose-only' to follow
out test script naming convention.

Leave the other tests in 't0000' unchanged: changing the names of
their test scripts would be only pointless code churn.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-05 15:21:31 -07:00
Ævar Arnfjörð Bjarmason
c740039921 tests: make GIT_TEST_FAIL_PREREQS a boolean
Change the GIT_TEST_FAIL_PREREQS variable from being "non-empty?" to
being a more standard boolean variable. I recently added the variable
in dfe1a17df9 ("tests: add a special setup where prerequisites fail",
2019-05-13), having to add another "non-empty?" special-case is what
prompted me to write the "git env--helper" utility being used here.

Converting this one is a bit tricky since we use it so early and
frequently in the guts of the test code itself, so let's set a
GIT_TEST_FAIL_PREREQS_INTERNAL which can be tested with the old "test
-n" for the purposes of the shell code, and change the user-exposed
and documented GIT_TEST_FAIL_PREREQS variable to a boolean.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-21 09:42:49 -07:00
Ævar Arnfjörð Bjarmason
dfe1a17df9 tests: add a special setup where prerequisites fail
As discussed in [1] there's a regression in the "pu" branch now
because a new test implicitly assumed that a previous test guarded by
a prerequisite had been run. Add a "GIT_TEST_FAIL_PREREQS" special
test setup where we'll skip (nearly) all tests guarded by
prerequisites, allowing us to easily emulate those platform where we
don't run these tests.

As noted in the documentation I'm adding I'm whitelisting the SYMLINKS
prerequisite for now. A lot of tests started failing if we lied about
not supporting symlinks. It's also unlikely that we'll have a failing
test due to a hard dependency on symlinks without that being the
obvious cause, so for now it's not worth the effort to make it work.

1. https://public-inbox.org/git/nycvar.QRO.7.76.6.1905131531000.44@tvgsbejvaqbjf.bet/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-14 16:48:17 +09:00
Johannes Schindelin
900721e15c test-lib: introduce 'test_atexit'
When running Apache, 'git daemon', or p4d, we want to kill them at the
end of the test script, otherwise a leftover daemon process will keep
its port open indefinitely, and thus will interfere with subsequent
executions of the same test script.

So far, we stop these daemon processes "manually", i.e.:

  - by registering functions or commands in the trap on EXIT to stop
    the daemon while preserving the last seen exit code before the
    trap (to deal with a failure when run with '--immediate' or with
    interrupts by ctrl-C),

  - and by invoking these functions/commands last thing before
    'test_done' (and sometimes restoring the test framework's default
    trap on EXIT, to prevent the daemons from being killed twice).

On one hand, we do this inconsistently, e.g. 'git p4' tests invoke
different functions in the trap on EXIT and in the last test before
'test_done', and they neither restore the test framework's default trap
on EXIT nor preserve the last seen exit code.  On the other hand, this
is error prone, because, as shown in a previous patch in this series,
any output from the cleanup commands in the trap on EXIT can prevent a
proper cleanup when a test script run with '--verbose-log' and certain
shells, notably 'dash', is interrupted.

Let's introduce 'test_atexit', which is loosely modeled after
'test_when_finished', but has a broader scope: rather than running the
commands after the current test case, run them when the test script
finishes, and also run them when the test is interrupted, or exits
early in case of a failure while the '--immediate' option is in
effect.

When running the cleanup commands at the end of a successful test,
then they will be run in 'test_done' before it removes the trash
directory, i.e. the cleanup commands will still be able to access any
pidfiles or socket files in there.  When running the cleanup commands
after an interrupt or failure with '--immediate', then they will be
run in the trap on EXIT.  In both cases they will be run in
'test_eval_', i.e. both standard error and output of all cleanup
commands will go where they should according to the '-v' or
'--verbose-log' options, and thus won't cause any troubles when
interrupting a test script run with '--verbose-log'.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14 12:34:39 +09:00
Ævar Arnfjörð Bjarmason
6cdccfce1e i18n: make GETTEXT_POISON a runtime option
Change the GETTEXT_POISON compile-time + runtime GIT_GETTEXT_POISON
test parameter to only be a GIT_TEST_GETTEXT_POISON=<non-empty?>
runtime parameter, to be consistent with other parameters documented
in "Running tests with special setups" in t/README.

When I added GETTEXT_POISON in bb946bba76 ("i18n: add GETTEXT_POISON
to simulate unfriendly translator", 2011-02-22) I was concerned with
ensuring that the _() function would get constant folded if NO_GETTEXT
was defined, and likewise that GETTEXT_POISON would be compiled out
unless it was defined.

But as the benchmark in my [1] shows doing a one-off runtime
getenv("GIT_TEST_[...]") is trivial, and since GETTEXT_POISON was
originally added the GIT_TEST_* env variables have become the common
idiom for turning on special test setups.

So change GETTEXT_POISON to work the same way. Now the
GETTEXT_POISON=YesPlease compile-time option is gone, and running the
tests with GIT_TEST_GETTEXT_POISON=[YesPlease|] can be toggled on/off
without recompiling.

This allows for conditionally amending tests to test with/without
poison, similar to what 859fdc0c3c ("commit-graph: define
GIT_TEST_COMMIT_GRAPH", 2018-08-29) did for GIT_TEST_COMMIT_GRAPH. Do
some of that, now we e.g. always run the t0205-gettext-poison.sh test.

I did enough there to remove the GETTEXT_POISON prerequisite, but its
inverse C_LOCALE_OUTPUT is still around, and surely some tests using
it can be converted to e.g. always set GIT_TEST_GETTEXT_POISON=.

Notes on the implementation:

 * We still compile a dedicated GETTEXT_POISON build in Travis
   CI. Perhaps this should be revisited and integrated into the
   "linux-gcc" build, see ae59a4e44f ("travis: run tests with
   GIT_TEST_SPLIT_INDEX", 2018-01-07) for prior art in that area. Then
   again maybe not, see [2].

 * We now skip a test in t0000-basic.sh under
   GIT_TEST_GETTEXT_POISON=YesPlease that wasn't skipped before. This
   test relies on C locale output, but due to an edge case in how the
   previous implementation of GETTEXT_POISON worked (reading it from
   GIT-BUILD-OPTIONS) wasn't enabling poison correctly. Now it does,
   and needs to be skipped.

 * The getenv() function is not reentrant, so out of paranoia about
   code of the form:

       printf(_("%s"), getenv("some-env"));

   call use_gettext_poison() in our early setup in git_setup_gettext()
   so we populate the "poison_requested" variable in a codepath that's
   won't suffer from that race condition.

 * We error out in the Makefile if you're still saying
   GETTEXT_POISON=YesPlease to prompt users to change their
   invocation.

 * We should not print out poisoned messages during the test
   initialization itself to keep it more readable, so the test library
   hides the variable if set in $GIT_TEST_GETTEXT_POISON_ORIG during
   setup. See [3].

See also [4] for more on the motivation behind this patch, and the
history of the GETTEXT_POISON facility.

1. https://public-inbox.org/git/871s8gd32p.fsf@evledraar.gmail.com/
2. https://public-inbox.org/git/20181102163725.GY30222@szeder.dev/
3. https://public-inbox.org/git/20181022202241.18629-2-szeder.dev@gmail.com/
4. https://public-inbox.org/git/878t2pd6yu.fsf@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-09 11:25:19 +09:00
Junio C Hamano
340fde61be Merge branch 'bp/rename-test-env-var'
Some environment variables that control the runtime options of Git
used during tests are getting renamed for consistency.

* bp/rename-test-env-var:
  t0000: do not get self-test disrupted by environment warnings
  preload-index: update GIT_FORCE_PRELOAD_TEST support
  read-cache: update TEST_GIT_INDEX_VERSION support
  fsmonitor: update GIT_TEST_FSMONITOR support
  preload-index: use git_env_bool() not getenv() for customization
  t/README: correct spelling of "uncommon"
2018-10-19 13:34:03 +09:00
Junio C Hamano
f2e2136ad7 Merge branch 'md/test-cleanup'
Various test scripts have been updated for style and also correct
handling of exit status of various commands.

* md/test-cleanup:
  tests: order arguments to git-rev-list properly
  t9109: don't swallow Git errors upstream of pipes
  tests: don't swallow Git errors upstream of pipes
  t/*: fix ordering of expected/observed arguments
  tests: standardize pipe placement
  Documentation: add shell guidelines
  t/README: reformat Do, Don't, Keep in mind lists
2018-10-16 16:16:01 +09:00
Matthew DeVore
dcbaa0b361 t/*: fix ordering of expected/observed arguments
Fix various places where the ordering was obviously wrong, meaning it
was easy to find with grep.

Signed-off-by: Matthew DeVore <matvore@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-07 08:51:18 +09:00
Junio C Hamano
4231d1ba99 t0000: do not get self-test disrupted by environment warnings
The test framework test-lib.sh itself would want to give warnings
and hints, e.g. when it sees a deprecated environment variable is in
use that we want to encourage users to migrate to another variable.

The self-test of test framework done in t0000 however do not expect
to see these warnings and hints, so depending on the settings of
environment variables, a running test may or may not produce these
messages to the standard error output, breaking the expectations of
self-test test framework does on itself.  Here is what we see:

    $ TEST_GIT_INDEX_VERSION=4 sh t0000-basic.sh -i -v
    ...
    'err' is not empty, it contains:
    warning: TEST_GIT_INDEX_VERSION is now GIT_TEST_INDEX_VERSION
    hint: set GIT_TEST_INDEX_VERSION too during the transition period
    not ok 5 - pretend we have a fully passing test suite

The following quick attempt to work it around does not work, because
some tests in t0000 do want to see expected errors from the test
framework itself.

         t/t0000-basic.sh | 2 +-
         1 file changed, 1 insertion(+), 1 deletion(-)

        diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
        index 850f651e4e..88c6ed4696 100755
        --- a/t/t0000-basic.sh
        +++ b/t/t0000-basic.sh
        @@ -88,7 +88,7 @@ _run_sub_test_lib_test_common () {
                        '

                        # Point to the t/test-lib.sh, which isn't in ../ as usual
        -		. "\$TEST_DIRECTORY"/test-lib.sh
        +		. "\$TEST_DIRECTORY"/test-lib.sh >/dev/null 2>&1
                        EOF
                        cat >>"$name.sh" &&
                        chmod +x "$name.sh" &&

There are a few possible ways to work this around:

 * We could strip the warning: and hint: unconditionally from the
   error output before the error messages are checked in the
   self-test (helper functions check_sub_test_lib_test_err and
   check_sub_test_lib_test); the problem with this approach is that
   it will make it impossible to write self-tests to ensure that
   right warnings and hints are given.

 * We could force a sane environment settings before the test helper
   _run_sub_test_lib_test_common dot-sources test-lib.sh; the
   problem with this approach is that _run_sub_test_lib_test_common
   now needs to be aware of what pairs of environment variables are
   checked in test-lib.sh using check_var_migration helper.

The final patch I came up with is probably the solution that is
least bad.  Set a variable to tell test-lib.sh that we are running
a self-test, so that various pieces in test-lib.sh can react to keep
the output stable.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-28 11:41:01 -07:00
brian m. carlson
e483e1441a t0000: update tests for SHA-256
Test t0000 tests the "basics of the basics" and as such, checks that we
have various fixed hard-coded object IDs.  The tests relying on these
assertions have been marked with the SHA1 prerequisite, as they will
obviously not function in their current form with SHA-256.

Use the test_oid helper to update these assertions and provide values
for both SHA-1 and SHA-256.

These object IDs were synthesized using a set of scripts that created
the objects for both SHA-1 and SHA-256 using the same method to ensure
that they are indeed the correct values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-13 14:15:24 -07:00
brian m. carlson
cdd1e17f87 t0000: use hash translation table
If the hash we're using is 32 bytes in size, attempting to insert a
20-byte object name won't work.  Since these are synthesized objects
that are almost all zeros, look them up in a translation table.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-13 14:15:24 -07:00
brian m. carlson
2c02b110da t: add test functions to translate hash-related values
Add several test functions to make working with various hash-related
values easier.

Add test_oid_init, which loads common hash-related constants and
placeholder object IDs from the newly added files in t/oid-info.
Provide values for these constants for both SHA-1 and SHA-256.

Add test_oid_cache, which accepts data on standard input in the form of
hash-specific key-value pairs that can be looked up later, using the
same format as the files in t/oid-info.  Document this format in a
t/oid-info/README directory so that it's easier to use in the future.

Add test_oid, which is used to specify look up a per-hash value
(produced on standard output) based on the key specified as its
argument.  Usually the data to be looked up will be a hash-related
constant (such as the size of the hash in binary or hexadecimal), a
well-known or placeholder object ID (such as the all-zeros object ID or
one consisting of "deadbeef" repeated), or something similar.  For these
reasons, test_oid will usually be used within a command substitution.
Consequently, redirect the error output to standard error, since
otherwise it will not be displayed.

Add test_detect_hash, which currently only detects SHA-1, and
test_set_hash, which can be used to set a different hash algorithm for
test purposes.  In the future, test_detect_hash will learn to actually
detect the hash depending on how the testsuite is to be run.

Use the local keyword within these functions to avoid overwriting other
shell variables.  We have had a test balloon in place for a couple of
releases to catch shells that don't have this keyword and have not
received any reports of failure.  Note that the varying usages of local
used here are supported by all common open-source shells supporting the
local keyword.

Test these new functions as part of t0000, which also serves to
demonstrate basic usage of them.  In addition, add documentation on how
to format the lookup data and how to use the test functions.

Implement two basic lookup charts, one for common invalid or synthesized
object IDs, and one for various facts about the hash function in use.
Provide versions of the data for both SHA-1 and SHA-256.

Since we use shell variables for storage, names used for lookup can
currently consist only of shell identifier characters.  If this is a
problem in the future, we can hash the names before use.

Improved-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-13 14:15:24 -07:00
Junio C Hamano
986c518107 Merge branch 'sg/test-must-be-empty'
Test fixes.

* sg/test-must-be-empty:
  tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>'
  tests: use 'test_must_be_empty' instead of 'test_cmp /dev/null <out>'
  tests: use 'test_must_be_empty' instead of 'test ! -s'
  tests: use 'test_must_be_empty' instead of '! test -s'
2018-08-27 14:33:43 -07:00
SZEDER Gábor
ec10b018e7 tests: use 'test_must_be_empty' instead of '! test -s'
Using 'test_must_be_empty' is preferable to '! test -s', because it
gives a helpful error message if the given file is unexpectedly not
empty, while the latter remains completely silent.  Furthermore, it
also catches cases when the given file unexpectedly does not exist at
all.

This patch was basically created by:

  sed -i -e 's/! test -s/test_must_be_empty/' t[0-9]*.sh

with the following notable exceptions:

  - The '! test -s' check in '.gitmodules ignore=dirty suppresses
    submodules with untracked content' in 't7508-status.sh' is left
    as-is, because it's bogus and, therefore, it's subject of a
    dedicated patch.

  - The '! test -s' checks in 't9131-git-svn-empty-symlink.sh' and
    't9135-git-svn-moved-branch-empty-file.sh' are immediately
    preceeded by a 'test -f' to ensure that the files exist in the
    first place.  'test_must_be_empty' ensures that as well, so those
    'test -f' commands are removed as well.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 11:48:29 -07:00
Eric Sunshine
75651fd783 t0000-t0999: fix broken &&-chains
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 14:38:47 -07:00
brian m. carlson
5b9ba9bd80 t0000: annotate with SHA1 prerequisite
Since this is a core test that tests basic functionality, annotate the
assertions that have dependencies on SHA-1 with the appropriate
prerequisite.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-14 11:02:01 +09:00
Michael Haggerty
01d3a526ad t0000: check whether the shell supports the "local" keyword
Add a test balloon to see if we get complaints from anybody who is
using a shell that doesn't support the "local" keyword. If so, this
test can be reverted. If not, we might want to consider using "local"
in shell code throughout the git code base.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-31 11:41:39 +09:00
Nguyễn Thái Ngọc Duy
f9e7d9f8c3 test-lib.sh: introduce and use $EMPTY_TREE
This is a special SHA1. Let's keep it at one place, easier to replace
later when the hash change comes, easier to recognize.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-18 13:45:32 -07:00
Li Peng
832c0e5e63 typofix: assorted typofixes in comments, documentation and messages
Many instances of duplicate words (e.g. "the the path") and
a few typoes are fixed, originally in multiple patches.

    wildmatch: fix duplicate words of "the"
    t: fix duplicate words of "output"
    transport-helper: fix duplicate words of "read"
    Git.pm: fix duplicate words of "return"
    path: fix duplicate words of "look"
    pack-protocol.txt: fix duplicate words of "the"
    precompose-utf8: fix typo of "sequences"
    split-index: fix typo
    worktree.c: fix typo
    remote-ext: fix typo
    utf8: fix duplicate words of "the"
    git-cvsserver: fix duplicate words

Signed-off-by: Li Peng <lip@dtdream.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06 13:16:37 -07:00
Jeff King
99094a7ad4 t: fix trivial &&-chain breakage
These are tests which are missing a link in their &&-chain,
but during a setup phase. We may fail to notice failure in
commands that build the test environment, but these are
typically not expected to fail at all (but it's still good
to double-check that our test environment is what we
expect).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20 10:20:14 -07:00
Ramsay Jones
7e28c16fdb t0000-*.sh: fix the GIT_SKIP_TESTS sub-tests
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06 13:48:00 -07:00
Ilya Bobyr
0445e6f0a1 test-lib: '--run' to run only specific tests
Allow better control of the set of tests that will be executed for a
single test suite.  Mostly useful while debugging or developing as it
allows to focus on a specific test.

Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06 13:48:00 -07:00
Ilya Bobyr
ef2ac68def test-lib: tests skipped by GIT_SKIP_TESTS say so
We used to show "(missing )" next to tests skipped because they are
specified in GIT_SKIP_TESTS.  Use "(GIT_SKIP_TESTS)" instead.

Plus tests that check basic GIT_SKIP_TESTS functions.

Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06 13:48:00 -07:00
Jeff King
738a8beac4 t0000: drop "known breakage" test
Having a simulated "known breakage" test means that the test
suite will always tell us there is a bug to be fixed, even
though it is only simulated.

The right way to test this is in a sub-test, that can also
check that we provide the correct exit status and output.
Fortunately, we already have such a test (added much later
by 5ebf89e).

We could arguably get rid of the simulated success test
immediately above, as well, as it is also redundant with the
tests added in 5ebf89e. However, it does not have the
annoying behavior of the "known breakage" test. It may also
be easier to debug if the test suite is truly broken, since
it is not a test-within-a-test, as the later tests are.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-02 14:43:12 -08:00
Jeff King
a63c12c9be t0000: simplify HARNESS_ACTIVE hack
Commit 517cd55 set HARNESS_ACTIVE unconditionally in
sub-tests, because that value affects the output of
"--verbose". t0000 needs stable output from its sub-tests,
and we may or may not be running under a TAP harness.

That commit made the decision to always set the variable,
since it has another useful side effect, which is
suppressing writes to t/test-results by the sub-tests (which
would just pollute the real results).

Since the last commit, though, the sub-tests have their own
test-results directories, so this is no longer an issue. We
can now update a few comments that are no longer accurate
nor necessary.

We can also revisit the choice of HARNESS_ACTIVE. Since we
must choose one value for stability, it's probably saner to
have it off. This means that future patches could test
things like the test-results writing, or the "--quiet"
option, which is currently ignored when run under a harness.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-02 14:43:11 -08:00
Jeff King
6883047071 t0000: set TEST_OUTPUT_DIRECTORY for sub-tests
Running t0000 produces more trash directories than expected and does
not clean up after itself:

    $ ./t0000-basic.sh
    [...]
    $ ls -d trash\ directory.*
    trash directory.failing-cleanup
    trash directory.mixed-results1
    trash directory.mixed-results2
    trash directory.partial-pass
    trash directory.test-verbose
    trash directory.test-verbose-only-2

These scratch areas for sub-tests should be under the t0000 trash
directory, but because TEST_OUTPUT_DIRECTORY defaults to
TEST_DIRECTORY, which is exported to help sub-tests find
test-lib.sh, the sub-test trash directories are created under the
toplevel t/ directory instead.  Because some of the sub-tests
simulate failures, their trash directories are kept around.

Fix it by explicitly setting TEST_OUTPUT_DIRECTORY appropriately for
sub-tests.

An alternative fix would be to pass the --root parameter that only
specifies where to put the trash directories, which would also work.
However, using TEST_OUTPUT_DIRECTORY is more futureproof in case
tests want to write more output in addition to the test-results/
(which are already suppressed in sub-tests using the HARNESS_ACTIVE
setting) and trash directories.

This fixes a regression introduced by 38b074d (t/test-lib.sh: fix
TRASH_DIRECTORY handling, 2013-04-14).  Before that commit, the
TEST_OUTPUT_DIRECTORY setting was not respected consistently so most
tests did their work in a "trash" subdirectory of the current
directory instead of the output dir.

Signed-off-by: Jeff King <peff@peff.net>
Clarified-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-02 14:40:03 -08:00
Junio C Hamano
f3930e4389 Merge branch 'tr/test-v-and-v-subtest-only'
Finishing touches to a topic that is already in master for the
upcoming release.

* tr/test-v-and-v-subtest-only:
  t0000: do not use export X=Y
2013-07-11 13:06:02 -07:00
Torsten Bögershausen
e66681eb94 t0000: do not use export X=Y
The shell syntax "export X=Y A=B" is not understood by all shells.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Acked-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-08 08:22:12 -07:00
Junio C Hamano
04f2ddda84 Merge branch 'tr/test-v-and-v-subtest-only'
Allows N instances of tests run in parallel, each running 1/N parts
of the test suite under Valgrind, to speed things up.

* tr/test-v-and-v-subtest-only:
  perf-lib: fix start/stop of perf tests
  test-lib: support running tests under valgrind in parallel
  test-lib: allow prefixing a custom string before "ok N" etc.
  test-lib: valgrind for only tests matching a pattern
  test-lib: verbose mode for only tests matching a pattern
  test-lib: self-test that --verbose works
  test-lib: rearrange start/end of test_expect_* and test_skip
  test-lib: refactor $GIT_SKIP_TESTS matching
  test-lib: enable MALLOC_* for the actual tests
2013-07-05 01:15:48 -07:00
Thomas Rast
ff09af3fb8 test-lib: verbose mode for only tests matching a pattern
With the new --verbose-only=<pattern> option, one can enable --verbose
at a per-test granularity.  The pattern is matched against the test
number, e.g.

  ./t0000-basic.sh --verbose-only='2[0-2]'

to see only the full output of test 20-22, while showing the rest in the
one-liner format.

As suggested by Jeff King, this takes care to wrap the entire
test_expect_* block, but nothing else, in the verbose toggling.  We
can use the test_start/end functions from the previous commit for the
purpose.

This is arguably not *too* useful on its own, but makes the next patch
easier to follow.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-23 12:24:06 -07:00
Thomas Rast
517cd55fd5 test-lib: self-test that --verbose works
t0000 contains some light self-tests of test-lib.sh, but --verbose was
not covered.  Add a test.

The only catch is that the presence of a test harness influences the
output (specifically, the presence of some empty lines).  So we need
to unset TEST_HARNESS or set it to a known value.  Leaving it unset
leads to spurious test failures in the final summary, which come from
the subtest.  So we always set it.

Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-23 12:24:05 -07:00
Johannes Sixt
c723a76d4d t0000: use test_ln_s_add to remove SYMLINKS prerequisite
t0000-basic hard-codes many object IDs. To cater to file systems that do
not support symbolic links, different IDs are used depending on the
SYMLINKS prerequisite. But we can observe the symbolic links are only
needed to generate index entries. Use test_ln_s_add to generate the
index entries and get rid of explicit SYMLINKS checks.

This undoes the special casing introduced in this test by 704a3143
(Use prerequisite tags to skip tests that depend on symbolic links,
2009-03-04).

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-07 15:01:48 -07:00
Junio C Hamano
6fedcd8188 Merge branch 'as/test-tweaks'
Output from the tests is coloured using "green is okay, yellow is
questionable, red is bad and blue is informative" scheme.

* as/test-tweaks:
  tests: paint unexpectedly fixed known breakages in bold red
  tests: test the test framework more thoroughly
  tests: refactor mechanics of testing in a sub test-lib
  tests: change info messages from yellow/brown to cyan
  tests: paint skipped tests in blue
  tests: paint known breakages in yellow
  tests: test number comes first in 'not ok $count - $message'
2013-01-03 10:29:12 -08:00
Adam Spiers
b73d9a2363 tests: paint unexpectedly fixed known breakages in bold red
Change color of unexpectedly fixed known breakages to bold red.  An
unexpectedly passing test indicates that the test code is somehow
broken or out of sync with the code it is testing.  Either way this is
an error which is potentially as bad as a failing test, and as such is
no longer portrayed as a pass in the output.

Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-20 14:22:12 -08:00
Adam Spiers
5ebf89e886 tests: test the test framework more thoroughly
Add 5 new full test suite runs each with a different number of
passing/failing/broken/fixed tests, in order to ensure that the
correct exit code and output are generated in each case.  As before,
these are run in a subdirectory to avoid disrupting the metrics for
the parent tests.

Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-20 14:22:12 -08:00
Adam Spiers
565b6fa87b tests: refactor mechanics of testing in a sub test-lib
This will allow us to test the test framework more thoroughly
without disrupting the top-level test metrics.

Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-20 14:22:12 -08:00
Adam Spiers
5e5c006eb7 tests: test number comes first in 'not ok $count - $message'
The old output to say "not ok - 1 messsage" was working by accident
only because the test numbers are optional in TAP.

Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-16 12:01:58 -08:00