The test framework has been updated so that most tests will run
with predictable (artificial) timestamps.
* jk/tests-timestamp-fix:
t9100: stop depending on commit timestamps
test-lib: set deterministic default author/committer date
t9100: explicitly unset GIT_COMMITTER_DATE
t5539: make timestamp requirements more explicit
t9700: loosen ident timezone regex
t6000: use test_tick consistently
Updates to the changed-paths bloom filter.
* ds/commit-graph-bloom-updates:
commit-graph: check all leading directories in changed path Bloom filters
revision: empty pathspecs should not use Bloom filters
revision.c: fix whitespace
commit-graph: check chunk sizes after writing
commit-graph: simplify chunk writes into loop
commit-graph: unify the signatures of all write_graph_chunk_*() functions
commit-graph: persist existence of changed-paths
bloom: fix logic in get_bloom_filter()
commit-graph: change test to die on parse, not load
commit-graph: place bloom_settings in context
The changed-path Bloom filter is improved using ideas from an
independent implementation.
* sg/commit-graph-cleanups:
commit-graph: simplify write_commit_graph_file() #2
commit-graph: simplify write_commit_graph_file() #1
commit-graph: simplify parse_commit_graph() #2
commit-graph: simplify parse_commit_graph() #1
commit-graph: clean up #includes
diff.h: drop diff_tree_oid() & friends' return value
commit-slab: add a function to deep free entries on the slab
commit-graph-format.txt: all multi-byte numbers are in network byte order
commit-graph: fix parsing the Chunk Lookup table
tree-walk.c: don't match submodule entries for 'submod/anything'
The "reference transaction" hook was introduced in commit 6754159767
(refs: implement reference transaction hook, 2020-06-19). The name of
the hook is declared as "reference-transaction" in "refs.c" and
testcases, but the name declared in "githooks.txt" is different.
Signed-off-by: Bojun Chen <bojun.cbj@alibaba-inc.com>
Reviewed-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When using git-prompt.sh with zsh, __git_ps1 currently errs
when inside a repo with:
__git_ps1:96: = not found
Avoid using non-portable "==" that is only understood by bash
and not zsh. Change to "=" so that the prompt script becomes
usable with zsh again.
Signed-off-by: David J. Malan <malan@harvard.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While we're at it, fix some minor misspelling
and improve translation for 3-way-merging.
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Tran Ngoc Quan <vnwildman@gmail.com>
In 6b7093064a ("t3200: test for specific errors", 2020-06-15), we
learned to grep stderr to ensure that the failing `git branch`
invocations fail for the right reason. In two of these tests, we grep
for "File exists", expecting the string to show up there since config.c
calls `error_errno()`, which ends up including `strerror(errno)` in the
error message.
But as we saw in 4605a73073 ("t1091: don't grep for `strerror()`
string", 2020-03-08), there exists at least one implementation where
`strerror()` yields a slightly different string than the one we're
grepping for. In particular, these tests fail on the NonStop platform.
Similar to 4605a73073, grep for the beginning of the string instead to
avoid relying on `strerror()` behavior.
Reported-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
AsciiDoctor renders the "~~~~~~~~~" literally. That's not our intention:
it is supposed to indicate a level 2 subsection. In 828197de8f ("docs:
adjust for the recent rename of `pu` to `seen`", 2020-06-25), the length
of this section header grew by two characters but we didn't adjust the
number of ~ characters accordingly. AsciiDoc handles this discrepancy ok
and still picks this up as a subsection title, but Asciidoctor is not as
forgiving.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the two-patch series for regression fix, to the users from 2.27
days, there is no visible behaviour change---we do not warn and fail
use of v0 repositories with newer extensions yet, so there is nothing
to note in the backward compatibility section.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 2.28-rc0, we corrected a bug that some repository extensions are
honored by mistake even in a version 0 repositories (these
configuration variables in extensions.* namespace were supposed to
have special meaning in repositories whose version numbers are 1 or
higher), but this was a bit too big a change.
* jn/v0-with-extensions-fix:
repository: allow repository format upgrade with extensions
Revert "check_repository_format_gently(): refuse extensions for old repositories"
Now that we officially permit repository extensions in repository
format v0, permit upgrading a repository with extensions from v0 to v1
as well.
For example, this means a repository where the user has set
"extensions.preciousObjects" can use "git fetch --filter=blob:none
origin" to upgrade the repository to use v1 and the partial clone
extension.
To avoid mistakes, continue to forbid repository format upgrades in v0
repositories with an unrecognized extension. This way, a v0 user
using a misspelled extension field gets a chance to correct the
mistake before updating to the less forgiving v1 format.
While we're here, make the error message for failure to upgrade the
repository format a bit shorter, and present it as an error, not a
warning.
Reported-by: Huan Huan Chen <huanhuanchen@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 14c7fa269e.
The core.repositoryFormatVersion field was introduced in ab9cb76f66
(Repository format version check., 2005-11-25), providing a welcome
bit of forward compatibility, thanks to some welcome analysis by
Martin Atukunda. The semantics are simple: a repository with
core.repositoryFormatVersion set to 0 should be comprehensible by all
Git implementations in active use; and Git implementations should
error out early instead of trying to act on Git repositories with
higher core.repositoryFormatVersion values representing new formats
that they do not understand.
A new repository format did not need to be defined until 00a09d57eb
(introduce "extensions" form of core.repositoryformatversion,
2015-06-23). This provided a finer-grained extension mechanism for
Git repositories. In a repository with core.repositoryFormatVersion
set to 1, Git implementations can act on "extensions.*" settings that
modify how a repository is interpreted. In repository format version
1, unrecognized extensions settings cause Git to error out.
What happens if a user sets an extension setting but forgets to
increase the repository format version to 1? The extension settings
were still recognized in that case; worse, unrecognized extensions
settings do *not* cause Git to error out. So combining repository
format version 0 with extensions settings produces in some sense the
worst of both worlds.
To improve that situation, since 14c7fa269e
(check_repository_format_gently(): refuse extensions for old
repositories, 2020-06-05) Git instead ignores extensions in v0 mode.
This way, v0 repositories get the historical (pre-2015) behavior and
maintain compatibility with Git implementations that do not know about
the v1 format. Unfortunately, users had been using this sort of
configuration and this behavior change came to many as a surprise:
- users of "git config --worktree" that had followed its advice
to enable extensions.worktreeConfig (without also increasing the
repository format version) would find their worktree configuration
no longer taking effect
- tools such as copybara[*] that had set extensions.partialClone in
existing repositories (without also increasing the repository format
version) would find that setting no longer taking effect
The behavior introduced in 14c7fa269e might be a good behavior if we
were traveling back in time to 2015, but we're far too late. For some
reason I thought that it was what had been originally implemented and
that it had regressed. Apologies for not doing my research when
14c7fa269e was under development.
Let's return to the behavior we've had since 2015: always act on
extensions.* settings, regardless of repository format version. While
we're here, include some tests to describe the effect on the "upgrade
repository version" code path.
[*] ca76c0b1e1
Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix to the code to produce progress bar, which is new in the
upcoming release.
* tb/commit-graph-no-check-oids:
commit-graph: fix "Collecting commits from input" progress line
The code to produce progress output from "git commit-graph --write"
had a few breakages, which have been fixed.
* sg/commit-graph-progress-fix:
commit-graph: fix "Writing out commit graph" progress counter
commit-graph: fix progress of reachable commits
When an aliased command, whose output is piped to a pager by git,
gets killed by a signal, the pager got into a funny state, which
has been corrected (again).
* ta/wait-on-aliased-commands-upon-signal:
Wait for child on signal death for aliases to externals
Wait for child on signal death for aliases to builtins
To display a progress line while reading commits from standard input
and looking them up, 5b6653e523 (builtin/commit-graph.c: dereference
tags in builtin, 2020-05-13) should have added a pair of
start_delayed_progress() and stop_progress() calls around the loop
reading stdin. Alas, the stop_progress() call ended up at the wrong
place, after write_commit_graph(), which does all the commit-graph
computation and writing, and has several progress lines of its own.
Consequently, that new
Collecting commits from input: 1234
progress line is overwritten by the first progress line shown by
write_commit_graph(), and its final "done" line is shown last, after
everything is finished:
$ { sleep 3 ; git rev-list -3 HEAD ; sleep 1 ; } | ~/src/git/git commit-graph write --stdin-commits
Expanding reachable commits in commit graph: 873402, done.
Writing out commit graph in 4 passes: 100% (3493608/3493608), done.
Collecting commits from input: 3, done.
Furthermore, that stop_progress() call was added after the 'cleanup'
label, where that loop reading stdin jumps in case of an error. In
case of invalid input this then results in the "done" line shown after
the error message:
$ { sleep 3 ; git rev-list -3 HEAD ; echo junk ; } | ~/src/git/git commit-graph write --stdin-commits
error: unexpected non-hex object ID: junk
Collecting commits from input: 3, done.
Move that stop_progress() call to the right place.
While at it, drop the unnecessary 'if (progress)' condition protecting
the stop_progress() call, because that function is prepared to handle
a NULL progress struct.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
An earlier "fix" to this script gave up updating it not to rely on
the current time because we cannot control what timestamp subversion
gives its commits. We however could solve the issue in a different
way and still use deterministic timestamps on Git commits.
One fix would be to sort the list of trees before removing duplicates,
but that loses information:
- we do care that the fetched history is in the same order
- there's a tree which appears twice in the history, and we'd want to
make sure that it's there both times
So instead, let's de-duplicate using a hash (preserving the order), and
drop only lines with identical trees and subjects (preserving the tree
which appears twice, since it has different subjects each time).
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We always set the name and email for committer and author idents to make
the test suite more deterministic, but not timestamps. Many scripts use
test_tick to get consistent and sensibly incrementing timestamps as they
create commits. But other scripts don't particularly care about the
timestamp, and are happy to use whatever the current system time is.
This non-determinism can be annoying:
- when debugging a test, comparing results between two runs can be
difficult, because the commit ids change
- this can sometimes cause tests to be racy. E.g., traversal order
depends on timestamp order. Even in a well-ordered set of commands,
because our timestamp granularity is one second, two commits might
sometimes have the same timestamp and sometimes differ.
Let's set a default timestamp for all scripts to use. Any that use
test_tick already will be unaffected (because their first test_tick call
will overwrite our default), but it will make things a bit more
deterministic for those that don't.
We should be able to choose any time we want here. I picked this one
because:
- it differs from the initial test_tick default, which may make it
easier to distinguish when debugging tests. I picked "April 1st
13:14:15" in the hope that it might stand out.
- it's slightly before the test_tick default. Some tests create some
commits before the first call to test_tick, so using an older
timestamps for those makes sense chronologically. Note that this
isn't how things currently work (where system times are usually more
recent than test_tick), but that also allows us to flush out a few
hidden timestamp dependencies (like the one recently fixed in
t5539).
- we could likewise pick any timezone we want. Choosing +0000 would
have required fixing up fewer tests, but we're more likely to turn
up interesting cases by not matching $TZ exactly. And since
test_tick already checks "-0700", let's try something in the "+"
zone range for variety.
It's possible that the non-deterministic times could help flush out bugs
(e.g., if something broke when the clock flipped over to 2021, our test
suite would let us know). But historically that hasn't been the case;
all time-dependent outcomes we've seen turned out to be accidentally
flaky tests (which we fixed by using test_tick). If we do want to cover
handling the current time, we should dedicate one script to doing so,
and have it unset GIT_COMMITTER_DATE explicitly.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The early part of t9100 creates an unusual "doubled" history in the
"git-svn" ref. When we get to t9100.17, it looks like this:
$ git log --oneline --graph git-svn
[...]
* efd0303 detect node change from file to directory #2
|\
* | 3e727c0 detect node change from file to directory #2
|/
* 3b00468 try a deep --rmdir with a commit
|\
* | b4832d8 try a deep --rmdir with a commit
|/
* f0d7bd5 import for git svn
Each commit we make with "git commit" is paired with one from "git svn
set-tree", with the latter as a merge of the first and its grandparent.
Later, t9100.17 wants to check that "git svn fetch" gets the same trees.
And it does, but just one copy of each. So it uses rev-list to get the
tree of each commit and pipes it to "uniq" to drop the duplicates. Our
input isn't sorted, but it will find adjacent duplicates. This works
reliably because the order of commits from rev-list always shows the
duplicates next to each other. For any one of those merges, we could
choose to show its duplicate or the grandparent first. But barring
clocks running backwards, the duplicate will always have a time equal to
or greater than the grandparent. Even if equal, we break ties by showing
the first-parent first, so the duplicates remain adjacent.
But this would break if the timestamps stopped moving in chronological
order. Normally we would rely on test_tick for this, but we have _two_
sources of time here:
- "git commit" creates one commit based on GIT_COMMITTER_DATE (which
respects test_tick)
- the "svn set-tree" one is based on subversion, which does not have
an easy way to specify a timestamp
So using test_tick actually breaks the test, because now the duplicates
are far in the past, and we'll show the grandparent before the
duplicate. And likewise, a proposed change to set GIT_COMMITTER_DATE in
all scripts will break it.
We _could_ fix this by sorting before removing duplicates, but
presumably it's a useful part of the test to make sure the trees appear
in the same order in both spots. Likewise, we could use something like:
perl -ne 'print unless $seen{$_}++'
to remove duplicates without impacting the order. But that doesn't work
either, because there are actually multiple (non-duplicate) commits with
the same trees (we change a file mode and then change it back). So we'd
actually have to de-duplicate the combination of subject and tree. Which
then further throws off t9100.18, which compares the tree hashes
exactly; we'd have to strip the result back down.
Since this test _isn't_ buggy, the simplest thing is to just work around
the proposed change by documenting our expectation that git-created
commits are correctly interleaved using the current time.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The description of `git diff` goes through several different invocations
(numbering added by me):
1. git diff [<options>] [--] [<path>...]
2. git diff [<options>] --no-index [--] <path> <path>
3. git diff [<options>] --cached [<commit>] [--] [<path>...]
4. git diff [<options>] <commit> [--] [<path>...]
5. git diff [<options>] <commit> <commit> [--] [<path>...]
6. git diff [<options>] <commit>..<commit> [--] [<path>...]
7. git diff [<options>] <commit> <commit>... <commit> [--] [<path>...]
8. git diff [<options>] <commit>...<commit> [--] [<path>...]
It then goes on to say that "all of the <commit> in the above
description, except in the last two forms that use '..' notations, can
be any <tree>". The "last two" actually refers to 6 and 8. This got out
of sync in commit b7e10b2ca2 ("Documentation: usage for diff combined
commits", 2020-06-12) which added item 7 to the mix.
As a further complication, after b7e10b2ca2 we also have some potential
confusion around "the '..' notation". The "..[.]" in items 6 and 8 are
part of the rev notation, whereas the "..." in item 7 is manpage
language for "one or more".
Move item 6 down, i.e., to between 7 and 8, to restore the ordering.
Because 6 refers to 5 ("synonymous to the previous form") we need to
tweak the language a bit.
An added bonus of this commit is that we're trying to steer users away
from `git diff <commit>..<commit>` and moving it further down probably
doesn't hurt.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit b7e10b2ca2 ("Documentation: usage for diff combined commits",
2020-06-12) modified the synopsis by adding an optional "[<commit>...]"
to
'git diff' [<options>] <commit> <commit> [--] [<path>...]
to effectively add
'git diff' [<options>] <commit> <commit>... <commit> [--] [<path>...]
as another valid invocation. Which makes sense.
Further down, in the description, it left the existing entry for
'git diff' [<options>] <commit> <commit> [--] [<path>...]
intact and added a new entry on
'git diff' [<options>] <commit> [<commit>...] <commit> [--] [<path>...]
where it says that "[t]his form is to view the results of a merge
commit" and details how "the first listed commit must be the merge
itself". But one possible instantiation of this form is `git diff
<commit> <commit>` for which the added text doesn't really apply.
Remove the brackets so that we lose this overlap between the two
descriptions. We can still use the more compact representation in the
synopsis.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test for "no shallow lines after receiving ACK ready" is very
sensitive to the timestamps of the commits we create. It's looking for
the fetch negotiation to send a "ready", which in turn depends on the
order in which we traverse commits during the negotiation.
It works reliably now because the base commit "7" is created without
test_commit, and thus gets a commit time matching the current system
clock. Whereas the new commits created in this test do use test_commit,
and get the usual test_tick time from 2005. So the fetch into the
"clone" repository results in a commit graph like this (I omitted some
of the "unrelated" commits for clarity; they're all just a sequence of
test_ticks):
$ git log --graph --format='%ct %s %d'
* 1112912953 new (origin/master, origin/HEAD)
* 1594322236 7 (grafted, master)
* 1112912893 unrelated15 (origin/unrelated15, unrelated15)
[...]
* 1112912053 unrelated1 (origin/unrelated1, unrelated1)
* 1112911993 new-too (HEAD -> newnew, tag: new-too)
The important things to see are:
- "7" is way in the future compared to the other commits
- "new-too" in the fetching repo is older than "new" (and its
"unrelated" ancestors) in the shallow repo
If we change our "setup shallow clone" step to use test_tick, too (and
get rid of the dependency on the system clock), then the test will fail.
The resulting graph looks like this:
$ git log --graph --format='%ct %s %d'
* 1112913373 new (origin/master, origin/HEAD)
* 1112912353 7 (grafted, master)
* 1112913313 unrelated15 (origin/unrelated15, unrelated15)
[...]
* 1112912473 unrelated1 (origin/unrelated1, unrelated1)
* 1112912413 new-too (HEAD -> newnew, tag: new-too)
Our "new-too" is still older than "new" and "unrelated", but now "7" is
older than all of them (because it advanced test_tick, which the other
tests built on top of). In the original, we advertised "7" as the first
"have" before anything else, but now "new-too" is more recent. You'd see
the same thing in the unlikely event that the system clock was set
before our test_tick default in 2005.
Let's make the timing requirements more explicit. The important thing is
that the client advertise all of its shared commits first, before
presenting its unique "new-too" commit. We can do that and get rid of
the system clock dependency at the same time by creating all of the
shared commits around time X (using test_tick), and then creating
"new-too" with some time long before X. The resulting graph looks like
this:
$ git log --graph --format='%ct %s %d'
* 1500001380 new (origin/master, origin/HEAD)
* 1500000420 7 (grafted, master)
* 1500001320 unrelated15 (origin/unrelated15, unrelated15)
[...]
* 1500000480 unrelated1 (origin/unrelated1, unrelated1)
* 1400000060 new-too (HEAD -> newnew, tag: new-too)
That also lets us get rid of the hacky test_tick added by f0e802ca20
(t5539: update a flaky test, 2014-07-14). That was clearly dancing
around the same problem, but only addressed the relationship between
commits created in the two subshells (which did use test_tick, but
overlapped because increments of test_tick in subshells are lost). Now
that we're using consistent and well-placed times for both lines of
history, we don't have to care about a one-tick difference between the
two sides.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A few of the perl tests in t9700 ask for the author and committer ident,
and then make sure we get something sensible. For the timestamp portion,
we just match [0-9]+, because the actual value will depend on when the
test is run. However, we do require that the timezone be "+0000". This
works reliably because we set $TZ in test-lib.sh. But in preparation for
changing the default timezone, let's be a bit more flexible. We don't
actually care about the exact value here, just that we were able to get
a sensible output from the perl module's access methods.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
