Commit Graph

16819 Commits

Author SHA1 Message Date
Elijah Newren
a1d8b01775 t6422: fix multiple errors with the mod6 test expectations
This test had multiple issues causing it to fail for the wrong
reason(s):

  * rename/rename(1to2) conflicts have always left the original source
    path present in the working directory and index (at stage 1).  Thus,
    the triple rename/rename(1to2) should result in 9 unstaged files,
    not 6.
  * It messed up the three-way content merge for checking the results of
    merging for one of the renames, accidentally turning it into a
    two-way merge.
  * It got the contents of the base files it was using to compare
    against wrong, due to an off-by-one error, and overwrite-redirection
    ('>') instead of append-redirection ('>>').
  * It used slightly too-long conflict markers
  * It didn't include filenames in the conflict marker hunks (granted,
    that was a shortcoming of the merge-recursive backend for rename/add
    and rename/rename(2to1) conflicts, but since it's
    test_expect_failure anyway we might as well make it expect our
    preferred behavior rather than some compromise that we can't yet
    reach anyway).

Fix these issues so that a merge backend which correctly handles these
kinds of nested conflicts will pass the test.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-10 15:59:01 -07:00
Elijah Newren
a0601b2eb3 t6423: fix test setup for a couple tests
Commit da1e295e00 ("t604[236]: do not run setup in separate tests",
2019-10-22) removed approximately half the tests (which were setup-only
tests) in t6043 by turning them into functions that the subsequent test
would call as their first step.  This ensured that any test from this
file could be run entirely independently of all the other tests in the
file.  Unfortunately, the call to the new setup function was missed in
two of the test_expect_failure cases.  Add them in.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-10 15:59:00 -07:00
Elijah Newren
3df4e3bb09 t6416, t6422: fix incorrect untracked file count
Apparently I don't know how to count untracked files, and since the
tests in question were marked as test_expect_failure, no one ever
noticed it until now.  Correct the count, as these tests clearly create
three untracked files ('out', 'err', and 'file_count').

(I believe this problem arose because earlier incarnations counted lines
via a pipe to 'wc -l'.  Reviewers asked that it be replaced by writing
the output to a file and using test_line_count, but when the temporary
output was added to a separate file, the count of untracked files should
have increased.)

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-10 15:59:00 -07:00
Elijah Newren
3b6eb15d2b t6422: fix bad check against missing file
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-10 15:59:00 -07:00
Elijah Newren
bc29dffe59 t6418: tighten delete/normalize conflict testcase
The testcase only required that the merge complete without conflict,
without specifying what the correct resolution was.  Since normalization
changed this from a modify/delete to a not-modified/delete, the correct
resolution is to have the file be removed at the end.  Add a check for
this resolution.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-10 15:59:00 -07:00
Elijah Newren
919df31955 Collect merge-related tests to t64xx
The tests for the merge machinery are spread over several places.
Collect them into t64xx for simplicity.  Some notes:

t60[234]*.sh:
  Merge tests started in t602*, overgrew bisect and remote tracking
  tests in t6030, t6040, and t6041, and nearly overtook replace tests
  in t6050.  This made picking out relevant tests that I wanted to run
  in a tighter loop slightly more annoying for years.

t303*.sh:
  These started out as tests for the 'merge-recursive' toplevel command,
  but did not restrict to that and had lots of overlap with the
  underlying merge machinery.
t7405, t7613:
  submodule-specific merge logic started out in submodule.c but was
  moved to merge-recursive.c in commit 18cfc08866 ("submodule.c: move
  submodule merging to merge-recursive.c", 2018-05-15).  Since these
  tests are about the logic found in the merge machinery, moving these
  tests to be with the merge tests makes sense.

t7607, t7609:
  Having tests spread all over the place makes it more likely that
  additional tests related to a certain piece of logic grow in all those
  other places.  Much like t303*.sh, these two tests were about the
  underlying merge machinery rather than outer levels.

Tests that were NOT moved:

t76[01]*.sh:
  Other than the four tests mentioned above, the remaining tests in
  t76[01]*.sh are related to non-recursive merge strategies, parameter
  parsing, and other stuff associated with the highlevel builtin/merge.c
  rather than the recursive merge machinery.

t3[45]*.sh:
  The rebase testcases in t34*.sh also test the merge logic pretty
  heavily; sometimes changes I make only trigger failures in the rebase
  tests.  The rebase tests are already nicely coupled together, though,
  and I didn't want to mess that up.  Similar comments apply for the
  cherry-pick tests in t35*.sh.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-10 15:59:00 -07:00
Junio C Hamano
4339259d5f Merge branch 'en/eol-attrs-gotchas'
All "mergy" operations that internally use the merge-recursive
machinery should honor the merge.renormalize configuration, but
many of them didn't.

* en/eol-attrs-gotchas:
  checkout: support renormalization with checkout -m <paths>
  merge: make merge.renormalize work for all uses of merge machinery
  t6038: remove problematic test
  t6038: make tests fail for the right reason
2020-08-10 10:24:02 -07:00
Junio C Hamano
5b53175b7a Merge branch 'ma/t1450-quotefix'
Test fix.

* ma/t1450-quotefix:
  t1450: fix quoting of NUL byte when corrupting pack
2020-08-10 10:23:59 -07:00
Junio C Hamano
46b225f153 Merge branch 'jk/strvec'
The argv_array API is useful for not just managing argv but any
"vector" (NULL-terminated array) of strings, and has seen adoption
to a certain degree.  It has been renamed to "strvec" to reduce the
barrier to adoption.

* jk/strvec:
  strvec: rename struct fields
  strvec: drop argv_array compatibility layer
  strvec: update documention to avoid argv_array
  strvec: fix indentation in renamed calls
  strvec: convert remaining callers away from argv_array name
  strvec: convert more callers away from argv_array name
  strvec: convert builtin/ callers away from argv_array name
  quote: rename sq_dequote_to_argv_array to mention strvec
  strvec: rename files from argv-array to strvec
  argv-array: rename to strvec
  argv-array: use size_t for count and alloc
2020-08-10 10:23:57 -07:00
Junio C Hamano
5b137e8441 Merge branch 'jt/pretend-object-never-come-from-elsewhere'
The pretend-object mechanism checks if the given object already
exists in the object store before deciding to keep the data
in-core, but the check would have triggered lazy fetching of such
an object from a promissor remote.

* jt/pretend-object-never-come-from-elsewhere:
  sha1-file: make pretend_object_file() not prefetch
2020-08-04 13:53:58 -07:00
Junio C Hamano
5c454b3825 Merge branch 'jt/pack-objects-prefetch-in-batch'
While packing many objects in a repository with a promissor remote,
lazily fetching missing objects from the promissor remote one by
one may be inefficient---the code now attempts to fetch all the
missing objects in batch (obviously this won't work for a lazy
clone that lazily fetches tree objects as you cannot even enumerate
what blobs are missing until you learn which trees are missing).

* jt/pack-objects-prefetch-in-batch:
  pack-objects: prefetch objects to be packed
  pack-objects: refactor to oid_object_info_extended
2020-08-04 13:53:57 -07:00
Elijah Newren
8d552258f4 merge: make merge.renormalize work for all uses of merge machinery
The 'merge' command is not the only one that does merges; other commands
like checkout -m or rebase do as well.  Unfortunately, the only area of
the code that checked for the "merge.renormalize" config setting was in
builtin/merge.c, meaning it could only affect merges performed by the
"merge" command.  Move the handling of this config setting to
merge_recursive_config() so that other commands can benefit from it as
well.  Fixes a few tests in t6038.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-03 11:48:15 -07:00
Elijah Newren
6f6e7cfb52 t6038: remove problematic test
t6038.11, 'cherry-pick patch from after text=auto' was a test of
undefined behavior.  To make matters worse, while there are a couple
possible correct answers, this test was coded to only check for an
obviously incorrect answer.  And the final cherry on top is that the
test is marked test_expect_failure, meaning it can't provide much value,
other than possibly confusing future folks who come along and try to
work on attributes and look at existing tests.  Because of all these
problems, just remove the test.

But for any future code spelunkers, here's my understanding of the two
possible correct answers:

This test was set up so that on a branch with no .gitattributes file,
you cherry-picked a patch from a branch that had a .gitattributes file
(containing '* text=auto').  Further, the two branches had a file which
differed only in line endings.  In this situation, correct behavior is
not well defined: should the .gitattributes file affect the merge or
not?

If the .gitattributes file on the other branch should not affect the
merge, then we would have a content conflict with all three stages
different (the merge base didn't match either side).

If the .gitattributes file from the other branch should affect the
merge, then we would expect the line endings to be normalized to LF for
the version to be recorded in the repository.  This would mean that when
doing a three-way content merge on the file that differed in line
endings, that the three-way content merge would see that the versions on
both sides matched and so the cherry-pick has no conflicts and can
succeed.  The line endings in the file as recorded in the repository
will change from CRLF to LF.  The version checked out in the working
copy will depend on the platform (since there's no eol attribute defined
for the file).

Also, as a final side note, this test expected an error message that was
built assuming cherry-pick was the old scripted version, because
cherry-pick no longer uses the error message that was encoded in this
test.  So it was wrong for yet another reason.

Given that the handling of .gitattributes is not well defined and this
test was obviously broken and could do nothing but confuse future
readers, just remove it.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-03 11:48:14 -07:00
Elijah Newren
fe48efb5fd t6038: make tests fail for the right reason
t6038 had a pair of tests that were expected to fail, but weren't
failing for the expected reason.  Both were meant to do a merge that
could be done cleanly after renormalization, but were supposed to fail
for lack of renormalization.  Unfortunately, both tests had staged
changes, and checkout -m would abort due to the presence of those staged
changes before even attempting a merge.

Fix this first issue by utilizing git-restore instead of git-checkout,
so that the index is left alone and just the working directory gets the
changes we want.

However, there is a second issue with these tests.  Technically, they
just wanted to verify that after renormalization, no conflicts would be
present.  This could have been checked for by grepping for a lack of
conflict markers, but the test instead tried to compare the working
directory files to an expected result.  Unfortunately, the setting of
"text=auto" without setting core.eol to any value meant that the content
of the file (in particular, the line endings) would be
platform-dependent and the tests could only pass on some platforms.
Replace the existing comparison with a call to 'git diff --no-index
--ignore-cr-at-eol' to verify that the contents, other than possible
carriage returns in the file, match the expected results and in
particular that the file has no conflicts from the checkout -m
operation.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-03 11:48:13 -07:00
Martin Ågren
dc156bc31f t1450: fix quoting of NUL byte when corrupting pack
We use

  printf '\0'

to generate a NUL byte which we then `dd` into the packfile to ensure
that we modify the first byte of the first object, thereby
(probabilistically) invalidating the checksum. Except the single quotes
we're using are interpreted to match with the ones we enclose the whole
test in. So we actually execute

  printf \0

and end up injecting the ASCII code for "0", 0x30, instead.

The comment right above this `printf` invocation says that "at least one
of [the type bits] is not zero, so setting the first byte to 0 is
sufficient". Substituting "0x30" for "0" in that comment won't do: we'd
need to reason about which bits go where and just what the packfile
looks like that we're modifying in this test.

Let's avoid all of that by actually executing

  printf "\0"

to generate a NUL byte, as intended.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-01 17:46:42 -07:00
Junio C Hamano
4083971673 Merge branch 'cc/pretty-contents-size' into master
Brown-paper-bag fix.

* cc/pretty-contents-size:
  t6300: fix issues related to %(contents:size)
2020-08-01 13:49:14 -07:00
Junio C Hamano
341a196ab6 Merge branch 'jc/fmt-merge-msg-suppress-destination' into master
"git merge" learned to selectively omit " into <branch>" at the end
of the title of default merge message with merge.suppressDest
configuration.

* jc/fmt-merge-msg-suppress-destination:
  fmt-merge-msg: allow merge destination to be omitted again
  Revert "fmt-merge-msg: stop treating `master` specially"
2020-08-01 13:49:13 -07:00
Alban Gruin
3db796c1c0 t6300: fix issues related to %(contents:size)
b6839fda68 (ref-filter: add support for %(contents:size), 2020-07-16)
added a new format for ref-filter, and added a function to generate
tests for this new feature in t6300.  Unfortunately, it tries to run
`test_expect_sucess' instead of `test_expect_success', and writes
$expect to `expected', but tries to read `expect'.  Those two issues
were probably unnoticed because the script only printed errors, but did
not crash.  This fixes these issues.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-31 13:26:19 -07:00
Junio C Hamano
5d4e13f6df Merge branch 'en/typofixes' into master
* en/typofixes:
  hashmap: fix typo in usage docs
  Remove doubled words in various comments
2020-07-30 21:34:30 -07:00
Jeff King
d70a9eb611 strvec: rename struct fields
The "argc" and "argv" names made sense when the struct was argv_array,
but now they're just confusing. Let's rename them to "nr" (which we use
for counts elsewhere) and "v" (which is rather terse, but reads well
when combined with typical variable names like "args.v").

Note that we have to update all of the callers immediately. Playing
tricks with the preprocessor is hard here, because we wouldn't want to
rewrite unrelated tokens.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30 19:18:06 -07:00
Junio C Hamano
82fafc77ba Merge branch 'en/fill-directory-exponential' into master
Fix to a regression introduced during 2.27 cycle.

* en/fill-directory-exponential:
  dir: check pathspecs before returning `path_excluded`
2020-07-30 13:20:36 -07:00
Junio C Hamano
be2dab9c80 Merge branch 'ct/mv-unmerged-path-error' into master
"git mv src dst", when src is an unmerged path, errored out
correctly but with an incorrect error message to claim that src is
not tracked, which has been clarified.

* ct/mv-unmerged-path-error:
  git-mv: improve error message for conflicted file
2020-07-30 13:20:35 -07:00
Junio C Hamano
c2796ac1c2 Merge branch 'bc/push-cas-cquoted-refname' into master
Pushing a ref whose name contains non-ASCII character with the
"--force-with-lease" option did not work over smart HTTP protocol,
which has been corrected.

* bc/push-cas-cquoted-refname:
  remote-curl: make --force-with-lease work with non-ASCII ref names
2020-07-30 13:20:34 -07:00
Junio C Hamano
be537062af Merge branch 'cc/pretty-contents-size' into master
"git for-each-ref --format=<>" learned %(contents:size).

* cc/pretty-contents-size:
  ref-filter: add support for %(contents:size)
  t6300: test refs pointing to tree and blob
  Documentation: clarify %(contents:XXXX) doc
2020-07-30 13:20:33 -07:00
Junio C Hamano
37f382a924 Merge branch 'jt/avoid-lazy-fetching-upon-have-check' into master
Fetching from a lazily cloned repository resulted at the server
side in attempts to lazy fetch objects that the client side has,
many of which will not be available from the third-party anyway.

* jt/avoid-lazy-fetching-upon-have-check:
  upload-pack: do not lazy-fetch "have" objects
2020-07-30 13:20:33 -07:00
Junio C Hamano
e163cff400 Merge branch 'dl/test-must-fail-fixes-6' into master
Dev support to limit the use of test_must_fail to only git commands.

* dl/test-must-fail-fixes-6:
  test-lib-functions: restrict test_must_fail usage
  t9400: don't use test_must_fail with cvs
  t9834: remove use of `test_might_fail p4`
  t7107: don't use test_must_fail()
  t5324: reorder `run_with_limited_open_files test_might_fail`
  t3701: stop using `env` in force_color()
2020-07-30 13:20:32 -07:00
Junio C Hamano
c28a2d0c12 Merge branch 'jk/reject-newer-extensions-in-v0' into master
With the base fix to 2.27 regresion, any new extensions in a v0
repository would still be silently honored, which is not quite
right.  Instead, complain and die loudly.

* jk/reject-newer-extensions-in-v0:
  verify_repository_format(): complain about new extensions in v0 repo
2020-07-30 13:20:32 -07:00
Junio C Hamano
3161cc6e6b Merge branch 'hn/reftable' into master
Preliminary clean-up of the refs API in preparation for adding a
new refs backend "reftable".

* hn/reftable:
  reflog: cleanse messages in the refs.c layer
  bisect: treat BISECT_HEAD as a pseudo ref
  t3432: use git-reflog to inspect the reflog for HEAD
  lib-t6000.sh: write tag using git-update-ref
2020-07-30 13:20:32 -07:00
Junio C Hamano
f175e9b845 Merge branch 'bw/fail-cloning-into-non-empty' into master
"git clone --separate-git-dir=$elsewhere" used to stomp on the
contents of the existing directory $elsewhere, which has been
taught to fail when $elsewhere is not an empty directory.

* bw/fail-cloning-into-non-empty:
  git clone: don't clone into non-empty directory
2020-07-30 13:20:32 -07:00
Junio C Hamano
6fc5542564 Merge branch 'jk/tests-timestamp-fix' into master
The test framework has been updated so that most tests will run
with predictable (artificial) timestamps.

* jk/tests-timestamp-fix:
  t9100: stop depending on commit timestamps
  test-lib: set deterministic default author/committer date
  t9100: explicitly unset GIT_COMMITTER_DATE
  t5539: make timestamp requirements more explicit
  t9700: loosen ident timezone regex
  t6000: use test_tick consistently
2020-07-30 13:20:31 -07:00
Junio C Hamano
70cdbbe3a7 Merge branch 'ds/commit-graph-bloom-updates' into master
Updates to the changed-paths bloom filter.

* ds/commit-graph-bloom-updates:
  commit-graph: check all leading directories in changed path Bloom filters
  revision: empty pathspecs should not use Bloom filters
  revision.c: fix whitespace
  commit-graph: check chunk sizes after writing
  commit-graph: simplify chunk writes into loop
  commit-graph: unify the signatures of all write_graph_chunk_*() functions
  commit-graph: persist existence of changed-paths
  bloom: fix logic in get_bloom_filter()
  commit-graph: change test to die on parse, not load
  commit-graph: place bloom_settings in context
2020-07-30 13:20:31 -07:00
Junio C Hamano
de6dda0dc3 Merge branch 'sg/commit-graph-cleanups' into master
The changed-path Bloom filter is improved using ideas from an
independent implementation.

* sg/commit-graph-cleanups:
  commit-graph: simplify write_commit_graph_file() #2
  commit-graph: simplify write_commit_graph_file() #1
  commit-graph: simplify parse_commit_graph() #2
  commit-graph: simplify parse_commit_graph() #1
  commit-graph: clean up #includes
  diff.h: drop diff_tree_oid() & friends' return value
  commit-slab: add a function to deep free entries on the slab
  commit-graph-format.txt: all multi-byte numbers are in network byte order
  commit-graph: fix parsing the Chunk Lookup table
  tree-walk.c: don't match submodule entries for 'submod/anything'
2020-07-30 13:20:30 -07:00
Junio C Hamano
6e6029a82a fmt-merge-msg: allow merge destination to be omitted again
In Git 2.28, we stopped special casing 'master' when producing the
default merge message by just removing the code to squelch "into
'master'" at the end of the message.

Introduce multi-valued merge.suppressDest configuration variable
that gives a set of globs to match against the name of the branch
into which the merge is being made, to let users specify for which
branch fmt-merge-msg's output should be shortened.  When it is not
set, 'master' is used as the sole value of the variable by default.

The above move mostly reverts the pre-2.28 default in repositories
that have no relevant configuration.

Add a few tests to protect the behaviour with the new configuration
variable from future regression.

Helped-by: Linus Torvalds <torvalds@linux-foundation.org>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30 12:43:10 -07:00
Junio C Hamano
21531927e4 Revert "fmt-merge-msg: stop treating master specially"
This reverts commit 489947cee5, which
stopped treating merges into the 'master' branch as special when
preparing the default merge message.  As the goal was not to have
any single branch designated as special, it solved it by leaving the
"into <branchname>" at the end of the title of the default merge
message for any and all branches.  An obvious and easy alternative
to treat everybody equally could have been to remove it for every
branch, but that involves loss of information.

We'll introduce a new mechanism to let end-users specify merges into
which branches would omit the "into <branchname>" from the title of
the default merge message, and make the mechanism, when unconfigured,
treat the traditional 'master' special again, so all the changes to
the tests we made earlier will become unnecessary, as these tests
will be run without configuring the said new mechanism.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30 12:41:49 -07:00
Jeff King
f6d8942b1f strvec: fix indentation in renamed calls
Code which split an argv_array call across multiple lines, like:

  argv_array_pushl(&args, "one argument",
                   "another argument", "and more",
		   NULL);

was recently mechanically renamed to use strvec, which results in
mis-matched indentation like:

  strvec_pushl(&args, "one argument",
                   "another argument", "and more",
		   NULL);

Let's fix these up to align the arguments with the opening paren. I did
this manually by sifting through the results of:

  git jump grep 'strvec_.*,$'

and liberally applying my editor's auto-format. Most of the changes are
of the form shown above, though I also normalized a few that had
originally used a single-tab indentation (rather than our usual style of
aligning with the open paren). I also rewrapped a couple of obvious
cases (e.g., where previously too-long lines became short enough to fit
on one), but I wasn't aggressive about it. In cases broken to three or
more lines, the grouping of arguments is sometimes meaningful, and it
wasn't worth my time or reviewer time to ponder each case individually.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28 15:02:18 -07:00
Jeff King
c972bf4cf5 strvec: convert remaining callers away from argv_array name
We eventually want to drop the argv_array name and just use strvec
consistently. There's no particular reason we have to do it all at once,
or care about interactions between converted and unconverted bits.
Because of our preprocessor compat layer, the names are interchangeable
to the compiler (so even a definition and declaration using different
names is OK).

This patch converts all of the remaining files, as the resulting diff is
reasonably sized.

The conversion was done purely mechanically with:

  git ls-files '*.c' '*.h' |
  xargs perl -i -pe '
    s/ARGV_ARRAY/STRVEC/g;
    s/argv_array/strvec/g;
  '

We'll deal with any indentation/style fallouts separately.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28 15:02:18 -07:00
Jeff King
dbbcd44fb4 strvec: rename files from argv-array to strvec
This requires updating #include lines across the code-base, but that's
all fairly mechanical, and was done with:

  git ls-files '*.c' '*.h' |
  xargs perl -i -pe 's/argv-array.h/strvec.h/'

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28 15:02:17 -07:00
Elijah Newren
6d12b533b7 Remove doubled words in various comments
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28 14:28:14 -07:00
Jonathan Tan
a64d2aae5a sha1-file: make pretend_object_file() not prefetch
When pretend_object_file() is invoked with an object that does not exist
(as is the typical case), there is no need to fetch anything from the
promisor remote, because the caller already knows what the object is
supposed to contain. Therefore, suppress the fetch. (The
OBJECT_INFO_QUICK flag is added for the same reason.)

This was noticed at $DAYJOB when "blame" was run on a file that had
uncommitted modifications.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-21 16:27:22 -07:00
Jonathan Tan
e00549aa9b pack-objects: prefetch objects to be packed
When an object to be packed is noticed to be missing, prefetch all
to-be-packed objects in one batch.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-21 14:29:42 -07:00
brian m. carlson
cd85b447bf remote-curl: make --force-with-lease work with non-ASCII ref names
When we invoke a remote transport helper and pass an option with an
argument, we quote the argument as a C-style string if necessary.  This
is the case for the cas option, which implements the --force-with-lease
command-line flag, when we're passing a non-ASCII refname.

However, the remote curl helper isn't designed to parse such an
argument, meaning that if we try to use --force-with-lease with an HTTP
push and a non-ASCII refname, we get an error like this:

  error: cannot parse expected object name '0000000000000000000000000000000000000000"'

Note the double quote, which get_oid has reminded us is not valid in an
hex object ID.

Even if we had been able to parse it, we would send the wrong data to
the server: we'd send an escaped ref, which would not behave as the user
wanted and might accidentally result in updating or deleting a ref we
hadn't intended.

Since we need to expect a quoted C-style string here, just check if the
first argument is a double quote, and if so, unquote it.  Note that if
the refname contains a double quote, then we will have double-quoted it
already, so there is no ambiguity.

We test for this case only in the smart protocol, since the DAV-based
protocol is not capable of handling this capability.  We use UTF-8
because this is nicer in our tests and friendlier to Windows, but the
code should work for all non-ASCII refs.

While we're at it, since the name of the option is now well established
and isn't going to change, let's inline it instead of using the #define
constant.

Reported-by: Frej Bjon <frej.bjon@nemit.fi>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-20 21:05:16 -07:00
Chris Torek
9b906af657 git-mv: improve error message for conflicted file
'git mv' has always complained about renaming a conflicted
file, as it cannot handle multiple index entries for one file.
However, the error message it uses has been the same as the
one for an untracked file:

    fatal: not under version control, src=...

which is patently wrong.  Distinguish the two cases and
add a test to make sure we produce the correct message.

Signed-off-by: Chris Torek <chris.torek@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-20 14:35:43 -07:00
Martin Ågren
cada7308ad dir: check pathspecs before returning path_excluded
In 95c11ecc73 ("Fix error-prone fill_directory() API; make it only
return matches", 2020-04-01), we taught `fill_directory()`, or more
specifically `treat_path()`, to check against any pathspecs so that we
could simplify the callers.

But in doing so, we added a slightly-too-early return for the "excluded"
case. We end up not checking the pathspecs, meaning we return
`path_excluded` when maybe we should return `path_none`. As a result,
`git status --ignored -- pathspec` might show paths that don't actually
match "pathspec".

Move the "excluded" check down to after we've checked any pathspecs.

Reported-by: Andreas Schwab <schwab@linux-m68k.org>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-20 13:25:07 -07:00
Junio C Hamano
ae46588be0 Merge branch 'dl/branch-cleanup' into master
Last minute fix-up to tests for portability.

* dl/branch-cleanup:
  t3200: don't grep for `strerror()` string
2020-07-18 16:35:22 -07:00
Martin Ågren
d223e85407 t3200: don't grep for strerror() string
In 6b7093064a ("t3200: test for specific errors", 2020-06-15), we
learned to grep stderr to ensure that the failing `git branch`
invocations fail for the right reason. In two of these tests, we grep
for "File exists", expecting the string to show up there since config.c
calls `error_errno()`, which ends up including `strerror(errno)` in the
error message.

But as we saw in 4605a73073 ("t1091: don't grep for `strerror()`
string", 2020-03-08), there exists at least one implementation where
`strerror()` yields a slightly different string than the one we're
grepping for. In particular, these tests fail on the NonStop platform.

Similar to 4605a73073, grep for the beginning of the string instead to
avoid relying on `strerror()` behavior.

Reported-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-18 13:47:05 -07:00
Junio C Hamano
d13b7f2198 Merge branch 'jn/v0-with-extensions-fix' into master
In 2.28-rc0, we corrected a bug that some repository extensions are
honored by mistake even in a version 0 repositories (these
configuration variables in extensions.* namespace were supposed to
have special meaning in repositories whose version numbers are 1 or
higher), but this was a bit too big a change.

* jn/v0-with-extensions-fix:
  repository: allow repository format upgrade with extensions
  Revert "check_repository_format_gently(): refuse extensions for old repositories"
2020-07-16 17:58:42 -07:00
Jonathan Tan
77aa0941ce upload-pack: do not lazy-fetch "have" objects
When upload-pack receives a request containing "have" hashes, it (among
other things) checks if the served repository has the corresponding
objects. However, it does not do so with the
OBJECT_INFO_SKIP_FETCH_OBJECT flag, so if serving a partial clone, a
lazy fetch will be triggered first.

This was discovered at $DAYJOB when a user fetched from a partial clone
(into another partial clone - although this would also happen if the
repo to be fetched into is not a partial clone).

Therefore, whenever "have" hashes are checked for existence, pass the
OBJECT_INFO_SKIP_FETCH_OBJECT flag. Also add the OBJECT_INFO_QUICK flag
to improve performance, as it is typical that such objects do not exist
in the serving repo, and the consequences of a false negative are minor
(usually, a slightly larger pack sent).

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-16 14:07:19 -07:00
Christian Couder
b6839fda68 ref-filter: add support for %(contents:size)
It's useful and efficient to be able to get the size of the
contents directly without having to pipe through `wc -c`.

Also the result of the following:

`git for-each-ref --format='%(contents)' refs/heads/my-branch | wc -c`

is off by one as `git for-each-ref` appends a newline character
after the contents, which can be seen by comparing its output
with the output from `git cat-file`.

As with %(contents), %(contents:size) is silently ignored, if a
ref points to something other than a commit or a tag:

```
$ git update-ref refs/mytrees/first HEAD^{tree}
$ git for-each-ref --format='%(contents)' refs/mytrees/first

$ git for-each-ref --format='%(contents:size)' refs/mytrees/first

```

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-16 10:46:55 -07:00
Jeff King
ec91ffca04 verify_repository_format(): complain about new extensions in v0 repo
We made the mistake in the past of respecting extensions.* even when the
repository format version was set to 0. This is bad because forgetting
to bump the repository version means that older versions of Git (which
do not know about our extensions) won't complain. I.e., it's not a
problem in itself, but it means your repository is in a state which does
not give you the protection you think you're getting from older
versions.

For compatibility reasons, we are stuck with that decision for existing
extensions. However, we'd prefer not to extend the damage further. We
can do that by catching any newly-added extensions and complaining about
the repository format.

Note that this is a pretty heavy hammer: we'll refuse to work with the
repository at all. A lesser option would be to ignore (possibly with a
warning) any new extensions. But because of the way the extensions are
handled, that puts the burden on each new extension that is added to
remember to "undo" itself (because they are handled before we know
for sure whether we are in a v1 repo or not, since we don't insist on a
particular ordering of config entries).

So one option would be to rewrite that handling to record any new
extensions (and their values) during the config parse, and then only
after proceed to handle new ones only if we're in a v1 repository. But
I'm not sure if it's worth the trouble:

  - ignoring extensions is likely to end up with broken results anyway
    (e.g., ignoring a proposed objectformat extension means parsing any
    object data is likely to encounter errors)

  - this is a sign that whatever tool wrote the extension field is
    broken. We may be better off notifying immediately and forcefully so
    that such tools don't even appear to work accidentally.

The only downside is that fixing the situation is a little tricky,
because programs like "git config" won't want to work with the
repository. But:

  git config --file=.git/config core.repositoryformatversion 1

should still suffice.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-16 10:39:45 -07:00
Jonathan Nieder
62f2eca606 repository: allow repository format upgrade with extensions
Now that we officially permit repository extensions in repository
format v0, permit upgrading a repository with extensions from v0 to v1
as well.

For example, this means a repository where the user has set
"extensions.preciousObjects" can use "git fetch --filter=blob:none
origin" to upgrade the repository to use v1 and the partial clone
extension.

To avoid mistakes, continue to forbid repository format upgrades in v0
repositories with an unrecognized extension.  This way, a v0 user
using a misspelled extension field gets a chance to correct the
mistake before updating to the less forgiving v1 format.

While we're here, make the error message for failure to upgrade the
repository format a bit shorter, and present it as an error, not a
warning.

Reported-by: Huan Huan Chen <huanhuanchen@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-16 09:36:39 -07:00