Commit Graph

14073 Commits

Author SHA1 Message Date
Jonathan Tan
4c7f9567ea fetch-pack: exclude blobs when lazy-fetching trees
A partial clone with missing trees can be obtained using "git clone
--filter=tree:none <repo>". In such a repository, when a tree needs to
be lazily fetched, any tree or blob it directly or indirectly references
is fetched as well, regardless of whether the original command required
those objects, or if the local repository already had some of them.

This is because the fetch protocol, which the lazy fetch uses, does not
allow clients to request that only the wanted objects be sent, which
would be the ideal solution. This patch implements a partial solution:
specify the "blob:none" filter, somewhat reducing the fetch payload.

This change has no effect when lazily fetching blobs (due to how filters
work). And if lazily fetching a commit (such repositories are difficult
to construct and is not a use case we support very well, but it is
possible), referenced commits and trees are still fetched - only the
blobs are not fetched.

The necessary code change is done in fetch_pack() instead of somewhere
closer to where the "filter" instruction is written to the wire so that
only one part of the code needs to be changed in order for users of all
protocol versions to benefit from this optimization.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-04 06:03:49 -07:00
Alexander Pyhalov
b1492bf315 t7005-editor: quote filename to fix whitespace-issue
Commit 4362da078e (t7005-editor: get rid of the SPACES_IN_FILENAMES
prereq, 2018-05-14) removed code for detecting whether spaces in
filenames work. Since we rely on spaces throughout the test suite
("trash directory.t1234-foo"), testing whether we can use the filename
"e space.sh" was redundant and unnecessary.

In simplifying the code, though, this introduced a regression around how
spaces are handled, not in the /name/ of the editor script, but /in/ the
script itself. The script just does `echo space >$1`, where $1 is for
example "/foo/t/trash directory.t7005-editor/.git/COMMIT_EDITMSG".

With most shells, or with Bash in posix mode, $1 will not be subjected
to field splitting. But if we invoke Bash directly, which will happen if
we build Git with SHELL_PATH=/bin/bash, it will detect and complain
about an "ambiguous redirect". More details can be found in [1], thanks
to SZEDER Gábor.

Make sure that the editor script quotes "$1" to remove the ambiguity.

[1] https://public-inbox.org/git/20180926121107.GH27036@localhost/

Signed-off-by: Alexander Pyhalov <apyhalov@gmail.com>
Commit-message-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-28 14:43:52 -07:00
Sam McKelvie
c5cbb27cb5 rev-parse: --show-superproject-working-tree should work during a merge
Invoking 'git rev-parse --show-superproject-working-tree' exits with

    "fatal: BUG: returned path string doesn't match cwd?"

when the superproject has an unmerged entry for the current submodule,
instead of displaying the superproject's working tree.

The problem is due to the fact that when a merge of the submodule reference
is in progress, "git ls-files --stage —full-name <submodule-relative-path>”
returns three seperate entries for the submodule (one for each stage) rather
than a single entry; e.g.,

  $ git ls-files --stage --full-name submodule-child-test
  160000 dbbd2766fa330fa741ea59bb38689fcc2d283ac5 1       submodule-child-test
  160000 f174d1dbfe863a59692c3bdae730a36f2a788c51 2       submodule-child-test
  160000 e6178f3a58b958543952e12824aa2106d560f21d 3       submodule-child-test

The code in get_superproject_working_tree() expected exactly one entry to
be returned; this patch makes it use the first entry if multiple entries
are returned.

Test t1500-rev-parse is extended to cover this case.

Signed-off-by: Sam McKelvie <sammck@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-28 14:22:42 -07:00
Martin Ågren
fc0503b04e t1400: drop debug echo to actually execute test
Instead of running `test "foo" = "$(bar)"`, we prefix the whole thing
with `echo`. Comparing to nearby tests makes it clear that this is just
debug leftover. This line has actually been modified four times since it
was introduced in e52290428b (General ref log reading improvements.,
2006-05-19) and the `echo` has always survived. Let's finally drop it.

This script could need some more cleanups. This is just an immediate fix
so that we actually test what we intend to.

All other hits for `git grep "\<echo test " -- t/` seem fine. They want
to create some input or expected output data.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-28 11:45:31 -07:00
SZEDER Gábor
18c765e0dd t1700-split-index: document why FSMONITOR is disabled in this test script
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-28 10:44:08 -07:00
Elijah Newren
3e73cc62c0 commit: fix erroneous BUG, 'multiple renames on the same target? how?'
builtin/commit.c:prepare_to_commit() can call run_status() twice if
using the editor, including status, and the user attempts to record a
non-merge empty commit without explicit --allow-empty.  If there is also
a rename involved as well (due to using 'git add -N'), then a BUG in
wt-status.c is triggered:

  BUG: wt-status.c:476: multiple renames on the same target? how?

The reason we hit this bug is that both run_status() calls use the same
struct wt_status * (named s), and s->change is not freed between runs.
Changes are inserted into s with string_list_insert, which usually means
that the second run just recomputes all the same results and overwrites
what was computed the first time.  However, ever since commit
176ea74793 ("wt-status.c: handle worktree renames", 2017-12-27),
wt-status started checking for renames and copies but also added a
preventative check that d->rename_status wasn't already set and output a
BUG message if it was.  The problem isn't that there are multiple rename
targets to a single path as the error implies, the problem is that 's'
is not freed/cleared between the two run_status() calls.

Ever since commit dc6b1d92ca ("wt-status: use settings from
git_diff_ui_config", 2018-05-04), which stopped hardcoding
DIFF_DETECT_RENAME and allowed users to ask for copy detection, this bug
has also been triggerable with a copy instead of a rename.

Fix the bug by clearing s->change.  A better change might be to clean up
all of s between the two run_status() calls.  A good first step towards
such a goal might be writing a function to free the necessary fields in
the wt_status * struct; a cursory glance at the code suggests all of its
allocated data is probably leaked.  However, doing all that cleanup is a
bigger task for someone else interested to tackle; just fix the bug for
now.

Reported-by: Andrea Stacchiotti <andreastacchiotti@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-27 15:22:34 -07:00
Junio C Hamano
1958ad504b Sync with 2.18.1
* maint-2.18:
  Git 2.18.1
  Git 2.17.2
  fsck: detect submodule paths starting with dash
  fsck: detect submodule urls starting with dash
  Git 2.16.5
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:50:45 -07:00
Junio C Hamano
44f87dac99 Sync with 2.17.2
* maint-2.17:
  Git 2.17.2
  fsck: detect submodule paths starting with dash
  fsck: detect submodule urls starting with dash
  Git 2.16.5
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:45:01 -07:00
Jeff King
1a7fd1fb29 fsck: detect submodule paths starting with dash
As with urls, submodule paths with dashes are ignored by
git, but may end up confusing older versions. Detecting them
via fsck lets us prevent modern versions of git from being a
vector to spread broken .gitmodules to older versions.

Compared to blocking leading-dash urls, though, this
detection may be less of a good idea:

  1. While such paths provide confusing and broken results,
     they don't seem to actually work as option injections
     against anything except "cd". In particular, the
     submodule code seems to canonicalize to an absolute
     path before running "git clone" (so it passes
     /your/clone/-sub).

  2. It's more likely that we may one day make such names
     actually work correctly. Even after we revert this fsck
     check, it will continue to be a hassle until hosting
     servers are all updated.

On the other hand, it's not entirely clear that the behavior
in older versions is safe. And if we do want to eventually
allow this, we may end up doing so with a special syntax
anyway (e.g., writing "./-sub" in the .gitmodules file, and
teaching the submodule code to canonicalize it when
comparing).

So on balance, this is probably a good protection.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-27 11:41:31 -07:00
Jeff King
a124133e1e fsck: detect submodule urls starting with dash
Urls with leading dashes can cause mischief on older
versions of Git. We should detect them so that they can be
rejected by receive.fsckObjects, preventing modern versions
of git from being a vector by which attacks can spread.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-27 11:41:26 -07:00
Junio C Hamano
e43aab778c Sync with 2.16.5
* maint-2.16:
  Git 2.16.5
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:41:02 -07:00
Junio C Hamano
424aac653a Sync with 2.15.3
* maint-2.15:
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:35:43 -07:00
Junio C Hamano
902df9f5c4 Sync with Git 2.14.4
* maint-2.14:
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:20:22 -07:00
Jeff King
273c61496f submodule-config: ban submodule paths that start with a dash
We recently banned submodule urls that look like
command-line options. This is the matching change to ban
leading-dash paths.

As with the urls, this should not break any use cases that
currently work. Even with our "--" separator passed to
git-clone, git-submodule.sh gets confused. Without the code
portion of this patch, the clone of "-sub" added in t7417
would yield results like:

    /path/to/git-submodule: 410: cd: Illegal option -s
    /path/to/git-submodule: 417: cd: Illegal option -s
    /path/to/git-submodule: 410: cd: Illegal option -s
    /path/to/git-submodule: 417: cd: Illegal option -s
    Fetched in submodule path '-sub', but it did not contain b56243f8f4eb91b2f1f8109452e659f14dd3fbe4. Direct fetching of that commit failed.

Moreover, naively adding such a submodule doesn't work:

  $ git submodule add $url -sub
  The following path is ignored by one of your .gitignore files:
  -sub

even though there is no such ignore pattern (the test script
hacks around this with a well-placed "git mv").

Unlike leading-dash urls, though, it's possible that such a
path _could_ be useful if we eventually made it work. So
this commit should be seen not as recommending a particular
policy, but rather temporarily closing off a broken and
possibly dangerous code-path. We may revisit this decision
later.

There are two minor differences to the tests in t7416 (that
covered urls):

  1. We don't have a "./-sub" escape hatch to make this
     work, since the submodule code expects to be able to
     match canonical index names to the path field (so you
     are free to add submodule config with that path, but we
     would never actually use it, since an index entry would
     never start with "./").

  2. After this patch, cloning actually succeeds. Since we
     ignore the submodule.*.path value, we fail to find a
     config stanza for our submodule at all, and simply
     treat it as inactive. We still check for the "ignoring"
     message.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-27 09:34:59 -07:00
Jeff King
f6adec4e32 submodule-config: ban submodule urls that start with dash
The previous commit taught the submodule code to invoke our
"git clone $url $path" with a "--" separator so that we
aren't confused by urls or paths that start with dashes.

However, that's just one code path. It's not clear if there
are others, and it would be an easy mistake to add one in
the future. Moreover, even with the fix in the previous
commit, it's quite hard to actually do anything useful with
such an entry. Any url starting with a dash must fall into
one of three categories:

 - it's meant as a file url, like "-path". But then any
   clone is not going to have the matching path, since it's
   by definition relative inside the newly created clone. If
   you spell it as "./-path", the submodule code sees the
   "/" and translates this to an absolute path, so it at
   least works (assuming the receiver has the same
   filesystem layout as you). But that trick does not apply
   for a bare "-path".

 - it's meant as an ssh url, like "-host:path". But this
   already doesn't work, as we explicitly disallow ssh
   hostnames that begin with a dash (to avoid option
   injection against ssh).

 - it's a remote-helper scheme, like "-scheme::data". This
   _could_ work if the receiver bends over backwards and
   creates a funny-named helper like "git-remote--scheme".
   But normally there would not be any helper that matches.

Since such a url does not work today and is not likely to do
anything useful in the future, let's simply disallow them
entirely. That protects the existing "git clone" path (in a
belt-and-suspenders way), along with any others that might
exist.

Our tests cover two cases:

  1. A file url with "./" continues to work, showing that
     there's an escape hatch for people with truly silly
     repo names.

  2. A url starting with "-" is rejected.

Note that we expect case (2) to fail, but it would have done
so even without this commit, for the reasons given above.
So instead of just expecting failure, let's also check for
the magic word "ignoring" on stderr. That lets us know that
we failed for the right reason.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-27 09:34:58 -07:00
Thomas Gummerer
29e8dc50ad t5551: compare sorted cookies files
In t5551 we check that we save cookies correctly to a file when
http.cookiefile and http.savecookies are set.  To do so we create an
expect file that expects the cookies in a certain order.

However after e2ef8d6fa ("cookies: support creation-time attribute for
cookies", 2018-08-28) in curl.git (released in curl 7.61.1) that order
changed.

We document the file format as "Netscape/Mozilla cookie file
format (see curl(1))", so any format produced by libcurl should be
fine here.  Sort the files, to be agnostic to the order of the
cookies, and make the test pass with both curl versions > 7.61.1 and
earlier curl versions.

Reported-by: Todd Zullinger <tmz@pobox.com>
Helped-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-24 08:35:06 -07:00
Thomas Gummerer
92b7fd87bb t5551: move setup code inside test_expect blocks
Move setup code inside test_expect blocks, to catch unexpected
failures in the setup steps, and bring the test scripts in line with
our modern test style.

Suggested-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-24 08:35:04 -07:00
Nguyễn Thái Ngọc Duy
84d938b732 add: do not accept pathspec magic 'attr'
Commit b0db704652 (pathspec: allow querying for attributes -
2017-03-13) adds new pathspec magic 'attr' but only with
match_pathspec(). "git add" has some pathspec related code that still
does not know about 'attr' and will bail out:

    $ git add ':(attr:foo)'
    fatal: BUG:dir.c:1584: unsupported magic 40

A better solution would be making this code support 'attr'. But I
don't know how much work is needed (I'm not familiar with this new
magic). For now, let's simply reject this magic with a friendlier
message:

    $ git add ':(attr:foo)'
    fatal: :(attr:foo): pathspec magic not supported by this command: 'attr'

Update t6135 so that the expected error message is from the
"graceful" rejection codepath, not "oops, we were supposed to reject
the request to trigger this magic" codepath.

Reported-by: smaudet@sebastianaudet.com
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-21 09:17:02 -07:00
Shulhan
5025425dff builtin/remote: quote remote name on error to display empty name
When adding new remote name with empty string, git will print the
following error message,

  fatal: '' is not a valid remote name\n

But when removing remote name with empty string as input, git shows the
empty string without quote,

  fatal: No such remote: \n

To make these error messages consistent, quote the name of the remote
that we tried and failed to find.

Signed-off-by: Shulhan <m.shulhan@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-14 09:38:18 -07:00
Thomas Gummerer
e467a90c7a linear-assignment: fix potential out of bounds memory access
Currently the 'compute_assignment()' function may read memory out
of bounds, even if used correctly.  Namely this happens when we only
have one column.  In that case we try to calculate the initial
minimum cost using '!j1' as column in the reduction transfer code.
That in turn causes us to try and get the cost from column 1 in the
cost matrix, which does not exist, and thus results in an out of
bounds memory read.

In the original paper [1], the example code initializes that minimum
cost to "infinite".  We could emulate something similar by setting the
minimum cost to INT_MAX, which would result in the same minimum cost
as the current algorithm, as we'd always go into the if condition at
least once, except when we only have one column, and column_count thus
equals 1.

If column_count does equal 1, the condition in the loop would always
be false, and we'd end up with a minimum of INT_MAX, which may lead to
integer overflows later in the algorithm.

For a column count of 1, we however do not even really need to go
through the whole algorithm.  A column count of 1 means that there's
no possible assignments, and we can just zero out the column2row and
row2column arrays, and return early from the function, while keeping
the reduction transfer part of the function the same as it is
currently.

Another solution would be to just not call the 'compute_assignment()'
function from the range diff code in this case, however it's better to
make the compute_assignment function more robust, so future callers
don't run into this potential problem.

Note that the test only fails under valgrind on Linux, but the same
command has been reported to segfault on Mac OS.

[1]: Jonker, R., & Volgenant, A. (1987). A shortest augmenting path
     algorithm for dense and sparse linear assignment
     problems. Computing, 38(4), 325–340.

Reported-by: ryenus <ryenus@gmail.com>
Helped-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-14 09:10:26 -07:00
Jonathan Tan
e68302011c fetch-object: set exact_oid when fetching
fetch_objects() currently does not set exact_oid in struct ref when
invoking transport_fetch_refs(). If the server supports ref-in-want,
fetch_pack() uses this field to determine whether a wanted ref should be
requested as a "want-ref" line or a "want" line; without the setting of
exact_oid, the wrong line will be sent.

Set exact_oid, so that the correct line is sent.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-13 13:57:31 -07:00
Elijah Newren
a3ec9eaf38 sequencer: fix --allow-empty-message behavior, make it smarter
In commit b00bf1c9a8 ("git-rebase: make --allow-empty-message the
default", 2018-06-27), several arguments were given for transplanting
empty commits without halting and asking the user for confirmation on
each commit.  These arguments were incomplete because the logic clearly
assumed the only cases under consideration were transplanting of commits
with empty messages (see the comment about "There are two sources for
commits with empty messages).  It didn't discuss or even consider
rewords, squashes, etc. where the user is explicitly asked for a new
commit message and provides an empty one.  (My bad, I totally should
have thought about that at the time, but just didn't.)

Rewords and squashes are significantly different, though, as described
by SZEDER:

    Let's suppose you start an interactive rebase, choose a commit to
    squash, save the instruction sheet, rebase fires up your editor, and
    then you notice that you mistakenly chose the wrong commit to
    squash.  What do you do, how do you abort?

    Before [that commit] you could clear the commit message, exit the
    editor, and then rebase would say "Aborting commit due to empty
    commit message.", and you get to run 'git rebase --abort', and start
    over.

    But [since that commit, ...] saving the commit message as is would
    let rebase continue and create a bunch of unnecessary objects, and
    then you would have to use the reflog to return to the pre-rebase
    state.

Also, he states:

    The instructions in the commit message template, which is shown for
    'reword' and 'squash', too, still say...

    # Please enter the commit message for your changes. Lines starting
    # with '#' will be ignored, and an empty message aborts the commit.

These are sound arguments that when editing commit messages during a
sequencer operation, that if the commit message is empty then the
operation should halt and ask the user to correct.  The arguments in
commit b00bf1c9a8 (referenced above) still apply when transplanting
previously created commits with empty commit messages, so the sequencer
should not halt for those.

Furthermore, all rationale so far applies equally for cherry-pick as for
rebase.  Therefore, make the code default to --allow-empty-message when
transplanting an existing commit, and to default to halting when the
user is asked to edit a commit message and provides an empty one -- for
both rebase and cherry-pick.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-13 13:25:08 -07:00
Elijah Newren
d345e9fbe7 update-ref: allow --no-deref with --stdin
If passed both --no-deref and --stdin, update-ref would error out with a
general usage message that did not at all suggest these options were
incompatible.  The manpage for update-ref did suggest through its
synopsis line that --no-deref and --stdin were incompatible, but it sadly
also incorrectly suggested that -d and --no-deref were incompatible.  So
the help around the --no-deref option is buggy in a few ways.

The --stdin option did provide a different mechanism for avoiding
dereferencing symbolic-refs: adding a line reading
  option no-deref
before every other directive in the input.  (Technically, if the user
wants to do the extra work of first determining which refs they want to
update or delete are symbolic, then they only need to put the extra
"option no-deref" lines before the updates of those refs.  But in some
cases, that's more work than just adding the "option no-deref" before
every other directive.)

It's easier to allow the user to just pass --no-deref along with --stdin
in order to tell update-ref that the user doesn't want any symbolic ref
to be dereferenced.  It also makes the update-ref documentation simpler.
Implement that, and update the documentation to match.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12 15:17:17 -07:00
SZEDER Gábor
456d7cd3a9 t0090: disable GIT_TEST_SPLIT_INDEX for the test checking split index
The test 'switching trees does not invalidate shared index' in
't0090-cache-tree.sh' is about verifying the behaviour of the split
index feature, therefore it should be in full control of when index
splitting is performed, like all the tests in 't1700-split-index.sh'.

Unset GIT_TEST_SPLIT_INDEX for this test to avoid unintended random
index splitting.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12 14:07:25 -07:00
SZEDER Gábor
acdee9e9e8 t1700-split-index: drop unnecessary 'grep'
The test 'disable split index' in 't1700-split-index.sh' runs the
following pipeline:

  cmd | grep <pattern> | sed s///

Drop that 'grep' from the pipeline, and let 'sed' take over its
duties.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12 14:06:07 -07:00
Jeff Hostetler
eeaf7ddac7 mingw: fix mingw_open_append to work with named pipes
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 13:54:54 -07:00
Jeff Hostetler
06ba9d03e3 t0051: test GIT_TRACE to a windows named pipe
Create a test-tool helper to create the server side of
a windows named pipe, wait for a client connection, and
copy data written to the pipe to stdout.

Create t0051 test to route GIT_TRACE output of a command
to a named pipe using the above test-tool helper.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 13:54:25 -07:00
Elijah Newren
ad2bf0d9b4 rerere: avoid buffer overrun
check_one_conflict() compares `i` to `active_nr` in two places to avoid
buffer overruns, but left out an important third location.

The code did used to have a check here comparing i to active_nr, back
before commit fb70a06da2 ("rerere: fix an off-by-one non-bug",
2015-06-28), however the code at the time used an 'if' rather than a
'while' meaning back then that this loop could not have read past the
end of the array, making the check unnecessary and it was removed.
Unfortunately, in commit 5eda906b28 ("rerere: handle conflicts with
multiple stage #1 entries", 2015-07-24), the 'if' was changed to a
'while' and the check comparing i and active_nr was not re-instated,
leading to this problem.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 13:43:23 -07:00
Elijah Newren
38c93c4d9d t4200: demonstrate rerere segfault on specially crafted merge
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 13:43:21 -07:00
SZEDER Gábor
79336116f5 t3701-add-interactive: tighten the check of trace output
The test 'add -p does not expand argument lists' in
't3701-add-interactive.sh', added in 7288e12cce (add--interactive: do
not expand pathspecs with ls-files, 2017-03-14), checks the GIT_TRACE
of 'git add -p' to ensure that the name of a tracked file wasn't
passed around as argument to any of the commands executed as a result
of undesired pathspec expansion.  This check is done with 'grep' using
the filename on its own as the pattern, which is too loose a pattern,
and would match any occurrences of the filename in the trace output,
not just those as command arguments.  E.g. if a developer were to
litter the index handling code with trace_printf()s printing, among
other things, the name of the just processed cache entry, then that
pattern would mistakenly match these as well, and would fail the test.

Tighten this 'grep' pattern to only match trace lines that show the
executed commands.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 13:38:50 -07:00
Junio C Hamano
f38a45b9ab Merge branch 'jn/submodule-core-worktree-revert'
* jn/submodule-core-worktree-revert:
  Revert "Merge branch 'sb/submodule-core-worktree'"
2018-09-10 10:38:58 -07:00
Junio C Hamano
fe468efff5 Merge branch 'mk/http-backend-content-length'
The earlier attempt barfed when given a CONTENT_LENGTH that is
set to an empty string.  RFC 3875 is fairly clear that in this
case we should not read any message body, but we've been reading
through to the EOF in previous versions (which did not even pay
attention to the environment variable), so keep that behaviour for
now in this late update.

* mk/http-backend-content-length:
  http-backend: allow empty CONTENT_LENGTH
2018-09-10 10:35:42 -07:00
Jonathan Nieder
f178c13fda Revert "Merge branch 'sb/submodule-core-worktree'"
This reverts commit 7e25437d35, reversing
changes made to 00624d608c.

v2.19.0-rc0~165^2~1 (submodule: ensure core.worktree is set after
update, 2018-06-18) assumes an "absorbed" submodule layout, where the
submodule's Git directory is in the superproject's .git/modules/
directory and .git in the submodule worktree is a .git file pointing
there.  In particular, it uses $GIT_DIR/modules/$name to find the
submodule to find out whether it already has core.worktree set, and it
uses connect_work_tree_and_git_dir if not, resulting in

	fatal: could not open sub/.git for writing

The context behind that patch: v2.19.0-rc0~165^2~2 (submodule: unset
core.worktree if no working tree is present, 2018-06-12) unsets
core.worktree when running commands like "git checkout
--recurse-submodules" to switch to a branch without the submodule.  If
a user then uses "git checkout --no-recurse-submodules" to switch back
to a branch with the submodule and runs "git submodule update", this
patch is needed to ensure that commands using the submodule directly
are aware of the path to the worktree.

It is late in the release cycle, so revert the whole 3-patch series.
We can try again later for 2.20.

Reported-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Helped-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-07 19:05:20 -07:00
Max Kirillov
574c513e8d http-backend: allow empty CONTENT_LENGTH
According to RFC3875, empty environment variable is equivalent to unset,
and for CONTENT_LENGTH it should mean zero body to read.

However, unset CONTENT_LENGTH is also used for chunked encoding to indicate
reading until EOF. At least, the test "large fetch-pack requests can be split
across POSTs" from t5551 starts faliing, if unset or empty CONTENT_LENGTH is
treated as zero length body. So keep the existing behavior as much as possible.

Add a test for the case.

Reported-By: Jelmer Vernooij <jelmer@jelmer.uk>
Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-07 12:35:51 -07:00
Jeff King
6c003d6ffb reopen_tempfile(): truncate opened file
We provide a reopen_tempfile() function, which is in turn
used by reopen_lockfile().  The idea is that a caller may
want to rewrite the tempfile without letting go of the lock.
And that's what our one caller does: after running
add--interactive, "commit -p" will update the cache-tree
extension of the index and write out the result, all while
holding the lock.

However, because we open the file with only the O_WRONLY
flag, the existing index content is left in place, and we
overwrite it starting at position 0. If the new index after
updating the cache-tree is smaller than the original, those
final bytes are not overwritten and remain in the file. This
results in a corrupt index, since those cruft bytes are
interpreted as part of the trailing hash (or even as an
extension, if there are enough bytes).

This bug actually pre-dates reopen_tempfile(); the original
code from 9c4d6c0297 (cache-tree: Write updated cache-tree
after commit, 2014-07-13) has the same bug, and those lines
were eventually refactored into the tempfile module. Nobody
noticed until now for two reasons:

 - the bug can only be triggered in interactive mode
   ("commit -p" or "commit -i")

 - the size of the index must shrink after updating the
   cache-tree, which implies a non-trivial deletion. Notice
   that the included test actually has to create a 2-deep
   hierarchy. A single level is not enough to actually cause
   shrinkage.

The fix is to truncate the file before writing out the
second index. We can do that at the caller by using
ftruncate(). But we shouldn't have to do that. There is no
other place in Git where we want to open a file and
overwrite bytes, making reopen_tempfile() a confusing and
error-prone interface. Let's pass O_TRUNC there, which gives
callers the same state they had after initially opening the
file or lock.

It's possible that we could later add a caller that wants
something else (e.g., to open with O_APPEND). But this is
the only caller we've had in the history of the codebase.
Let's punt on doing anything more clever until another one
comes along.

Reported-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-05 09:46:16 -07:00
Junio C Hamano
e9983f8965 Merge branch 'es/chain-lint-more'
The test linter code has learned that the end of here-doc mark
"EOF" can be quoted in a double-quote pair, not just in a
single-quote pair.

* es/chain-lint-more:
  chainlint: match "quoted" here-doc tags
2018-09-04 14:31:40 -07:00
Junio C Hamano
28d294a5ea Merge branch 'ab/portable-more'
Portability fix.

* ab/portable-more:
  tests: fix non-portable iconv invocation
  tests: fix non-portable "${var:-"str"}" construct
  tests: fix and add lint for non-portable grep --file
  tests: fix version-specific portability issue in Perl JSON
  tests: use shorter labels in chainlint.sed for AIX sed
  tests: fix comment syntax in chainlint.sed for AIX sed
  tests: fix and add lint for non-portable seq
  tests: fix and add lint for non-portable head -c N
2018-09-04 14:31:40 -07:00
Junio C Hamano
ca676b9bd3 Merge branch 'en/directory-renames-nothanks'
Recent addition of "directory rename" heuristics to the
merge-recursive backend makes the command susceptible to false
positives and false negatives.  In the context of "git am -3",
which does not know about surrounding unmodified paths and thus
cannot inform the merge machinery about the full trees involved,
this risk is particularly severe.  As such, the heuristic is
disabled for "git am -3" to keep the machinery "more stupid but
predictable".

* en/directory-renames-nothanks:
  am: avoid directory rename detection when calling recursive merge machinery
  merge-recursive: add ability to turn off directory rename detection
  t3401: add another directory rename testcase for rebase and am
2018-09-04 14:31:38 -07:00
Junio C Hamano
064e0b2d4c Merge branch 'pw/rebase-i-author-script-fix'
Recent "git rebase -i" update started to write bogusly formatted
author-script, with a matching broken reading code.  These are
fixed.

* pw/rebase-i-author-script-fix:
  sequencer: fix quoting in write_author_script
  sequencer: handle errors from read_author_ident()
2018-09-04 14:31:38 -07:00
Johannes Schindelin
10d2f35436 rebase -i: be careful to wrap up fixup/squash chains
When an interactive rebase was stopped at the end of a fixup/squash
chain, the user might have edited the commit manually before continuing
(with either `git rebase --skip` or `git rebase --continue`, it does not
really matter which).

We need to be very careful to wrap up the fixup/squash chain also in
this scenario: otherwise the next fixup/squash chain would try to pick
up where the previous one was left.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-09-04 08:59:33 -07:00
Johannes Schindelin
2f3eb68f10 rebase -i --autosquash: demonstrate a problem skipping the last squash
The `git commit --squash` command can be used not only to amend commit
messages and changes, but also to record notes for an upcoming rebase.

For example, when the author information of a given commit is incorrect,
a user might call `git commit --allow-empty -m "Fix author" --squash
<commit>`, to remind them to fix that during the rebase. When the editor
would pop up, the user would simply delete the commit message to abort
the rebase at this stage, fix the author information, and continue with
`git rebase --skip`. (This is a real-world example from the rebase of
Git for Windows onto v2.19.0-rc1.)

However, there is a bug in `git rebase` that will cause the squash
message *not* to be forgotten in this case. It will therefore be reused
in the next fixup/squash chain (if any).

This patch adds a test case to demonstrate this breakage.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-09-04 08:59:33 -07:00
Elijah Newren
6aba117d5c am: avoid directory rename detection when calling recursive merge machinery
Let's say you have the following three trees, where Base is from one commit
behind either master or branch:

   Base  : bar_v1, foo/{file1, file2, file3}
   branch: bar_v2, foo/{file1, file2},       goo/file3
   master: bar_v3, foo/{file1, file2, file3}

Using git-am (or am-based rebase) to apply the changes from branch onto
master results in the following tree:

   Result: bar_merged, goo/{file1, file2, file3}

This is not what users want; they did not rename foo/ -> goo/, they only
renamed one file within that directory.  The reason this happens is am
constructs fake trees (via build_fake_ancestor()) of the following form:

   Base_bfa  : bar_v1, foo/file3
   branch_bfa: bar_v2, goo/file3

Combining these two trees with master's tree:

   master: bar_v3, foo/{file1, file2, file3},

You can see that merge_recursive_generic() would see branch_bfa as renaming
foo/ -> goo/, and master as just adding both foo/file1 and foo/file2.  As
such, it ends up with goo/{file1, file2, file3}

The core problem is that am does not have access to the original trees; it
can only construct trees using the blobs involved in the patch.  As such,
it is not safe to perform directory rename detection within am -3.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30 07:58:59 -07:00
Elijah Newren
e7588c9652 t3401: add another directory rename testcase for rebase and am
Similar to commit 16346883ab ("t3401: add directory rename testcases for
rebase and am", 2018-06-27), add another testcase for directory rename
detection.  This new testcase differs in that it showcases a situation
where no directory rename was performed, but which some backends
incorrectly detect.

As with the other testcase, run this in conjunction with each of the
types of rebases:
  git-rebase--interactive
  git-rebase--am
  git-rebase--merge
and also use the same testcase for
  git am --3way

Reported-by: Nikolay Kasyanov <corrmage@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30 07:58:59 -07:00
Eric Sunshine
3042b6bb59 chainlint: match "quoted" here-doc tags
A here-doc tag can be quoted ('EOF'/"EOF") or escaped (\EOF) to suppress
interpolation within the body. chainlint recognizes single-quoted and
escaped tags, but does not know about double-quoted tags. For
completeness, teach it to recognize double-quoted tags, as well.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-29 10:57:38 -07:00
Ævar Arnfjörð Bjarmason
f6af6f9970 tests: fix non-portable iconv invocation
The iconv that comes with a FreeBSD 11.2-RELEASE-p2 box I have access
to doesn't support the SHIFT-JIS encoding. Guard a test added in
e92d62253 ("convert: add round trip check based on
'core.checkRoundtripEncoding'", 2018-04-15) first released with Git
v2.18.0 with a prerequisite that checks for its availability.

The iconv command is in POSIX, and we have numerous tests
unconditionally relying on its ability to convert ASCII, UTF-8 and
UTF-16, but unconditionally relying on the presence of more obscure
encodings isn't portable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-29 10:35:05 -07:00
Ævar Arnfjörð Bjarmason
de231e577b tests: fix non-portable "${var:-"str"}" construct
On both AIX 7200-00-01-1543 and FreeBSD 11.2-RELEASE-p2 the
"${var:-"str"}" syntax means something different than what it does
under the bash or dash shells.

Both will consider the start of the new unescaped quotes to be a new
argument to test_expect_success, resulting in the following error:

    error: bug in the test script: 'git diff-tree initial # magic
    is (not' does not look like a prereq

Fix this by removing the redundant quotes. There's no need for them,
and the resulting code works under all the aforementioned shells. This
fixes a regression in c2f1d3989 ("t4013: test new output from diff
--abbrev --raw", 2017-12-03) first released with Git v2.16.0.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-29 10:34:10 -07:00
Junio C Hamano
926107db6d Merge branch 'sg/test-rebase-editor-fix'
Test fix.

* sg/test-rebase-editor-fix:
  t/lib-rebase.sh: support explicit 'pick' commands in 'fake_editor.sh'
2018-08-27 14:33:50 -07:00
Junio C Hamano
99fb11d15b Merge branch 'ab/test-must-be-empty-for-master'
Test fixes.

* ab/test-must-be-empty-for-master:
  t6018-rev-list-glob: fix 'empty stdin' test
2018-08-27 14:33:49 -07:00
Junio C Hamano
56ce87daff Merge branch 'sg/t3420-autostash-fix'
Test fixes.

* sg/t3420-autostash-fix:
  t3420-rebase-autostash: don't try to grep non-existing files
2018-08-27 14:33:49 -07:00
Junio C Hamano
5f0ed3e204 Merge branch 'sg/t3903-missing-fix'
Test fixes.

* sg/t3903-missing-fix:
  t3903-stash: don't try to grep non-existing file
2018-08-27 14:33:48 -07:00
Junio C Hamano
1d27164f1a Merge branch 'sg/t7501-thinkofix'
Test fixes.

* sg/t7501-thinkofix:
  t7501-commit: drop silly command substitution
2018-08-27 14:33:48 -07:00
Junio C Hamano
df19317f4f Merge branch 'sg/t0020-conversion-fix'
Test fixes.

* sg/t0020-conversion-fix:
  t0020-crlf: check the right file
2018-08-27 14:33:47 -07:00
Junio C Hamano
603160b17e Merge branch 'sg/t4051-fix'
Test fixes.

* sg/t4051-fix:
  t4051-diff-function-context: read the right file
2018-08-27 14:33:45 -07:00
Junio C Hamano
6d9276ea13 Merge branch 'jk/use-compat-util-in-test-tool'
Dev tool update.

* jk/use-compat-util-in-test-tool:
  test-tool.h: include git-compat-util.h
2018-08-27 14:33:43 -07:00
Junio C Hamano
986c518107 Merge branch 'sg/test-must-be-empty'
Test fixes.

* sg/test-must-be-empty:
  tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>'
  tests: use 'test_must_be_empty' instead of 'test_cmp /dev/null <out>'
  tests: use 'test_must_be_empty' instead of 'test ! -s'
  tests: use 'test_must_be_empty' instead of '! test -s'
2018-08-27 14:33:43 -07:00
Junio C Hamano
a988ce9a58 Merge branch 'ep/worktree-quiet-option'
"git worktree" command learned "--quiet" option to make it less
verbose.

* ep/worktree-quiet-option:
  worktree: add --quiet option
2018-08-27 14:33:42 -07:00
Junio C Hamano
d89db6f4c3 Merge branch 'sm/branch-sort-config'
"git branch --list" learned to take the default sort order from the
'branch.sort' configuration variable, just like "git tag --list"
pays attention to 'tag.sort'.

* sm/branch-sort-config:
  branch: support configuring --sort via .gitconfig
2018-08-27 14:33:42 -07:00
Ævar Arnfjörð Bjarmason
4a3ed63802 tests: fix and add lint for non-portable grep --file
The --file option to grep isn't in POSIX[1], but -f is[1]. Let's check
for that in the future, and fix the portability regression in
f237c8b6fe ("commit-graph: implement git-commit-graph write",
2018-04-02) that broke e.g. AIX.

1. http://pubs.opengroup.org/onlinepubs/009695399/utilities/grep.html

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27 14:07:32 -07:00
Ævar Arnfjörð Bjarmason
8c97e38731 tests: fix version-specific portability issue in Perl JSON
The test guarded by PERLJSON added in 75459410ed ("json_writer: new
routines to create JSON data", 2018-07-13) assumed that a JSON boolean
value like "true" or "false" would be represented as "1" or "0" in
Perl.

This behavior can't be relied upon, e.g. with JSON.pm 2.50 and
JSON::PP.  A JSON::PP::Boolean object will be represented as "true"
or "false". To work around this let's check if we have any refs left
after we check for hashes and arrays, assume those are JSON objects,
and coerce them to a known boolean value.

The behavior of this test still looks odd to me. Why implement our own
ad-hoc encoder just for some one-off test, as opposed to say Perl's
own Data::Dumper with Sortkeys et al? But with this change it works,
so let's leave it be.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27 14:07:32 -07:00
Ævar Arnfjörð Bjarmason
a3c4c8841c tests: use shorter labels in chainlint.sed for AIX sed
Improve the portability of chainlint by using shorter labels. On
AIX sed will complain about:

    sed: 0602-417 The label :hereslurp is greater than eight
    characters

This, in combination with the previous fix to this file makes
GIT_TEST_CHAIN_LINT=1 (which is the default) working again on AIX
without issues, and the "gmake check-chainlint" test also passes.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27 14:07:18 -07:00
Ævar Arnfjörð Bjarmason
2d9ded8acc tests: fix comment syntax in chainlint.sed for AIX sed
Change a comment in chainlint.sed to appease AIX sed, which would
previously print this error:

    sed:    # stash for later printing is not a recognized function

1. https://public-inbox.org/git/CAPig+cTTbU5HFMKgNyrxTp3+kcK46-Fn=4ZH6zDt1oQChAc3KA@mail.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27 11:31:43 -07:00
Ævar Arnfjörð Bjarmason
b2fa7a2372 tests: fix and add lint for non-portable seq
The seq command is not in POSIX, and doesn't exist on
e.g. OpenBSD. We've had the test_seq wrapper since d17cf5f3a3 ("tests:
Introduce test_seq", 2012-08-04), but use of it keeps coming back,
e.g. in the recently added "fetch negotiator" tests being added here.

So let's also add a check to "make test-lint". The regex is aiming to
capture the likes of $(seq ..) and "seq" as a stand-alone command,
without capturing some existing cases where we e.g. have files called
"seq", as \bseq\b would do.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27 11:31:18 -07:00
Ævar Arnfjörð Bjarmason
2a59a6ef20 tests: fix and add lint for non-portable head -c N
The "head -c BYTES" option is non-portable (not in POSIX[1]). Change
such invocations to use the test_copy_bytes wrapper added in
48860819e8 ("t9300: factor out portable "head -c" replacement",
2016-06-30).

This fixes a test added in 9d2e330b17 ("ewah_read_mmap: bounds-check
mmap reads", 2018-06-14), which has been breaking
t5310-pack-bitmaps.sh on OpenBSD since 2.18.0. The OpenBSD ports
already have a similar workaround after their upgrade to 2.18.0[2].

I have not tested this on IRIX, but according to 4de0bbd898 ("t9300:
use perl "head -c" clone in place of "dd bs=1 count=16000" kluge",
2010-12-13) this invocation would have broken things there too.

Also, change a valgrind-specific codepath in test-lib.sh to use this
wrapper. Given where valgrind runs I don't think this would ever
become a portability issue in practice, but it's easier to just use
the wrapper than introduce some exception for the "make test-lint"
check being added here.

1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
2. 08d5d82eae (diff-f7d3c4fabeed1691620d608f1534f5e5)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27 11:31:18 -07:00
Jeff King
ffce7f590f sequencer: ignore "---" divider when parsing trailers
When the sequencer code appends a signoff or cherry-pick
origin, it uses the default trailer-parsing options, which
treat "---" as the end of the commit message. As a result,
it may be fooled by a commit message that contains that
string and fail to find the existing trailer block. Even
more confusing, the actual append code does not know about
"---", and always appends to the end of the string. This can
lead to bizarre results. E.g., appending a signoff to a
commit message like this:

  subject

  body
  ---
  these dashes confuse the parser!

  Signed-off-by: A

results in output with a final block like:

  Signed-off-by: A

  Signed-off-by: A

The parser thinks the final line of the message is "body",
and ignores everything else, claiming there are no trailers.
So we output an extra newline separator (wrong) and add a
duplicate signoff (also wrong).

Since we know we are feeding a pure commit message, we can
simply tell the parser to ignore the "---" divider.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-23 10:08:51 -07:00
Jeff King
e5fba5d558 pretty, ref-filter: format %(trailers) with no_divider option
In both of these cases we know that we are feeding the
trailer-parsing code a pure commit message. We should tell
it so, which avoids false positives for a commit message
that contains a "---" line.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-23 10:08:51 -07:00
Jeff King
1688c9a489 interpret-trailers: allow suppressing "---" divider
Even with the newly-tightened "---" parser, it's still
possible for a commit message to trigger a false positive if
it contains something like "--- foo". If the caller knows
that it has only a single commit message, it can now tell us
with the "--no-divider" option, eliminating any false
positives.

If we were designing this from scratch, I'd probably make
this the default. But we've advertised the "---" behavior in
the documentation since interpret-trailers has existed.
Since it's meant to be scripted, breaking that would be a
bad idea.

Note that the logic is in the underlying trailer.c code,
which is used elsewhere. The default there will keep the
current behavior, but many callers will benefit from setting
this new option. That's left for future patches.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-23 10:08:51 -07:00
Jeff King
c188668e38 interpret-trailers: tighten check for "---" patch boundary
The interpret-trailers command accepts not only raw commit
messages, but it also can manipulate trailers in
format-patch output. That means it must find the "---"
boundary separating the commit message from the patch.
However, it does so by looking for any line starting with
"---", regardless of whether there is further content.

This is overly lax compared to the parsing done in
mailinfo.c's patchbreak(), and may cause false positives
(e.g., t/perf output tables uses dashes; if you cut and
paste them into your commit message, it fools the parser).

We could try to reuse patchbreak() here, but it actually has
several heuristics that are not of interest to us (e.g.,
matching "diff -" without a three-dash separator or even a
CVS "Index:" line). We're not interested in taking in
whatever random cruft people may send, but rather handling
git-formatted patches.

Note that the existing documentation was written in a loose
way, so technically we are changing the behavior from what
it said. But this should implement the original intent in a
more accurate way.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-23 10:08:51 -07:00
SZEDER Gábor
7afb0d6777 t/lib-rebase.sh: support explicit 'pick' commands in 'fake_editor.sh'
The verbose output of the test 'reword without issues functions as
intended' in 't3423-rebase-reword.sh', added in a9279c6785 (sequencer:
do not squash 'reword' commits when we hit conflicts, 2018-06-19),
contains the following error output:

  sed: -e expression #1, char 2: extra characters after command

This error comes from within the 'fake-editor.sh' script created by
'lib-rebase.sh's set_fake_editor() function, and the root cause is the
FAKE_LINES="pick 1 reword 2" variable in the test in question, in
particular the "pick" word.  'fake-editor.sh' assumes 'pick' to be the
default rebase command and doesn't support an explicit 'pick' command
in FAKE_LINES.  As a result, 'pick' will be used instead of a line
number when assembling the following 'sed' script:

  sed -n picks/^pick/pick/p

which triggers the aforementioned error.

Luckily, this didn't affect the test's correctness: the erroring 'sed'
command doesn't write anything to the todo script, and processing the
rest of FAKE_LINES generates the desired todo script, as if that
'pick' command were not there at all.

The minimal fix would be to remove the 'pick' word from FAKE_LINES,
but that would leave us susceptible to similar issues in the future.

Instead, teach the fake-editor script to recognize an explicit 'pick'
command, which is still a fairly trivial change.

In the future we might want to consider reinforcing this fake editor
script with an &&-chain and stricter parsing of the FAKE_LINES
variable (e.g. to error out when encountering unknown rebase commands
or commands and line numbers in the wrong order).

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-23 09:23:34 -07:00
SZEDER Gábor
2745817028 t3420-rebase-autostash: don't try to grep non-existing files
Several tests in 't3420-rebase-autostash.sh' start various rebase
processes that are expected to fail because of merge conflicts.  These
tests then run '! grep' to ensure that the autostash feature did its
job, and the dirty contents of a file is gone.  However, due to the
test repo's history and the choice of upstream branch that file
shouldn't exist in the conflicted state at all.  Consequently, this
'grep' doesn't fail as expected, because it can't find the dirty
content, but it fails because it can't open the file.

Tighten this check by using 'test_path_is_missing' instead, thereby
avoiding unexpected errors from 'grep' as well.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-22 11:52:51 -07:00
SZEDER Gábor
79b04f9b60 t3903-stash: don't try to grep non-existing file
The test 'store updates stash ref and reflog' in 't3903-stash.sh'
creates a stash from a new file, runs 'git reset --hard' to throw away
any modifications to the work tree, and then runs '! grep' to ensure
that the staged contents are gone.  Since the file didn't exist
before, it shouldn't exist after 'git reset' either.  Consequently,
this 'grep' doesn't fail as expected, because it can't find the staged
content, but it fails because it can't open the file.

Tighten this check by using 'test_path_is_missing' instead, thereby
avoiding an unexpected error from 'grep' as well.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-22 11:52:35 -07:00
Junio C Hamano
29d9e3e2c4 Merge branch 'nd/pack-deltify-regression-fix'
In a recent update in 2.18 era, "git pack-objects" started
producing a larger than necessary packfiles by missing
opportunities to use large deltas.

* nd/pack-deltify-regression-fix:
  pack-objects: fix performance issues on packing large deltas
2018-08-22 11:17:05 -07:00
SZEDER Gábor
b89b4a660c t6018-rev-list-glob: fix 'empty stdin' test
Prior to d3c6751b18 (tests: make use of the test_must_be_empty
function, 2018-07-27), in the test 'rev-list should succeed with empty
output on empty stdin' in 't6018-rev-list-glob' the empty 'expect'
file served dual purpose: besides specifying the expected output, as
usual, it also served as empty input for 'git rev-list --stdin'.

Then d3c6751b18 came along, and, as part of the conversion to
'test_must_be_empty', removed this empty 'expect' file, not realizing
its secondary purpose.  Redirecting stdin from the now non-existing
file failed the test, but since this test expects failure in the first
place, this issue went unnoticed.

Redirect 'git rev-list's stdin explicitly from /dev/null to provide
empty input.  (Strictly speaking we don't need this redirection,
because the test script's stdin is already redirected from /dev/null
anyway, but I think it's better to be explicit about it.)

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-22 11:02:07 -07:00
SZEDER Gábor
c8b35b95e1 t4051-diff-function-context: read the right file
The test ' context does not include preceding empty lines' in the
block of tests 'change with long common tail and no context' in
't4051-diff-function-context.sh' tries to read the file
'long_common_tail.diff.diff', but that file doesn't exist as its name
contains one more '.diff' suffixes than necessary.

Despite this error the test still succeeded without checking what it's
supposed to, because this erroneous read is done on the line:

  test "$(first_context_line <long_common_tail.diff.diff)" != " "

which means that:

  - the command substitution hides the error, so it won't fail the
    test, and

  - the result of the command substitution is the empty string, which
    is, of course, not equal to a single space character, so the
    condition is fulfilled, and the test succeeds.

As a minimal fix, fix the name of the file to be read.

In the future we might want to reorganize this test script (1) to use
'test_cmp' instead of 'test's and command substitutions to catch
failing commands and to provide helpful error messages, and (2) to
specify what the expected result actually _is_ instead of what it
isn't.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-22 09:14:24 -07:00
SZEDER Gábor
30612cb670 t0020-crlf: check the right file
In the test 'checkout with autocrlf=input' in 't0020-crlf.sh', one of
the 'has_cr' checks looks at the non-existing file 'two' instead of
'dir/two'.  The test still succeeds, without actually checking what it
was supposed to, because this check is expected to fail anyway.

As a minimal fix, fix the name of the file to be checked.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-22 09:08:08 -07:00
SZEDER Gábor
15da753709 t7501-commit: drop silly command substitution
The test '--dry-run with conflicts fixed from a merge' in
't7501-commit.sh', added in 8dc874b2ee (wt-status.c: set commitable
bit if there is a meaningful merge., 2016-02-15), runs the following
unnecessary and downright bogus command substitution:

  ! $(git merge --no-commit commit-1) &&

I.e. after 'git merge ...' is executed and expectedly fails, the test
attempts to execute its output:

  Merging:
  80f2ea2 commit 2
  virtual commit-1
  found 1 common ancestor:
  e60d113 Initial commit
  Auto-merging test-file
  CONFLICT (content): Merge conflict in test-file
  Automatic merge failed; fix conflicts and then commit the result.

as a command, which most likely fails, because there is no such
command as "Merging:".  Then '!' negates the failed exit status, the
test continues, and eventually succeeds.

Remove this command substitution and use 'test_must_fail' to ensure
that 'git merge' fails.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-22 08:39:31 -07:00
Jeff King
69d846f053 test-tool.h: include git-compat-util.h
The test-tool programs include "test-tool.h" as their first
include, which breaks our CodingGuideline of "the first
include must be git-compat-util.h or an equivalent".

Rather than change them all, let's instead make test-tool.h
one of those equivalents, just like we do for builtin.h
(which many of the actual git builtins include first).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 12:11:40 -07:00
SZEDER Gábor
1c5e94f459 tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>'
Using 'test_must_be_empty' is shorter and more idiomatic than

  >empty &&
  test_cmp empty out

as it saves the creation of an empty file.  Furthermore, sometimes the
expected empty file doesn't have such a descriptive name like 'empty',
and its creation is far away from the place where it's finally used
for comparison (e.g. in 't7600-merge.sh', where two expected empty
files are created in the 'setup' test, but are used only about 500
lines later).

These cases were found by instrumenting 'test_cmp' to error out the
test script when it's used to compare empty files, and then converted
manually.

Note that even after this patch there still remain a lot of cases
where we use 'test_cmp' to check empty files:

  - Sometimes the expected output is not hard-coded in the test, but
    'test_cmp' is used to ensure that two similar git commands produce
    the same output, and that output happens to be empty, e.g. the
    test 'submodule update --merge  - ignores --merge  for new
    submodules' in 't7406-submodule-update.sh'.

  - Repetitive common tasks, including preparing the expected results
    and running 'test_cmp', are often extracted into a helper
    function, and some of this helper's callsites expect no output.

  - For the same reason as above, the whole 'test_expect_success'
    block is within a helper function, e.g. in 't3070-wildmatch.sh'.

  - Or 'test_cmp' is invoked in a loop, e.g. the test 'cvs update
    (-p)' in 't9400-git-cvsserver-server.sh'.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 11:48:36 -07:00
SZEDER Gábor
ec21ac8c18 tests: use 'test_must_be_empty' instead of 'test_cmp /dev/null <out>'
Using 'test_must_be_empty' is more idiomatic than 'test_cmp /dev/null
out', and its message on error is perhaps a bit more to the point.

This patch was basically created by running:

  sed -i -e 's%test_cmp /dev/null%test_must_be_empty%' t[0-9]*.sh

with the exception of the change in 'should not fail in an empty repo'
in 't7401-submodule-summary.sh', where it was 'test_cmp output
/dev/null'.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 11:48:34 -07:00
SZEDER Gábor
f0dc593a95 tests: use 'test_must_be_empty' instead of 'test ! -s'
Using 'test_must_be_empty' is preferable to 'test ! -s', because it
gives a helpful error message if the given file is unexpectedly no
empty, while the latter remains completely silent.  Furthermore, it
also catches cases when the given file unexpectedly does not exist at
all.

This patch was created by:

  sed -i -e 's/test ! -s/test_must_be_empty/' t[0-9]*.sh

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 11:48:31 -07:00
SZEDER Gábor
ec10b018e7 tests: use 'test_must_be_empty' instead of '! test -s'
Using 'test_must_be_empty' is preferable to '! test -s', because it
gives a helpful error message if the given file is unexpectedly not
empty, while the latter remains completely silent.  Furthermore, it
also catches cases when the given file unexpectedly does not exist at
all.

This patch was basically created by:

  sed -i -e 's/! test -s/test_must_be_empty/' t[0-9]*.sh

with the following notable exceptions:

  - The '! test -s' check in '.gitmodules ignore=dirty suppresses
    submodules with untracked content' in 't7508-status.sh' is left
    as-is, because it's bogus and, therefore, it's subject of a
    dedicated patch.

  - The '! test -s' checks in 't9131-git-svn-empty-symlink.sh' and
    't9135-git-svn-moved-branch-empty-file.sh' are immediately
    preceeded by a 'test -f' to ensure that the files exist in the
    first place.  'test_must_be_empty' ensures that as well, so those
    'test -f' commands are removed as well.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 11:48:29 -07:00
Derrick Stolee
20fd6d5799 commit-graph: not compatible with grafts
Augment commit_graph_compatible(r) to return false when the given
repository r has commit grafts or is a shallow clone. Test that in these
situations we ignore existing commit-graph files and we do not write new
commit-graph files.

Helped-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 10:22:51 -07:00
Derrick Stolee
d6538246d3 commit-graph: not compatible with replace objects
Create new method commit_graph_compatible(r) to check if a given
repository r is compatible with the commit-graph feature. Fill the
method with a check to see if replace-objects exist. Test this
interaction succeeds, including ignoring an existing commit-graph and
failing to write a new commit-graph. However, we do ensure that
we write a new commit-graph by setting read_replace_refs to 0, thereby
ignoring the replace refs.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 10:22:51 -07:00
Derrick Stolee
b775896342 test-repository: properly init repo
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21 10:22:50 -07:00
Junio C Hamano
273b0a1f58 Merge branch 'ab/checkout-default-remote'
* ab/checkout-default-remote:
  t2024: mark test using "checkout -p" with PERL prerequisite
2018-08-20 12:53:46 -07:00
Junio C Hamano
d28017005f Merge branch 'hn/highlight-sideband-keywords'
The sideband code learned to optionally paint selected keywords at
the beginning of incoming lines on the receiving end.

* hn/highlight-sideband-keywords:
  sideband: do not read beyond the end of input
  sideband: highlight keywords in remote sideband output
2018-08-20 12:41:34 -07:00
Junio C Hamano
39e415cfd1 Merge branch 'nd/cherry-pick-quit-fix'
"git cherry-pick --quit" failed to remove CHERRY_PICK_HEAD even
though we won't be in a cherry-pick session after it returns, which
has been corrected.

* nd/cherry-pick-quit-fix:
  cherry-pick: fix --quit not deleting CHERRY_PICK_HEAD
2018-08-20 12:41:34 -07:00
Junio C Hamano
85c54ecc5f Merge branch 'sb/submodule-cleanup'
A few preliminary minor clean-ups in the area around submodules.

* sb/submodule-cleanup:
  builtin/submodule--helper: remove stray new line
  t7410: update to new style
2018-08-20 12:41:33 -07:00
Junio C Hamano
5a5c5e9565 Merge branch 'pw/rebase-i-merge-segv-fix'
"git rebase -i", when a 'merge <branch>' insn in its todo list
fails, segfaulted, which has been (minimally) corrected.

* pw/rebase-i-merge-segv-fix:
  rebase -i: fix SIGSEGV when 'merge <branch>' fails
  t3430: add conflicting commit
2018-08-20 12:41:33 -07:00
Junio C Hamano
36fd1e843b Merge branch 'pw/rebase-i-squash-number-fix'
When "git rebase -i" is told to squash two or more commits into
one, it labeled the log message for each commit with its number.
It correctly called the first one "1st commit", but the next one
was "commit #1", which was off-by-one.  This has been corrected.

* pw/rebase-i-squash-number-fix:
  rebase -i: fix numbering in squash message
2018-08-20 12:41:33 -07:00
Junio C Hamano
2a2c18f1c3 Merge branch 'sb/config-write-fix'
Recent update to "git config" broke updating variable in a
subsection, which has been corrected.

* sb/config-write-fix:
  git-config: document accidental multi-line setting in deprecated syntax
  config: fix case sensitive subsection names on writing
  t1300: document current behavior of setting options
2018-08-20 12:41:32 -07:00
Junio C Hamano
87aa1595e7 Merge branch 'ab/submodule-relative-url-tests'
Test updates.

* ab/submodule-relative-url-tests:
  submodule: add more exhaustive up-path testing
2018-08-20 12:41:32 -07:00
Junio C Hamano
36f0f344e7 Merge branch 'jt/repack-promisor-packs'
After a partial clone, repeated fetches from promisor remote would
have accumulated many packfiles marked with .promisor bit without
getting them coalesced into fewer packfiles, hurting performance.
"git repack" now learned to repack them.

* jt/repack-promisor-packs:
  repack: repack promisor objects if -a or -A is set
  repack: refactor setup of pack-objects cmd
2018-08-20 12:40:31 -07:00
Junio C Hamano
e72db08f15 Merge branch 'wc/make-funnynames-shared-lazy-prereq'
A test prerequisite defined by various test scripts with slightly
different semantics has been consolidated into a single copy and
made into a lazily defined one.

* wc/make-funnynames-shared-lazy-prereq:
  t: factor out FUNNYNAMES as shared lazy prereq
2018-08-20 11:33:55 -07:00
Junio C Hamano
4601516b41 Merge branch 'js/chain-lint-attrfix'
Test fix.

* js/chain-lint-attrfix:
  chainlint: fix for core.autocrlf=true
2018-08-20 11:33:54 -07:00
Junio C Hamano
81eab6871e Merge branch 'js/range-diff'
"git tbdiff" that lets us compare individual patches in two
iterations of a topic has been rewritten and made into a built-in
command.

* js/range-diff: (21 commits)
  range-diff: use dim/bold cues to improve dual color mode
  range-diff: make --dual-color the default mode
  range-diff: left-pad patch numbers
  completion: support `git range-diff`
  range-diff: populate the man page
  range-diff --dual-color: skip white-space warnings
  range-diff: offer to dual-color the diffs
  diff: add an internal option to dual-color diffs of diffs
  color: add the meta color GIT_COLOR_REVERSE
  range-diff: use color for the commit pairs
  range-diff: add tests
  range-diff: do not show "function names" in hunk headers
  range-diff: adjust the output of the commit pairs
  range-diff: suppress the diff headers
  range-diff: indent the diffs just like tbdiff
  range-diff: right-trim commit messages
  range-diff: also show the diff between patches
  range-diff: improve the order of the shown commits
  range-diff: first rudimentary implementation
  Introduce `range-diff` to compare iterations of a topic branch
  ...
2018-08-20 11:33:53 -07:00
Junio C Hamano
ace1f99cc8 Merge branch 'es/chain-lint-more'
Improve built-in facility to catch broken &&-chain in the tests.

* es/chain-lint-more:
  chainlint: add test of pathological case which triggered false positive
  chainlint: recognize multi-line quoted strings more robustly
  chainlint: let here-doc and multi-line string commence on same line
  chainlint: recognize multi-line $(...) when command cuddled with "$("
  chainlint: match 'quoted' here-doc tags
  chainlint: match arbitrary here-docs tags rather than hard-coded names
2018-08-20 11:33:53 -07:00
Junio C Hamano
a15bfa517d Merge branch 'sg/t5310-empty-input-fix'
Test fix.

* sg/t5310-empty-input-fix:
  t5310-pack-bitmaps: fix bogus 'pack-objects to file can use bitmap' test
2018-08-20 11:33:52 -07:00
Junio C Hamano
0c54cdaf65 Merge branch 'jk/for-each-object-iteration'
The API to iterate over all objects learned to optionally list
objects in the order they appear in packfiles, which helps locality
of access if the caller accesses these objects while as objects are
enumerated.

* jk/for-each-object-iteration:
  for_each_*_object: move declarations to object-store.h
  cat-file: use a single strbuf for all output
  cat-file: split batch "buf" into two variables
  cat-file: use oidset check-and-insert
  cat-file: support "unordered" output for --batch-all-objects
  cat-file: rename batch_{loose,packed}_object callbacks
  t1006: test cat-file --batch-all-objects with duplicates
  for_each_packed_object: support iterating in pack-order
  for_each_*_object: give more comprehensive docstrings
  for_each_*_object: take flag arguments as enum
  for_each_*_object: store flag definitions in a single location
2018-08-20 11:33:52 -07:00
Junio C Hamano
42a6274b62 Merge branch 'ab/fetch-tags-noclobber'
Test and doc clean-ups.

* ab/fetch-tags-noclobber:
  pull doc: fix a long-standing grammar error
  fetch tests: correct a comment "remove it" -> "remove them"
  push tests: assert re-pushing annotated tags
  push tests: add more testing for forced tag pushing
  push tests: fix logic error in "push" test assertion
  push tests: remove redundant 'git push' invocation
  fetch tests: change "Tag" test tag to "testTag"
2018-08-20 11:33:52 -07:00
Junio C Hamano
3bc484af74 Merge branch 'jt/commit-graph-per-object-store'
Test update.

* jt/commit-graph-per-object-store:
  t5318: avoid unnecessary command substitutions
2018-08-20 11:33:51 -07:00