"git add -p <pathspec>" unnecessarily expanded the pathspec to a
list of individual files that matches the pathspec by running "git
ls-files <pathspec>", before feeding it to "git diff-index" to see
which paths have changes, because historically the pathspec
language supported by "diff-index" was weaker. These days they are
equivalent and there is no reason to internally expand it. This
helps both performance and avoids command line argument limit on
some platforms.
* jk/add-i-use-pathspecs:
add--interactive: do not expand pathspecs with ls-files
When we want to get the list of modified files, we first
expand any user-provided pathspecs with "ls-files", and then
feed the resulting list of paths as arguments to
"diff-index" and "diff-files". If your pathspec expands into
a large number of paths, you may run into one of two
problems:
1. The OS may complain about the size of the argument
list, and refuse to run. For example:
$ (ulimit -s 128 && git add -p drivers)
Can't exec "git": Argument list too long at .../git-add--interactive line 177.
Died at .../git-add--interactive line 177.
That's on the linux.git repository, which has about 20K
files in the "drivers" directory (none of them modified
in this case). The "ulimit -s" trick is necessary to
show the problem on Linux even for such a gigantic set
of paths. Other operating systems have much smaller
limits (e.g., a real-world case was seen with only 5K
files on OS X).
2. Even when it does work, it's really slow. The pathspec
code is not optimized for huge numbers of paths. Here's
the same case without the ulimit:
$ time git add -p drivers
No changes.
real 0m16.559s
user 0m53.140s
sys 0m0.220s
We can improve this by skipping "ls-files" completely, and
just feeding the original pathspecs to the diff commands.
This solution was discussed in 2010:
http://public-inbox.org/git/20100105041438.GB12574@coredump.intra.peff.net/
but at the time the diff code's pathspecs were more
primitive than those used by ls-files (e.g., they did not
support globs). Making the change would have caused a
user-visible regression, so we didn't.
Since then, the pathspec code has been unified, and the diff
commands natively understand pathspecs like '*.c'.
This patch implements that solution. That skips the
argument-list limits, and the result runs much faster:
$ time git add -p drivers
No changes.
real 0m0.149s
user 0m0.116s
sys 0m0.080s
There are two new tests. The first just exercises the
globbing behavior to confirm that we are not causing a
regression there. The second checks the actual argument
behavior using GIT_TRACE. We _could_ do it with the "ulimit
-s" trick, as above. But that would mean the test could only
run where "ulimit -s" works. And tests of that sort are
expensive, because we have to come up with enough files to
actually bust the limit (we can't just shrink the "128" down
infinitely, since it is also the in-program stack size).
Finally, two caveats and possibilities for future work:
a. This fixes one argument-list expansion, but there may
be others. In fact, it's very likely that if you run
"git add -i" and select a large number of modified
files that the script would try to feed them all to a
single git command.
In practice this is probably fine. The real issue here
is that the argument list was growing with the _total_
number of files, not the number of modified or selected
files.
b. If the repository contains filenames with literal wildcard
characters (e.g., "foo*"), the original code expanded
them via "ls-files" and then fed those wildcard names
to "diff-index", which would have treated them as
wildcards. This was a bug, which is now fixed (though
unless you really go through some contortions with
":(literal)", it's likely that your original pathspec
would match whatever the accidentally-expanded wildcard
would anyway).
So this takes us one step closer to working correctly
with files whose names contain wildcard characters, but
it's likely that others remain (e.g., if "git add -i"
feeds the selected paths to "git add").
Reported-by: Wincent Colaiuta <win@wincent.com>
Reported-by: Mislav Marohnić <mislav.marohnic@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When invoked as "git add -i", each menu interactive menu
option prompts the user to select a list of files. This
includes the "patch" option, which gets the list before
starting the hunk-selection loop.
As "git add -p", it behaves differently, and jumps straight
to the hunk selection loop.
Since 0539d5e6d (i18n: add--interactive: mark patch prompt
for translation, 2016-12-14), the "add -i" case mistakenly
jumps to straight to the hunk-selection loop. Prior to that
commit the distinction between the two cases was managed by
the $patch_mode variable. That commit used $patch_mode for
something else, and moved the old meaning to the "$cmd"
variable. But it forgot to update the $patch_mode check
inside patch_update_cmd() which controls the file-list
behavior.
The simplest fix would be to change that line to check $cmd.
But while we're here, let's use a less obscure name for this
flag: $patch_mode_only, a boolean which tells whether we are
in full-interactive mode or only in patch-mode.
Reported-by: Henrik Grubbström <grubba@grubba.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Prior to c2f41bf52 (color.c: fix color_parse_mem() with
value_len == 0, 2017-01-19), the empty string was
interpreted as a color "reset". This was an accidental
outcome, and that commit turned it into an error.
However, scripts may pass the empty string as a default
value to "git config --get-color" to disable color when the
value is not defined. The git-add--interactive script does
this. As a result, the script is unusable since c2f41bf52
unless you have color.diff.plain defined (if it is defined,
then we don't parse the empty default at all).
Our test scripts didn't notice the recent breakage because
they run without a terminal, and thus without color. They
never hit this code path at all. And nobody noticed the
original buggy "reset" behavior, because it was effectively
a noop.
Let's fix the code to have an empty color name produce an
empty sequence of color codes. The tests need a few fixups:
- we'll add a new test in t4026 to cover this case. But
note that we need to tweak the color() helper. While
we're there, let's factor out the literal ANSI ESC
character. Otherwise it makes the diff quite hard to
read.
- we'll add a basic sanity-check in t4026 that "git add
-p" works at all when color is enabled. That would have
caught this bug, as well as any others that are specific
to the color code paths.
- 73c727d69 (log --graph: customize the graph lines with
config log.graphColors, 2017-01-19) added a test to
t4202 that checks some "invalid" graph color config.
Since ",, blue" before yielded only "blue" as valid, and
now yields "empty, empty, blue", we don't match the
expected output.
One way to fix this would be to change the expectation
to the empty color strings. But that makes the test much
less interesting, since we show only two graph lines,
both of which would be colorless.
Since the empty-string case is now covered by t4026,
let's remove them entirely here. They're just in the way
of the primary thing the test is supposed to be
checking.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test passes if one replaces the 'e' command with a 'y' command in
the 'add -p' session.
Reported-by: Tanky Woo <wtq1990@gmail.com>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This way, test authors don't need to remember to source
lib-prereq-FILEMODE.sh before using the FILEMODE prereq to guard tests
that rely on the executable bit being honored when checking out files.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is too easy to forget to add the PERL prerequisite for new
"add -i" tests, especially given that many people do not test with
NO_PERL so the missing prereq is not always noticed quickly.
The test had used the skip_all mechanism since 1b19ccd2 (2009-04-03)
but switched to explicit PERL prereqs in f0459319 (2010-10-13) in hope
of helping people see how many tests were skipped, perhaps to motivate
them to tweak their platform or tests to improve test coverage. That
didn't pan out much in practice, so let's move back to the simpler
skip_all method.
Reported-by: Kacper Kornet <draenog@pld-linux.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of these were found using Lucas De Marchi's codespell tool.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When "add -p" sees an unmerged entry, it shows the combined
diff and then immediately skips the hunk. This can be
confusing in a variety of ways, depending on whether there
are other changes to stage (in which case you get the
superfluous combined diff output in between other hunks) or
not (in which case you get the combined diff and the program
exits immediately, rather than seeing "No changes").
The current behavior was not planned, and is just what the
implementation happens to do. Instead, let's explicitly
remove unmerged entries from our list of modified files, and
print a warning that we are ignoring them.
We can cheaply find which entries are unmerged by adding
"--raw" output to the "diff-files --numstat" we already run.
There is one non-obvious thing we must change when parsing
this combined output. Before this patch, when we saw a
numstat line for a file that did not have index changes, we
would create a new record with 'unchanged' in the 'INDEX'
field. Because "--raw" comes before "--numstat", we must
move this special-case down to the raw-line case (and it is
sufficient to move it rather than handle it in both places,
since any file which has a --numstat will also have a --raw
entry).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/maint-add-p-overlapping-hunks:
t3701: add-p-fix makes the last test to pass
"add -p": work-around an old laziness that does not coalesce hunks
add--interactive.perl: factor out repeated --recount option
t3701: Editing a split hunk in an "add -p" session
add -p: 'q' should really quit
A broken here-document was not caught because end of file is taken by
an implicit end of the here document (POSIX does not seem to say it is
an error to lack the delimiter), and everything in the test just turned
into a single "cat into a file".
Noticed-by: Kacper Kornet <draenog@pld-linux.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Arnaud Lacombe reported that with the recent change to reject overlapping
hunks fed to "git apply", the edit mode of an "add -p" session that lazily
feeds overlapping hunks without coalescing adjacent ones claim that the
patch does not apply. Expose the problem to be fixed.
Cf. http://thread.gmane.org/gmane.comp.version-control.git/170685/focus=171000
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change this test to skip test with test prerequisites, and to do setup
work in tests. This improves the skipped statistics on platforms where
the test isn't run.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the five tests that were all checking "git config --bool
core.filemode" to use a new FILEMODE prerequisite in
lib-prereq-FILEMODE.sh.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
SKIP messages are now part of the TAP plan. A TAP harness now knows
why a particular test was skipped and can report that information. The
non-TAP harness built into Git's test-lib did nothing special with
these messages, and is unaffected by these changes.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 24ab81a fixed the deletion of empty files, but broke
deletion of non-empty files. The approach it took was to
factor out the "deleted" line from the patch header into its
own hunk, the same way we do for mode changes. However,
unlike mode changes, we only showed the special "delete this
file" hunk if there were no other hunks. Otherwise, the user
would annoyingly be presented with _two_ hunks: one for
deleting the file and one for deleting the content.
This meant that in the non-empty case, we forgot about the
deleted line entirely, and we submitted a bogus patch to
git-apply (with "/dev/null" as the destination file, but not
marked as a deletion).
Instead, this patch combines the file deletion hunk and the
content deletion hunk (if there is one) into a single
deletion hunk which is either staged or not.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Usually we show deletion as a big hunk deleting all of the
file's text. However, for files with no content, the diff
shows just the 'deleted file mode ...' line. This patch
cause "add -p" (and related commands) to recognize that line
and explicitly ask about deleting the file.
We only add the "stage this deletion" hunk for empty files,
since other files will already ask about the big content
deletion hunk. We could also change those files to simply
display "stage this deletion", but showing the actual
deleted content is probably what an interactive user wants.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make "git add -p" to not skip files that are in index even if they are
excluded (by .gitignore etc.). This fixes the contradictory behavior
that "git status" and "git commit -a" listed such files as modified, but
"git add -p FILENAME" ignored them.
Signed-off-by: Pauli Virtanen <pav@iki.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 0392513 (add-interactive: refactor mode hunk handling, 2009-04-16),
we merged the interaction loops for mode changes and hunk staging.
This was fine at the time, because 0beee4c (git-add--interactive:
remove hunk coalescing, 2008-07-02) removed hunk coalescing.
However, in 7a26e65 (Revert "git-add--interactive: remove hunk
coalescing", 2009-05-16), we resurrected it. Since then, the code
would attempt in vain to merge mode changes with diff hunks,
corrupting both in the process.
We add a check to the coalescing loop to ensure it only looks at diff
hunks, thus skipping mode changes.
Noticed-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When trying to stage changes to file which has also pending `chmod +x`,
`git add -p` produces lots of 'Use of uninitialized value ...' warnings
and fails to do the job:
$ echo content >> file
$ chmod +x file
$ git add -p
diff --git a/file b/file
index e69de29..d95f3ad
--- a/file
+++ b/file
old mode 100644
new mode 100755
Stage mode change [y,n,q,a,d,/,j,J,g,?]? y
@@ -0,0 +1 @@
+content
Stage this hunk [y,n,q,a,d,/,K,g,e,?]? y
Use of uninitialized value $o_ofs in addition (+) at .../git-add--interactive line 776.
Use of uninitialized value $ofs in numeric le (<=) at .../git-add--interactive line 806.
Use of uninitialized value $o0_ofs in concatenation (.) or string at .../git-add--interactive line 830.
Use of uninitialized value $n0_ofs in concatenation (.) or string at .../git-add--interactive line 830.
Use of uninitialized value $o_ofs in addition (+) at .../git-add--interactive line 776.
fatal: corrupt patch at line 5
diff --git a/file b/file
index e69de29..d95f3ad
--- a/file
+++ b/file
@@ -,0 + @@
+content
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are two tests that are skipped if file modes are not obeyed by the
file system. In this case, the subsequent test failed because the
repository was in an unexpected state. This corrects it.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 0beee4c6de but with a
bit of twist, as we have added "edit hunk manually" hack and we cannot
rely on the original line numbers of the hunks that were manually edited.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Splitting a hunk into two in add -p doesn't work for a diff that adds a
new line at the top of the file with other add in the same hunk.
Signed-off-by: Matthew Graham <mdg149@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These scripts all test git programs that are written in
perl, and thus obviously won't work if NO_PERL is defined.
We pass NO_PERL to the scripts from the building Makefile
via the GIT-BUILD-OPTIONS file.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Adds a new option 'e' to the 'add -p' command loop that lets you edit
the current hunk in your favourite editor.
If the resulting patch applies cleanly, the edited hunk will
immediately be marked for staging. If it does not apply cleanly, you
will be given an opportunity to edit again. If all lines of the hunk
are removed, then the edit is aborted and the hunk is left unchanged.
Applying the changed hunk(s) relies on Johannes Schindelin's new
--recount option for git-apply.
Note that the "real patch" test intentionally uses
(echo e; echo n; echo d) | git add -p
even though the 'n' and 'd' are superfluous at first sight. They
serve to get out of the interaction loop if git add -p wrongly
concludes the patch does not apply.
Many thanks to Jeff King <peff@peff.net> for lots of help and
suggestions.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When using the 'p'atch command, instead of just throwing out any mode
change, present it to the user in the same way that we show hunks.
This way, the mode change can be staged independently from the changes
to the contents.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a path is examined in the patch subcommand, any mode changes in
the file are given to use in the diff header by git-diff. If no hunks
are staged, then we throw out that header and do not touch the
path. But if _any_ hunks are staged, we use the header, and the mode
is changed together with the contents.
Since the 'p'atch command should just be dealing with hunks that are
shown to the user, it makes sense to just ignore mode changes
entirely. We do squirrel away the mode, though, since the next patch
will allow users to select the mode update separately.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Many scripts compare actual and expected output using
"diff -u". This is nicer than "cmp" because the output shows
how the two differ. However, not all versions of diff
understand -u, leading to unnecessary test failure.
This adds a test_cmp function to the test scripts and
switches all "diff -u" invocations to use it. The function
uses the contents of "$GIT_TEST_CMP" to compare its
arguments; the default is "diff -u".
On systems with a less-capable diff, you can do:
GIT_TEST_CMP=cmp make test
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There were several points where we looked at the HEAD
commit; for initial commits, this is meaningless. So instead
we:
- show staged status data as a diff against the empty tree
instead of HEAD
- show file diffs as creation events
- use "git rm --cached" to revert instead of going back to
the HEAD commit
We magically reference the empty tree to implement this.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
