Previous to commit 5d8f084a5 (pathspec: simpler logic to prefix original
pathspec elements, 2017-01-04), we were always using the computed
`match` variable to perform pathspec matching whenever
`PATHSPEC_PREFIX_ORIGIN` is set. This is for example useful when passing
the parsed pathspecs to other commands, as the computed `match` may
contain a pathspec relative to the repository root. The commit changed
this logic to only do so when we do have an actual prefix and when
literal pathspecs are deactivated.
But this change may actually break some commands which expect passed
pathspecs to be relative to the repository root. One such case is `git
add --patch`, which now fails when using relative paths from a
subdirectory. For example if executing "git add -p ../foo.c" in a
subdirectory, the `git-add--interactive` command will directly pass
"../foo.c" to `git-ls-files`. As ls-files is executed at the
repository's root, the command will notice that "../foo.c" is outside
the repository and fail.
Fix the issue by again using the computed `match` variable when
`PATHSPEC_PREFIX_ORIGIN` is set and global literal pathspecs are
deactivated. Note that in contrast to previous behavior, we will now
always call `prefix_magic` regardless of whether a prefix is actually
set. But this is the right thing to do: when the `match` variable has
been resolved to the repository's root, it will be set to an empty
string. When passing the empty string directly to other commands, it
will result in a warning regarding deprecated empty pathspecs. By always
adding the prefix magic, we will end up with at least the string
":(prefix:0)" and thus avoid the warning.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Acked-by: Brandon Williams <bmwill@google.com>
Reviewed-by: Duy Nguyen <pclouds@gmail.com>
"git add -p <pathspec>" unnecessarily expanded the pathspec to a
list of individual files that matches the pathspec by running "git
ls-files <pathspec>", before feeding it to "git diff-index" to see
which paths have changes, because historically the pathspec
language supported by "diff-index" was weaker. These days they are
equivalent and there is no reason to internally expand it. This
helps both performance and avoids command line argument limit on
some platforms.
* jk/add-i-use-pathspecs:
add--interactive: do not expand pathspecs with ls-files
"Dumb http" transport used to misparse a nonsense http-alternates
response, which has been fixed.
* jk/http-walker-buffer-underflow-fix:
http-walker: fix buffer underflow processing remote alternates
The pathspec mechanism learned to further limit the paths that
match the pattern to those that have specified attributes attached
via the gitattributes mechanism.
* bw/attr-pathspec:
pathspec: allow escaped query values
pathspec: allow querying for attributes
"git tag --contains" used to (ab)use the object bits to keep track
of the state of object reachability without clearing them after
use; this has been cleaned up and made to use the newer commit-slab
facility.
* jk/ref-filter-flags-cleanup:
ref-filter: use separate cache for contains_tag_algo
ref-filter: die on parse_commit errors
ref-filter: use contains_result enum consistently
ref-filter: move ref_cbdata definition into ref-filter.c
From a working tree of a repository, a new option of "rev-parse"
lets you ask if the repository is used as a submodule of another
project, and where the root level of the working tree of that
project (i.e. your superproject) is.
* sb/rev-parse-show-superproject-root:
rev-parse: add --show-superproject-working-tree
Picking two versions of Git and running tests to make sure the
older one and the newer one interoperate happily has now become
possible.
* jk/interop-test:
t/interop: add test of old clients against modern git-daemon
t: add an interoperability test harness
The experimental "split index" feature has gained a few
configuration variables to make it easier to use.
* cc/split-index-config: (22 commits)
Documentation/git-update-index: explain splitIndex.*
Documentation/config: add splitIndex.sharedIndexExpire
read-cache: use freshen_shared_index() in read_index_from()
read-cache: refactor read_index_from()
t1700: test shared index file expiration
read-cache: unlink old sharedindex files
config: add git_config_get_expiry() from gc.c
read-cache: touch shared index files when used
sha1_file: make check_and_freshen_file() non static
Documentation/config: add splitIndex.maxPercentChange
t1700: add tests for splitIndex.maxPercentChange
read-cache: regenerate shared index if necessary
config: add git_config_get_max_percent_split_change()
Documentation/git-update-index: talk about core.splitIndex config var
Documentation/config: add information for core.splitIndex
t1700: add tests for core.splitIndex
update-index: warn in case of split-index incoherency
read-cache: add and then use tweak_split_index()
split-index: add {add,remove}_split_index() functions
config: add git_config_get_split_index()
...
Git v2.12 was shipped with an embarrassing breakage where various
operations that verify paths given from the user stopped dying when
seeing an issue, and instead later triggering segfault.
... and then to down to 'maint'.
* js/realpath-pathdup-fix:
real_pathdup(): fix callsites that wanted it to die on error
t1501: demonstrate NULL pointer access with invalid GIT_WORK_TREE
Code clean-up and a string truncation fix.
* mm/two-more-xstrfmt:
bisect_next_all: convert xsnprintf to xstrfmt
stop_progress_msg: convert xsnprintf to xstrfmt
The command-line parsing of "git log -L" copied internal data
structures using incorrect size on ILP32 systems.
* vn/line-log-memcpy-size-fix:
line-log: use COPY_ARRAY to fix mis-sized memcpy
The code to parse "git log -L..." command line was buggy when there
are many ranges specified with -L; overrun of the allocated buffer
has been fixed.
* ax/line-log-range-merge-fix:
line-log.c: prevent crash during union of too many ranges
The patch subcommand of "git add -i" was meant to have paths
selection prompt just like other subcommand, unlike "git add -p"
directly jumps to hunk selection. Recently, this was broken and
"add -i" lost the paths selection dialog, but it now has been
fixed.
* jk/add-i-patch-do-prompt:
add--interactive: fix missing file prompt for patch mode with "-i"
When a redirected http transport gets an error during the
redirected request, we ignored the error we got from the server,
and ended up giving a not-so-useful error message.
* jt/http-base-url-update-upon-redirect:
http: attempt updating base URL only if no error
Reduce authentication round-trip over HTTP when the server supports
just a single authentication method.
* jk/http-auth:
http: add an "auto" mode for http.emptyauth
http: restrict auth methods to what the server advertises
"git branch --list" takes the "--abbrev" and "--no-abbrev" options
to control the output of the object name in its "-v"(erbose)
output, but a recent update started ignoring them; this fixes it
before the breakage reaches to any released version.
* kn/ref-filter-branch-list:
branch: honor --abbrev/--no-abbrev in --list mode
"git push" had a handful of codepaths that could lead to a deadlock
when unexpected error happened, which has been fixed.
* jk/push-deadlock-regression-fix:
send-pack: report signal death of pack-objects
send-pack: read "unpack" status even on pack-objects failure
send-pack: improve unpack-status error messages
send-pack: use skip_prefix for parsing unpack status
send-pack: extract parsing of "unpack" response
receive-pack: fix deadlock when we cannot create tmpdir
Recent versions of Git treats http alternates (used in dumb http
transport) just like HTTP redirects and requires the client to
enable following it, due to security concerns. But we forgot to
give a warning when we decide not to honor the alternates.
* ew/http-alternates-as-redirects-warning:
http: release strbuf on disabled alternates
http: inform about alternates-as-redirects behavior
"git filter-branch --prune-empty" drops a single-parent commit that
becomes a no-op, but did not drop a root commit whose tree is empty.
* dp/filter-branch-prune-empty:
p7000: add test for filter-branch with --prune-empty
filter-branch: fix --prune-empty on parentless commits
t7003: ensure --prune-empty removes entire branch when applicable
t7003: ensure --prune-empty can prune root commit
The t/perf performance test suite was not prepared to test not so
old versions of Git, but now it covers versions of Git that are not
so ancient.
* jt/perf-updates:
t/perf: add fallback for pre-bin-wrappers versions of git
t/perf: use $MODERN_GIT for all repo-copying steps
t/perf: export variable used in other blocks
"git fetch" that requests a commit by object name, when the other
side does not allow such an request, failed without much
explanation.
* mm/fetch-show-error-message-on-unadvertised-object:
fetch-pack: add specific error for fetching an unadvertised object
fetch_refs_via_pack: call report_unmatched_refs
fetch-pack: move code to report unmatched refs to a function
"git branch @" created refs/heads/@ as a branch, and in general the
code that handled @{-1} and @{upstream} was a bit too loose in
disambiguating.
* jk/interpret-branch-name:
checkout: restrict @-expansions when finding branch
strbuf_check_ref_format(): expand only local branches
branch: restrict @-expansions when deleting
t3204: test git-branch @-expansion corner cases
interpret_branch_name: allow callers to restrict expansions
strbuf_branchname: add docstring
strbuf_branchname: drop return value
interpret_branch_name: move docstring to header file
interpret_branch_name(): handle auto-namelen for @{-1}
A few tests were run conditionally under (rare) conditions where
they cannot be run (like running cvs tests under 'root' account).
* ab/cond-skip-tests:
gitweb tests: skip tests when we don't have Time::HiRes
gitweb tests: change confusing "skip_all" phrasing
cvs tests: skip tests that call "cvs commit" when running as root
When we want to get the list of modified files, we first
expand any user-provided pathspecs with "ls-files", and then
feed the resulting list of paths as arguments to
"diff-index" and "diff-files". If your pathspec expands into
a large number of paths, you may run into one of two
problems:
1. The OS may complain about the size of the argument
list, and refuse to run. For example:
$ (ulimit -s 128 && git add -p drivers)
Can't exec "git": Argument list too long at .../git-add--interactive line 177.
Died at .../git-add--interactive line 177.
That's on the linux.git repository, which has about 20K
files in the "drivers" directory (none of them modified
in this case). The "ulimit -s" trick is necessary to
show the problem on Linux even for such a gigantic set
of paths. Other operating systems have much smaller
limits (e.g., a real-world case was seen with only 5K
files on OS X).
2. Even when it does work, it's really slow. The pathspec
code is not optimized for huge numbers of paths. Here's
the same case without the ulimit:
$ time git add -p drivers
No changes.
real 0m16.559s
user 0m53.140s
sys 0m0.220s
We can improve this by skipping "ls-files" completely, and
just feeding the original pathspecs to the diff commands.
This solution was discussed in 2010:
http://public-inbox.org/git/20100105041438.GB12574@coredump.intra.peff.net/
but at the time the diff code's pathspecs were more
primitive than those used by ls-files (e.g., they did not
support globs). Making the change would have caused a
user-visible regression, so we didn't.
Since then, the pathspec code has been unified, and the diff
commands natively understand pathspecs like '*.c'.
This patch implements that solution. That skips the
argument-list limits, and the result runs much faster:
$ time git add -p drivers
No changes.
real 0m0.149s
user 0m0.116s
sys 0m0.080s
There are two new tests. The first just exercises the
globbing behavior to confirm that we are not causing a
regression there. The second checks the actual argument
behavior using GIT_TRACE. We _could_ do it with the "ulimit
-s" trick, as above. But that would mean the test could only
run where "ulimit -s" works. And tests of that sort are
expensive, because we have to come up with enough files to
actually bust the limit (we can't just shrink the "128" down
infinitely, since it is also the in-program stack size).
Finally, two caveats and possibilities for future work:
a. This fixes one argument-list expansion, but there may
be others. In fact, it's very likely that if you run
"git add -i" and select a large number of modified
files that the script would try to feed them all to a
single git command.
In practice this is probably fine. The real issue here
is that the argument list was growing with the _total_
number of files, not the number of modified or selected
files.
b. If the repository contains filenames with literal wildcard
characters (e.g., "foo*"), the original code expanded
them via "ls-files" and then fed those wildcard names
to "diff-index", which would have treated them as
wildcards. This was a bug, which is now fixed (though
unless you really go through some contortions with
":(literal)", it's likely that your original pathspec
would match whatever the accidentally-expanded wildcard
would anyway).
So this takes us one step closer to working correctly
with files whose names contain wildcard characters, but
it's likely that others remain (e.g., if "git add -i"
feeds the selected paths to "git add").
Reported-by: Wincent Colaiuta <win@wincent.com>
Reported-by: Mislav Marohnić <mislav.marohnic@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In our own .gitattributes file we have attributes such as:
*.[ch] whitespace=indent,trail,space
When querying for attributes we want to be able to ask for the exact
value, i.e.
git ls-files :(attr:whitespace=indent,trail,space)
should work, but the commas are used in the attr magic to introduce
the next attr, such that this query currently fails with
fatal: Invalid pathspec magic 'trail' in ':(attr:whitespace=indent,trail,space)'
This change allows escaping characters by a backslash, such that the query
git ls-files :(attr:whitespace=indent\,trail\,space)
will match all path that have the value "indent,trail,space" for the
whitespace attribute. To accomplish this, we need to modify two places.
First `parse_long_magic` needs to not stop early upon seeing a comma or
closing paren that is escaped. As a second step we need to remove any
escaping from the attr value.
Based on a patch by Stefan Beller <sbeller@google.com>
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The pathspec mechanism is extended via the new
":(attr:eol=input)pattern/to/match" syntax to filter paths so that it
requires paths to not just match the given pattern but also have the
specified attrs attached for them to be chosen.
Based on a patch by Stefan Beller <sbeller@google.com>
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we parse a remote alternates (or http-alternates), we
expect relative lines like:
../../foo.git/objects
which we convert into "$URL/../foo.git/" (and then use that
as a base for fetching more objects).
But if the remote feeds us nonsense like just:
../
we will try to blindly strip the last 7 characters, assuming
they contain the string "objects". Since we don't _have_ 7
characters at all, this results in feeding a small negative
value to strbuf_add(), which converts it to a size_t,
resulting in a big positive value. This should consistently
fail (since we can't generall allocate the max size_t minus
7 bytes), so there shouldn't be any security implications.
Let's fix this by using strbuf_strip_suffix() to drop the
characters we want. If they're not present, we'll ignore the
alternate (in theory we could use it as-is, but the rest of
the http-walker code unconditionally tacks "objects/" back
on, so it is it not prepared to handle such a case).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git diff --quiet" relies on the size field in diff_filespec to be
correctly populated, but diff_populate_filespec() helper function
made an incorrect short-cut when asked only to populate the size
field for paths that need to go through convert_to_git() (e.g. CRLF
conversion).
* jc/diff-populate-filespec-size-only-fix:
diff: do not short-cut CHECK_SIZE_ONLY check in diff_populate_filespec()
The command-line parsing of "git log -L" copied internal data
structures using incorrect size on ILP32 systems.
* vn/line-log-memcpy-size-fix:
line-log: use COPY_ARRAY to fix mis-sized memcpy
The code to parse "git log -L..." command line was buggy when there
are many ranges specified with -L; overrun of the allocated buffer
has been fixed.
* ax/line-log-range-merge-fix:
line-log.c: prevent crash during union of too many ranges
There is no need for Python only to give a few messages to the
standard error stream, but we somehow did.
* ss/remote-bzr-hg-placeholder-wo-python:
contrib: git-remote-{bzr,hg} placeholders don't need Python
Git v2.12 was shipped with an embarrassing breakage where various
operations that verify paths given from the user stopped dying when
seeing an issue, and instead later triggering segfault.
* js/realpath-pathdup-fix:
real_pathdup(): fix callsites that wanted it to die on error
t1501: demonstrate NULL pointer access with invalid GIT_WORK_TREE
The patch subcommand of "git add -i" was meant to have paths
selection prompt just like other subcommand, unlike "git add -p"
directly jumps to hunk selection. Recently, this was broken and
"add -i" lost the paths selection dialog, but it now has been
fixed.
* jk/add-i-patch-do-prompt:
add--interactive: fix missing file prompt for patch mode with "-i"
All callers of add_blame_entry() allocate and copy the second argument.
Let the function do it for them, reducing code duplication.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This test just checks that old clients can clone and fetch
from a newer git-daemon. The opposite should also be true,
but it's hard to test ancient versions of git-daemon because
they lack basic options like "--listen".
Note that we have to make a slight tweak to the
lib-git-daemon helper from the regular tests, so that it
starts the daemon with our correct git.a version.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current test suite is good at letting you test a
particular version of Git. But it's not very good at letting
you test _two_ versions and seeing how they interact (e.g.,
one cloning from the other).
This commit adds a test harness that will build two
arbitrary versions of git and make it easy to call them from
inside your tests. See the README and the example script for
details.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
