Commit Graph

57226 Commits

Author SHA1 Message Date
Denton Liu
b1b16bba96 completion: add --skip for cherry-pick and revert
Even though `--skip` is a valid command-line option for cherry-pick and
revert while they are in progress, it is not completed. Add this missing
option to the completion script.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-27 14:47:09 -07:00
Denton Liu
deaa65a754 completion: merge options for cherry-pick and revert
Since revert and cherry-pick share the same sequencer code, they should
both accept the same command-line options. Derive the
`__git_cherry_pick_inprogress_options` and
`__git_revert_inprogress_options` variables from
`__git_sequencer_inprogress_options` so that the options aren't
unnecessarily duplicated twice.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-27 14:47:08 -07:00
Ævar Arnfjörð Bjarmason
4336d36512 t3432: distinguish "noop-same" v.s. "work-same" in "same head" tests
Change "same head" introduced in the preceding commit to check whether
the rebase.c code lands in the can_fast_forward() case in, and thus
prints out an "is up to date" and aborts early.

In some of these cases we make it past that and to "rewinding head",
then do a rebase, only to find out there's nothing to change so HEAD
stays at the same OID.

These tests presumed these two cases were the same thing. In terms of
where HEAD ends up they are, but we're not only interested in rebase
semantics, but also whether or not we're needlessly doing work when we
could avoid it entirely.

I'm adding "same" and "diff" here because I'll follow-up and add
--no-ff tests, where some of those will be "diff"-erent, so add the
"diff" code already.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 12:34:56 -07:00
Denton Liu
793ac7e309 t3432: test rebase fast-forward behavior
When rebase is run on a branch that can be fast-forwarded, this should
automatically be done. Create test to ensure this behavior happens.

There are some cases that currently don't pass. The first case is where
a feature and master have diverged, running
"git rebase master... master" causes a full rebase to happen even though
a fast-forward should happen.

The second case is when we are doing "git rebase --fork-point" and a
fork-point commit is found. Once again, a full rebase happens even
though a fast-forward should happen.

Mark these cases as failure so we can fix it later.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 12:34:56 -07:00
Denton Liu
359ecebc34 t3431: add rebase --fork-point tests
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 12:34:56 -07:00
SZEDER Gábor
502c386ff9 t7300-clean: demonstrate deleting nested repo with an ignored file breakage
'git clean -fd' must not delete an untracked directory if it belongs
to a different Git repository or worktree.  Unfortunately, if a
'.gitignore' rule in the outer repository happens to match a file in a
nested repository or worktree, then something goes awry and 'git clean
-fd' does delete the content of the nested repository's worktree
except that ignored file, potentially leading to data loss.

Add a test to 't7300-clean.sh' to demonstrate this breakage.

This issue is a regression introduced in 6b1db43109 (clean: teach
clean -d to preserve ignored paths, 2017-05-23).

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 12:01:33 -07:00
Carlo Marcelo Arenas Belón
ff61681b46 grep: refactor and simplify PCRE1 support
The code used both a macro and a variable to keep track if JIT
support was desired and relied on the fact that a non JIT
enabled library will ignore a request for JIT compilation
(as defined by the second parameter of the call to pcre_study)

Cleanup the multiple levels of macros used and call pcre_study
with the right parameter after JIT support has been confirmed
and unless it was requested to be disabled with NO_LIBPCRE1_JIT

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 11:37:02 -07:00
Carlo Marcelo Arenas Belón
8991da6a38 grep: make sure NO_LIBPCRE1_JIT disable JIT in PCRE1
e87de7cab4 ("grep: un-break building with PCRE < 8.32", 2017-05-25)
added a restriction for JIT support that is no longer needed after
pcre_jit_exec() calls were removed.

Reorganize the definitions in grep.h so that JIT support could be
detected early and NO_LIBPCRE1_JIT could be used reliably to enforce
JIT doesn't get used.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 11:37:01 -07:00
René Scharfe
1fd881d404 trace2: use warning() directly in tr2_dst_malformed_warning()
Let warning() format the message instead of using an intermediate strbuf
for that.  This is shorter, easier to read and avoids an allocation.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 11:34:23 -07:00
René Scharfe
fd99c2dd9b grep: use return value of strbuf_detach()
Append the strbuf buffer only after detaching it.  There is no practical
difference here, as the strbuf is not empty and no strbuf_ function is
called between storing the pointer to the still attached buffer and
calling strbuf_detach(), so that pointer is valid, but make sure to
follow the standard sequence anyway for consistency.

Signed-off-by: René Scharfe <l.s.r@web.de>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 11:32:28 -07:00
René Scharfe
82f51af345 log-tree: always use return value of strbuf_detach()
strbuf_detach() has been returning a pointer to a buffer even for empty
strbufs since 08ad56f3f0 ("strbuf: always return a non-NULL value from
strbuf_detach", 2012-10-18).  Use that feature in show_log() instead of
having it handle empty strbufs specially.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 11:29:43 -07:00
Mike Hommey
7e92756751 http: don't leak urlmatch_config.vars
Signed-off-by: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 11:10:09 -07:00
Mike Hommey
9784f97321 commit: free the right buffer in release_commit_memory
The index field in the commit object is used to find the buffer
corresponding to that commit in the buffer_slab. Resetting it first
means free_commit_buffer is not going to free the right buffer.

Signed-off-by: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:53:25 -07:00
brian m. carlson
ce17feb1b3 path: add a function to check for path suffix
We have a function to strip the path suffix from a commit, but we don't
have one to check for a path suffix. For a plain filename, we can use
basename, but that requires an allocation, since POSIX allows it to
modify its argument. Refactor strip_path_suffix into a helper function
and a new function, ends_with_path_components, to meet this need.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:36:13 -07:00
Taylor Blau
60d198d022 banned.h: fix vsprintf()'s ban message
In cc8fdaee1e (banned.h: mark sprintf() as banned, 2018-07-24), both
'sprintf()' and 'vsprintf()' were marked as banned functions. The
non-variadic macro to ban 'vsprintf' has a typo which says that
'sprintf', not 'vsprintf' is banned. The variadic version does not have
the same typo.

Fix this by updating the explicit form of 'vsprintf' as the banned
version of itself, not 'sprintf'.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:32:45 -07:00
Jeff King
60fe477a0b notes: avoid potential use-after-free during insertion
The note_tree_insert() function may free the leaf_node struct we pass in
(e.g., if it's a duplicate, or if it needs to be combined with an
existing note).

Most callers are happy with this, as they assume that ownership of the
struct is handed off. But in load_subtree(), if we see an error we'll
use the handed-off struct's key_oid to generate the die() message,
potentially accessing freed memory.

We can easily fix this by instead using the original oid that we copied
into the leaf_node struct.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:29:56 -07:00
Mike Hommey
779ad6641b notes: avoid leaking duplicate entries
When add_note is called multiple times with the same key/value pair, the
leaf_node it creates is leaked by notes_tree_insert.

Signed-off-by: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:29:38 -07:00
Adam Roben
4e1a641ee3 mingw: fix launching of externals from Unicode paths
If Git were installed in a path containing non-ASCII characters,
commands such as `git am` and `git submodule`, which are implemented as
externals, would fail to launch with the following error:

> fatal: 'am' appears to be a git command, but we were not
> able to execute it. Maybe git-am is broken?

This was due to lookup_prog not being Unicode-aware. It was somehow
missed in 85faec9d3a (Win32: Unicode file name support (except dirent),
2012-03-15).

Note that the only problem in this function was calling
`GetFileAttributes()` instead of `GetFileAttributesW()`. The calls to
`access()` were fine because `access()` is a macro which resolves to
`mingw_access()`, which already handles Unicode correctly. But
`lookup_prog()` was changed to use `_waccess()` directly so that we only
convert the path to UTF-16 once.

To make things work correctly, we have to maintain UTF-8 and UTF-16
versions in tandem in `lookup_prog()`.

Signed-off-by: Adam Roben <adam@roben.org>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:06:03 -07:00
Johannes Schindelin
5cf7b3b1ac setup_git_directory(): handle UNC root paths correctly
When working in the root directory of a file share (this is only
possible in Git Bash and Powershell, but not in CMD), the current
directory is reported without a trailing slash.

This is different from Unix and standard Windows directories: both / and
C:\ are reported with a trailing slash as current directories.

If a Git worktree is located there, Git is not quite prepared for that:
while it does manage to find the .git directory/file, it returns as
length of the top-level directory's path *one more* than the length of
the current directory, and setup_git_directory_gently() would then
return an undefined string as prefix.

In practice, this undefined string usually points to NUL bytes, and does
not cause much harm. Under rare circumstances that are really involved
to reproduce (and not reliably so), the reported prefix could be a
suffix string of Git's exec path, though.

A careful analysis determined that this bug is unlikely to be
exploitable, therefore we mark this as a regular bug fix.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:03:41 -07:00
Johannes Schindelin
e2683d51d9 Fix .git/ discovery at the root of UNC shares
A very common assumption in Git's source code base is that
offset_1st_component() returns either 0 for relative paths, or 1 for
absolute paths that start with a slash. In other words, the return value
is either 0 or points just after the dir separator.

This assumption is not fulfilled when calling offset_1st_component()
e.g. on UNC paths on Windows, e.g. "//my-server/my-share". In this case,
offset_1st_component() returns the length of the entire string (which is
correct, because stripping the last "component" would not result in a
valid directory), yet the return value still does not point just after a
dir separator.

This assumption is most prominently seen in the
setup_git_directory_gently_1() function, where we want to append a
".git" component and simply assume that there is already a dir
separator. In the UNC example given above, this assumption is incorrect.

As a consequence, Git will fail to handle a worktree at the top of a UNC
share correctly.

Let's fix this by adding a dir separator specifically for that case: we
found that there is no first component in the path and it does not end
in a dir separator? Then add it.

This fixes https://github.com/git-for-windows/git/issues/1320

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:03:41 -07:00
Johannes Schindelin
d17f2124a7 setup_git_directory(): handle UNC paths correctly
The first offset in a UNC path is not the host name, but the folder name after that.

This fixes https://github.com/git-for-windows/git/issues/1181

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:03:41 -07:00
Torsten Bögershausen
ebb8d2c90f mingw: support UNC in git clone file://server/share/repo
Extend the parser to accept file://server/share/repo in the way that
Windows users expect it to be parsed who are used to referring to file
shares by UNC paths of the form \\server\share\folder.

[jes: tightened check to avoid handling file://C:/some/path as a UNC
path.]

This closes https://github.com/git-for-windows/git/issues/1264.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 10:00:33 -07:00
brian m. carlson
0c37c41d13 t4009: make hash size independent
Instead of hard-coding object IDs, compute them and use those in the
comparison.  Note that the comparison code ignores the actual object
IDs, but does check that they're the right size, so computing them is
the easiest way to ensure that they are.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 08:55:45 -07:00
brian m. carlson
c1f3dfcc23 t4002: make hash independent
Factor out the hard-coded object IDs and use test_oid to provide values
for both SHA-1 and SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 08:55:45 -07:00
brian m. carlson
8cc5ff83c3 t4000: make hash size independent
Use $ZERO_OID instead of hard-coding a fixed size all-zeros object ID.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 08:55:45 -07:00
brian m. carlson
c784815073 t3903: abstract away SHA-1-specific constants
Abstract away the SHA-1-specific constants by sanitizing diff output to
remove the index lines, since it's clear from the assertions in question
that we are not interested in the specific object IDs.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26 08:55:45 -07:00
Pratyush Yadav
2ccdfb1c78 git-gui: return early when patch fails to apply
In the procedure apply_or_revert_range_or_line, if the patch does not
apply successfully, a dialog is shown, but execution proceeds after
that. Instead, return early on error so the parts that come after this
don't work on top of an error state.

Signed-off-by: Pratyush Yadav <me@yadavpratyush.com>
2019-08-26 04:23:13 +05:30
Pratyush Yadav
62bd99934b git-gui: allow reverting selected hunk
Just like the user can select a hunk to stage or unstage, add the
ability to revert hunks.

Signed-off-by: Pratyush Yadav <me@yadavpratyush.com>
2019-08-26 03:53:15 +05:30
Pratyush Yadav
5f0a516de9 git-gui: allow reverting selected lines
Just like the user can select lines to stage or unstage, add the
ability to revert selected lines.

Signed-off-by: Pratyush Yadav <me@yadavpratyush.com>
2019-08-26 01:35:27 +05:30
Jonathan Tan
fddf2ebe38 transport: teach all vtables to allow fetch first
The only transport that does not allow fetch() to be called before
get_refs_list() is the bundle transport. Clean up the code by teaching
the bundle transport the ability to do this, and removing support for
transports that don't support this order of invocation.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-22 14:20:39 -07:00
Jonathan Tan
ac3fda82bf transport-helper: skip ls-refs if unnecessary
Commit e70a3030e7 ("fetch: do not list refs if fetching only hashes",
2018-10-07) and its ancestors taught Git, as an optimization, to skip
the ls-refs step when it is not necessary during a protocol v2 fetch
(for example, when lazy fetching a missing object in a partial clone, or
when running "git fetch --no-tags <remote> <SHA-1>"). But that was only
done for natively supported protocols; in particular, HTTP was not
supported.

Teach Git to skip ls-refs when using remote helpers that support connect
or stateless-connect. To do this, fetch() is made an acceptable entry
point. Because fetch() can now be the first function in the vtable
called, "get_helper(transport);" has to be added to the beginning of
that function to set the transport up (if not yet set up) before
process_connect() is invoked.

When fetch() is called, the transport could be taken over (this happens
if "connect" or "stateless-connect" is successfully run without any
"fallback" response), or not. If the transport is taken over, execution
continues like execution for natively supported protocols
(fetch_refs_via_pack() is executed, which will fetch refs using ls-refs
if needed). If not, the remote helper interface will invoke
get_refs_list() if it hasn't been invoked yet, preserving existing
behavior.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-22 14:20:35 -07:00
Junio C Hamano
745f681289 First batch after Git 2.23
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-22 12:41:04 -07:00
Junio C Hamano
d4b12b9e07 Merge branch 'sg/worktree-remove-errormsg'
Error message update/clarification.

* sg/worktree-remove-errormsg:
  worktree remove: clarify error message on dirty worktree
2019-08-22 12:34:12 -07:00
Junio C Hamano
22e86e85cb Merge branch 'en/fast-import-merge-doc'
Doc update.

* en/fast-import-merge-doc:
  git-fast-import.txt: clarify that multiple merge commits are allowed
2019-08-22 12:34:12 -07:00
Junio C Hamano
9c7573581c Merge branch 'jk/perf-no-dups'
Test & perf scripts must use unique numeric prefix, but a pair
shared the same number, which is fixed here.

* jk/perf-no-dups:
  t/perf: rename duplicate-numbered test script
2019-08-22 12:34:11 -07:00
Junio C Hamano
4336fdb2ef Merge branch 'rs/nedalloc-fixlets'
Compilation fix.

* rs/nedalloc-fixlets:
  nedmalloc: avoid compiler warning about unused value
  nedmalloc: do assignments only after the declaration section
2019-08-22 12:34:11 -07:00
Junio C Hamano
8ae7a46c4d Merge branch 'sg/show-failed-test-names'
The first line of verbose output from each test piece now carries
the test name and number to help scanning with eyeballs.

* sg/show-failed-test-names:
  tests: show the test name and number at the start of verbose output
  t0000-basic: use realistic test script names in the verbose tests
2019-08-22 12:34:11 -07:00
Junio C Hamano
6ba06b582b Merge branch 'sg/commit-graph-validate'
The code to write commit-graph over given commit object names has
been made a bit more robust.

* sg/commit-graph-validate:
  commit-graph: error out on invalid commit oids in 'write --stdin-commits'
  commit-graph: turn a group of write-related macro flags into an enum
  t5318-commit-graph: use 'test_expect_code'
2019-08-22 12:34:11 -07:00
Junio C Hamano
072735ea58 Merge branch 'vn/restore-empty-ita-corner-case-fix'
"git checkout" and "git restore" to re-populate the index from a
tree-ish (typically HEAD) did not work correctly for a path that
was removed and then added again with the intent-to-add bit, when
the corresponding working tree file was empty.  This has been
corrected.

* vn/restore-empty-ita-corner-case-fix:
  restore: add test for deleted ita files
  checkout.c: unstage empty deleted ita files
2019-08-22 12:34:11 -07:00
Junio C Hamano
207ad3cb20 Merge branch 'sc/pack-refs-deletion-racefix'
"git pack-refs" can lose refs that are created while running, which
is getting corrected.

* sc/pack-refs-deletion-racefix:
  pack-refs: always refresh after taking the lock file
2019-08-22 12:34:10 -07:00
Junio C Hamano
77067b6ce8 Merge branch 'sg/do-not-skip-non-httpd-tests'
Test fix.

* sg/do-not-skip-non-httpd-tests:
  t: warn against adding non-httpd-specific tests after sourcing 'lib-httpd'
  t5703: run all non-httpd-specific tests before sourcing 'lib-httpd.sh'
  t5510-fetch: run non-httpd-specific test before sourcing 'lib-httpd.sh'
2019-08-22 12:34:10 -07:00
Junio C Hamano
1b01cdbf2e Merge branch 'jk/tree-walk-overflow'
Codepaths to walk tree objects have been audited for integer
overflows and hardened.

* jk/tree-walk-overflow:
  tree-walk: harden make_traverse_path() length computations
  tree-walk: add a strbuf wrapper for make_traverse_path()
  tree-walk: accept a raw length for traverse_path_len()
  tree-walk: use size_t consistently
  tree-walk: drop oid from traverse_info
  setup_traverse_info(): stop copying oid
2019-08-22 12:34:10 -07:00
Junio C Hamano
8aa76abba5 Merge branch 'sg/t5510-test-i18ngrep-fix'
Test fix.

* sg/t5510-test-i18ngrep-fix:
  t5510-fetch: fix negated 'test_i18ngrep' invocation
2019-08-22 12:34:10 -07:00
Junio C Hamano
307179732d Merge branch 'mt/grep-submodules-working-tree'
"git grep --recurse-submodules" that looks at the working tree
files looked at the contents in the index in submodules, instead of
files in the working tree.

* mt/grep-submodules-working-tree:
  grep: fix worktree case in submodules
2019-08-22 12:34:10 -07:00
Thomas Gummerer
58166c2e9d t0021: make sure clean filter runs
In t0021.15 one of the things we are checking is that the clean filter
is run when checking out empty-branch.  The clean filter needs to be
run to make sure there are no modifications on the file system for the
test.r file, and thus it isn't dangerous to overwrite it.

However in the current test setup it is not always necessary to run
the clean filter, and thus the test sometimes fails, as debug.log
isn't written.

This happens when test.r has an older mtime than the index itself.
That mtime is also recorded as stat data for test.r in the index, and
based on the heuristic we're using for index entries, git correctly
assumes this file is up-to-date.

Usually this test succeeds because the mtime of test.r is the same as
the mtime of the index.  In this case test.r is racily clean, so git
actually checks the contents, for which the clean filter is run.

Fix the test by updating the mtime of test.r, so git is forced to
check the contents of the file, and the clean filter is run as the
test expects.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-22 12:32:59 -07:00
Mischa POSLAWSKY
8b3f33ef11 ref-filter: initialize empty name or email fields
Formatting $(taggername) on headerless tags such as v0.99 in Git
causes a SIGABRT with error "munmap_chunk(): invalid pointer",
because of an oversight in commit f0062d3b74 (ref-filter: free
item->value and item->value->s, 2018-10-19).

Signed-off-by: Mischa POSLAWSKY <git@shiar.nl>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-22 11:14:45 -07:00
Stephen Boyd
3c81760bc6 userdiff: add a builtin pattern for dts files
The Linux kernel receives many patches to the devicetree files each
release. The hunk header for those patches typically show nothing,
making it difficult to figure out what node is being modified without
applying the patch or opening the file and seeking to the context. Let's
add a builtin 'dts' pattern to git so that users can get better diff
output on dts files when they use the diff=dts driver.

The regex has been constructed based on the spec at devicetree.org[1]
and with some help from Johannes Sixt.

[1] https://github.com/devicetree-org/devicetree-specification/releases/latest

Cc: Rob Herring <robh+dt@kernel.org>
Cc: Frank Rowand <frowand.list@gmail.com>
Signed-off-by: Stephen Boyd <sboyd@kernel.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-21 15:09:34 -07:00
Denton Liu
fe49814409 t4014: drop unnecessary blank lines from test cases
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-21 10:56:44 -07:00
SZEDER Gábor
a2bb801f6a line-log: avoid unnecessary full tree diffs
With rename detection enabled the line-level log is able to trace the
evolution of line ranges across whole-file renames [1].  Alas, to
achieve that it uses the diff machinery very inefficiently, making the
operation very slow [2].  And since rename detection is enabled by
default, the line-level log is very slow by default.

When the line-level log processes a commit with rename detection
enabled, it currently does the following (see queue_diffs()):

  1. Computes a full tree diff between the commit and (one of) its
     parent(s), i.e. invokes diff_tree_oid() with an empty
     'diffopt->pathspec'.
  2. Checks whether any paths in the line ranges were modified.
  3. Checks whether any modified paths in the line ranges are missing
     in the parent commit's tree.
  4. If there is such a missing path, then calls diffcore_std() to
     figure out whether the path was indeed renamed based on the
     previously computed full tree diff.
  5. Continues doing stuff that are unrelated to the slowness.

So basically the line-level log computes a full tree diff for each
commit-parent pair in step (1) to be used for rename detection in step
(4) in the off chance that an interesting path is missing from the
parent.

Avoid these expensive and mostly unnecessary full tree diffs by
limiting the diffs to paths in the line ranges.  This is much cheaper,
and makes step (2) unnecessary.  If it turns out that an interesting
path is missing from the parent, then fall back and compute a full
tree diff, so the rename detection will still work.

Care must be taken when to update the pathspec used to limit the diff
in case of renames.  A path might be renamed on one branch and
modified on several parallel running branches, and while processing
commits on these branches the line-level log might have to alternate
between looking at a path's new and old name.  However, at any one
time there is only a single 'diffopt->pathspec'.

So add a step (0) to the above to ensure that the paths in the
pathspec match the paths in the line ranges associated with the
currently processed commit, and re-parse the pathspec from the paths
in the line ranges if they differ.

The new test cases include a specially crafted piece of history with
two merged branches and two files, where each branch modifies both
files, renames on of them, and then modifies both again.  Then two
separate 'git log -L' invocations check the line-level log of each of
those two files, which ensures that at least one of those invocations
have to do that back-and-forth between the file's old and new name (no
matter which branch is traversed first).  't/t4211-line-log.sh'
already contains two tests involving renames, they don't don't trigger
this back-and-forth.

Avoiding these unnecessary full tree diffs can have huge impact on
performance, especially in big repositories with big trees and mergy
history.  Tracing the evolution of a function through the whole
history:

  # git.git
  $ time git --no-pager log -L:read_alternate_refs:sha1-file.c v2.23.0

  Before:

    real    0m8.874s
    user    0m8.816s
    sys     0m0.057s

  After:

    real    0m2.516s
    user    0m2.456s
    sys     0m0.060s

  # linux.git
  $ time ~/src/git/git --no-pager log \
    -L:build_restore_work_registers:arch/mips/mm/tlbex.c v5.2

  Before:

    real    3m50.033s
    user    3m48.041s
    sys     0m0.300s

  After:

    real    0m2.599s
    user    0m2.466s
    sys     0m0.157s

That's just over 88x speedup.

[1] Line-level log's rename following is quite similar to 'git log
    --follow path', with the notable differences that it does handle
    multiple paths at once as well, and that it doesn't show the
    commit performing the rename if it's an exact rename.

[2] This slowness might not have been apparent initially, because back
    when the line-level log feature was introduced rename detection
    was not yet enabled by default; 12da1d1f6f (Implement line-history
    search (git log -L), 2013-03-28) and 5404c116aa (diff: activate
    diff.renames by default, 2016-02-25).

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-21 10:17:54 -07:00
SZEDER Gábor
eef5204190 line-log: extract pathspec parsing from line ranges into a helper function
A helper function to parse the paths involved in the line ranges and
to turn them into a pathspec will be useful in the next patch.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-21 10:17:52 -07:00