Commit Graph

7595 Commits

Author SHA1 Message Date
Paul-Sebastian Ungureanu
ef0f0b4509 stash: optimize get_untracked_files() and check_changes()
This commits introduces a optimization by avoiding calling the
same functions again. For example, `git stash push -u`
would call at some points the following functions:

 * `check_changes()` (inside `do_push_stash()`)
 * `do_create_stash()`, which calls: `check_changes()` and
`get_untracked_files()`

Note that `check_changes()` also calls `get_untracked_files()`.
So, `check_changes()` is called 2 times and `get_untracked_files()`
3 times.

The old function `check_changes()` now consists of two functions:
`get_untracked_files()` and `check_changes_tracked_files()`.

These are the call chains for `push` and `create`:

 * `push_stash()` -> `do_push_stash()` -> `do_create_stash()`

 * `create_stash()` -> `do_create_stash()`

To prevent calling the same functions over and over again,
`check_changes()` inside `do_create_stash()` is now placed
in the caller functions (`create_stash()` and `do_push_stash()`).
This way `check_changes()` and `get_untracked files()` are called
only one time.

Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Paul-Sebastian Ungureanu
64fe9c26a4 stash: convert save to builtin
Add stash save to the helper and delete functions which are no
longer needed (`show_help()`, `save_stash()`, `push_stash()`,
`create_stash()`, `clear_stash()`, `untracked_files()` and
`no_changes()`).

Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Paul-Sebastian Ungureanu
1ac528c0b0 stash: make push -q quiet
There is a change in behaviour with this commit. When there was
no initial commit, the shell version of stash would still display
a message. This commit makes `push` to not display any message if
`--quiet` or `-q` is specified. Add tests for `--quiet`.

Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Paul-Sebastian Ungureanu
d553f538b8 stash: convert push to builtin
Add stash push to the helper.

Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Paul-Sebastian Ungureanu
d4788af875 stash: convert create to builtin
Add stash create to the helper.

Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Helped-by: Matthew Kraai <mkraai@its.jnj.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Paul-Sebastian Ungureanu
41e0dd55c4 stash: convert store to builtin
Add stash store to the helper and delete the store_stash function
from the shell script.

Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Paul-Sebastian Ungureanu
dc7bd382b1 stash: convert show to builtin
Add stash show to the helper and delete the show_stash, have_stash,
assert_stash_like, is_stash_like and parse_flags_and_rev functions
from the shell script now that they are no longer needed.

In shell version, although `git stash show` accepts `--index` and
`--quiet` options, it ignores them. In C, both options are passed
further to `git diff`.

Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Paul-Sebastian Ungureanu
130f2697da stash: convert list to builtin
Add stash list to the helper and delete the list_stash function
from the shell script.

Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Joel Teichroeb
c4de61d7a9 stash: convert pop to builtin
Add stash pop to the helper and delete the pop_stash, drop_stash,
assert_stash_ref functions from the shell script now that they
are no longer needed.

Signed-off-by: Joel Teichroeb <joel@teichroeb.net>
Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Joel Teichroeb
577c1995e8 stash: convert branch to builtin
Add stash branch to the helper and delete the apply_to_branch
function from the shell script.

Checkout does not currently provide a function for checking out
a branch as cmd_checkout does a large amount of sanity checks
first that we require here.

Signed-off-by: Joel Teichroeb <joel@teichroeb.net>
Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Joel Teichroeb
4e2dd39303 stash: convert drop and clear to builtin
Add the drop and clear commands to the builtin helper. These two
are each simple, but are being added together as they are quite
related.

We have to unfortunately keep the drop and clear functions in the
shell script as functions are called with parameters internally
that are not valid when the commands are called externally. Once
pop is converted they can both be removed.

Signed-off-by: Joel Teichroeb <joel@teichroeb.net>
Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Joel Teichroeb
8a0fc8d19d stash: convert apply to builtin
Add a builtin helper for performing stash commands. Converting
all at once proved hard to review, so starting with just apply
lets conversion get started without the other commands being
finished.

The helper is being implemented as a drop in replacement for
stash so that when it is complete it can simply be renamed and
the shell script deleted.

Delete the contents of the apply_stash shell function and replace
it with a call to stash--helper apply until pop is also
converted.

Signed-off-by: Joel Teichroeb <joel@teichroeb.net>
Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:41:40 +09:00
Alban Gruin
ed35d18841 rebase--interactive: move transform_todo_file()
As transform_todo_file() is only needed inside of
rebase--interactive.c for `rebase -p', it is moved there from
sequencer.c.

The parameter r (repository) is dropped along the way.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:17:57 +09:00
Alban Gruin
a930eb03a8 rebase-interactive: rewrite edit_todo_list() to handle the initial edit
edit_todo_list() is changed to work on a todo_list, and to handle the
initial edition of the todo list (ie. making a backup of the todo
list).

It does not check for dropped commits yet, as todo_list_check() does not
take the commits that have already been processed by the rebase (ie. the
todo list is edited in the middle of a rebase session).

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:17:57 +09:00
Alban Gruin
79d7e883bb rebase--interactive: move rearrange_squash_in_todo_file()
As rearrange_squash_in_todo_file() is only needed inside of
rebase--interactive.c for `rebase -p', it is moved there from
sequencer.c.

The parameter r (repository) is dropped along the way, and the error
handling is slightly improved.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:17:57 +09:00
Alban Gruin
1ba204de69 rebase--interactive: move sequencer_add_exec_commands()
As sequencer_add_exec_commands() is only needed inside of
rebase--interactive.c for `rebase -p', it is moved there from
sequencer.c.

The parameter r (repository) is dropped along the way.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:17:57 +09:00
Alban Gruin
94bcad7979 sequencer: change complete_action() to use the refactored functions
complete_action() used functions that read the todo-list file, made some
changes to it, and wrote it back to the disk.

The previous commits were dedicated to separate the part that deals with
the file from the actual logic of these functions.  Now that this is
done, we can call directly the "logic" functions to avoid useless file
access.

The parsing of the list has to be done by the caller.  If the buffer of
the todo list provided by the caller is empty, a `noop' command is
directly added to the todo list, without touching the buffer.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:17:57 +09:00
Alban Gruin
d358fc286d sequencer: make sequencer_make_script() write its script to a strbuf
This makes sequencer_make_script() write its script to a strbuf (ie. the
buffer of a todo_list) instead of a FILE.  This reduce the amount of
read/write made by rebase interactive.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:17:57 +09:00
Alban Gruin
f2a04904be sequencer: refactor rearrange_squash() to work on a todo_list
This refactors rearrange_squash() to work on a todo_list to avoid
redundant reads and writes.  The function is renamed
todo_list_rearrange_squash().

The old version created a new buffer, which was directly written to the
disk.  This new version creates a new item list by just copying items
from the old item list, without creating a new buffer.  This eliminates
the need to reparse the todo list, but this also means its buffer cannot
be directly written to the disk.

As rebase -p still need to check the todo list from the disk, a new
function is introduced, rearrange_squash_in_todo_file().

complete_action() still uses rearrange_squash_in_todo_file() for now.
This will be changed in a future commit.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:17:57 +09:00
Alban Gruin
683153a438 sequencer: refactor sequencer_add_exec_commands() to work on a todo_list
This refactors sequencer_add_exec_commands() to work on a todo_list to
avoid redundant reads and writes to the disk.

Instead of inserting the `exec' commands between the other commands and
re-parsing the buffer at the end, they are appended to the buffer once,
and a new list of items is created.  Items from the old list are copied
across and new `exec' items are appended when necessary.  This
eliminates the need to reparse the buffer, but this also means we have
to use todo_list_write_to_disk() to write the file.

todo_list_add_exec_commands() and sequencer_add_exec_commands() are
modified to take a string list instead of a string -- one item for each
command.  This makes it easier to insert a new command to the todo list
for each command to execute.

sequencer_add_exec_commands() still reads the todo list from the disk,
as it is needed by rebase -p.

complete_action() still uses sequencer_add_exec_commands() for now.
This will be changed in a future commit.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07 09:17:57 +09:00
Jeff King
8d8c2a5aef fsck: always compute USED flags for unreachable objects
The --connectivity-only option avoids opening every object, and instead
just marks reachable objects with a flag and compares this to the set
of all objects. This strategy is discussed in more detail in 3e3f8bd608
(fsck: prepare dummy objects for --connectivity-check, 2017-01-17).

This means that we report _every_ unreachable object as dangling.
Whereas in a full fsck, we'd have actually opened and parsed each of
those unreachable objects, marking their child objects with the USED
flag, to mean "this was mentioned by another object". And thus we can
report only the tip of an unreachable segment of the object graph as
dangling.

You can see this difference with a trivial example:

  tree=$(git hash-object -t tree -w /dev/null)
  one=$(echo one | git commit-tree $tree)
  two=$(echo two | git commit-tree -p $one $tree)

Running `git fsck` will report only $two as dangling, but with
--connectivity-only, both commits (and the tree) are reported. Likewise,
using --lost-found would write all three objects.

We can make --connectivity-only work like the normal case by taking a
separate pass over the unreachable objects, parsing them and marking
objects they refer to as USED. That still avoids parsing any blobs,
though we do pay the cost to access any unreachable commits and trees
(which may or may not be noticeable, depending on how many you have).

If neither --dangling nor --lost-found is in effect, then we can skip
this step entirely, just like we do now. That makes "--connectivity-only
--no-dangling" just as fast as the current "--connectivity-only". I.e.,
we do the correct thing always, but you can still tweak the options to
make it faster if you don't care about dangling objects.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-05 22:55:57 +09:00
Jeff King
f06ab027ef rev-list: allow cached objects in existence check
This fixes a regression in 7c0fe330d5 (rev-list: handle missing tree
objects properly, 2018-10-05) where rev-list will now complain about the
empty tree when it doesn't physically exist on disk.

Before that commit, we relied on the traversal code in list-objects.c to
walk through the trees. Since it uses parse_tree(), we'd do a normal
object lookup that includes looking in the set of "cached" objects
(which is where our magic internal empty-tree kicks in).

After that commit, we instead tell list-objects.c not to die on any
missing trees, and we check them ourselves using has_object_file(). But
that function uses OBJECT_INFO_SKIP_CACHED, which means we won't use our
internal empty tree.

This normally wouldn't come up. For most operations, Git will try to
write out the empty tree object as it would any other object. And
pack-objects in a push or fetch will send the empty tree (even if it's
virtual on the sending side). However, there are cases where this can
matter. One I found in the wild:

  1. The root tree of a commit became empty by deleting all files,
     without using an index. In this case it was done using libgit2's
     tree builder API, but as the included test shows, it can easily be
     done with regular git using hash-object.

     The resulting repo works OK, as we'd avoid walking over our own
     reachable commits for a connectivity check.

  2. Cloning with --reference pointing to the repository from (1) can
     trigger the problem, because we tell the other side we already have
     that commit (and hence the empty tree), but then walk over it
     during the connectivity check (where we complain about it missing).

Arguably the workflow in step (1) should be more careful about writing
the empty tree object if we're referencing it. But this workflow did
work prior to 7c0fe330d5, so let's restore it.

This patch makes the minimal fix, which is to swap out a direct call to
oid_object_info_extended(), minus the SKIP_CACHED flag, instead of
calling has_object_file(). This is all that has_object_file() is doing
under the hood. And there's little danger of unrelated fallout from
other unexpected "cached" objects, since there's only one call site that
ends such a cached object, and it's in git-blame.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-05 22:28:29 +09:00
Jeff King
143588949c fetch: ignore SIGPIPE during network operation
The default SIGPIPE behavior can be useful for a command that generates
a lot of output: if the receiver of our output goes away, we'll be
notified asynchronously to stop generating it (typically by killing the
program).

But for a command like fetch, which is primarily concerned with
receiving data and writing it to disk, an unexpected SIGPIPE can be
awkward. We're already checking the return value of all of our write()
calls, and dying due to the signal takes away our chance to gracefully
handle the error.

On Linux, we wouldn't generally see SIGPIPE at all during fetch. If the
other side of the network connection hangs up, we'll see ECONNRESET. But
on OS X, we get a SIGPIPE, and the process is killed. This causes t5570
to racily fail, as we sometimes die by signal (instead of the expected
die() call) when the server side hangs up.

Let's ignore SIGPIPE during the network portion of the fetch, which will
cause our write() to return EPIPE, giving us consistent behavior across
platforms.

This fixes the test flakiness, but note that it stops short of fixing
the larger problem. The server side hit a fatal error, sent us an "ERR"
packet, and then hung up. We notice the failure because we're trying to
write to a closed socket. But by dying immediately, we never actually
read the ERR packet and report its content to the user. This is a (racy)
problem on all platforms. So this patch lays the groundwork from which
that problem might be fixed consistently, but it doesn't actually fix
it.

Note the placement of the SIGPIPE handling. The absolute minimal change
would be to ignore SIGPIPE only when we're writing. But twiddling the
signal handler for each write call is inefficient and maintenance
burden. On the opposite end of the spectrum, we could simply declare
that fetch does not need SIGPIPE handling, since it doesn't generate a
lot of output, and we could just ignore it at the start of cmd_fetch().

This patch takes a middle ground. It ignores SIGPIPE during the network
operation (which is admittedly most of the program, since the actual
network operations are all done under the hood by the transport code).
So it's still pretty coarse.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-05 15:02:18 +09:00
Johannes Schindelin
cbd29ead92 built-in rebase: set ORIG_HEAD just once, before the rebase
Technically, the scripted version set ORIG_HEAD only in two spots (which
really could have been one, because it called `git checkout $onto^0` to
start the rebase and also if it could take a shortcut, and in both cases
it called `git update-ref $orig_head`).

Practically, it *implicitly* reset ORIG_HEAD whenever `git reset --hard`
was called.

However, what we really want is that it is set exactly once, at the
beginning of the rebase.

So let's do that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-04 13:31:04 +09:00
Johannes Schindelin
eaf81605b8 built-in rebase: use the correct reflog when switching branches
By mistake, we used the reflog intended for ORIG_HEAD.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-04 13:31:04 +09:00
Johannes Schindelin
e6aac8177d built-in rebase: no need to check out onto twice
In the case that the rebase boils down to a fast-forward, the built-in
rebase reset the working tree twice: once to start the rebase at `onto`,
then realizing that the original (pre-rebase) HEAD was an ancestor and
we basically already fast-forwarded to the post-rebase HEAD,
`reset_head()` was called to update the original ref and to point HEAD
back to it.

That second `reset_head()` call does not need to touch the working tree,
though, as it does not change the actual tip commit (and therefore the
working tree should stay unchanged anyway): only the ref needs to be
updated (because the rebase detached the HEAD, and we want to go back to
the branch on which the rebase was started).

But that second `reset_head()` was called without the flag to leave the
working tree alone (the reason: when that call was introduced, that flag
was not yet even thought of). Let's avoid that unnecessary work by
passing that flag.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-04 13:31:03 +09:00
Martin Ågren
e8805af1c3 setup: fix memory leaks with struct repository_format
After we set up a `struct repository_format`, it owns various pieces of
allocated memory. We then either use those members, because we decide we
want to use the "candidate" repository format, or we discard the
candidate / scratch space. In the first case, we transfer ownership of
the memory to a few global variables. In the latter case, we just
silently drop the struct and end up leaking memory.

Introduce an initialization macro `REPOSITORY_FORMAT_INIT` and a
function `clear_repository_format()`, to be used on each side of
`read_repository_format()`. To have a clear and simple memory ownership,
let all users of `struct repository_format` duplicate the strings that
they take from it, rather than stealing the pointers.

Call `clear_...()` at the start of `read_...()` instead of just zeroing
the struct, since we sometimes enter the function multiple times. Thus,
it is important to initialize the struct before calling `read_...()`, so
document that. It's also important because we might not even call
`read_...()` before we call `clear_...()`, see, e.g., builtin/init-db.c.

Teach `read_...()` to clear the struct on error, so that it is reset to
a safe state, and document this. (In `setup_git_directory_gently()`, we
look at `repo_fmt.hash_algo` even if `repo_fmt.version` is -1, which we
weren't actually supposed to do per the API. After this commit, that's
ok.)

We inherit the existing code's combining "error" and "no version found".
Both are signalled through `version == -1` and now both cause us to
clear any partial configuration we have picked up. For "extensions.*",
that's fine, since they require a positive version number. For
"core.bare" and "core.worktree", we're already verifying that we have a
non-negative version number before using them.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-01 08:52:00 +09:00
Jeff King
287ab28bfa diff: reuse diff setup for --no-index case
When "--no-index" is in effect (or implied by the arguments), git-diff
jumps early to a special code path to perform that diff. This means we
miss out on some settings like enabling --ext-diff and --textconv by
default.

Let's jump to the no-index path _after_ we've done more setup on
rev.diffopt. Since some of the options don't affect us (e.g., items
related to the index), let's re-order the setup into two blocks (see the
in-code comments).

Note that we also need to stop re-initializing the diffopt struct in
diff_no_index(). This should not be necessary, as it will already have
been initialized by cmd_diff() (and there are no other callers). That in
turn lets us drop the "repository" argument from diff_no_index (which
never made much sense, since the whole point is that you don't need a
repository).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-24 07:08:34 -08:00
Jeff Hostetler
b3a5d5a80c trace2:data: add subverb for rebase
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 15:28:21 -08:00
Jeff Hostetler
c18b6c1a2b trace2:data: add subverb to reset command
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 15:28:21 -08:00
Jeff Hostetler
e27dd8ae9f trace2:data: add subverb to checkout command
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 15:28:21 -08:00
Derrick Stolee
ae417807b0 trace2:data: pack-objects: add trace2 regions
When studying the performance of 'git push' we would like to know
how much time is spent at various parts of the command. One area
that could cause performance trouble is 'git pack-objects'.

Add trace2 regions around the three main actions taken in this
command:

1. Enumerate objects.
2. Prepare pack.
3. Write pack-file.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 15:28:21 -08:00
Jeff Hostetler
6206286e49 trace2:data: add trace2 hook classification
Classify certain child processes as hooks.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 15:28:21 -08:00
Jeff Hostetler
ee4512ed48 trace2: create new combined trace facility
Create a new unified tracing facility for git.  The eventual intent is to
replace the current trace_printf* and trace_performance* routines with a
unified set of git_trace2* routines.

In addition to the usual printf-style API, trace2 provides higer-level
event verbs with fixed-fields allowing structured data to be written.
This makes post-processing and analysis easier for external tools.

Trace2 defines 3 output targets.  These are set using the environment
variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT".  These may be
set to "1" or to an absolute pathname (just like the current GIT_TRACE).

* GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command
  summary data.

* GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE.
  It extends the output with columns for the command process, thread,
  repo, absolute and relative elapsed times.  It reports events for
  child process start/stop, thread start/stop, and per-thread function
  nesting.

* GIT_TR2_EVENT is a new structured format. It writes event data as a
  series of JSON records.

Calls to trace2 functions log to any of the 3 output targets enabled
without the need to call different trace_printf* or trace_performance*
routines.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 15:27:59 -08:00
Junio C Hamano
2fe95f494c format-patch: notice failure to open cover letter for writing
The make_cover_letter() function is supposed to open a new file for
writing, and let the caller write into it via FILE *rev->diffopt.file
but because the function does not return anything, the caller does not
bother checking the return value.

Make sure it dies, instead of keep going with a NULL output
filestream and relying on it to cause a crash, when it fails to
open the file.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 14:25:35 -08:00
Junio C Hamano
bc208ae314 builtin/log: downcase the beginning of error messages
Also drop full-stop at the end of error messages, per
Documentation/CodingGuidelines.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22 14:19:08 -08:00
Ævar Arnfjörð Bjarmason
9903623761 receive-pack: fix use-after-free bug
The resolve_ref_unsafe() function can, and sometimes will in the case
of this codepath, return the char * passed to it to the caller. In
this case we construct a strbuf, free it, and then continue using the
dst_name after that free().

The code being fixed dates back to da3efdb17b ("receive-pack: detect
aliased updates which can occur with symrefs", 2010-04-19). When it
was originally added it didn't have this bug, it was introduced when
it was subsequently modified to use strbuf in 6b01ecfe22 ("ref
namespaces: Support remote repositories via upload-pack and
receive-pack", 2011-07-08).

This is theoretically a security issue, the C standard makes no
guarantees that a value you use after free() hasn't been poked at or
changed by something else on the system, but in practice modern OSs
will have mapped the relevant page to this process, so nothing else
would have used it. We do no further allocations between the free()
and use-after-free, so we ourselves didn't corrupt or change the
value.

Jeff investigated that and found: "It probably would be an issue if
the allocation were larger. glibc at least will use mmap()/munmap()
after some cutoff[1], in which case we'd get a segfault from hitting
the unmapped page. But for small allocations, it just bumps brk() and
the memory is still available for further allocations after
free(). [...] If you had a sufficiently large refname you might be
able to trigger the bug [...]. I tried to push such a ref. I had to
manually make a packed-refs file with the long name to avoid
filesystem limits (though probably you could have a long a/b/c/ name
on ext4).  But the result can't actually be pushed, because it all has
to fit into a 64k pkt-line as part of the push protocol.".

An a alternative and more succinct way of implementing this would have
been to do the strbuf_release() at the end of check_aliased_update()
and use "goto out" instead of the early "return" statements. Hopefully
this approach of using a helper instead makes it easier to follow.

1. Jeff: "Weirdly, the mmap() cutoff on my glibc system is 135168
   bytes. Which is...2^17 + 2^12? 33 pages? I'm sure there's a good
   reason for that, but I didn't dig into it."

Reported-by: 王健强 <jianqiang.wang@securitygossip.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-20 15:02:12 -08:00
Nguyễn Thái Ngọc Duy
5a59a2301f completion: add more parameter value completion
This adds value completion for a couple more paramters. To make it
easier to maintain these hard coded lists, add a comment at the original
list/code to remind people to update git-completion.bash too.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-20 12:31:56 -08:00
Ben Peart
1956ecd0ab read-cache: add post-index-change hook
Add a post-index-change hook that is invoked after the index is written in
do_write_locked_index().

This hook is meant primarily for notification, and cannot affect
the outcome of git commands that trigger the index write.

The hook is passed a flag to indicate whether the working directory was
updated or not and a flag indicating if a skip-worktree bit could have
changed.  These flags enable the hook to optimize its response to the
index change notification.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-15 11:00:33 -08:00
Jeff King
c2bf473d0d prune: check SEEN flag for reachability
The git-prune command checks reachability by doing a traversal, and then
checking whether a given object exists in the global object hash. This
can yield false positives if any other part of the code had to create an
object struct for some reason. It's not clear whether this is even
possible, but it's more robust to rely on something a little more
concrete: the SEEN flag set by our traversal.

Note that there is a slight possibility of regression here, as we're
relying on mark_reachable_objects() to consistently set the flag.
However, it has always done so, and we're already relying on that fact
in prune_shallow(), which is called as part of git-prune. So this is
making these two parts of the prune operation more consistent.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14 15:25:33 -08:00
Jeff King
d55a30bb1d prune: lazily perform reachability traversal
The general strategy of "git prune" is to do a full reachability walk,
then for each loose object see if we found it in our walk. But if we
don't have any loose objects, we don't need to do the expensive walk in
the first place.

This patch postpones that walk until the first time we need to see its
results.

Note that this is really a specific case of a more general optimization,
which is that we could traverse only far enough to find the object under
consideration (i.e., stop the traversal when we find it, then pick up
again when asked about the next object, etc). That could save us in some
instances from having to do a full walk. But it's actually a bit tricky
to do with our traversal code, and you'd need to do a full walk anyway
if you have even a single unreachable object (which you generally do, if
any objects are actually left after running git-repack).

So in practice this lazy-load of the full walk catches one easy but
common case (i.e., you've just repacked via git-gc, and there's nothing
unreachable).

The perf script is fairly contrived, but it does show off the
improvement:

  Test                            HEAD^             HEAD
  -------------------------------------------------------------------------
  5304.4: prune with no objects   3.66(3.60+0.05)   0.00(0.00+0.00) -100.0%

and would let us know if we accidentally regress this optimization.

Note also that we need to take special care with prune_shallow(), which
relies on us having performed the traversal. So this optimization can
only kick in for a non-shallow repository. Since this is easy to get
wrong and is not covered by existing tests, let's add an extra test to
t5304 that covers this case explicitly.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14 15:25:32 -08:00
Ramsay Jones
9b0bd87ed2 prune-packed: check for too many arguments
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11 13:11:20 -08:00
Jean-Noël Avila
32ceace39f Fix typos in translatable strings for v2.21.0
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11 12:58:06 -08:00
Junio C Hamano
b966813e71 Merge branch 'js/rebase-i-redo-exec-fix'
For "rebase -i --reschedule-failed-exec", we do not want the "-y"
shortcut after all.

* js/rebase-i-redo-exec-fix:
  Revert "rebase: introduce a shortcut for --reschedule-failed-exec"
2019-02-08 20:44:52 -08:00
Junio C Hamano
87c9831df0 Merge branch 'nd/checkout-noisy-unmerge'
"git checkout [<tree-ish>] <pathspec>" started reporting the number
of paths that have got updated recently, but the same messages were
given when "git checkout -m <pathspec>" to unresolve conflicts that
have just been resolved.  The message now reports these unresolved
paths separately from the paths that are checked out from the index.

* nd/checkout-noisy-unmerge:
  checkout: count and print -m paths separately
  checkout: update count-checkouts messages
2019-02-08 20:44:51 -08:00
Denton Liu
c89c494240 submodule--helper: teach config subcommand --unset
This teaches submodule--helper config the --unset option, which removes
the specified configuration key from the .gitmodule file.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08 12:02:58 -08:00
Elijah Newren
d76ce4f734 log,diff-tree: add --combined-all-paths option
The combined diff format for merges will only list one filename, even if
rename or copy detection is active.  For example, with raw format one
might see:

  ::100644 100644 100644 fabadb8 cc95eb0 4866510 MM	describe.c
  ::100755 100755 100755 52b7a2d 6d1ac04 d2ac7d7 RM	bar.sh
  ::100644 100644 100644 e07d6c5 9042e82 ee91881 RR	phooey.c

This doesn't let us know what the original name of bar.sh was in the
first parent, and doesn't let us know what either of the original names
of phooey.c were in either of the parents.  In contrast, for non-merge
commits, raw format does provide original filenames (and a rename score
to boot).  In order to also provide original filenames for merge
commits, add a --combined-all-paths option (which must be used with
either -c or --cc, and is likely only useful with rename or copy
detection active) so that we can print tab-separated filenames when
renames are involved.  This transforms the above output to:

  ::100644 100644 100644 fabadb8 cc95eb0 4866510 MM	desc.c	desc.c	desc.c
  ::100755 100755 100755 52b7a2d 6d1ac04 d2ac7d7 RM	foo.sh	bar.sh	bar.sh
  ::100644 100644 100644 e07d6c5 9042e82 ee91881 RR	fooey.c	fuey.c	phooey.c

Further, in patch format, this changes the from/to headers so that
instead of just having one "from" header, we get one for each parent.
For example, instead of having

  --- a/phooey.c
  +++ b/phooey.c

we would see

  --- a/fooey.c
  --- a/fuey.c
  +++ b/phooey.c

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-07 20:15:25 -08:00
Junio C Hamano
a1e19004e1 Merge branch 'ss/describe-dirty-in-the-right-directory'
"git --work-tree=$there --git-dir=$here describe --dirty" did not
work correctly as it did not pay attention to the location of the
worktree specified by the user by mistake, which has been
corrected.

* ss/describe-dirty-in-the-right-directory:
  t6120: test for describe with a bare repository
  describe: setup working tree for --dirty
2019-02-06 22:05:29 -08:00
Junio C Hamano
3a14fdec88 Merge branch 'sl/const'
Code cleanup.

* sl/const:
  various: tighten constness of some local variables
2019-02-06 22:05:28 -08:00
Junio C Hamano
cba595ab1a Merge branch 'jk/loose-object-cache-oid'
Code clean-up.

* jk/loose-object-cache-oid:
  prefer "hash mismatch" to "sha1 mismatch"
  sha1-file: avoid "sha1 file" for generic use in messages
  sha1-file: prefer "loose object file" to "sha1 file" in messages
  sha1-file: drop has_sha1_file()
  convert has_sha1_file() callers to has_object_file()
  sha1-file: convert pass-through functions to object_id
  sha1-file: modernize loose header/stream functions
  sha1-file: modernize loose object file functions
  http: use struct object_id instead of bare sha1
  update comment references to sha1_object_info()
  sha1-file: fix outdated sha1 comment references
2019-02-06 22:05:27 -08:00
Junio C Hamano
96e6547c2e Merge branch 'pw/rebase-x-sanity-check'
"git rebase -x $cmd" did not reject multi-line command, even though
the command is incapable of handling such a command.  It now is
rejected upfront.

* pw/rebase-x-sanity-check:
  rebase -x: sanity check command
2019-02-06 22:05:26 -08:00
Junio C Hamano
e7b120bfa5 Merge branch 'br/commit-tree-fully-spelled-gpg-sign-option'
The documentation of "git commit-tree" said that the command
understands "--gpg-sign" in addition to "-S", but the command line
parser did not know about the longhand, which has been corrected.

* br/commit-tree-fully-spelled-gpg-sign-option:
  commit-tree: add missing --gpg-sign flag
  t7510: invoke git as part of &&-chain
2019-02-06 22:05:25 -08:00
Junio C Hamano
5fda343321 Merge branch 'ds/push-sparse-tree-walk'
"git pack-objects" learned another algorithm to compute the set of
objects to send, that trades the resulting packfile off to save
traversal cost to favor small pushes.

* ds/push-sparse-tree-walk:
  pack-objects: create GIT_TEST_PACK_SPARSE
  pack-objects: create pack.useSparse setting
  revision: implement sparse algorithm
  list-objects: consume sparse tree walk
  revision: add mark_tree_uninteresting_sparse
2019-02-06 22:05:25 -08:00
Junio C Hamano
ecbe1beb8e Merge branch 'lt/date-human'
A new date format "--date=human" that morphs its output depending
on how far the time is from the current time has been introduced.
"--date=auto" can be used to use this new format when the output is
going to the pager or to the terminal and otherwise the default
format.

* lt/date-human:
  Add `human` date format tests.
  Add `human` format to test-tool
  Add 'human' date format documentation
  Replace the proposed 'auto' mode with 'auto:'
  Add 'human' date format
2019-02-06 22:05:24 -08:00
Junio C Hamano
b2fc9d2fb0 Merge branch 'jk/unused-parameter-cleanup'
Code cleanup.

* jk/unused-parameter-cleanup:
  convert: drop path parameter from actual conversion functions
  convert: drop len parameter from conversion checks
  config: drop unused parameter from maybe_remove_section()
  show_date_relative(): drop unused "tz" parameter
  column: drop unused "opts" parameter in item_length()
  create_bundle(): drop unused "header" parameter
  apply: drop unused "def" parameter from find_name_gnu()
  match-trees: drop unused path parameter from score functions
2019-02-06 22:05:23 -08:00
Junio C Hamano
7589e63648 Merge branch 'nd/the-index-final'
The assumption to work on the single "in-core index" instance has
been reduced from the library-ish part of the codebase.

* nd/the-index-final:
  cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch
  read-cache.c: remove the_* from index_has_changes()
  merge-recursive.c: remove implicit dependency on the_repository
  merge-recursive.c: remove implicit dependency on the_index
  sha1-name.c: remove implicit dependency on the_index
  read-cache.c: replace update_index_if_able with repo_&
  read-cache.c: kill read_index()
  checkout: avoid the_index when possible
  repository.c: replace hold_locked_index() with repo_hold_locked_index()
  notes-utils.c: remove the_repository references
  grep: use grep_opt->repo instead of explict repo argument
2019-02-06 22:05:23 -08:00
Junio C Hamano
e52c6bbd13 Merge branch 'js/rebase-am'
Instead of going through "git-rebase--am" scriptlet to use the "am"
backend, the built-in version of "git rebase" learned to drive the
"am" backend directly.

* js/rebase-am:
  built-in rebase: call `git am` directly
  rebase: teach `reset_head()` to optionally skip the worktree
  rebase: avoid double reflog entry when switching branches
  rebase: move `reset_head()` into a better spot
2019-02-06 22:05:22 -08:00
Junio C Hamano
09a9c1f427 Merge branch 'tt/bisect-in-c'
More code in "git bisect" has been rewritten in C.

* tt/bisect-in-c:
  bisect--helper: `bisect_start` shell function partially in C
  bisect--helper: `get_terms` & `bisect_terms` shell function in C
  bisect--helper: `bisect_next_check` shell function in C
  bisect--helper: `check_and_set_terms` shell function in C
  wrapper: move is_empty_file() and rename it as is_empty_or_missing_file()
  bisect--helper: `bisect_write` shell function in C
  bisect--helper: `bisect_reset` shell function in C
2019-02-06 22:05:22 -08:00
Junio C Hamano
cfd9167c15 Merge branch 'dt/cat-file-batch-ambiguous'
"git cat-file --batch" reported a dangling symbolic link by
mistake, when it wanted to report that a given name is ambiguous.

* dt/cat-file-batch-ambiguous:
  t1512: test ambiguous cat-file --batch and --batch-output
  Do not print 'dangling' for cat-file in case of ambiguity
2019-02-06 22:05:21 -08:00
Junio C Hamano
8fe9c3f21d Merge branch 'en/rebase-merge-on-sequencer'
"git rebase --merge" as been reimplemented by reusing the internal
machinery used for "git rebase -i".

* en/rebase-merge-on-sequencer:
  rebase: implement --merge via the interactive machinery
  rebase: define linearization ordering and enforce it
  git-legacy-rebase: simplify unnecessary triply-nested if
  git-rebase, sequencer: extend --quiet option for the interactive machinery
  am, rebase--merge: do not overlook --skip'ed commits with post-rewrite
  t5407: add a test demonstrating how interactive handles --skip differently
  rebase: fix incompatible options error message
  rebase: make builtin and legacy script error messages the same
2019-02-06 22:05:20 -08:00
Nguyễn Thái Ngọc Duy
1d1f689bef checkout: count and print -m paths separately
Since 0f086e6dca (checkout: print something when checking out paths -
2018-11-13), this command reports how many paths have been updated
from what source (either from a tree, or from the index). I forget
that there's a third source: when -m is used, the merge conflict is
re-created (granted, also from the index, but it's not a straight copy
from the index).

Count and report unmerged paths separately. There's a bit more update
to avoid reporting:

   Recreated X merge conflicts
   Updated 0 paths from the index

The second line is unnecessary. Though if there's no conflict
recreation, we still report

   Updated 0 paths from the index

to make it clear we're not really doing anything.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06 12:46:08 -08:00
Nguyễn Thái Ngọc Duy
3c5883b3c9 checkout: update count-checkouts messages
Commit 0f086e6dca [1] counts the number of files updated by "git
checkout -- <paths>" command and prints it. Later on 536ec1839d [2]
adds the ability to remove files in "git checkout -- <paths>". This is
still an update on worktree and should be reported to the user.

To prepare for such an update since that commit is on track to
'master' now, the messages are rephrased to avoid "checked out" which
does not imply file deletion.

[1] 0f086e6dca (checkout: print something when checking out paths -
    2018-11-13)
[2] 536ec1839d (entry: support CE_WT_REMOVE flag in checkout_entry -
    2018-12-20)

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06 12:46:07 -08:00
Johannes Schindelin
e11ff8975b Revert "rebase: introduce a shortcut for --reschedule-failed-exec"
This patch was contributed only as a tentative "we could introduce a
convenient short option if we do not want to change the default behavior
in the long run" patch, opening the discussion whether other people
agree with deprecating the current behavior in favor of the rescheduling
behavior.

But the consensus on the Git mailing list was that it would make sense
to show a warning in the near future, and flip the default
rebase.rescheduleFailedExec to reschedule failed `exec` commands by
default. See e.g.
<CAGZ79kZL5CRqCDRb6B-EedUm8Z_i4JuSF2=UtwwdRXMitrrOBw@mail.gmail.com>

So let's back out that patch that added the `-y` short option that we
agreed was not necessary or desirable.

This reverts commit 81ef8ee75d.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06 11:27:17 -08:00
Junio C Hamano
d6cc13602b Merge branch 'nd/fetch-compact-update'
"git fetch" output cleanup.

* nd/fetch-compact-update:
  fetch: prefer suffix substitution in compact fetch.output
2019-02-05 14:26:18 -08:00
Junio C Hamano
69dd6e5737 Merge branch 'pw/no-editor-in-rebase-i-implicit'
When GIT_SEQUENCE_EDITOR is set, the command was incorrectly
started when modes of "git rebase" that implicitly uses the
machinery for the interactive rebase are run, which has been
corrected.

* pw/no-editor-in-rebase-i-implicit:
  implicit interactive rebase: don't run sequence editor
2019-02-05 14:26:17 -08:00
Junio C Hamano
5ad3550f02 Merge branch 'bp/checkout-new-branch-optim'
"git checkout -b <new> [HEAD]" to create a new branch from the
current commit and check it out ought to be a no-op in the index
and the working tree in normal cases, but there are corner cases
that do require updates to the index and the working tree.  Running
it immediately after "git clone --no-checkout" is one of these
cases that an earlier optimization kicked in incorrectly, which has
been fixed.

* bp/checkout-new-branch-optim:
  checkout: fix regression in checkout -b on intitial checkout
  checkout: add test demonstrating regression with checkout -b on initial commit
2019-02-05 14:26:17 -08:00
Junio C Hamano
d243a323a5 Merge branch 'ph/pack-objects-mutex-fix'
"git pack-objects" incorrectly used uninitialized mutex, which has
been corrected.

* ph/pack-objects-mutex-fix:
  pack-objects: merge read_lock and lock in packing_data struct
  pack-objects: move read mutex to packing_data struct
2019-02-05 14:26:16 -08:00
Junio C Hamano
e5eac57356 Merge branch 'ab/commit-graph-write-progress'
The codepath to show progress meter while writing out commit-graph
file has been improved.

* ab/commit-graph-write-progress:
  commit-graph write: emit a percentage for all progress
  commit-graph write: add itermediate progress
  commit-graph write: remove empty line for readability
  commit-graph write: add more descriptive progress output
  commit-graph write: show progress for object search
  commit-graph write: more descriptive "writing out" output
  commit-graph write: add "Writing out" progress output
  commit-graph: don't call write_graph_chunk_extra_edges() unnecessarily
  commit-graph: rename "large edges" to "extra edges"
2019-02-05 14:26:14 -08:00
Junio C Hamano
1c418243a5 Merge branch 'jk/add-ignore-errors-bit-assignment-fix'
"git add --ignore-errors" did not work as advertised and instead
worked as an unintended synonym for "git add --renormalize", which
has been fixed.

* jk/add-ignore-errors-bit-assignment-fix:
  add: use separate ADD_CACHE_RENORMALIZE flag
2019-02-05 14:26:13 -08:00
Junio C Hamano
5f8b86db94 Merge branch 'jt/fetch-v2-sideband'
"git fetch" and "git upload-pack" learned to send all exchange over
the sideband channel while talking the v2 protocol.

* jt/fetch-v2-sideband:
  tests: define GIT_TEST_SIDEBAND_ALL
  {fetch,upload}-pack: sideband v2 fetch response
  sideband: reverse its dependency on pkt-line
  pkt-line: introduce struct packet_writer
  pack-protocol.txt: accept error packets in any context
  Use packet_reader instead of packet_read_line
2019-02-05 14:26:11 -08:00
Junio C Hamano
073312b4c7 Merge branch 'js/filter-options-should-use-plain-int'
Update the protocol message specification to allow only the limited
use of scaled quantities.  This is ensure potential compatibility
issues will not go out of hand.

* js/filter-options-should-use-plain-int:
  filter-options: expand scaled numbers
  tree:<depth>: skip some trees even when collecting omits
  list-objects-filter: teach tree:# how to handle >0
2019-02-05 14:26:10 -08:00
Junio C Hamano
b99a579f8e Merge branch 'sb/more-repo-in-api'
The in-core repository instances are passed through more codepaths.

* sb/more-repo-in-api: (23 commits)
  t/helper/test-repository: celebrate independence from the_repository
  path.h: make REPO_GIT_PATH_FUNC repository agnostic
  commit: prepare free_commit_buffer and release_commit_memory for any repo
  commit-graph: convert remaining functions to handle any repo
  submodule: don't add submodule as odb for push
  submodule: use submodule repos for object lookup
  pretty: prepare format_commit_message to handle arbitrary repositories
  commit: prepare logmsg_reencode to handle arbitrary repositories
  commit: prepare repo_unuse_commit_buffer to handle any repo
  commit: prepare get_commit_buffer to handle any repo
  commit-reach: prepare in_merge_bases[_many] to handle any repo
  commit-reach: prepare get_merge_bases to handle any repo
  commit-reach.c: allow get_merge_bases_many_0 to handle any repo
  commit-reach.c: allow remove_redundant to handle any repo
  commit-reach.c: allow merge_bases_many to handle any repo
  commit-reach.c: allow paint_down_to_common to handle any repo
  commit: allow parse_commit* to handle any repo
  object: parse_object to honor its repository argument
  object-store: prepare has_{sha1, object}_file to handle any repo
  object-store: prepare read_object_file to deal with any repo
  ...
2019-02-05 14:26:09 -08:00
Jiang Xin
0e37abd2e8 pack-redundant: consistent sort method
SZEDER reported that test case t5323 has different test result on MacOS.
This is because `cmp_pack_list_reverse` cannot give identical result
when two pack being sorted has the same size of remaining_objects.

Changes to the sorting function will make consistent test result for
t5323.

The new algorithm to find redundant packs is a trade-off to save memory
resources, and the result of it may be different with old one, and may
be not the best result sometimes.  Update t5323 for the new algorithm.

Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:25 -08:00
Jiang Xin
4bc0cc12c1 pack-redundant: rename pack_list.all_objects
New algorithm uses `pack_list.all_objects` to track remaining objects,
so rename it to `pack_list.remaining_objects`.

Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:24 -08:00
Sun Chao
3084a01e5e pack-redundant: new algorithm to find min packs
When calling `git pack-redundant --all`, if there are too many local
packs and too many redundant objects within them, the too deep iteration
of `get_permutations` will exhaust all the resources, and the process of
`git pack-redundant` will be killed.

The following script could create a repository with too many redundant
packs, and running `git pack-redundant --all` in the `test.git` repo
will die soon.

    #!/bin/sh

    repo="$(pwd)/test.git"
    work="$(pwd)/test"
    i=1
    max=199

    if test -d "$repo" || test -d "$work"; then
    	echo >&2 "ERROR: '$repo' or '$work' already exist"
    	exit 1
    fi

    git init -q --bare "$repo"
    git --git-dir="$repo" config gc.auto 0
    git --git-dir="$repo" config transfer.unpackLimit 0
    git clone -q "$repo" "$work" 2>/dev/null

    while :; do
        cd "$work"
        echo "loop $i: $(date +%s)" >$i
        git add $i
        git commit -q -sm "loop $i"
        git push -q origin HEAD:master
        printf "\rCreate pack %4d/%d\t" $i $max
        if test $i -ge $max; then break; fi

        cd "$repo"
        git repack -q
        if test $(($i % 2)) -eq 0; then
            git repack -aq
            pack=$(ls -t $repo/objects/pack/*.pack | head -1)
            touch "${pack%.pack}.keep"
        fi
        i=$((i+1))
    done
    printf "\ndone\n"

To get the `min` unique pack list, we can replace the iteration in
`minimize` function with a new algorithm, and this could solve this
issue:

1. Get the unique and non_uniqe packs, add the unique packs to the
   `min` list.

2. Remove the objects of unique packs from non_unique packs, then each
   object left in the non_unique packs will have at least two copies.

3. Sort the non_unique packs by the objects' size, more objects first,
   and add the first non_unique pack to `min` list.

4. Drop the duplicated objects from other packs in the ordered
   non_unique pack list, and repeat step 3.

Some test cases will fail on Mac OS X. Mark them and will resolve in
later commit.

Original PR and discussions: https://github.com/jiangxin/git/pull/25

Signed-off-by: Sun Chao <sunchao9@huawei.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:24 -08:00
Sun Chao
8822859363 pack-redundant: delete redundant code
The objects in alt-odb are removed from `all_objects` twice in `load_all_objects`
and `scan_alt_odb_packs`, remove it from the later function.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:24 -08:00
Jiang Xin
3011177640 pack-redundant: delay creation of unique_objects
Instead of initializing unique_objects in `add_pack()`, copy from
all_objects in `cmp_two_packs()`, when unwanted objects are removed from
all_objects.

This will save memory (no allocate memory for alt-odb packs), and run
`llist_sorted_difference_inplace()` only once when removing ignored
objects and removing objects in alt-odb in `scan_alt_odb_packs()`.

Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:24 -08:00
Thomas Gummerer
e92aa0e4ef revert "checkout: introduce checkout.overlayMode config"
This reverts 1495ff7da5 ("checkout: introduce checkout.overlayMode
config", 2019-01-08) and thus removes the checkout.overlayMode config
option.

The option was originally introduced to give users the option to make
the new no-overlay behaviour the default.  However users may be using
'git checkout' in scripts, even though it is porcelain.  Users setting
the option to false may actually end up accidentally breaking scripts.

With the introduction of a new subcommand that will make the behaviour
the default, the config option will not be needed anymore anyway.
Revert the commit and remove the config option, so we don't risk
breaking scripts.

Suggested-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 13:30:14 -08:00
William Hubbs
39ab4d0951 config: allow giving separate author and committer idents
The author.email, author.name, committer.email and committer.name
settings are analogous to the GIT_AUTHOR_* and GIT_COMMITTER_*
environment variables, but for the git config system. This allows them
to be set separately for each repository.

Git supports setting different authorship and committer
information with environment variables. However, environment variables
are set in the shell, so if different authorship and committer
information is needed for different repositories an external tool is
required.

This adds support to git config for author.email, author.name,
committer.email and committer.name  settings so this information
can be set per repository.

Also, it generalizes the fmt_ident function so it can handle author vs
committer identification.

Signed-off-by: William Hubbs <williamh@gentoo.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 12:18:13 -08:00
Sebastian Staudt
2ed5c8e174 describe: setup working tree for --dirty
We don't use NEED_WORK_TREE when running the git-describe builtin,
since you should be able to describe a commit even in a bare repository.
However, the --dirty flag does need a working tree. Since we don't call
setup_work_tree(), it uses whatever directory we happen to be in. That's
unlikely to match our index, meaning we'd say "dirty" even when the real
working tree is clean.

We can fix that by calling setup_work_tree() once we know that the user
has asked for --dirty.

The --broken option also needs a working tree. But because its
implementation calls git-diff-index we don‘t have to setup the working
tree in the git-describe process.

Signed-off-by: Sebastian Staudt <koraktor@gmail.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 10:27:54 -08:00
Shahzad Lone
33de80b1d5 various: tighten constness of some local variables
Signed-off-by: Shahzad Lone <shahzadlone@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 09:57:10 -08:00
Phillip Wood
c762aada1a rebase -x: sanity check command
If the user gives an empty argument to --exec then git creates a todo
list that it cannot parse. The rebase starts to run before erroring out
with

  error: missing arguments for exec
  error: invalid line 2: exec
  You can fix this with 'git rebase --edit-todo' and then run 'git rebase --continue'.
  Or you can abort the rebase with 'git rebase --abort'.

Instead check for empty commands before starting the rebase.

Also check that the command does not contain any newlines as the
todo-list format is unable to cope with multiline commands. Note that
this changes the behavior, before this change one could do

git rebase --exec='echo one
exec echo two'

and it would insert two exec lines in the todo list, now it will error
out.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 13:34:51 -08:00
Junio C Hamano
a562a11983 Merge branch 'it/log-format-source'
Custom userformat "log --format" learned %S atom that stands for
the tip the traversal reached the commit from, i.e. --source.

* it/log-format-source:
  log: add %S option (like --source) to log --format
2019-01-29 12:47:57 -08:00
Junio C Hamano
7fa92ba40a Merge branch 'js/add-e-clear-patch-before-stating'
"git add -e" got confused when the change it wants to let the user
edit is smaller than the previous change that was left over in a
temporary file.

* js/add-e-clear-patch-before-stating:
  add --edit: truncate the patch file
2019-01-29 12:47:56 -08:00
Junio C Hamano
371820d5f1 Merge branch 'bc/tree-walk-oid'
The code to walk tree objects has been taught that we may be
working with object names that are not computed with SHA-1.

* bc/tree-walk-oid:
  cache: make oidcpy always copy GIT_MAX_RAWSZ bytes
  tree-walk: store object_id in a separate member
  match-trees: use hashcpy to splice trees
  match-trees: compute buffer offset correctly when splicing
  tree-walk: copy object ID before use
2019-01-29 12:47:56 -08:00
Junio C Hamano
d94ade7f1f Merge branch 'os/rebase-runs-post-checkout-hook'
"git rebase" internally runs "checkout" to switch between branches,
and the command used to call the post-checkout hook, but the
reimplementation stopped doing so, which is getting fixed.

* os/rebase-runs-post-checkout-hook:
  rebase: run post-checkout hook on checkout
  t5403: simplify by using a single repository
2019-01-29 12:47:55 -08:00
Junio C Hamano
5d3635db19 Merge branch 'sb/submodule-recursive-fetch-gets-the-tip'
"git fetch --recurse-submodules" may not fetch the necessary commit
that is bound to the superproject, which is getting corrected.

* sb/submodule-recursive-fetch-gets-the-tip:
  fetch: ensure submodule objects fetched
  submodule.c: fetch in submodules git directory instead of in worktree
  submodule: migrate get_next_submodule to use repository structs
  repository: repo_submodule_init to take a submodule struct
  submodule: store OIDs in changed_submodule_names
  submodule.c: tighten scope of changed_submodule_names struct
  submodule.c: sort changed_submodule_names before searching it
  submodule.c: fix indentation
  sha1-array: provide oid_array_filter
2019-01-29 12:47:54 -08:00
Junio C Hamano
f33989464e Merge branch 'jt/fetch-pack-v2'
"git fetch-pack" now can talk the version 2 protocol.

* jt/fetch-pack-v2:
  fetch-pack: support protocol version 2
2019-01-29 12:47:54 -08:00
Junio C Hamano
773e408881 Merge branch 'jk/save-getenv-result'
There were many places the code relied on the string returned from
getenv() to be non-volatile, which is not true, that have been
corrected.

* jk/save-getenv-result:
  builtin_diff(): read $GIT_DIFF_OPTS closer to use
  merge-recursive: copy $GITHEAD strings
  init: make a copy of $GIT_DIR string
  config: make a copy of $GIT_CONFIG string
  commit: copy saved getenv() result
  get_super_prefix(): copy getenv() result
2019-01-29 12:47:54 -08:00
Junio C Hamano
d9d9ab0876 Merge branch 'js/rebase-i-redo-exec'
"git rebase -i" learned to re-execute a command given with 'exec'
to run after it failed the last time.

* js/rebase-i-redo-exec:
  rebase: introduce a shortcut for --reschedule-failed-exec
  rebase: add a config option to default to --reschedule-failed-exec
  rebase: introduce --reschedule-failed-exec
2019-01-29 12:47:53 -08:00
Junio C Hamano
61c51acf7d Merge branch 'cc/fetch-error-message-fix'
Error message fix.

* cc/fetch-error-message-fix:
  fetch: fix extensions.partialclone name in error message
2019-01-29 12:47:53 -08:00
Alban Gruin
6ca89c6f39 sequencer: refactor check_todo_list() to work on a todo_list
This refactors check_todo_list() to work on a todo_list to avoid
redundant reads and writes to the disk.  The function is renamed
todo_list_check().  The parsing of the two todo lists is left to the
caller.

As rebase -p still need to check the todo list from the disk, a new
function is introduced, check_todo_list_from_file().  It reads the file
from the disk, parses it, pass the todo_list to todo_list_check(), and
writes it back to the disk.

As get_missing_commit_check_level() and the enum
missing_commit_check_level are no longer needed inside of sequencer.c,
they are moved to rebase-interactive.c, and made static again.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 12:09:24 -08:00
Alban Gruin
cbef27d61c sequencer: refactor transform_todos() to work on a todo_list
This refactors transform_todos() to work on a todo_list.  The function
is renamed todo_list_transform().

As rebase -p still need to check the todo list from the disk, a new
function is introduced, transform_todo_file().  It is still used by
complete_action() and edit_todo_list() for now, but they will be
replaced in a future commit.

todo_list_transform() is not a static function, because it will be used
by edit_todo_list() from rebase-interactive.c in a future commit.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 12:09:24 -08:00
Patrick Hogg
edb673cf10 pack-objects: merge read_lock and lock in packing_data struct
Rename the packing_data lock to obd_lock and upgrade it to a recursive
mutex to make it suitable for current read_lock usages. Additionally
remove the superfluous #ifndef NO_PTHREADS guard around mutex
initialization in prepare_packing_data as the mutex functions
themselves are already protected.

Signed-off-by: Patrick Hogg <phogg@novamoon.net>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 11:22:12 -08:00
Patrick Hogg
459307b139 pack-objects: move read mutex to packing_data struct
ac77d0c37 ("pack-objects: shrink size field in struct object_entry",
2018-04-14) added an extra usage of read_lock/read_unlock in the newly
introduced oe_get_size_slow for thread safety in parallel calls to
try_delta(). Unfortunately oe_get_size_slow is also used in serial
code, some of which is called before the first invocation of
ll_find_deltas. As such the read mutex is not guaranteed to be
initialized.

Resolve this by moving the read mutex to packing_data and initializing
it in prepare_packing_data which is initialized in cmd_pack_objects.

Signed-off-by: Patrick Hogg <phogg@novamoon.net>
Reviewed-by: Duy Nguyen <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 11:22:06 -08:00
Phillip Wood
891d4a0313 implicit interactive rebase: don't run sequence editor
If GIT_SEQUENCE_EDITOR is set then rebase runs it when executing
implicit interactive rebases which are supposed to appear
non-interactive to the user. Fix this by setting GIT_SEQUENCE_EDITOR=:
rather than GIT_EDITOR=:. A couple of tests relied on the old behavior
so they are updated to work with the new regime.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 10:25:39 -08:00
Nguyễn Thái Ngọc Duy
3ebbe28989 parse-options: allow ll_callback with OPTION_CALLBACK
OPTION_CALLBACK is much simpler/safer to use, but parse_opt_cb does
not allow access to parse_opt_ctx_t, which sometimes is useful
(e.g. to obtain the prefix).

Extending parse_opt_cb to take parse_opt_cb could result in a lot of
changes. Instead let's just allow ll_callback to be used with
OPTION_CALLBACK. The user will have to be careful, not to change
anything in ctx, or return wrong result code. But that's the price for
ll_callback.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-27 16:28:18 -08:00
Nguyễn Thái Ngọc Duy
f41179f16b parse-options: avoid magic return codes
Give names to these magic negative numbers. Make parse_opt_ll_cb
return an enum to make clear it can actually control parse_options()
with different return values (parse_opt_cb can too, but nobody needs
it).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-27 16:28:18 -08:00
Nguyễn Thái Ngọc Duy
bf3ff338a2 parse-options: stop abusing 'callback' for lowlevel callbacks
Lowlevel callbacks have different function signatures. Add a new field
in 'struct option' with the right type for lowlevel callbacks.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-27 16:28:18 -08:00
Nguyễn Thái Ngọc Duy
dc40b24df4 fetch: prefer suffix substitution in compact fetch.output
I have a remote named "jch" and it has a branch with the same name. And
fetch.output is set to "compact". Fetching this remote looks like this

 From https://github.com/gitster/git
  + eb7fd39f6b...835363af2f jch                -> */jch  (forced update)
    6f11fd5edb..59b12ae96a  nd/config-move-to  -> jch/*
  * [new branch]            nd/diff-parseopt   -> jch/*
  * [new branch]            nd/the-index-final -> jch/*

Notice that the local side of branch jch starts with "*" instead of
ending with it like the rest. It's not exactly wrong. It just looks
weird.

This patch changes the find-and-replace code a bit to try finding prefix
first before falling back to strstr() which finds a substring from left
to right. Now we have something less OCD

 From https://github.com/gitster/git
  + eb7fd39f6b...835363af2f jch                -> jch/*  (forced update)
    6f11fd5edb..59b12ae96a  nd/config-move-to  -> jch/*
  * [new branch]            nd/diff-parseopt   -> jch/*
  * [new branch]            nd/the-index-final -> jch/*

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-27 16:20:34 -08:00
Jeff King
fcb133e978 create_bundle(): drop unused "header" parameter
There's no need to pass a header struct to create_bundle(); it writes
the header information directly to a descriptor (and does not report
back details to the caller).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 12:35:44 -08:00
Nguyễn Thái Ngọc Duy
f8adbec9fe cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch
By default, index compat macros are off from now on, because they
could hide the_index dependency.

Only those in builtin can use it.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 11:55:06 -08:00
Ben Peart
8424bfd45b checkout: fix regression in checkout -b on intitial checkout
When doing a 'checkout -b' do a full checkout including updating the working
tree when doing the initial checkout. As the new test involves an filesystem
access, do it later in the sequence to give chance to other cheaper tests to
leave early. This fixes the regression in behavior caused by fa655d8411
(checkout: optimize "git checkout -b <new_branch>", 2018-08-16).

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-23 13:22:48 -08:00
SZEDER Gábor
5af7417bd8 commit-graph: rename "large edges" to "extra edges"
The optional 'Large Edge List' chunk of the commit graph file stores
parent information for commits with more than two parents, and the
names of most of the macros, variables, struct fields, and functions
related to this chunk contain the term "large edges", e.g.
write_graph_chunk_large_edges().  However, it's not a really great
term, as the edges to the second and subsequent parents stored in this
chunk are not any larger than the edges to the first and second
parents stored in the "main" 'Commit Data' chunk.  It's the number of
edges, IOW number of parents, that is larger compared to non-merge and
"regular" two-parent merge commits.  And indeed, two functions in
'commit-graph.c' have a local variable called 'num_extra_edges' that
refer to the same thing, and this "extra edges" term is much better at
describing these edges.

So let's rename all these references to "large edges" in macro,
variable, function, etc. names to "extra edges".  There is a
GRAPH_OCTOPUS_EDGES_NEEDED macro as well; for the sake of consistency
rename it to GRAPH_EXTRA_EDGES_NEEDED.

We can do so safely without causing any incompatibility issues,
because the term "large edges" doesn't come up in the file format
itself in any form (the chunk's magic is {'E', 'D', 'G', 'E'}, there
is no 'L' in there), but only in the specification text.  The string
"large edges", however, does come up in the output of 'git
commit-graph read' and in tests looking at its input, but that command
is explicitly documented as debugging aid, so we can change its output
and the affected tests safely.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-22 11:33:46 -08:00
Brandon Richardson
70ddbd7767 commit-tree: add missing --gpg-sign flag
Add --gpg-sign option in commit-tree, which was documented, but not
implemented, in 55ca3f99ae. Add tests for the --gpg-sign option.

Signed-off-by: Brandon Richardson <brandon1024.br@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-22 11:08:35 -08:00
David Turner
d1dd94b308 Do not print 'dangling' for cat-file in case of ambiguity
The return values -1 and -2 from get_oid could mean two different
things, depending on whether they were from an enum returned by
get_tree_entry_follow_symlinks, or from a different code path.  This
caused 'dangling' to be printed from a git cat-file in the case of an
ambiguous (-2) result.

Unify the results of get_oid* and get_tree_entry_follow_symlinks to be
one common type, with unambiguous values.

Signed-off-by: David Turner <novalis@novalis.org>
Reported-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18 15:22:02 -08:00
Junio C Hamano
5104f8f1ac Merge branch 'js/gc-repack-close-before-remove'
"git gc" and "git repack" did not close the open packfiles that
they found unneeded before removing them, which didn't work on a
platform incapable of removing an open file.  This has been
corrected.

* js/gc-repack-close-before-remove:
  gc/repack: release packs when needed
2019-01-18 13:49:57 -08:00
Junio C Hamano
726f89c2dd Merge branch 'nd/worktree-remove-with-uninitialized-submodules'
"git worktree remove" and "git worktree move" refused to work when
there is a submodule involved.  This has been loosened to ignore
uninitialized submodules.

* nd/worktree-remove-with-uninitialized-submodules:
  worktree: allow to (re)move worktrees with uninitialized submodules
2019-01-18 13:49:54 -08:00
Junio C Hamano
3942920966 Merge branch 'sb/submodule-unset-core-worktree-when-worktree-is-lost'
The core.worktree setting in a submodule repository should not be
pointing at a directory when the submodule loses its working tree
(e.g. getting deinit'ed), but the code did not properly maintain
this invariant.

* sb/submodule-unset-core-worktree-when-worktree-is-lost:
  submodule deinit: unset core.worktree
  submodule--helper: fix BUG message in ensure_core_worktree
  submodule: unset core.worktree if no working tree is present
  submodule update: add regression test with old style setups
2019-01-18 13:49:53 -08:00
Junio C Hamano
ec27a94013 Merge branch 'jn/stripspace-wo-repository'
"git stripspace" should be usable outside a git repository, but
under the "-s" or "-c" mode, it didn't.

* jn/stripspace-wo-repository:
  stripspace: allow -s/-c outside git repository
2019-01-18 13:49:53 -08:00
Junio C Hamano
4744d03a47 Merge branch 'sb/submodule-fetchjobs-default-to-one'
"git submodule update" ought to use a single job unless asked, but
by mistake used multiple jobs, which has been fixed.

* sb/submodule-fetchjobs-default-to-one:
  submodule update: run at most one fetch job unless otherwise set
2019-01-18 13:49:52 -08:00
Junio C Hamano
3434569fc2 Merge branch 'nd/style-opening-brace'
Code clean-up.

* nd/style-opening-brace:
  style: the opening '{' of a function is in a separate line
2019-01-18 13:49:52 -08:00
Linus Torvalds
acdd37769d Add 'human' date format
This adds --date=human, which skips the timezone if it matches the
current time-zone, and doesn't print the whole date if that matches (ie
skip printing year for dates that are "this year", but also skip the
whole date itself if it's in the last few days and we can just say what
weekday it was).

For really recent dates (same day), use the relative date stamp, while
for old dates (year doesn't match), don't bother with time and timezone.

Also add 'auto' date mode, which defaults to human if we're using the
pager.  So you can do

	git config --add log.date auto

and your "git log" commands will show the human-legible format unless
you're scripting things.

Note that this time format still shows the timezone for recent enough
events (but not so recent that they show up as relative dates).  You can
combine it with the "-local" suffix to never show timezones for an even
more simplified view.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Stephen P. Smith <ischis2@cox.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18 10:31:23 -08:00
Johannes Schindelin
21853626ea built-in rebase: call git am directly
While the scripted `git rebase` still has to rely on the
`git-rebase--am.sh` script to implement the glue between the `rebase`
and the `am` commands, we can go a more direct route in the built-in
rebase and avoid using a shell script altogether.

This patch represents a straight-forward port of `git-rebase--am.sh` to
C, along with the glue code to call it directly from within
`builtin/rebase.c`.

This reduces the chances of Git for Windows running into trouble due to
problems with the POSIX emulation layer (known as "MSYS2 runtime",
itself a derivative of the Cygwin runtime): when no shell script is
called, the POSIX emulation layer is avoided altogether.

Note: we pass an empty action to `reset_head()` here when moving back to
the original branch, as no other action is applicable, really. This
parameter is used to initialize `unpack_trees()`' messages.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18 10:11:45 -08:00
Johannes Schindelin
414f336069 rebase: teach reset_head() to optionally skip the worktree
This is what the legacy (scripted) rebase does in
`move_to_original_branch`, and we will need this functionality in the
next commit.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18 10:11:45 -08:00
Johannes Schindelin
5b2237a876 rebase: avoid double reflog entry when switching branches
When switching a branch *and* updating said branch to a different
revision, let's avoid a double entry in HEAD's reflog by first updating
the branch and then adjusting the symbolic ref HEAD.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18 10:11:45 -08:00
Johannes Schindelin
c5233708c5 rebase: move reset_head() into a better spot
Over the next commits, we want to make use of it in `run_am()` (i.e.
running the `--am` backend directly, without detouring to Unix shell
script code) which in turn will be called from `run_specific_rebase()`.

So let's move it before that latter function.

This commit is best viewed using --color-moved.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18 10:11:45 -08:00
Derrick Stolee
99dbbfa8dd pack-objects: create GIT_TEST_PACK_SPARSE
Create a test variable GIT_TEST_PACK_SPARSE to enable the sparse
object walk algorithm by default during the test suite. Enabling
this variable ensures coverage in many interesting cases, such as
shallow clones, partial clones, and missing objects.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 13:44:44 -08:00
Derrick Stolee
3d036eb0d2 pack-objects: create pack.useSparse setting
The '--sparse' flag in 'git pack-objects' changes the algorithm
used to enumerate objects to one that is faster for individual
users pushing new objects that change only a small cone of the
working directory. The sparse algorithm is not recommended for a
server, which likely sends new objects that appear across the
entire working directory.

Create a 'pack.useSparse' setting that enables this new algorithm.
This allows 'git push' to use this algorithm without passing a
'--sparse' flag all the way through four levels of run_command()
calls.

If the '--no-sparse' flag is set, then this config setting is
overridden.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 13:44:43 -08:00
Derrick Stolee
4f6d26b167 list-objects: consume sparse tree walk
When creating a pack-file using 'git pack-objects --revs' we provide
a list of interesting and uninteresting commits. For example, a push
operation would make the local topic branch be interesting and the
known remote refs as uninteresting. We want to discover the set of
new objects to send to the server as a thin pack.

We walk these commits until we discover a frontier of commits such
that every commit walk starting at interesting commits ends in a root
commit or unintersting commit. We then need to discover which
non-commit objects are reachable from  uninteresting commits. This
commit walk is not changing during this series.

The mark_edges_uninteresting() method in list-objects.c iterates on
the commit list and does the following:

* If the commit is UNINTERSTING, then mark its root tree and every
  object it can reach as UNINTERESTING.

* If the commit is interesting, then mark the root tree of every
  UNINTERSTING parent (and all objects that tree can reach) as
  UNINTERSTING.

At the very end, we repeat the process on every commit directly
given to the revision walk from stdin. This helps ensure we properly
cover shallow commits that otherwise were not included in the
frontier.

The logic to recursively follow trees is in the
mark_tree_uninteresting() method in revision.c. The algorithm avoids
duplicate work by not recursing into trees that are already marked
UNINTERSTING.

Add a new 'sparse' option to the mark_edges_uninteresting() method
that performs this logic in a slightly different way. As we iterate
over the commits, we add all of the root trees to an oidset. Then,
call mark_trees_uninteresting_sparse() on that oidset. Note that we
include interesting trees in this process. The current implementation
of mark_trees_unintersting_sparse() will walk the same trees as
the old logic, but this will be replaced in a later change.

Add a '--sparse' flag in 'git pack-objects' to call this new logic.
Add a new test script t/t5322-pack-objects-sparse.sh that tests this
option. The tests currently demonstrate that the resulting object
list is the same as the old algorithm. This includes a case where
both algorithms pack an object that is not needed by a remote due to
limits on the explored set of trees. When the sparse algorithm is
changed in a later commit, we will add a test that demonstrates a
change of behavior in some cases.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 13:44:39 -08:00
Jeff King
9e5da3d055 add: use separate ADD_CACHE_RENORMALIZE flag
Commit 9472935d81 (add: introduce "--renormalize", 2017-11-16) taught
git-add to pass HASH_RENORMALIZE to add_to_index(), which then passes
the flag along to index_path(). However, the flags taken by
add_to_index() and the ones taken by index_path() are distinct
namespaces. We cannot take HASH_* flags in add_to_index(), because they
overlap with the ADD_CACHE_* flags we already take (in this case,
HASH_RENORMALIZE conflicts with ADD_CACHE_IGNORE_ERRORS).

We can solve this by adding a new ADD_CACHE_RENORMALIZE flag, and using
it to set HASH_RENORMALIZE within add_to_index(). In order to make it
clear that these two flags come from distinct sets, let's also change
the name "newflags" in the function to "hash_flags".

Reported-by: Dmitriy Smirnov <dmitriy.smirnov@jetbrains.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 13:40:21 -08:00
Josh Steadmon
87c2d9d310 filter-options: expand scaled numbers
When communicating with a remote server or a subprocess, use
expanded numbers rather than numbers with scaling suffix in the
object filter spec (e.g.  "limit:blob=1k" becomes
"limit:blob=1024").

Update the protocol docs to note that clients should always perform this
expansion, to allow for more compatibility between server
implementations.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-15 15:42:31 -08:00
Junio C Hamano
0aa9d8aa6c Merge branch 'nd/the-index' into md/list-objects-filter-by-depth 2019-01-15 15:38:29 -08:00
Johannes Schindelin
fa6f225e01 add --edit: truncate the patch file
If there is already a .git/ADD_EDIT.patch file, we fail to truncate it
properly, which could result in very funny errors.

Of course, this file should not be left lying around. But at least in
one case, there was a stale copy, larger than the current diff. So the
result was a corrupt diff.

Let's just truncate the file when we write it and not worry about it too
much.

Reported by J Wyman.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-15 10:51:21 -08:00
brian m. carlson
ea82b2a085 tree-walk: store object_id in a separate member
When parsing a tree, we read the object ID directly out of the tree
buffer. This is normally fine, but such an object ID cannot be used with
oidcpy, which copies GIT_MAX_RAWSZ bytes, because if we are using SHA-1,
there may not be that many bytes to copy.

Instead, store the object ID in a separate struct member. Since we can
no longer efficiently compute the path length, store that information as
well in struct name_entry. Ensure we only copy the object ID into the
new buffer if the path length is nonzero, as some callers will pass us
an empty path with no object ID following it, and we will not want to
read past the end of the buffer.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-15 09:57:41 -08:00
Junio C Hamano
f2b6aa98be Merge branch 'nd/indentation-fix'
Code cleanup.

* nd/indentation-fix:
  Indent code with TABs
2019-01-14 15:29:32 -08:00
Junio C Hamano
6e5be1f2d5 Merge branch 'md/exclude-promisor-objects-fix-cleanup'
Code clean-up.

* md/exclude-promisor-objects-fix-cleanup:
  revision.c: put promisor option in specialized struct
2019-01-14 15:29:31 -08:00
Junio C Hamano
4084df42c2 Merge branch 'nd/checkout-noisy'
"git checkout [<tree-ish>] path..." learned to report the number of
paths that have been checked out of the index or the tree-ish,
which gives it the same degree of noisy-ness as the case in which
the command checks out a branch.

* nd/checkout-noisy:
  t0027: squelch checkout path run outside test_expect_* block
  checkout: print something when checking out paths
2019-01-14 15:29:29 -08:00
Junio C Hamano
d6f05a435f Merge branch 'nd/attr-pathspec-in-tree-walk'
The traversal over tree objects has learned to honor
":(attr:label)" pathspec match, which has been implemented only for
enumerating paths on the filesystem.

* nd/attr-pathspec-in-tree-walk:
  tree-walk: support :(attr) matching
  dir.c: move, rename and export match_attrs()
  pathspec.h: clean up "extern" in function declarations
  tree-walk.c: make tree_entry_interesting() take an index
  tree.c: make read_tree*() take 'struct repository *'
2019-01-14 15:29:28 -08:00
Nguyễn Thái Ngọc Duy
150fe065f7 read-cache.c: remove the_* from index_has_changes()
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 12:13:05 -08:00
Nguyễn Thái Ngọc Duy
0d6caa2d08 merge-recursive.c: remove implicit dependency on the_index
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 12:13:04 -08:00
Nguyễn Thái Ngọc Duy
3a7a698e93 sha1-name.c: remove implicit dependency on the_index
This kills the_index dependency in get_oid_with_context() but for
get_oid() and friends, they still assume the_repository (which also
means the_index).

Unfortunately the widespread use of get_oid() will make it hard to
make the conversion now. We probably will add repo_get_oid() at some
point and limit the use of get_oid() in builtin/ instead of forcing
all get_oid() call sites to carry struct repository.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 12:13:04 -08:00
Nguyễn Thái Ngọc Duy
1b0d968b34 read-cache.c: replace update_index_if_able with repo_&
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 12:13:04 -08:00
Nguyễn Thái Ngọc Duy
e1ff0a32e4 read-cache.c: kill read_index()
read_index() shares the same problem as hold_locked_index(): it
assumes $GIT_DIR/index. Move all call sites to repo_read_index()
instead. read_index_preload() and read_index_unmerged() are also
killed as a consequence.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 12:13:04 -08:00
Nguyễn Thái Ngọc Duy
fb4a8464a6 checkout: avoid the_index when possible
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 12:13:04 -08:00
Nguyễn Thái Ngọc Duy
1d18d7581c notes-utils.c: remove the_repository references
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 12:13:04 -08:00
Nguyễn Thái Ngọc Duy
dba093ddc0 grep: use grep_opt->repo instead of explict repo argument
This command is probably the first one that operates on a repository
other than the_repository, in f9ee2fcdfa (grep: recurse in-process
using 'struct repository' - 2017-08-02). An explicit 'struct
repository *' was added in that commit to pass around the repository
that we're supposed to grep from.

Since 38bbc2ea39 (grep.c: remove implicit dependency on the_index -
2018-09-21). 'struct grep_opt *' carries in itself a repository
parameter for grepping. We should now be able to reuse grep_opt to
hold the submodule repo instead of a separate argument, which is just
room for mistakes.

While at there, use the right reference instead of the_repository and
the_index in this code. I was a bit careless in my attempt to kick
the_repository / the_index out of library code. It's normally safe to
just stick the_repository / the_index in bultin/ code, but it's not
the case for grep.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 12:13:04 -08:00
Junio C Hamano
17069c7fae Merge branch 'ms/packet-err-check' into jt/fetch-v2-sideband
* ms/packet-err-check:
  pack-protocol.txt: accept error packets in any context
  Use packet_reader instead of packet_read_line
2019-01-14 11:16:04 -08:00
Christian Couder
e01378753d fetch: fix extensions.partialclone name in error message
There is "extensions.partialclone" and "core.partialCloneFilter", but
not "core.partialclone". Only "extensions.partialclone" is meant to
contain a remote name.

While at it, let's wrap the relevant code lines to keep them at a
reasonable length.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14 10:49:09 -08:00
Jeff King
d64bb065c0 merge-recursive: copy $GITHEAD strings
If $GITHEAD_1234abcd is set in the environment, we use its value as a
"better branch name" in generating conflict markers. However, we pick
these better names early in the process, and the return value from
getenv() is not guaranteed to stay valid.

Let's make a copy of the returned string. And to make memory management
easier, let's just always return an allocated string from
better_branch_name(), so we know that it must always be freed.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-11 18:48:59 -08:00
Jeff King
e5b07c539d init: make a copy of $GIT_DIR string
We pass the result of getenv("GIT_DIR") to init_db() and assume that the
string remains valid. But that's not guaranteed across calls to setenv()
or even getenv(), although it often works in practice. Let's make a copy
of the string so that we follow the rules.

Note that we need to mark it with UNLEAK(), since the value persists
until the end of program (but we have no opportunity to free it).

This patch also handles $GIT_WORK_TREE the same way. It actually doesn't
have as long a lifetime and is probably fine, but it's simpler to just
treat the two side-by-side variables the same.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-11 18:48:59 -08:00
Jeff King
423ff9bef0 config: make a copy of $GIT_CONFIG string
cmd_config() points our source filename pointer at the return value of
getenv(), but that value may be invalidated by further calls to
environment functions. Let's copy it to make sure it remains valid.

We don't need to bother freeing it, as it remains part of the
whole-process global state until we exit.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-11 18:48:58 -08:00
Jeff King
406bab3811 commit: copy saved getenv() result
We save the result of $GIT_INDEX_FILE so that we can restore it after
setting it to a new value and running add--interactive. However, the
pointer returned by getenv() is not guaranteed to be valid after calling
setenv(). This _usually_ works fine, but can fail if libc needs to
reallocate the environment block during the setenv().

Let's just duplicate the string, so we know that it remains valid.

In the long run it may be more robust to teach interactive_add() to take
a set of environment variables to pass along to run-command when it
execs add--interactive. And then we would not have to do this
save/restore dance at all. But this is an easy fix in the meantime.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-11 18:48:58 -08:00
Issac Trotts
ad6f028f06 log: add %S option (like --source) to log --format
Make it possible to write for example

        git log --format="%H,%S"

where the %S at the end is a new placeholder that prints out the ref
(tag/branch) for each commit.

Using %d might seem like an alternative but it only shows the ref for the last
commit in the branch.

Signed-off-by: Issac Trotts <issactrotts@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-11 10:28:11 -08:00
Johannes Schindelin
5bdece0d70 gc/repack: release packs when needed
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we introduced the
close_all_packs() function.

Earlier, we made sure that the packs are released just before `git gc`
is spawned, in case that gc wants to remove no-longer needed packs.

But this developer forgot that gc itself also needs to let go of packs,
e.g. when consolidating all packs via the --aggressive option.

Likewise, `git repack -d` wants to delete obsolete packs and therefore
needs to close all pack handles, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-11 09:22:39 -08:00
Jonathan Tan
4316ff3068 fetch-pack: support protocol version 2
When the scaffolding for protocol version 2 was initially added in
8f6982b4e1 ("protocol: introduce enum protocol_version value
protocol_v2", 2018-03-14). As seen in:

    git log -p -G'support for protocol v2 not implemented yet' --full-diff --reverse v2.17.0..v2.20.0

Many of those scaffolding "die" placeholders were removed, but we
hadn't gotten around to fetch-pack yet.

The test here for "fetch refs from cmdline" is very minimal. There's
much better coverage when running the entire test suite under the WIP
GIT_TEST_PROTOCOL_VERSION=2 mode[1], we should ideally have better
coverage without needing to invoke a special test mode.

1. https://public-inbox.org/git/20181213155817.27666-1-avarab@gmail.com/

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-10 11:18:36 -08:00
Thomas Gummerer
1495ff7da5 checkout: introduce checkout.overlayMode config
In the previous patch we introduced a new no-overlay mode for git
checkout.  Some users (such as the author of this commit) may want to
have this mode turned on by default as it matches their mental model
more closely.  Make that possible by introducing a new config option
to that extend.

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-08 14:49:28 -08:00
Thomas Gummerer
091e04bc8c checkout: introduce --{,no-}overlay option
Currently 'git checkout' is defined as an overlay operation, which
means that if in 'git checkout <tree-ish> -- [<pathspec>]' we have an
entry in the index that matches <pathspec>, but that doesn't exist in
<tree-ish>, that entry will not be removed from the index or the
working tree.

Introduce a new --{,no-}overlay option, which allows using 'git
checkout' in non-overlay mode, thus removing files from the working
tree if they do not exist in <tree-ish> but match <pathspec>.

Note that 'git checkout -p <tree-ish> -- [<pathspec>]' already works
this way, so no changes are needed for the patch mode.  We disallow
'git checkout --overlay -p' to avoid confusing users who would expect
to be able to force overlay mode in 'git checkout -p' this way.

Untracked files are not affected by this change, so 'git checkout
--no-overlay HEAD -- untracked' will not remove untracked from the
working tree.  This is so e.g. 'git checkout --no-overlay HEAD -- dir/'
doesn't delete all untracked files in dir/, but rather just resets the
state of files that are known to git.

Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-08 14:49:28 -08:00
Jeff King
98374a07c9 convert has_sha1_file() callers to has_object_file()
The only remaining callers of has_sha1_file() actually have an object_id
already. They can use the "object" variant, rather than dereferencing
the hash themselves.

The code changes here were completely generated by the included
coccinelle patch.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-08 09:41:06 -08:00
Jeff King
c93206b412 update comment references to sha1_object_info()
Commit abef9020e3 (sha1_file: convert sha1_object_info* to object_id,
2018-03-12) renamed the function to oid_object_info(), but missed some
comments which mention it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-08 09:40:19 -08:00