Commit Graph

8331 Commits

Author SHA1 Message Date
Matheus Tavares
faf123c730 grep: fix race conditions at grep_submodule()
There're currently two function calls in builtin/grep.c:grep_submodule()
which might result in race conditions:

- submodule_from_path(): it has config_with_options() in its call stack
  which, in turn, may have read_object_file() in its own. Therefore,
  calling the first function without acquiring grep_read_mutex may end
  up causing a race condition with other object read operations
  performed by worker threads (for example, at the fill_textconv()
  call in grep.c:fill_textconv_grep()).
- parse_object_or_die(): it falls into the same problem, having
  repo_has_object_file(the_repository, ...) in its call stack. Besides
  that, parse_object(), which is also called by parse_object_or_die(),
  is thread-unsafe and also called by object reading functions.

It's unlikely to really fall into a data race with these operations as
the volume of calls to them is usually very low. But we better protect
ourselves against this possibility, anyway. So, to solve these issues,
move both of these function calls into the critical section of
grep_read_mutex.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-17 13:52:14 -08:00
Hans Jerry Illikainen
54887b4689 gpg-interface: add minTrustLevel as a configuration option
Previously, signature verification for merge and pull operations checked
if the key had a trust-level of either TRUST_NEVER or TRUST_UNDEFINED in
verify_merge_signature().  If that was the case, the process die()d.

The other code paths that did signature verification relied entirely on
the return code from check_commit_signature().  And signatures made with
a good key, irregardless of its trust level, was considered valid by
check_commit_signature().

This difference in behavior might induce users to erroneously assume
that the trust level of a key in their keyring is always considered by
Git, even for operations where it is not (e.g. during a verify-commit or
verify-tag).

The way it worked was by gpg-interface.c storing the result from the
key/signature status *and* the lowest-two trust levels in the `result`
member of the signature_check structure (the last of these status lines
that were encountered got written to `result`).  These are documented in
GPG under the subsection `General status codes` and `Key related`,
respectively [1].

The GPG documentation says the following on the TRUST_ status codes [1]:

    """
    These are several similar status codes:

    - TRUST_UNDEFINED <error_token>
    - TRUST_NEVER     <error_token>
    - TRUST_MARGINAL  [0  [<validation_model>]]
    - TRUST_FULLY     [0  [<validation_model>]]
    - TRUST_ULTIMATE  [0  [<validation_model>]]

    For good signatures one of these status lines are emitted to
    indicate the validity of the key used to create the signature.
    The error token values are currently only emitted by gpgsm.
    """

My interpretation is that the trust level is conceptionally different
from the validity of the key and/or signature.  That seems to also have
been the assumption of the old code in check_signature() where a result
of 'G' (as in GOODSIG) and 'U' (as in TRUST_NEVER or TRUST_UNDEFINED)
were both considered a success.

The two cases where a result of 'U' had special meaning were in
verify_merge_signature() (where this caused git to die()) and in
format_commit_one() (where it affected the output of the %G? format
specifier).

I think it makes sense to refactor the processing of TRUST_ status lines
such that users can configure a minimum trust level that is enforced
globally, rather than have individual parts of git (e.g. merge) do it
themselves (except for a grace period with backward compatibility).

I also think it makes sense to not store the trust level in the same
struct member as the key/signature status.  While the presence of a
TRUST_ status code does imply that the signature is good (see the first
paragraph in the included snippet above), as far as I can tell, the
order of the status lines from GPG isn't well-defined; thus it would
seem plausible that the trust level could be overwritten with the
key/signature status if they were stored in the same member of the
signature_check structure.

This patch introduces a new configuration option: gpg.minTrustLevel.  It
consolidates trust-level verification to gpg-interface.c and adds a new
`trust_level` member to the signature_check structure.

Backward-compatibility is maintained by introducing a special case in
verify_merge_signature() such that if no user-configurable
gpg.minTrustLevel is set, then the old behavior of rejecting
TRUST_UNDEFINED and TRUST_NEVER is enforced.  If, on the other hand,
gpg.minTrustLevel is set, then that value overrides the old behavior.

Similarly, the %G? format specifier will continue show 'U' for
signatures made with a key that has a trust level of TRUST_UNDEFINED or
TRUST_NEVER, even though the 'U' character no longer exist in the
`result` member of the signature_check structure.  A new format
specifier, %GT, is also introduced for users that want to show all
possible trust levels for a signature.

Another approach would have been to simply drop the trust-level
requirement in verify_merge_signature().  This would also have made the
behavior consistent with other parts of git that perform signature
verification.  However, requiring a minimum trust level for signing keys
does seem to have a real-world use-case.  For example, the build system
used by the Qubes OS project currently parses the raw output from
verify-tag in order to assert a minimum trust level for keys used to
sign git tags [2].

[1] https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob;f=doc/doc/DETAILS;h=bd00006e933ac56719b1edd2478ecd79273eae72;hb=refs/heads/master
[2] 9674c1991d/scripts/verify-git-tag (L43)

Signed-off-by: Hans Jerry Illikainen <hji@dyntopia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15 14:06:06 -08:00
Heba Waly
bf66db37f1 add: use advise function to display hints
Use the advise function in advice.c to display hints to the users, as
it provides a neat and a standard format for hint messages, i.e: the
text is colored in yellow and the line starts by the word "hint:".

Also this will enable us to control the messages using advice.*
configuration variables.

Signed-off-by: Heba Waly <heba.waly@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15 12:15:04 -08:00
Junio C Hamano
4d924528d8 Revert "Merge branch 'ra/rebase-i-more-options'"
This reverts commit 5d9324e0f4, reversing
changes made to c58ae96fc4.

The topic turns out to be too buggy for real use.

cf. <f2fe7437-8a48-3315-4d3f-8d51fe4bb8f1@gmail.com>
2020-01-12 13:25:18 -08:00
Jeff King
e701bab3e9 restore: invalidate cache-tree when removing entries with --staged
When "git restore --staged <path>" removes a path that's in the index,
it marks the entry with CE_REMOVE, but we don't do anything to
invalidate the cache-tree. In the non-staged case, we end up in
checkout_worktree(), which calls remove_marked_cache_entries(). That
actually drops the entries from the index, as well as invalidating the
cache-tree and untracked-cache.

But with --staged, we never call checkout_worktree(), and the CE_REMOVE
entries remain. Interestingly, they are dropped when we write out the
index, but that means the resulting index is inconsistent: its
cache-tree will not match the actual entries, and running "git commit"
immediately after will create the wrong tree.

We can solve this by calling remove_marked_cache_entries() ourselves
before writing out the index. Note that we can't just hoist it out of
checkout_worktree(); that function needs to iterate over the CE_REMOVE
entries (to drop their matching worktree files) before removing them.

One curiosity about the test: without this patch, it actually triggers a
BUG() when running git-restore:

  BUG: cache-tree.c:810: new1 with flags 0x4420000 should not be in cache-tree

But in the original problem report, which used a similar recipe,
git-restore actually creates the bogus index (and the commit is created
with the wrong tree). I'm not sure why the test here behaves differently
than my out-of-suite reproduction, but what's here should catch either
symptom (and the fix corrects both cases).

Reported-by: Torsten Krah <krah.tm@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-08 09:03:36 -08:00
Alexandr Miloslavskiy
fa74180d08 checkout: don't revert file on ambiguous tracking branches
For easier understanding, here are the existing good scenarios:

  1) Have *no* file 'foo', *no* local branch 'foo' and a *single*
     remote branch 'foo'
  2) `git checkout foo` will create local branch foo, see [1]

  and

  1) Have *a* file 'foo', *no* local branch 'foo' and a *single*
     remote branch 'foo'
  2) `git checkout foo` will complain, see [3]

This patch prevents the following scenario:

  1) Have *a* file 'foo', *no* local branch 'foo' and *multiple*
     remote branches 'foo'
  2) `git checkout foo` will successfully... revert contents of
     file `foo`!

That is, adding another remote suddenly changes behavior significantly,
which is a surprise at best and could go unnoticed by user at worst.
Please see [3] which gives some real world complaints.

To my understanding, fix in [3] overlooked the case of multiple remotes,
and the whole behavior of falling back to reverting file was never
intended:

  [1] introduces the unexpected behavior. Before, there was fallback
  from not-a-ref to pathspec. This is reasonable fallback. After, there
  is another fallback from ambiguous-remote to pathspec. I understand
  that it was a copy&paste oversight.

  [2] noticed the unexpected behavior but chose to semi-document it
  instead of forbidding, because the goal of the patch series was
  focused on something else.

  [3] adds `die()` when there is ambiguity between branch and file. The
  case of multiple tracking branches is seemingly overlooked.

The new behavior: if there is no local branch and multiple remote
candidates, just die() and don't try reverting file whether it
exists (prevents surprise) or not (improves error message).

[1] Commit 70c9ac2f ("DWIM "git checkout frotz" to "git checkout -b frotz origin/frotz"" 2009-10-18)
    https://public-inbox.org/git/7vaazpxha4.fsf_-_@alter.siamese.dyndns.org/
[2] Commit ad8d5104 ("checkout: add advice for ambiguous "checkout <branch>"", 2018-06-05)
    https://public-inbox.org/git/20180502105452.17583-1-avarab@gmail.com/
[3] Commit be4908f1 ("checkout: disambiguate dwim tracking branches and local files", 2018-11-13)
    https://public-inbox.org/git/20181110120707.25846-1-pclouds@gmail.com/

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-07 13:15:38 -08:00
Alexandr Miloslavskiy
2957709bd4 parse_branchname_arg(): extract part as new function
This is done for the next commit to avoid crazy 7x tab code padding.

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-07 13:15:37 -08:00
Junio C Hamano
037f067587 Merge branch 'ds/commit-graph-set-size-mult'
The code to write split commit-graph file(s) upon fetching computed
bogus value for the parameter used in splitting the resulting
files, which has been corrected.

* ds/commit-graph-set-size-mult:
  commit-graph: prefer default size_mult when given zero
2020-01-06 14:17:51 -08:00
Junio C Hamano
c20d4fd44a Merge branch 'ds/sparse-list-in-cone-mode'
"git sparse-checkout list" subcommand learned to give its output in
a more concise form when the "cone" mode is in effect.

* ds/sparse-list-in-cone-mode:
  sparse-checkout: document interactions with submodules
  sparse-checkout: list directories in cone mode
2020-01-06 14:17:51 -08:00
Derrick Stolee
63020f175f commit-graph: prefer default size_mult when given zero
In 50f26bd ("fetch: add fetch.writeCommitGraph config setting",
2019-09-02), the fetch builtin added the capability to write a
commit-graph using the "--split" feature. This feature creates
multiple commit-graph files, and those can merge based on a set
of "split options" including a size multiple. The default size
multiple is 2, which intends to provide a log_2 N depth of the
commit-graph chain where N is the number of commits.

However, I noticed during dogfooding that my commit-graph chains
were becoming quite large when left only to builds by 'git fetch'.
It turns out that in split_graph_merge_strategy(), we default the
size_mult variable to 2 except we override it with the context's
split_opts if they exist. In builtin/fetch.c, we create such a
split_opts, but do not populate it with values.

This problem is due to two failures:

 1. It is unclear that we can add the flag COMMIT_GRAPH_WRITE_SPLIT
    with a NULL split_opts.
 2. If we have a non-NULL split_opts, then we override the default
    values even if a zero value is given.

Correct both of these issues. First, do not override size_mult when
the options provide a zero value. Second, stop creating a split_opts
in the fetch builtin.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-02 13:46:34 -08:00
Junio C Hamano
e0e1ac5db0 Merge branch 'en/rebase-signoff-fix'
"git rebase --signoff" stopped working when the command was written
in C, which has been corrected.

* en/rebase-signoff-fix:
  rebase: fix saving of --signoff state for am-based rebases
2020-01-02 12:38:30 -08:00
Derrick Stolee
de11951b03 sparse-checkout: list directories in cone mode
When core.sparseCheckoutCone is enabled, the 'git sparse-checkout set'
command takes a list of directories as input, then creates an ordered
list of sparse-checkout patterns such that those directories are
recursively included and all sibling entries along the parent directories
are also included. Listing the patterns is less user-friendly than the
directories themselves.

In cone mode, and as long as the patterns match the expected cone-mode
pattern types, change the output of 'git sparse-checkout list' to only
show the directories that created the patterns.

With this change, the following piped commands would not change the
working directory:

	git sparse-checkout list | git sparse-checkout set --stdin

The only time this would not work is if core.sparseCheckoutCone is
true, but the sparse-checkout file contains patterns that do not
match the expected pattern types for cone mode.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-30 09:07:18 -08:00
Junio C Hamano
65099bd775 Merge branch 'mr/bisect-save-pointer-to-const-string'
Code cleanup.

* mr/bisect-save-pointer-to-const-string:
  bisect--helper: convert `*_warning` char pointers to char arrays.
2019-12-25 11:22:01 -08:00
Junio C Hamano
45b96a6fa1 Merge branch 'js/add-p-in-c'
The effort to move "git-add--interactive" to C continues.

* js/add-p-in-c:
  built-in add -p: show helpful hint when nothing can be staged
  built-in add -p: only show the applicable parts of the help text
  built-in add -p: implement the 'q' ("quit") command
  built-in add -p: implement the '/' ("search regex") command
  built-in add -p: implement the 'g' ("goto") command
  built-in add -p: implement hunk editing
  strbuf: add a helper function to call the editor "on an strbuf"
  built-in add -p: coalesce hunks after splitting them
  built-in add -p: implement the hunk splitting feature
  built-in add -p: show different prompts for mode changes and deletions
  built-in app -p: allow selecting a mode change as a "hunk"
  built-in add -p: handle deleted empty files
  built-in add -p: support multi-file diffs
  built-in add -p: offer a helpful error message when hunk navigation failed
  built-in add -p: color the prompt and the help text
  built-in add -p: adjust hunk headers as needed
  built-in add -p: show colored hunks by default
  built-in add -i: wire up the new C code for the `patch` command
  built-in add -i: start implementing the `patch` functionality in C
2019-12-25 11:22:01 -08:00
Junio C Hamano
87cbb1ca66 Merge branch 'rs/ref-read-cleanup'
Code cleanup.

* rs/ref-read-cleanup:
  remote: pass NULL to read_ref_full() because object ID is not needed
  refs: pass NULL to refs_read_ref_full() because object ID is not needed
2019-12-25 11:22:00 -08:00
Junio C Hamano
4bfc9ccfb6 Merge branch 'mr/bisect-use-after-free'
Use-after-free fix.

* mr/bisect-use-after-free:
  bisect--helper: avoid use-after-free
2019-12-25 11:21:59 -08:00
Junio C Hamano
bd72a08d6c Merge branch 'ds/sparse-cone'
Management of sparsely checked-out working tree has gained a
dedicated "sparse-checkout" command.

* ds/sparse-cone: (21 commits)
  sparse-checkout: improve OS ls compatibility
  sparse-checkout: respect core.ignoreCase in cone mode
  sparse-checkout: check for dirty status
  sparse-checkout: update working directory in-process for 'init'
  sparse-checkout: cone mode should not interact with .gitignore
  sparse-checkout: write using lockfile
  sparse-checkout: use in-process update for disable subcommand
  sparse-checkout: update working directory in-process
  sparse-checkout: sanitize for nested folders
  unpack-trees: add progress to clear_ce_flags()
  unpack-trees: hash less in cone mode
  sparse-checkout: init and set in cone mode
  sparse-checkout: use hashmaps for cone patterns
  sparse-checkout: add 'cone' mode
  trace2: add region in clear_ce_flags
  sparse-checkout: create 'disable' subcommand
  sparse-checkout: add '--stdin' option to set subcommand
  sparse-checkout: 'set' subcommand
  clone: add --sparse mode
  sparse-checkout: create 'init' subcommand
  ...
2019-12-25 11:21:58 -08:00
Junio C Hamano
f3c520e17f Merge branch 'sg/name-rev-wo-recursion'
Redo "git name-rev" to avoid recursive calls.

* sg/name-rev-wo-recursion:
  name-rev: cleanup name_ref()
  name-rev: eliminate recursion in name_rev()
  name-rev: use 'name->tip_name' instead of 'tip_name'
  name-rev: drop name_rev()'s 'generation' and 'distance' parameters
  name-rev: restructure creating/updating 'struct rev_name' instances
  name-rev: restructure parsing commits and applying date cutoff
  name-rev: pull out deref handling from the recursion
  name-rev: extract creating/updating a 'struct name_rev' into a helper
  t6120: add a test to cover inner conditions in 'git name-rev's name_rev()
  name-rev: use sizeof(*ptr) instead of sizeof(type) in allocation
  name-rev: avoid unnecessary cast in name_ref()
  name-rev: use strbuf_strip_suffix() in get_rev_name()
  t6120-describe: modernize the 'check_describe' helper
  t6120-describe: correct test repo history graph in comment
2019-12-25 11:21:58 -08:00
Junio C Hamano
17066bea38 Merge branch 'dl/format-patch-notes-config-fixup'
"git format-patch" can take a set of configured format.notes values
to specify which notes refs to use in the log message part of the
output.  The behaviour of this was not consistent with multiple
--notes command line options, which has been corrected.

* dl/format-patch-notes-config-fixup:
  notes.h: fix typos in comment
  notes: break set_display_notes() into smaller functions
  config/format.txt: clarify behavior of multiple format.notes
  format-patch: move git_config() before repo_init_revisions()
  format-patch: use --notes behavior for format.notes
  notes: extract logic into set_display_notes()
  notes: create init_display_notes() helper
  notes: rename to load_display_notes()
2019-12-25 11:21:58 -08:00
Junio C Hamano
135365dd99 Merge branch 'am/pathspec-f-f-checkout'
A few more commands learned the "--pathspec-from-file" command line
option.

* am/pathspec-f-f-checkout:
  checkout, restore: support the --pathspec-from-file option
  doc: restore: synchronize <pathspec> description
  doc: checkout: synchronize <pathspec> description
  doc: checkout: fix broken text reference
  doc: checkout: remove duplicate synopsis
  add: support the --pathspec-from-file option
  cmd_add: prepare for next patch
2019-12-25 11:21:57 -08:00
Junio C Hamano
ff0cb70d45 Merge branch 'am/pathspec-from-file'
An earlier series to teach "--pathspec-from-file" to "git commit"
forgot to make the option incompatible with "--all", which has been
corrected.

* am/pathspec-from-file:
  commit: forbid --pathspec-from-file --all
2019-12-25 11:21:57 -08:00
Johannes Schindelin
c480eeb574 commit --interactive: make it work with the built-in add -i
The built-in `git add -i` machinery obviously has its `the_repository`
structure initialized at the point where `cmd_commit()` calls it, and
therefore does not look at the environment variable `GIT_INDEX_FILE`.

But when being called from `commit --interactive`, it has to, because
the index was already locked in that case, and we want to ask the
interactive add machinery to work on the `index.lock` file instead of
the `index` file.

Technically, we could teach `run_add_i()`, or for that matter
`run_add_p()`, to look specifically at that environment variable, but
the entire idea of passing in a parameter of type `struct repository *`
is to allow working on multiple repositories (and their index files)
independently.

So let's instead override the `index_file` field of that structure
temporarily.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-21 16:06:22 -08:00
Johannes Schindelin
cee6cb7300 built-in add -p: implement the "worktree" patch modes
This is a straight-forward port of 2f0896ec3a (restore: support
--patch, 2019-04-25) which added support for `git restore -p`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-21 16:06:22 -08:00
Johannes Schindelin
52628f94fc built-in add -p: implement the "checkout" patch modes
This patch teaches the built-in `git add -p` machinery all the tricks it
needs to know in order to act as the work horse for `git checkout -p`.

Apart from the minor changes (slightly reworded messages, different
`diff` and `apply --check` invocations), it requires a new function to
actually apply the changes, as `git checkout -p` is a bit special in
that respect: when the desired changes do not apply to the index, but
apply to the work tree, Git does not fail straight away, but asks the
user whether to apply the changes to the worktree at least.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-21 16:06:22 -08:00
Johannes Schindelin
6610e4628a built-in stash: use the built-in git add -p if so configured
The scripted version of `git stash` called directly into the Perl script
`git-add--interactive.perl`, and this was faithfully converted to C.

However, we have a much better way to do this now: call the internal API
directly, which will now incidentally also respect the
`add.interactive.useBuiltin` setting. Let's just do this.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-21 16:06:21 -08:00
Johannes Schindelin
90a6bb98d1 legacy stash -p: respect the add.interactive.usebuiltin setting
As `git add` traditionally did not expose the `--patch=<mode>` modes via
command-line options, the scripted version of `git stash` had to call
`git add--interactive` directly.

But this prevents the built-in `add -p` from kicking in, as
`add--interactive` is the scripted version (which does not have a
"fall-back" to the built-in version).

So let's introduce support for internal switch for `git add` that the
scripted `git stash` can use to call the appropriate backend (scripted
or built-in, depending on `add.interactive.useBuiltin`).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-21 16:06:21 -08:00
Johannes Schindelin
36bae1dc0e built-in add -p: implement the "stash" and "reset" patch modes
The `git stash` and `git reset` commands support a `--patch` option, and
both simply hand off to `git add -p` to perform that work. Let's teach
the built-in version of that command to be able to perform that work, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-21 16:06:21 -08:00
Johannes Schindelin
d2a233cb8b built-in add -p: prepare for patch modes other than "stage"
The Perl script backing `git add -p` is used not only for that command,
but also for `git stash -p`, `git reset -p` and `git checkout -p`.

In preparation for teaching the C version of `git add -p` to support
also the latter commands, let's abstract away what is "stage" specific
into a dedicated data structure describing the differences between the
patch modes.

Finally, please note that the Perl version tries to make sure that the
diffs are only generated for the modified files. This is not actually
necessary, as the calls to Git's diff machinery already perform that
work, and perform it well. This makes it unnecessary to port the
`FILTER` field of the `%patch_modes` struct, as well as the
`get_diff_reference()` function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-21 16:06:21 -08:00
Elijah Newren
4fe7e43c53 rebase: fix saving of --signoff state for am-based rebases
This was an error introduced in the conversion from shell in commit
21853626ea ("built-in rebase: call `git am` directly", 2019-01-18),
which was noticed by a random browsing of the code.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-20 11:29:49 -08:00
Heba Waly
5c4f55f1f6 commit: honor advice.statusHints when rejecting an empty commit
In ea9882bfc4 (commit: disable status hints when writing to
COMMIT_EDITMSG, 2013-09-12) the intent was to disable status hints
when writing to COMMIT_EDITMSG, because giving the hints in the "git
status" like output in the commit message template are too late to
be useful (they say things like "'git add' to stage", but that is
only possible after aborting the current "git commit" session).

But there is one case that the hints can be useful: When the current
attempt to commit is rejected because no change is recorded in the
index.  The message is given and "git commit" errors out, so the
hints can immediately be followed by the user.  Teach the codepath
to honor the configuration variable.

Signed-off-by: Heba Waly <heba.waly@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-19 11:58:08 -08:00
Alexandr Miloslavskiy
509efef789 commit: forbid --pathspec-from-file --all
I forgot this in my previous patch `--pathspec-from-file` for
`git commit` [1]. When both `--pathspec-from-file` and `--all` were
specified, `--all` took precedence and `--pathspec-from-file` was
ignored. Before `--pathspec-from-file` was implemented, this case was
prevented by this check in `parse_and_validate_options()` :

    die(_("paths '%s ...' with -a does not make sense"), argv[0]);

It is unfortunate that these two cases are disconnected. This came as
result of how the code was laid out before my patches, where `pathspec`
is parsed outside of `parse_and_validate_options()`. This branch is
already full of refactoring patches and I did not dare to go for another
one.

Fix by mirroring `die()` for `--pathspec-from-file` as well.

[1] Commit e440fc58 ("commit: support the --pathspec-from-file option" 2019-11-19)

Reported-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-18 14:14:14 -08:00
Tanushree Tumane
7c5cea7242 bisect--helper: convert *_warning char pointers to char arrays.
Instead of using a pointer that points at a constant string,
just give name directly to the constant string; this way, we
do not have to allocate a pointer variable in addition to
the string we want to use.

Let's convert `need_bad_and_good_revision_warning` and
`need_bisect_start_warning` char pointers to char arrays.

Mentored-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Tanushree Tumane <tanushreetumane@gmail.com>
Signed-off-by: Miriam Rubio <mirucam@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-17 14:55:36 -08:00
Junio C Hamano
59d0b3be45 Merge branch 'rs/patch-id-use-oid-to-hex'
Code cleanup.

* rs/patch-id-use-oid-to-hex:
  patch-id: use oid_to_hex() to print multiple object IDs
2019-12-16 13:14:48 -08:00
Junio C Hamano
e3b72391d1 Merge branch 'rs/commit-export-env-simplify'
Code cleanup.

* rs/commit-export-env-simplify:
  commit: use strbuf_add() to add a length-limited string
2019-12-16 13:14:47 -08:00
Junio C Hamano
3a44db2ed2 Merge branch 'dr/branch-usage-casefix'
Message fix.

* dr/branch-usage-casefix:
  l10n: minor case fix in 'git branch' '--unset-upstream' description
2019-12-16 13:14:46 -08:00
Junio C Hamano
d1c0fe8d9b Merge branch 'dl/range-diff-with-notes'
Code clean-up.

* dl/range-diff-with-notes:
  range-diff: clear `other_arg` at end of function
  range-diff: mark pointers as const
  t3206: fix incorrect test name
2019-12-16 13:08:46 -08:00
Junio C Hamano
71a7de7a99 Merge branch 'dl/rebase-with-autobase'
"git rebase" did not work well when format.useAutoBase
configuration variable is set, which has been corrected.

* dl/rebase-with-autobase:
  rebase: fix format.useAutoBase breakage
  format-patch: teach --no-base
  t4014: use test_config()
  format-patch: fix indentation
  t3400: demonstrate failure with format.useAutoBase
2019-12-16 13:08:32 -08:00
Junio C Hamano
37c2619d91 Merge branch 'ag/sequencer-todo-updates'
Reduce unnecessary reading of state variables back from the disk
during sequencer operation.

* ag/sequencer-todo-updates:
  sequencer: directly call pick_commits() from complete_action()
  rebase: fill `squash_onto' in get_replay_opts()
  sequencer: move the code writing total_nr on the disk to a new function
  sequencer: update `done_nr' when skipping commands in a todo list
  sequencer: update `total_nr' when adding an item to a todo list
2019-12-16 13:08:31 -08:00
Johannes Schindelin
f6aa7ecc34 built-in add -i: start implementing the patch functionality in C
In the previous steps, we re-implemented the main loop of `git add -i`
in C, and most of the commands.

Notably, we left out the actual functionality of `patch`, as the
relevant code makes up more than half of `git-add--interactive.perl`,
and is actually pretty independent of the rest of the commands.

With this commit, we start to tackle that `patch` part. For better
separation of concerns, we keep the code in a separate file,
`add-patch.c`. The new code is still guarded behind the
`add.interactive.useBuiltin` config setting, and for the moment,
it can only be called via `git add -p`.

The actual functionality follows the original implementation of
5cde71d64a (git-add --interactive, 2006-12-10), but not too closely
(for example, we use string offsets rather than copying strings around,
and after seeing whether the `k` and `j` commands are applicable, in the
C version we remember which previous/next hunk was undecided, and use it
rather than looking again when the user asked to jump).

As a further deviation from that commit, We also use a comma instead of
a slash to separate the available commands in the prompt, as the current
version of the Perl script does this, and we also add a line about the
question mark ("print help") to the help text.

While it is tempting to use this conversion of `git add -p` as an excuse
to work on `apply_all_patches()` so that it does _not_ want to read a
file from `stdin` or from a file, but accepts, say, an `strbuf` instead,
we will refrain from this particular rabbit hole at this stage.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-13 12:37:13 -08:00
Derrick Stolee
190a65f9db sparse-checkout: respect core.ignoreCase in cone mode
When a user uses the sparse-checkout feature in cone mode, they
add patterns using "git sparse-checkout set <dir1> <dir2> ..."
or by using "--stdin" to provide the directories line-by-line over
stdin. This behaviour naturally looks a lot like the way a user
would type "git add <dir1> <dir2> ..."

If core.ignoreCase is enabled, then "git add" will match the input
using a case-insensitive match. Do the same for the sparse-checkout
feature.

Perform case-insensitive checks while updating the skip-worktree
bits during unpack_trees(). This is done by changing the hash
algorithm and hashmap comparison methods to optionally use case-
insensitive methods.

When this is enabled, there is a small performance cost in the
hashing algorithm. To tease out the worst possible case, the
following was run on a repo with a deep directory structure:

	git ls-tree -d -r --name-only HEAD |
		git sparse-checkout set --stdin

The 'set' command was timed with core.ignoreCase disabled or
enabled. For the repo with a deep history, the numbers were

	core.ignoreCase=false: 62s
	core.ignoreCase=true:  74s (+19.3%)

For reproducibility, the equivalent test on the Linux kernel
repository had these numbers:

	core.ignoreCase=false: 3.1s
	core.ignoreCase=true:  3.6s (+16%)

Now, this is not an entirely fair comparison, as most users
will define their sparse cone using more shallow directories,
and the performance improvement from eb42feca97 ("unpack-trees:
hash less in cone mode" 2019-11-21) can remove most of the
hash cost. For a more realistic test, drop the "-r" from the
ls-tree command to store only the first-level directories.
In that case, the Linux kernel repository takes 0.2-0.25s in
each case, and the deep repository takes one second, plus or
minus 0.05s, in each case.

Thus, we _can_ demonstrate a cost to this change, but it is
unlikely to matter to any reasonable sparse-checkout cone.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-13 12:01:02 -08:00
Denton Liu
1d7297513d notes: break set_display_notes() into smaller functions
In 8164c961e1 (format-patch: use --notes behavior for format.notes,
2019-12-09), we introduced set_display_notes() which was a monolithic
function with three mutually exclusive branches. Break the function up
into three small and simple functions that each are only responsible for
one task.

This family of functions accepts an `int *show_notes` instead of
returning a value suitable for assignment to `show_notes`. This is for
two reasons. First of all, this guarantees that the external
`show_notes` variable changes in lockstep with the
`struct display_notes_opt`. Second, this prompts future developers to be
careful about doing something meaningful with this value. In fact, a
NULL check is intentionally omitted because causing a segfault here
would tell the future developer that they are meant to use the value for
something meaningful.

One alternative was making the family of functions accept a
`struct rev_info *` instead of the `struct display_notes_opt *`, since
the former contains the `show_notes` field as well. This does not work
because we have to call git_config() before repo_init_revisions().
However, if we had a `struct rev_info`, we'd need to initialize it before
it gets assigned values from git_config(). As a result, we break the
circular dependency by having standalone `int show_notes` and
`struct display_notes_opt notes_opt` variables which temporarily hold
values from git_config() until the values are copied over to `rev`.

To implement this change, we need to get a pointer to
`rev_info::show_notes`. Unfortunately, this is not possible with
bitfields and only direct-assignment is possible. Change
`rev_info::show_notes` to a non-bitfield int so that we can get its
address.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-13 11:07:15 -08:00
René Scharfe
99f86bde83 remote: pass NULL to read_ref_full() because object ID is not needed
read_ref_full() wraps refs_read_ref_full(), which in turn wraps
refs_resolve_ref_unsafe(), which handles a NULL oid pointer of callers
not interested in the resolved object ID.  Make use of that feature to
document that mv() is such a caller.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-11 13:48:46 -08:00
Tanushree Tumane
51a0a4ed95 bisect--helper: avoid use-after-free
In 5e82c3dd22 (bisect--helper: `bisect_reset` shell function in C,
2019-01-02), the `git bisect reset` subcommand was ported to C. When the
call to `git checkout` failed, an error message was reported to the
user.

However, this error message used the `strbuf` that had just been
released already. Let's switch that around: first use it, then release
it.

Mentored-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Mentored-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Tanushree Tumane <tanushreetumane@gmail.com>
Signed-off-by: Miriam Rubio <mirucam@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-11 09:24:27 -08:00
Junio C Hamano
930078ba39 Merge branch 'hi/gpg-use-check-signature'
Hide lower-level verify_signed-buffer() API as a pure helper to
implement the public check_signature() function, in order to
encourage new callers to use the correct and more strict
validation.

* hi/gpg-use-check-signature:
  gpg-interface: prefer check_signature() for GPG verification
2019-12-10 13:11:45 -08:00
Junio C Hamano
5dd1d59d35 Merge branch 'jt/clone-recursesub-ref-advise'
The interaction between "git clone --recurse-submodules" and
alternate object store was ill-designed.  The documentation and
code have been taught to make more clear recommendations when the
users see failures.

* jt/clone-recursesub-ref-advise:
  submodule--helper: advise on fatal alternate error
  Doc: explain submodule.alternateErrorStrategy
2019-12-10 13:11:43 -08:00
Junio C Hamano
c58ae96fc4 Merge branch 'am/pathspec-from-file'
A few commands learned to take the pathspec from the
standard input or a named file, instead of taking it as the command
line arguments.

* am/pathspec-from-file:
  commit: support the --pathspec-from-file option
  doc: commit: synchronize <pathspec> description
  reset: support the `--pathspec-from-file` option
  doc: reset: synchronize <pathspec> description
  pathspec: add new function to parse file
  parse-options.h: add new options `--pathspec-from-file`, `--pathspec-file-nul`
2019-12-10 13:11:41 -08:00
Junio C Hamano
5d9324e0f4 Merge branch 'ra/rebase-i-more-options'
"git rebase -i" learned a few options that are known by "git
rebase" proper.

* ra/rebase-i-more-options:
  rebase -i: finishing touches to --reset-author-date
  rebase: add --reset-author-date
  rebase -i: support --ignore-date
  sequencer: rename amend_author to author_to_rename
  rebase -i: support --committer-date-is-author-date
  sequencer: allow callers of read_author_script() to ignore fields
  rebase -i: add --ignore-whitespace flag
2019-12-10 13:11:41 -08:00
Junio C Hamano
7034cd094b Sync with Git 2.24.1 2019-12-09 22:17:55 -08:00
Denton Liu
09ac67a183 format-patch: move git_config() before repo_init_revisions()
In 13cdf78094 (format-patch: teach format.notes config option,
2019-05-16), the order in which git_config() and repo_init_revisions()
were swapped so that `rev.notes_opt` would be initialized before
git_config() was called. This is problematic, however, as git_config()
should generally be called before repo_init_revisions().

Break this circular dependency by creating `show_notes` and `notes_opt`
which git_config() reads into. Then, copy these values over to
`rev.show_notes` and `rev.notes_opt` after repo_init_revisions() is
called.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 13:37:21 -08:00
Denton Liu
8164c961e1 format-patch: use --notes behavior for format.notes
When we had multiple `format.notes` config values where we had `<ref1>`,
`false`, `<ref2>` (in that order), then we would print out the notes for
both `<ref1>` and `<ref2>`. This doesn't make sense, however, since we
parse the config in a top-down manner and a `false` should be able to
override previous configurations, just like how `--no-notes` will
override previous `--notes`.

Duplicate the logic that handles the `--[no-]notes[=]` option to
`format.notes` for consistency. As a result, when parsing the config
from top to bottom, `format.notes = true` will behave like `--notes`,
`format.notes = <ref>` will behave like `--notes=<ref>` and
`format.notes = false` will behave like `--no-notes`.

This change isn't strictly backwards compatible but since it is an edge
case where a sane user would not mix notes refs with `false` and this
feature is relatively new (released only in v2.23.0), this change should
be harmless.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 13:37:20 -08:00
Denton Liu
1e6ed5441a notes: rename to load_display_notes()
According to the function comment, init_display_notes() was supposed to
"Load the notes machinery for displaying several notes trees." Rename
this function to load_display_notes() so that its use is more accurately
represented.

This is done because, in a future commit, we will reuse the name
init_display_notes().

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 13:36:42 -08:00
SZEDER Gábor
2866fd284c name-rev: cleanup name_ref()
Earlier patches in this series moved a couple of conditions from the
recursive name_rev() function into its caller name_ref(), for no other
reason than to make eliminating the recursion a bit easier to follow.

Since the previous patch name_rev() is not recursive anymore, so let's
move all those conditions back into name_rev().

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 13:33:01 -08:00
SZEDER Gábor
49f7a2fde9 name-rev: eliminate recursion in name_rev()
The name_rev() function calls itself recursively for each interesting
parent of the commit it got as parameter, and, consequently, it can
segfault when processing a deep history if it exhausts the available
stack space.  E.g. running 'git name-rev --all' and 'git name-rev
HEAD~100000' in the gcc, gecko-dev, llvm, and WebKit repositories
results in segfaults on my machine ('ulimit -s' reports 8192kB of
stack size limit), and nowadays the former segfaults in the Linux repo
as well (it reached the necessasry depth sometime between v5.3-rc4 and
-rc5).

Eliminate the recursion by inserting the interesting parents into a
LIFO 'prio_queue' [1] and iterating until the queue becomes empty.

Note that the parent commits must be added in reverse order to the
LIFO 'prio_queue', so their relative order is preserved during
processing, i.e. the first parent should come out first from the
queue, because otherwise performance greatly suffers on mergy
histories [2].

The stacksize-limited test 'name-rev works in a deep repo' in
't6120-describe.sh' demonstrated this issue and expected failure.  Now
the recursion is gone, so flip it to expect success.  Also gone are
the dmesg entries logging the segfault of that segfaulting 'git
name-rev' process on every execution of the test suite.

Note that this slightly changes the order of lines in the output of
'git name-rev --all', usually swapping two lines every 35 lines in
git.git or every 150 lines in linux.git.  This shouldn't matter in
practice, because the output has always been unordered anyway.

This patch is best viewed with '--ignore-all-space'.

[1] Early versions of this patch used a 'commit_list', resulting in
    ~15% performance penalty for 'git name-rev --all' in 'linux.git',
    presumably because of the memory allocation and release for each
    insertion and removal. Using a LIFO 'prio_queue' has basically no
    effect on performance.

[2] We prefer shorter names, i.e. 'v0.1~234' is preferred over
    'v0.1^2~5', meaning that usually following the first parent of a
    merge results in the best name for its ancestors.  So when later
    we follow the remaining parent(s) of a merge, and reach an already
    named commit, then we usually find that we can't give that commit
    a better name, and thus we don't have to visit any of its
    ancestors again.

    OTOH, if we were to follow the Nth parent of the merge first, then
    the name of all its ancestors would include a corresponding '^N'.
    Those are not the best names for those commits, so when later we
    reach an already named commit following the first parent of that
    merge, then we would have to update the name of that commit and
    the names of all of its ancestors as well.  Consequently, we would
    have to visit many commits several times, resulting in a
    significant slowdown.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 13:33:01 -08:00
SZEDER Gábor
fee984bcab name-rev: use 'name->tip_name' instead of 'tip_name'
Following the previous patches in this series we can get the value of
'name_rev()'s 'tip_name' parameter from the 'struct rev_name'
associated with the commit as well.

So let's use 'name->tip_name' instead, which makes the patch
eliminating the recursion of name_rev() a bit easier to follow.

Note that at this point we could drop the 'tip_name' parameter as
well, but that parameter will be necessary later, after the recursion
is eliminated.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 13:33:01 -08:00
Dimitriy Ryazantcev
11de8dd7ef l10n: minor case fix in 'git branch' '--unset-upstream' description
Signed-off-by: Dimitriy Ryazantcev <dimitriy.ryazantcev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 12:30:55 -08:00
René Scharfe
4507ecc771 patch-id: use oid_to_hex() to print multiple object IDs
flush_current_id() prints the hexadecimal representation of two object
IDs.  When the code was added in f97672225b (Add "git-patch-id" program
to generate patch ID's., 2005-06-23), sha1_to_hex() had only a single
internal static buffer, so the result of one invocation had to be stored
in a local buffer.

Since dcb3450fd8 (sha1_to_hex() usage cleanup, 2006-05-03) it rotates
through four buffers, which allows to print up to four object IDs at the
same time.  1a876a69af (patch-id: convert to use struct object_id,
2015-03-13) replaced sha1_to_hex() with oid_to_hex(), which has the same
feature.  Use it to simplify the code.

Signed-off-by: René Scharfe <l.s.r@web.de>
Acked-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 12:26:40 -08:00
René Scharfe
147ee35558 commit: use strbuf_add() to add a length-limited string
This is shorter and simpler.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-09 11:25:27 -08:00
Junio C Hamano
391fb22ac7 Merge branch 'rs/use-skip-prefix-more'
Code cleanup.

* rs/use-skip-prefix-more:
  name-rev: use skip_prefix() instead of starts_with()
  push: use skip_prefix() instead of starts_with()
  shell: use skip_prefix() instead of starts_with()
  fmt-merge-msg: use skip_prefix() instead of starts_with()
  fetch: use skip_prefix() instead of starts_with()
2019-12-06 15:09:22 -08:00
SZEDER Gábor
8c5724c585 name-rev: drop name_rev()'s 'generation' and 'distance' parameters
Following the previous patches in this series we can get the values of
name_rev()'s 'generation' and 'distance' parameters from the 'stuct
rev_name' associated with the commit as well.

Let's simplify the function's signature and remove these two
unnecessary parameters.

Note that at this point we could do the same with the 'tip_name',
'taggerdate' and 'from_tag' parameters as well, but those parameters
will be necessary later, after the recursion is eliminated.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 13:29:04 -08:00
SZEDER Gábor
3a52150301 name-rev: restructure creating/updating 'struct rev_name' instances
At the beginning of the recursive name_rev() function it creates a new
'struct rev_name' instance for each previously unvisited commit or, if
this visit results in better name for an already visited commit, then
updates the 'struct rev_name' instance attached to the commit, or
returns early.

Restructure this so it's caller creates or updates the 'struct
rev_name' instance associated with the commit to be passed as
parameter, i.e. both name_ref() before calling name_rev() and
name_rev() itself as it iterates over the parent commits.

This makes eliminating the recursion a bit easier to follow, and the
condition moved to name_ref() will be moved back to name_rev() after
the recursion is eliminated.

This change also plugs the memory leak that was temporarily unplugged
in the earlier "name-rev: pull out deref handling from the recursion"
patch in this series.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 13:29:04 -08:00
SZEDER Gábor
dd432a6ecf name-rev: restructure parsing commits and applying date cutoff
At the beginning of the recursive name_rev() function it parses the
commit it got as parameter, and returns early if the commit is older
than a cutoff limit.

Restructure this so the caller parses the commit and checks its date,
and doesn't invoke name_rev() if the commit to be passed as parameter
is older than the cutoff, i.e. both name_ref() before calling
name_rev() and name_rev() itself as it iterates over the parent
commits.

This makes eliminating the recursion a bit easier to follow, and the
condition moved to name_ref() will be moved back to name_rev() after
the recursion is eliminated.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 13:29:04 -08:00
SZEDER Gábor
dd090a8a37 name-rev: pull out deref handling from the recursion
The 'if (deref) { ... }' condition near the beginning of the recursive
name_rev() function can only ever be true in the first invocation,
because the 'deref' parameter is always 0 in the subsequent recursive
invocations.

Extract this condition from the recursion into name_rev()'s caller and
drop the function's 'deref' parameter.  This makes eliminating the
recursion a bit easier to follow, and it will be moved back into
name_rev() after the recursion is eliminated.

Furthermore, drop the condition that die()s when both 'deref' and
'generation' are non-null (which should have been a BUG() to begin
with).

Note that this change reintroduces the memory leak that was plugged in
in commit 5308224633 (name-rev: avoid leaking memory in the `deref`
case, 2017-05-04), but a later patch (name-rev: restructure
creating/updating 'struct rev_name' instances) in this series will
plug it in again.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 13:29:04 -08:00
SZEDER Gábor
766f9e39c0 name-rev: extract creating/updating a 'struct name_rev' into a helper
In a later patch in this series we'll want to do this in two places.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 13:29:04 -08:00
SZEDER Gábor
bf43abc6e6 name-rev: use sizeof(*ptr) instead of sizeof(type) in allocation
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 13:29:04 -08:00
SZEDER Gábor
e0c4da6f2a name-rev: avoid unnecessary cast in name_ref()
Casting a 'struct object' to 'struct commit' is unnecessary there,
because it's already available in the local 'commit' variable.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 13:29:04 -08:00
René Scharfe
c3794d4ccb name-rev: use strbuf_strip_suffix() in get_rev_name()
get_name_rev() basically open-codes strip_suffix() before adding a
string to a strbuf.

Let's use the strbuf right from the beginning, i.e. add the whole
string to the strbuf and then use strbuf_strip_suffix(), making the
code more idiomatic.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 13:29:04 -08:00
Denton Liu
abcf857300 range-diff: clear other_arg at end of function
We were leaking memory by not clearing `other_arg` after we were done
using it. Clear it after we've finished using it.

Note that this isn't strictly necessary since the memory will be
reclaimed once the command exits. However, since we are releasing the
strbufs, we should also clear `other_arg` for consistency.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06 12:36:53 -08:00
Johannes Schindelin
67af91c47a Sync with 2.23.1
* maint-2.23: (44 commits)
  Git 2.23.1
  Git 2.22.2
  Git 2.21.1
  mingw: sh arguments need quoting in more circumstances
  mingw: fix quoting of empty arguments for `sh`
  mingw: use MSYS2 quoting even when spawning shell scripts
  mingw: detect when MSYS2's sh is to be spawned more robustly
  t7415: drop v2.20.x-specific work-around
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  ...
2019-12-06 16:31:39 +01:00
Johannes Schindelin
7fd9fd94fb Sync with 2.22.2
* maint-2.22: (43 commits)
  Git 2.22.2
  Git 2.21.1
  mingw: sh arguments need quoting in more circumstances
  mingw: fix quoting of empty arguments for `sh`
  mingw: use MSYS2 quoting even when spawning shell scripts
  mingw: detect when MSYS2's sh is to be spawned more robustly
  t7415: drop v2.20.x-specific work-around
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  ...
2019-12-06 16:31:30 +01:00
Johannes Schindelin
5421ddd8d0 Sync with 2.21.1
* maint-2.21: (42 commits)
  Git 2.21.1
  mingw: sh arguments need quoting in more circumstances
  mingw: fix quoting of empty arguments for `sh`
  mingw: use MSYS2 quoting even when spawning shell scripts
  mingw: detect when MSYS2's sh is to be spawned more robustly
  t7415: drop v2.20.x-specific work-around
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  ...
2019-12-06 16:31:23 +01:00
Johannes Schindelin
fc346cb292 Sync with 2.20.2
* maint-2.20: (36 commits)
  Git 2.20.2
  t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  t6130/t9350: prepare for stringent Win32 path validation
  quote-stress-test: allow skipping some trials
  quote-stress-test: accept arguments to test via the command-line
  tests: add a helper to stress test argument quoting
  mingw: fix quoting of arguments
  Disallow dubiously-nested submodule git directories
  ...
2019-12-06 16:31:12 +01:00
Jonathan Nieder
c154745074 submodule: defend against submodule.update = !command in .gitmodules
In v2.15.4, we started to reject `submodule.update` settings in
`.gitmodules`. Let's raise a BUG if it somehow still made it through
from anywhere but the Git config.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
2019-12-06 16:30:50 +01:00
Johannes Schindelin
d851d94151 Sync with 2.19.3
* maint-2.19: (34 commits)
  Git 2.19.3
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  t6130/t9350: prepare for stringent Win32 path validation
  quote-stress-test: allow skipping some trials
  quote-stress-test: accept arguments to test via the command-line
  tests: add a helper to stress test argument quoting
  mingw: fix quoting of arguments
  Disallow dubiously-nested submodule git directories
  protect_ntfs: turn on NTFS protection by default
  path: also guard `.gitmodules` against NTFS Alternate Data Streams
  ...
2019-12-06 16:30:49 +01:00
Johannes Schindelin
7c9fbda6e2 Sync with 2.18.2
* maint-2.18: (33 commits)
  Git 2.18.2
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  t6130/t9350: prepare for stringent Win32 path validation
  quote-stress-test: allow skipping some trials
  quote-stress-test: accept arguments to test via the command-line
  tests: add a helper to stress test argument quoting
  mingw: fix quoting of arguments
  Disallow dubiously-nested submodule git directories
  protect_ntfs: turn on NTFS protection by default
  path: also guard `.gitmodules` against NTFS Alternate Data Streams
  is_ntfs_dotgit(): speed it up
  ...
2019-12-06 16:30:38 +01:00
Johannes Schindelin
14af7ed5a9 Sync with 2.17.3
* maint-2.17: (32 commits)
  Git 2.17.3
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  t6130/t9350: prepare for stringent Win32 path validation
  quote-stress-test: allow skipping some trials
  quote-stress-test: accept arguments to test via the command-line
  tests: add a helper to stress test argument quoting
  mingw: fix quoting of arguments
  Disallow dubiously-nested submodule git directories
  protect_ntfs: turn on NTFS protection by default
  path: also guard `.gitmodules` against NTFS Alternate Data Streams
  is_ntfs_dotgit(): speed it up
  mingw: disallow backslash characters in tree objects' file names
  ...
2019-12-06 16:29:15 +01:00
Johannes Schindelin
bdfef0492c Sync with 2.16.6
* maint-2.16: (31 commits)
  Git 2.16.6
  test-drop-caches: use `has_dos_drive_prefix()`
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  t6130/t9350: prepare for stringent Win32 path validation
  quote-stress-test: allow skipping some trials
  quote-stress-test: accept arguments to test via the command-line
  tests: add a helper to stress test argument quoting
  mingw: fix quoting of arguments
  Disallow dubiously-nested submodule git directories
  protect_ntfs: turn on NTFS protection by default
  path: also guard `.gitmodules` against NTFS Alternate Data Streams
  is_ntfs_dotgit(): speed it up
  mingw: disallow backslash characters in tree objects' file names
  path: safeguard `.git` against NTFS Alternate Streams Accesses
  ...
2019-12-06 16:27:36 +01:00
Johannes Schindelin
9ac92fed5b Sync with 2.15.4
* maint-2.15: (29 commits)
  Git 2.15.4
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  t6130/t9350: prepare for stringent Win32 path validation
  quote-stress-test: allow skipping some trials
  quote-stress-test: accept arguments to test via the command-line
  tests: add a helper to stress test argument quoting
  mingw: fix quoting of arguments
  Disallow dubiously-nested submodule git directories
  protect_ntfs: turn on NTFS protection by default
  path: also guard `.gitmodules` against NTFS Alternate Data Streams
  is_ntfs_dotgit(): speed it up
  mingw: disallow backslash characters in tree objects' file names
  path: safeguard `.git` against NTFS Alternate Streams Accesses
  clone --recurse-submodules: prevent name squatting on Windows
  is_ntfs_dotgit(): only verify the leading segment
  ...
2019-12-06 16:27:18 +01:00
Johannes Schindelin
d3ac8c3f27 Sync with 2.14.6
* maint-2.14: (28 commits)
  Git 2.14.6
  mingw: handle `subst`-ed "DOS drives"
  mingw: refuse to access paths with trailing spaces or periods
  mingw: refuse to access paths with illegal characters
  unpack-trees: let merged_entry() pass through do_add_entry()'s errors
  quote-stress-test: offer to test quoting arguments for MSYS2 sh
  t6130/t9350: prepare for stringent Win32 path validation
  quote-stress-test: allow skipping some trials
  quote-stress-test: accept arguments to test via the command-line
  tests: add a helper to stress test argument quoting
  mingw: fix quoting of arguments
  Disallow dubiously-nested submodule git directories
  protect_ntfs: turn on NTFS protection by default
  path: also guard `.gitmodules` against NTFS Alternate Data Streams
  is_ntfs_dotgit(): speed it up
  mingw: disallow backslash characters in tree objects' file names
  path: safeguard `.git` against NTFS Alternate Streams Accesses
  clone --recurse-submodules: prevent name squatting on Windows
  is_ntfs_dotgit(): only verify the leading segment
  test-path-utils: offer to run a protectNTFS/protectHFS benchmark
  ...
2019-12-06 16:26:55 +01:00
Junio C Hamano
88cf80949e Merge branch 'mg/submodule-status-from-a-subdirectory'
"git submodule status" that is run from a subdirectory of the
superproject did not work well, which has been corrected.

* mg/submodule-status-from-a-subdirectory:
  submodule: fix 'submodule status' when called from a subdirectory
2019-12-05 12:52:47 -08:00
Junio C Hamano
6b3cb32f43 Merge branch 'nl/reset-patch-takes-a-tree'
"git reset --patch $object" without any pathspec should allow a
tree object to be given, but incorrectly required a committish,
which has been corrected.

* nl/reset-patch-takes-a-tree:
  reset: parse rev as tree-ish in patch mode
2019-12-05 12:52:47 -08:00
Junio C Hamano
36fd304d81 Merge branch 'jk/fail-show-toplevel-outside-working-tree'
"git rev-parse --show-toplevel" run outside of any working tree did
not error out, which has been corrected.

* jk/fail-show-toplevel-outside-working-tree:
  rev-parse: make --show-toplevel without a worktree an error
2019-12-05 12:52:45 -08:00
Junio C Hamano
cf91c31688 Merge branch 'sg/unpack-progress-throughput'
"git unpack-objects" used to show progress based only on the number
of received and unpacked objects, which stalled when it has to
handle an unusually large object.  It now shows the throughput as
well.

* sg/unpack-progress-throughput:
  builtin/unpack-objects.c: show throughput progress
2019-12-05 12:52:45 -08:00
Junio C Hamano
f3c7bfdde2 Merge branch 'dl/range-diff-with-notes'
"git range-diff" learned to take the "--notes=<ref>" and the
"--no-notes" options to control the commit notes included in the
log message that gets compared.

* dl/range-diff-with-notes:
  format-patch: pass notes configuration to range-diff
  range-diff: pass through --notes to `git log`
  range-diff: output `## Notes ##` header
  t3206: range-diff compares logs with commit notes
  t3206: s/expected/expect/
  t3206: disable parameter substitution in heredoc
  t3206: remove spaces after redirect operators
  pretty-options.txt: --notes accepts a ref instead of treeish
  rev-list-options.txt: remove reference to --show-notes
  argv-array: add space after `while`
2019-12-05 12:52:44 -08:00
Junio C Hamano
f7998d9793 Merge branch 'js/builtin-add-i'
The beginning of rewriting "git add -i" in C.

* js/builtin-add-i:
  built-in add -i: implement the `help` command
  built-in add -i: use color in the main loop
  built-in add -i: support `?` (prompt help)
  built-in add -i: show unique prefixes of the commands
  built-in add -i: implement the main loop
  built-in add -i: color the header in the `status` command
  built-in add -i: implement the `status` command
  diff: export diffstat interface
  Start to implement a built-in version of `git add --interactive`
2019-12-05 12:52:43 -08:00
Johannes Schindelin
a8dee3ca61 Disallow dubiously-nested submodule git directories
Currently it is technically possible to let a submodule's git
directory point right into the git dir of a sibling submodule.

Example: the git directories of two submodules with the names `hippo`
and `hippo/hooks` would be `.git/modules/hippo/` and
`.git/modules/hippo/hooks/`, respectively, but the latter is already
intended to house the former's hooks.

In most cases, this is just confusing, but there is also a (quite
contrived) attack vector where Git can be fooled into mistaking remote
content for file contents it wrote itself during a recursive clone.

Let's plug this bug.

To do so, we introduce the new function `validate_submodule_git_dir()`
which simply verifies that no git dir exists for any leading directories
of the submodule name (if there are any).

Note: this patch specifically continues to allow sibling modules names
of the form `core/lib`, `core/doc`, etc, as long as `core` is not a
submodule name.

This fixes CVE-2019-1387.

Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-12-05 15:36:51 +01:00
Denton Liu
cae0bc09ab rebase: fix format.useAutoBase breakage
With `format.useAutoBase = true`, running rebase resulted in an
error:

	fatal: failed to get upstream, if you want to record base commit automatically,
	please use git branch --set-upstream-to to track a remote branch.
	Or you could specify base commit by --base=<base-commit-id> manually
	error:
	git encountered an error while preparing the patches to replay
	these revisions:

	    ede2467cdedc63784887b587a61c36b7850ebfac..d8f581194799ae29bf5fa72a98cbae98a1198b12

	As a result, git cannot rebase them.

Fix this by always passing `--no-base` to format-patch from rebase so
that the effect of `format.useAutoBase` is negated.

Reported-by: Christian Biesinger <cbiesinger@google.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-05 06:06:18 -08:00
Denton Liu
945dc55dda format-patch: teach --no-base
If `format.useAutoBase = true`, there was no way to override this from
the command-line. Teach the `--no-base` option in format-patch to
override `format.useAutoBase`.

Helped-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-05 06:06:18 -08:00
Denton Liu
a749d01e1d format-patch: fix indentation
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-05 06:06:18 -08:00
Alexandr Miloslavskiy
a9aecc7abb checkout, restore: support the --pathspec-from-file option
Decisions taken for simplicity:
1) For now, `--pathspec-from-file` is declared incompatible with
   `--patch`, even when <file> is not `stdin`. Such use case it not
   really expected.
2) It is not allowed to pass pathspec in both args and file.

`you must specify path(s) to restore` block was moved down to be able to
test for `pathspec.nr` instead, because testing for `argc` is no longer
correct.

`git switch` does not support the new options because it doesn't expect
`<pathspec>` arguments.

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-04 10:10:37 -08:00
Alexandr Miloslavskiy
bebb5d6d6b add: support the --pathspec-from-file option
Decisions taken for simplicity:
1) For now, `--pathspec-from-file` is declared incompatible with
   `--interactive/--patch/--edit`, even when <file> is not `stdin`.
   Such use case it not really expected. Also, it would require changes
   to `interactive_add()` and `edit_patch()`.
2) It is not allowed to pass pathspec in both args and file.

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-04 10:10:37 -08:00
Alexandr Miloslavskiy
21bb3083c3 cmd_add: prepare for next patch
Some code blocks were moved down to be able to test for `pathspec.nr`
in the next patch. Blocks are moved as is without any changes. This
is done as separate patch to reduce the amount of diffs in next patch.

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-04 10:10:37 -08:00
Johannes Schindelin
0060fd1511 clone --recurse-submodules: prevent name squatting on Windows
In addition to preventing `.git` from being tracked by Git, on Windows
we also have to prevent `git~1` from being tracked, as the default NTFS
short name (also known as the "8.3 filename") for the file name `.git`
is `git~1`, otherwise it would be possible for malicious repositories to
write directly into the `.git/` directory, e.g. a `post-checkout` hook
that would then be executed _during_ a recursive clone.

When we implemented appropriate protections in 2b4c6efc82 (read-cache:
optionally disallow NTFS .git variants, 2014-12-16), we had analyzed
carefully that the `.git` directory or file would be guaranteed to be
the first directory entry to be written. Otherwise it would be possible
e.g. for a file named `..git` to be assigned the short name `git~1` and
subsequently, the short name generated for `.git` would be `git~2`. Or
`git~3`. Or even `~9999999` (for a detailed explanation of the lengths
we have to go to protect `.gitmodules`, see the commit message of
e7cb0b4455 (is_ntfs_dotgit: match other .git files, 2018-05-11)).

However, by exploiting two issues (that will be addressed in a related
patch series close by), it is currently possible to clone a submodule
into a non-empty directory:

- On Windows, file names cannot end in a space or a period (for
  historical reasons: the period separating the base name from the file
  extension was not actually written to disk, and the base name/file
  extension was space-padded to the full 8/3 characters, respectively).
  Helpfully, when creating a directory under the name, say, `sub.`, that
  trailing period is trimmed automatically and the actual name on disk
  is `sub`.

  This means that while Git thinks that the submodule names `sub` and
  `sub.` are different, they both access `.git/modules/sub/`.

- While the backslash character is a valid file name character on Linux,
  it is not so on Windows. As Git tries to be cross-platform, it
  therefore allows backslash characters in the file names stored in tree
  objects.

  Which means that it is totally possible that a submodule `c` sits next
  to a file `c\..git`, and on Windows, during recursive clone a file
  called `..git` will be written into `c/`, of course _before_ the
  submodule is cloned.

Note that the actual exploit is not quite as simple as having a
submodule `c` next to a file `c\..git`, as we have to make sure that the
directory `.git/modules/b` already exists when the submodule is checked
out, otherwise a different code path is taken in `module_clone()` that
does _not_ allow a non-empty submodule directory to exist already.

Even if we will address both issues nearby (the next commit will
disallow backslash characters in tree entries' file names on Windows,
and another patch will disallow creating directories/files with trailing
spaces or periods), it is a wise idea to defend in depth against this
sort of attack vector: when submodules are cloned recursively, we now
_require_ the directory to be empty, addressing CVE-2019-1349.

Note: the code path we patch is shared with the code path of `git
submodule update --init`, which must not expect, in general, that the
directory is empty. Hence we have to introduce the new option
`--force-init` and hand it all the way down from `git submodule` to the
actual `git submodule--helper` process that performs the initial clone.

Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-12-04 13:20:05 +01:00
Jonathan Tan
4f3e57ef13 submodule--helper: advise on fatal alternate error
When recursively cloning a superproject with some shallow modules
defined in its .gitmodules, then recloning with "--reference=<path>", an
error occurs. For example:

  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    master
  git clone --recurse-submodules --branch=master -j8 \
    https://android.googlesource.com/platform/superproject \
    --reference master master2

fails with:

  fatal: submodule '<snip>' cannot add alternate: reference repository
  '<snip>' is shallow

When a alternate computed from the superproject's alternate cannot be
added, whether in this case or another, advise about configuring the
"submodule.alternateErrorStrategy" configuration option and using
"--reference-if-able" instead of "--reference" when cloning.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-03 08:49:45 -08:00
Junio C Hamano
723a8adba5 Merge branch 'ds/test-read-graph'
Dev support for commit-graph feature.

* ds/test-read-graph:
  test-tool: use 'read-graph' helper
2019-12-01 09:04:39 -08:00
Junio C Hamano
fce9e836d3 Merge branch 'jt/fetch-remove-lazy-fetch-plugging'
"git fetch" codepath had a big "do not lazily fetch missing objects
when I ask if something exists" switch.  This has been corrected by
marking the "does this thing exist?" calls with "if not please do not
lazily fetch it" flag.

* jt/fetch-remove-lazy-fetch-plugging:
  promisor-remote: remove fetch_if_missing=0
  clone: remove fetch_if_missing=0
  fetch: remove fetch_if_missing=0
2019-12-01 09:04:38 -08:00
Junio C Hamano
3c3e5d0ea2 Merge branch 'tg/stash-refresh-index'
Recent update to "git stash pop" made the command empty the index
when run with the "--quiet" option, which has been corrected.

* tg/stash-refresh-index:
  stash: make sure we have a valid index before writing it
2019-12-01 09:04:37 -08:00
Junio C Hamano
ca5c8aa8e1 Merge branch 'rj/bundle-ui-updates'
"git bundle" has been taught to use the parse options API.  "git
bundle verify" learned "--quiet" and "git bundle create" learned
options to control the progress output.

* rj/bundle-ui-updates:
  bundle-verify: add --quiet
  bundle-create: progress output control
  bundle: framework for options before bundle file
2019-12-01 09:04:36 -08:00
Junio C Hamano
d3096d2ba6 Merge branch 'en/doc-typofix'
Docfix.

* en/doc-typofix:
  Fix spelling errors in no-longer-updated-from-upstream modules
  multimail: fix a few simple spelling errors
  sha1dc: fix trivial comment spelling error
  Fix spelling errors in test commands
  Fix spelling errors in messages shown to users
  Fix spelling errors in names of tests
  Fix spelling errors in comments of testcases
  Fix spelling errors in code comments
  Fix spelling errors in documentation outside of Documentation/
  Documentation: fix a bunch of typos, both old and new
2019-12-01 09:04:35 -08:00
Junio C Hamano
bcb06e204c Merge branch 'js/fetch-multi-lockfix'
Fetching from multiple remotes into the same repository in parallel
had a bad interaction with the recent change to (optionally) update
the commit-graph after a fetch job finishes, as these parallel
fetches compete with each other.  Which has been corrected.

* js/fetch-multi-lockfix:
  fetch: avoid locking issues between fetch.jobs/fetch.writeCommitGraph
  fetch: add the command-line option `--write-commit-graph`
2019-12-01 09:04:33 -08:00
Junio C Hamano
7ab2088255 Merge branch 'rt/fetch-message-fix'
A small message update.

* rt/fetch-message-fix:
  fetch.c: fix typo in a warning message
2019-12-01 09:04:32 -08:00
Junio C Hamano
05fc6471e3 Merge branch 'pb/no-recursive-reset-hard-in-worktree-add'
"git worktree add" internally calls "reset --hard" that should not
descend into submodules, even when submodule.recurse configuration
is set, but it was affected.  This has been corrected.

* pb/no-recursive-reset-hard-in-worktree-add:
  worktree: teach "add" to ignore submodule.recurse config
2019-12-01 09:04:31 -08:00
Junio C Hamano
532d983823 Merge branch 'sg/blame-indent-heuristics-is-now-the-default'
Message update.

* sg/blame-indent-heuristics-is-now-the-default:
  builtin/blame.c: remove '--indent-heuristic' from usage string
2019-12-01 09:04:30 -08:00
Junio C Hamano
dfc03e48ec Merge branch 'mr/clone-dir-exists-to-path-exists'
Code cleanup.

* mr/clone-dir-exists-to-path-exists:
  clone: rename static function `dir_exists()`.
2019-12-01 09:04:30 -08:00
Junio C Hamano
0e07c1cd83 Merge branch 'jk/cleanup-object-parsing-and-fsck'
Crufty code and logic accumulated over time around the object
parsing and low-level object access used in "git fsck" have been
cleaned up.

* jk/cleanup-object-parsing-and-fsck: (23 commits)
  fsck: accept an oid instead of a "struct tree" for fsck_tree()
  fsck: accept an oid instead of a "struct commit" for fsck_commit()
  fsck: accept an oid instead of a "struct tag" for fsck_tag()
  fsck: rename vague "oid" local variables
  fsck: don't require an object struct in verify_headers()
  fsck: don't require an object struct for fsck_ident()
  fsck: drop blob struct from fsck_finish()
  fsck: accept an oid instead of a "struct blob" for fsck_blob()
  fsck: don't require an object struct for report()
  fsck: only require an oid for skiplist functions
  fsck: only provide oid/type in fsck_error callback
  fsck: don't require object structs for display functions
  fsck: use oids rather than objects for object_name API
  fsck_describe_object(): build on our get_object_name() primitive
  fsck: unify object-name code
  fsck: require an actual buffer for non-blobs
  fsck: stop checking tag->tagged
  fsck: stop checking commit->parent counts
  fsck: stop checking commit->tree value
  commit, tag: don't set parsed bit for parse failures
  ...
2019-12-01 09:04:28 -08:00
Hans Jerry Illikainen
72b006f4bf gpg-interface: prefer check_signature() for GPG verification
This commit refactors the use of verify_signed_buffer() outside of
gpg-interface.c to use check_signature() instead.  It also turns
verify_signed_buffer() into a file-local function since it's now only
invoked internally by check_signature().

There were previously two globally scoped functions used in different
parts of Git to perform GPG signature verification:
verify_signed_buffer() and check_signature().  Now only
check_signature() is used.

The verify_signed_buffer() function doesn't guard against duplicate
signatures as described by Michał Górny [1].  Instead it only ensures a
non-erroneous exit code from GPG and the presence of at least one
GOODSIG status field.  This stands in contrast with check_signature()
that returns an error if more than one signature is encountered.

The lower degree of verification makes the use of verify_signed_buffer()
problematic if callers don't parse and validate the various parts of the
GPG status message themselves.  And processing these messages seems like
a task that should be reserved to gpg-interface.c with the function
check_signature().

Furthermore, the use of verify_signed_buffer() makes it difficult to
introduce new functionality that relies on the content of the GPG status
lines.

Now all operations that does signature verification share a single entry
point to gpg-interface.c.  This makes it easier to propagate changed or
additional functionality in GPG signature verification to all parts of
Git, without having odd edge-cases that don't perform the same degree of
verification.

[1] https://dev.gentoo.org/~mgorny/articles/attack-on-git-signature-verification.html

Signed-off-by: Hans Jerry Illikainen <hji@dyntopia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-30 13:52:35 -08:00
René Scharfe
2059e79c0d name-rev: use skip_prefix() instead of starts_with()
Let skip_prefix() advance refname to get rid of two magic numbers.

Signed-off-by: René Scharfe <l.s.r@web.de>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-27 11:21:18 +09:00
René Scharfe
1768aaf01d push: use skip_prefix() instead of starts_with()
Get rid of a magic number by using skip_prefix().

Signed-off-by: René Scharfe <l.s.r@web.de>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-27 11:18:39 +09:00
René Scharfe
7e412e8a34 fmt-merge-msg: use skip_prefix() instead of starts_with()
Get rid of two magic numbers by using skip_prefix().

Signed-off-by: René Scharfe <l.s.r@web.de>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-27 11:17:30 +09:00
René Scharfe
a6293f5d28 fetch: use skip_prefix() instead of starts_with()
Get rid of magic numbers by letting skip_prefix() set the pointer
"what".

Signed-off-by: René Scharfe <l.s.r@web.de>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-27 11:17:22 +09:00
Junio C Hamano
d82dfa7f5b rebase -i: finishing touches to --reset-author-date
Clarify the way the `--reset-author-date` option is described,
and mark its usage string translatable.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-25 15:09:29 +09:00
Manish Goregaokar
1f3aea22c7 submodule: fix 'submodule status' when called from a subdirectory
When calling `git submodule status` while in a subdirectory, we are
incorrectly not detecting modified submodules and
thus reporting that all of the submodules are unchanged.

This is because the submodule helper is calling `diff-index` with the
submodule path assuming the path is relative to the current prefix
directory, however the submodule path used is actually relative to the root.

Always pass NULL as the `prefix` when running diff-files on the
submodule, to make sure the submodule's path is interpreted as relative
to the superproject's repository root.

Signed-off-by: Manish Goregaokar <manishsmail@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-25 14:08:25 +09:00
Alban Gruin
a2dd67f105 rebase: fill `squash_onto' in get_replay_opts()
When sequencer_continue() is called by complete_action(), `opts' has
been filled by get_replay_opts().  Currently, it does not initialise the
`squash_onto' field (used by the `--root' mode), only
read_populate_opts() does.  It’s not a problem yet since
sequencer_continue() calls it before pick_commits(), but it would lead
to incorrect results once complete_action() is modified to call
pick_commits() directly.

Let’s change that.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-25 12:24:49 +09:00
Nika Layzell
0a8e3036a3 reset: parse rev as tree-ish in patch mode
Since 2f328c3d ("reset $sha1 $pathspec: require $sha1 only to be
treeish", 2013-01-14), we allowed "git reset $object -- $path" to reset
individual paths that match the pathspec to take the blob from a tree
object, not necessarily a commit, while the form to reset the tip of the
current branch to some other commit still must be given a commit.

Like resetting with paths, "git reset --patch" does not update HEAD, and
need not require a commit. The path-filtered form, "git reset --patch
$object -- $pathspec", has accepted a tree-ish since 2f328c3d.

"git reset --patch" is documented as accepting a <tree-ish> since
bf44142f ("reset: update documentation to require only tree-ish with
paths", 2013-01-16). Documentation changes are not required.

Loosen the restriction that requires a commit for the unfiltered "git
reset --patch $object".

Signed-off-by: Nika Layzell <nika@thelayzells.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-25 11:01:22 +09:00
Derrick Stolee
cff4e9138d sparse-checkout: check for dirty status
The index-merge performed by 'git sparse-checkout' will erase any staged
changes, which can lead to data loss. Prevent these attempts by requiring
a clean 'git status' output.

Helped-by: Szeder Gábor <szeder.dev@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:45 +09:00
Derrick Stolee
416adc8711 sparse-checkout: update working directory in-process for 'init'
The 'git sparse-checkout init' subcommand previously wrote directly
to the sparse-checkout file and then updated the working directory.
This may fail if there are modified files not included in the initial
pattern set. However, that left a populated sparse-checkout file.

Use the in-process working directory update to guarantee that the
init subcommand only changes the sparse-checkout file if the working
directory update succeeds.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:45 +09:00
Derrick Stolee
fb10ca5b54 sparse-checkout: write using lockfile
If two 'git sparse-checkout set' subcommands are launched at the
same time, the behavior can be unexpected as they compete to write
the sparse-checkout file and update the working directory.

Take a lockfile around the writes to the sparse-checkout file. In
addition, acquire this lock around the working directory update
to avoid two commands updating the working directory in different
ways.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:45 +09:00
Derrick Stolee
99dfa6f970 sparse-checkout: use in-process update for disable subcommand
The 'git sparse-checkout disable' subcommand returns a user to a
full working directory. The old process for doing this required
updating the sparse-checkout file with the "/*" pattern and then
updating the working directory with core.sparseCheckout enabled.
Finally, the sparse-checkout file could be removed and the config
setting disabled.

However, it is valuable to keep a user's sparse-checkout file
intact so they can re-enable the sparse-checkout they previously
used with 'git sparse-checkout init'. This is now possible with
the in-process mechanism for updating the working directory.

Reported-by: Szeder Gábor <szeder.dev@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:45 +09:00
Derrick Stolee
e091228e17 sparse-checkout: update working directory in-process
The sparse-checkout builtin used 'git read-tree -mu HEAD' to update the
skip-worktree bits in the index and to update the working directory.
This extra process is overly complex, and prone to failure. It also
requires that we write our changes to the sparse-checkout file before
trying to update the index.

Remove this extra process call by creating a direct call to
unpack_trees() in the same way 'git read-tree -mu HEAD' does. In
addition, provide an in-memory list of patterns so we can avoid
reading from the sparse-checkout file. This allows us to test a
proposed change to the file before writing to it.

An earlier version of this patch included a bug when the 'set' command
failed due to the "Sparse checkout leaves no entry on working directory"
error. It would not rollback the index.lock file, so the replay of the
old sparse-checkout specification would fail. A test in t1091 now
covers that scenario.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:44 +09:00
Derrick Stolee
e9de487aa3 sparse-checkout: sanitize for nested folders
If a user provides folders A/ and A/B/ for inclusion in a cone-mode
sparse-checkout file, the parsing logic will notice that A/ appears
both as a "parent" type pattern and as a "recursive" type pattern.
This is unexpected and hence will complain via a warning and revert
to the old logic for checking sparse-checkout patterns.

Prevent this from happening accidentally by sanitizing the folders
for this type of inclusion in the 'git sparse-checkout' builtin.
This happens in two ways:

1. Do not include any parent patterns that also appear as recursive
   patterns.

2. Do not include any recursive patterns deeper than other recursive
   patterns.

In order to minimize duplicate code for scanning parents, create
hashmap_contains_parent() method. It takes a strbuf buffer to
avoid reallocating a buffer when calling in a tight loop.

Helped-by: Eric Wong <e@80x24.org>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:44 +09:00
Derrick Stolee
af09ce24a9 sparse-checkout: init and set in cone mode
To make the cone pattern set easy to use, update the behavior of
'git sparse-checkout (init|set)'.

Add '--cone' flag to 'git sparse-checkout init' to set the config
option 'core.sparseCheckoutCone=true'.

When running 'git sparse-checkout set' in cone mode, a user only
needs to supply a list of recursive folder matches. Git will
automatically add the necessary parent matches for the leading
directories.

When testing 'git sparse-checkout set' in cone mode, check the
error stream to ensure we do not see any errors. Specifically,
we want to avoid the warning that the patterns do not match
the cone-mode patterns.

Helped-by: Eric Wong <e@80x24.org>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:44 +09:00
Derrick Stolee
72918c1ad9 sparse-checkout: create 'disable' subcommand
The instructions for disabling a sparse-checkout to a full
working directory are complicated and non-intuitive. Add a
subcommand, 'git sparse-checkout disable', to perform those
steps for the user.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:44 +09:00
Derrick Stolee
7bffca95ea sparse-checkout: add '--stdin' option to set subcommand
The 'git sparse-checkout set' subcommand takes a list of patterns
and places them in the sparse-checkout file. Then, it updates the
working directory to match those patterns. For a large list of
patterns, the command-line call can get very cumbersome.

Add a '--stdin' option to instead read patterns over standard in.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:44 +09:00
Derrick Stolee
f6039a9423 sparse-checkout: 'set' subcommand
The 'git sparse-checkout set' subcommand takes a list of patterns
as arguments and writes them to the sparse-checkout file. Then, it
updates the working directory using 'git read-tree -mu HEAD'.

The 'set' subcommand will replace the entire contents of the
sparse-checkout file. The write_patterns_and_update() method is
extracted from cmd_sparse_checkout() to make it easier to implement
'add' and/or 'remove' subcommands in the future.

If the core.sparseCheckout config setting is disabled, then enable
the config setting in the worktree config. If we set the config
this way and the sparse-checkout fails, then re-disable the config
setting.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:43 +09:00
Derrick Stolee
d89f09c828 clone: add --sparse mode
When someone wants to clone a large repository, but plans to work
using a sparse-checkout file, they either need to do a full
checkout first and then reduce the patterns they included, or
clone with --no-checkout, set up their patterns, and then run
a checkout manually. This requires knowing a lot about the repo
shape and how sparse-checkout works.

Add a new '--sparse' option to 'git clone' that initializes the
sparse-checkout file to include the following patterns:

	/*
	!/*/

These patterns include every file in the root directory, but
no directories. This allows a repo to include files like a
README or a bootstrapping script to grow enlistments from that
point.

During the 'git sparse-checkout init' call, we must first look
to see if HEAD is valid, since 'git clone' does not have a valid
HEAD at the point where it initializes the sparse-checkout. The
following checkout within the clone command will create the HEAD
ref and update the working directory correctly.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:43 +09:00
Derrick Stolee
bab3c35908 sparse-checkout: create 'init' subcommand
Getting started with a sparse-checkout file can be daunting. Help
users start their sparse enlistment using 'git sparse-checkout init'.
This will set 'core.sparseCheckout=true' in their config, write
an initial set of patterns to the sparse-checkout file, and update
their working directory.

Make sure to use the `extensions.worktreeConfig` setting and write
the sparse checkout config to the worktree-specific config file.
This avoids confusing interactions with other worktrees.

The use of running another process for 'git read-tree' is sub-
optimal. This will be removed in a later change.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:43 +09:00
Derrick Stolee
94c0956b60 sparse-checkout: create builtin with 'list' subcommand
The sparse-checkout feature is mostly hidden to users, as its
only documentation is supplementary information in the docs for
'git read-tree'. In addition, users need to know how to edit the
.git/info/sparse-checkout file with the right patterns, then run
the appropriate 'git read-tree -mu HEAD' command. Keeping the
working directory in sync with the sparse-checkout file requires
care.

Begin an effort to make the sparse-checkout feature a porcelain
feature by creating a new 'git sparse-checkout' builtin. This
builtin will be the preferred mechanism for manipulating the
sparse-checkout file and syncing the working directory.

The documentation provided is adapted from the "git read-tree"
documentation with a few edits for clarity in the new context.
Extra sections are added to hint toward a future change to
a more restricted pattern set.

Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:43 +09:00
Denton Liu
5b583e6a09 format-patch: pass notes configuration to range-diff
Since format-patch accepts `--[no-]notes`, one would expect the
range-diff generated to also respect the setting. Unfortunately, the
range-diff we currently generate only uses the default option (which
always outputs default notes, even when notes are not being used
elsewhere).

Pass the notes configuration to range-diff so that it can honor it.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-21 09:29:52 +09:00
Denton Liu
bd36191886 range-diff: pass through --notes to git log
When a commit being range-diff'd has a note attached to it, the note
will be compared as well. However, if a user has multiple notes refs or
if they want to suppress notes from being printed, there is currently no
way to do this.

Pass through `--[no-]notes[=<ref>]` to the `git log` call so that this
option is customizable.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-21 09:29:52 +09:00
Alexandr Miloslavskiy
e440fc5888 commit: support the --pathspec-from-file option
Decisions taken for simplicity:
1) For now, `--pathspec-from-file` is declared incompatible with
   `--interactive/--patch`, even when <file> is not `stdin`. Such use
   case it not really expected. Also, it would require changes to
   `interactive_add()`.
2) It is not allowed to pass pathspec in both args and file.

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-20 13:01:53 +09:00
Alexandr Miloslavskiy
64bac8df97 reset: support the --pathspec-from-file option
Decisions taken for simplicity:
1) For now, `--pathspec-from-file` is declared incompatible with
   `--patch`, even when <file> is not `stdin`. Such use case it not
   really expected. Also, it is harder to support in `git commit`, so
   I decided to make it incompatible in all places.
2) It is not allowed to pass pathspec in both args and file.

Co-authored-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-20 13:01:53 +09:00
Alexandr Miloslavskiy
d137b50756 doc: reset: synchronize <pathspec> description
`git add` shows an example of good writing, follow it.

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-20 13:01:53 +09:00
SZEDER Gábor
bae60ba7e9 builtin/unpack-objects.c: show throughput progress
'git unpack-objects' shows a progress line only counting the number of
unpacked objects, so if some of the received objects are unusually
large, then that progress might appear to be frozen while processing
such a larger object.  I just stared at a seemingly stuck progress
line for over half a minute, while 'git fetch' was busy receiving a
pack with only a couple of objects (i.e. fewer than
'fetch.unpackLimit'), with one of them being over 80MB.

Display throughput in 'git unpack-objects' progress line, so we show
that something is going on even when receiving and processing a large
object.

Counting the consumed bytes is far away from the place that
counts objects and displays progress, and to pass around the 'struct
progress' instance we would have to modify the signature of five
functions and 14 of their callsites: this is just too much churn, so
let's rather make it file-scope static.

'git index-pack', i.e. the non-unpacking cousin of 'git
unpack-objects' already includes throughput in its progress line, and
it uses a file-scope static 'struct progress' instance as well.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-20 10:30:18 +09:00
Jeff King
2d92ab32fd rev-parse: make --show-toplevel without a worktree an error
Ever since it was introduced in 7cceca5ccc (Add 'git rev-parse
--show-toplevel' option., 2010-01-12), the --show-toplevel option has
treated a missing working tree as a quiet success: it neither prints a
toplevel path, but nor does it report any kind of error.

While a caller could distinguish this case by looking for an empty
response, the behavior is rather confusing. We're better off complaining
that there is no working tree, as other internal commands would do in
similar cases (e.g., "git status" or any builtin with NEED_WORK_TREE set
would just die()). So let's do the same here.

While we're at it, let's clarify the documentation and add some tests,
both for the new behavior and for the more mundane case (which was not
covered).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-20 10:19:58 +09:00
Johannes Schindelin
f83dff60a7 Start to implement a built-in version of git add --interactive
Unlike previous conversions to C, where we started with a built-in
helper, we start this conversion by adding an interception in the
`run_add_interactive()` function when the new opt-in
`add.interactive.useBuiltin` config knob is turned on (or the
corresponding environment variable `GIT_TEST_ADD_I_USE_BUILTIN`), and
calling the new internal API function `run_add_i()` that is implemented
directly in libgit.a.

At this point, the built-in version of `git add -i` only states that it
cannot do anything yet. In subsequent patches/patch series, the
`run_add_i()` function will gain more and more functionality, until it
is feature complete. The whole arc of the conversion can be found in the
PRs #170-175 at https://github.com/gitgitgadget/git.

The "--helper approach" can unfortunately not be used here: on Windows
we face the very specific problem that a `system()` call in
Perl seems to close `stdin` in the parent process when the spawned
process consumes even one character from `stdin`. Which prevents us from
implementing the main loop in C and still trying to hand off to the Perl
script.

The very real downside of the approach we have to take here is that the
test suite won't pass with `GIT_TEST_ADD_I_USE_BUILTIN=true` until the
conversion is complete (the `--helper` approach would have let it pass,
even at each of the incremental conversion steps).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-14 11:10:04 +09:00
Thomas Gummerer
df53c80822 stash: make sure we have a valid index before writing it
In 'do_apply_stash()' we refresh the index in the end.  Since
34933d0eff ("stash: make sure to write refreshed cache", 2019-09-11),
we also write that refreshed index when --quiet is given to 'git stash
apply'.

However if '--index' is not given to 'git stash apply', we also
discard the index in the else clause just before.  We need to do so
because we use an external 'git update-index --add --stdin', which
leads to an out of date in-core index.

Later we call 'refresh_and_write_cache', which now leads to writing
the discarded index, which means we essentially write an empty index
file.  This is obviously not correct, or the behaviour the user
wanted.  We should not modify the users index without being asked to
do so.

Make sure to re-read the index after discarding the current in-core
index, to avoid dealing with outdated information.  Instead we could
also drop the 'discard_cache()' + 'read_cache()', however that would
make it easy to fall into the same trap as 34933d0eff did, so it's
better to avoid that.

We can also drop the 'refresh_and_write_cache' completely in the quiet
case.  Previously in legacy stash we relied on 'git status' to refresh
the index after calling 'git read-tree' when '--index' was passed to
'git apply'.  However the 'reset_tree()' call that replaced 'git
read-tree' always passes options that are equivalent to '-m', making
the refresh of the index unnecessary.

Reported-by: Grzegorz Rajchman <rayman17@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-14 11:08:25 +09:00
Jonathan Tan
e362fadcd0 clone: remove fetch_if_missing=0
Commit 6462d5eb9a ("fetch: remove fetch_if_missing=0", 2019-11-08)
strove to remove the need for fetch_if_missing=0 from the fetching
mechanism, so it is plausible to attempt removing fetch_if_missing=0
from clone as well. But doing so reveals a bug - when the server does
not send an object directly pointed to by a ref, this should be an
error, not a trigger for a lazy fetch. (This case in the fetching
mechanism was covered by a test using "git clone", not "git fetch",
which is why the aforementioned commit didn't uncover the bug.)

The bug can be fixed by suppressing lazy-fetching during the
connectivity check. Fix this bug, and remove fetch_if_missing from
clone.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-13 11:48:47 +09:00
Derrick Stolee
4bd0593e0f test-tool: use 'read-graph' helper
The 'git commit-graph read' subcommand is used in test scripts to check
that the commit-graph contents match the expected data. Mostly, this
helps check the header information and the list of chunks. Users do not
need this information, so move the functionality to a test helper.

Reported-by: Bryan Turner <bturner@atlassian.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-13 11:14:16 +09:00
Robin H. Johnson
e0eba649e8 bundle-verify: add --quiet
Add --quiet to git-bundle verify as proposed on the mailing list [1].

Reference: https://www.mail-archive.com/git@vger.kernel.org/msg182844.html <robbat2-20190806T191156-796782357Z@orbis-terrarum.net>
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-11 11:46:29 +09:00
Robin H. Johnson
79862b6b77 bundle-create: progress output control
Support the progress output options from pack-objects in git-bundle's
create subcommand. Most notably, this provides --quiet as requested on
the git mailing list per [1]

Reference: https://www.mail-archive.com/git@vger.kernel.org/msg182844.html <robbat2-20190806T191156-796782357Z@orbis-terrarum.net>
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-11 11:46:28 +09:00
Robin H. Johnson
73c3253d75 bundle: framework for options before bundle file
Make it possible for any of the git-bundle subcommands to include
options:
- before the sub-command
- after the sub-command, before the bundle filename

There is an immediate gain in support for help with all of the
sub-commands, where 'git bundle list-heads -h' previously returned an
error.

Downside here is an increase in code duplication that cannot be
trivially avoided short of shared global static options.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-11 11:46:26 +09:00
Junio C Hamano
28014c1084 Merge branch 'bc/hash-independent-tests-part-6'
Test updates to prepare for SHA-2 transition continues.

* bc/hash-independent-tests-part-6:
  t4048: abstract away SHA-1-specific constants
  t4045: make hash-size independent
  t4044: update test to work with SHA-256
  t4039: abstract away SHA-1-specific constants
  t4038: abstract away SHA-1 specific constants
  t4034: abstract away SHA-1-specific constants
  t4027: make hash-size independent
  t4015: abstract away SHA-1-specific constants
  t4011: abstract away SHA-1-specific constants
  t4010: abstract away SHA-1-specific constants
  t3429: remove SHA1 annotation
  t1305: avoid comparing extensions
  rev-parse: add a --show-object-format option
  t/oid-info: add empty tree and empty blob values
  t/oid-info: allow looking up hash algorithm name
2019-11-10 18:02:17 +09:00
Junio C Hamano
57b530125e Merge branch 'js/update-index-ignore-removal-for-skip-worktree'
"git stash save" in a working tree that is sparsely checked out
mistakenly removed paths that are outside the area of interest.

* js/update-index-ignore-removal-for-skip-worktree:
  stash: handle staged changes in skip-worktree files correctly
  update-index: optionally leave skip-worktree entries alone
2019-11-10 18:02:16 +09:00
Junio C Hamano
14b58c62bc Merge branch 'sg/commit-graph-usage-fix'
Message fix.

* sg/commit-graph-usage-fix:
  builtin/commit-graph.c: remove subcommand-less usage string
2019-11-10 18:02:15 +09:00
Junio C Hamano
8f1119b988 Merge branch 'wb/midx-progress'
The code to generate multi-pack index learned to show (or not to
show) progress indicators.

* wb/midx-progress:
  multi-pack-index: add [--[no-]progress] option.
  midx: honor the MIDX_PROGRESS flag in midx_repack
  midx: honor the MIDX_PROGRESS flag in verify_midx_file
  midx: add progress to expire_midx_packs
  midx: add progress to write_midx_file
  midx: add MIDX_PROGRESS flag
2019-11-10 18:02:14 +09:00
Junio C Hamano
0c51181ffb Merge branch 'js/rebase-deprecate-preserve-merges'
"git rebase --preserve-merges" has been marked as deprecated; this
release stops advertising it in the "git rebase -h" output.

* js/rebase-deprecate-preserve-merges:
  rebase: hide --preserve-merges option
2019-11-10 18:02:13 +09:00
Junio C Hamano
8f40d89783 Merge branch 'hv/bitshift-constants-in-blame'
Move the definition of a set of bitmask constants from 0ctal
literal to (1U<<count) notation.

* hv/bitshift-constants-in-blame:
  builtin/blame.c: constants into bit shift format
2019-11-10 18:02:12 +09:00
Junio C Hamano
d4a98e701f Merge branch 'dd/notes-copy-default-dst-to-head'
"git notes copy $original" ought to copy the notes attached to the
original object to HEAD, but a mistaken tightening to command line
parameter validation made earlier disabled that feature by mistake.

* dd/notes-copy-default-dst-to-head:
  notes: fix minimum number of parameters to "copy" subcommand
  t3301: test diagnose messages for too few/many paramters
2019-11-10 18:02:12 +09:00
Junio C Hamano
5c8c0a0d78 Merge branch 'pw/post-commit-from-sequencer'
"rebase -i" ceased to run post-commit hook by mistake in an earlier
update, which has been corrected.

* pw/post-commit-from-sequencer:
  sequencer: run post-commit hook
  move run_commit_hook() to libgit and use it there
  sequencer.h fix placement of #endif
  t3404: remove uneeded calls to set_fake_editor
  t3404: set $EDITOR in subshell
  t3404: remove unnecessary subshell
2019-11-10 18:02:12 +09:00
Junio C Hamano
b75ba9bbd1 Merge branch 'dl/format-patch-cover-from-desc'
The branch description ("git branch --edit-description") has been
used to fill the body of the cover letters by the format-patch
command; this has been enhanced so that the subject can also be
filled.

* dl/format-patch-cover-from-desc:
  format-patch: teach --cover-from-description option
  format-patch: use enum variables
  format-patch: replace erroneous and condition
2019-11-10 18:02:11 +09:00
Junio C Hamano
026587c793 Merge branch 'jt/fetch-pack-record-refs-in-the-dot-promisor'
Debugging support for lazy cloning has been a bit improved.

* jt/fetch-pack-record-refs-in-the-dot-promisor:
  fetch-pack: write fetched refs to .promisor
2019-11-10 18:02:10 +09:00
Elijah Newren
15beaaa3d1 Fix spelling errors in code comments
Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10 16:00:54 +09:00
Jonathan Tan
6462d5eb9a fetch: remove fetch_if_missing=0
In fetch_pack() (and all functions it calls), pass
OBJECT_INFO_SKIP_FETCH_OBJECT whenever we query an object that could be
a tree or blob that we do not want to be lazy-fetched even if it is
absent. Thus, the only lazy-fetches occurring for trees and blobs are
when resolving deltas.

Thus, we can remove fetch_if_missing=0 from builtin/fetch.c. Remove
this, and also add a test ensuring that such objects are not
lazy-fetched. (We might be able to remove fetch_if_missing=0 from other
places too, but I have limited myself to builtin/fetch.c in this commit
because I have not written tests for the other commands yet.)

Note that commits and tags may still be lazy-fetched. I limited myself
to objects that could be trees or blobs here because Git does not
support creating such commit- and tag-excluding clones yet, and even if
such a clone were manually created, Git does not have good support for
fetching a single commit (when fetching a commit, it and all its
ancestors would be sent).

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-08 15:26:44 +09:00
Johannes Schindelin
7d8e72b970 fetch: avoid locking issues between fetch.jobs/fetch.writeCommitGraph
When both `fetch.jobs` and `fetch.writeCommitGraph` is set, we currently
try to write the commit graph in each of the concurrent fetch jobs,
which frequently leads to error messages like this one:

fatal: Unable to create '.../.git/objects/info/commit-graphs/commit-graph-chain.lock': File exists.

Let's avoid this by holding off from writing the commit graph until all
fetch jobs are done.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-06 10:59:37 +09:00
Johannes Schindelin
c14e6e7903 fetch: add the command-line option --write-commit-graph
This option overrides the config setting `fetch.writeCommitGraph`, if
both are set.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-06 10:59:36 +09:00
Rohit Ashiwal
fe28ad8520 rebase: add --reset-author-date
The previous commit introduced --ignore-date flag to interactive
rebase, but the name is actually very vague in context of rebase -i
since there are two dates we can work with. Add an alias to convey
the precise purpose.

Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-02 15:37:12 +09:00
Rohit Ashiwal
08187b4cba rebase -i: support --ignore-date
rebase am already has this flag to "lie" about the author date
by changing it to the committer (current) date. Let's add the same
for interactive machinery.

Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-02 15:37:12 +09:00
Rohit Ashiwal
cbd8db17ac rebase -i: support --committer-date-is-author-date
rebase am already has this flag to "lie" about the committer date
by changing it to the author date. Let's add the same for
interactive machinery.

Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-02 15:36:23 +09:00
Rohit Ashiwal
ba51d2fb24 rebase -i: add --ignore-whitespace flag
There are two backends available for rebasing, viz, the am and the
interactive. Naturally, there shall be some features that are
implemented in one but not in the other. One such flag is
--ignore-whitespace which indicates merge mechanism to treat lines
with only whitespace changes as unchanged. Wire the interactive
rebase to also understand the --ignore-whitespace flag by
translating it to -Xignore-space-change.

Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-02 15:34:50 +09:00
Johannes Schindelin
4a58c3d7f7 stash: handle staged changes in skip-worktree files correctly
When calling `git stash` while changes were staged for files that are
marked with the `skip-worktree` bit (e.g. files that are excluded in a
sparse checkout), the files are recorded as _deleted_ instead.

The reason is that `git stash` tries to construct the tree reflecting
the worktree essentially by copying the index to a temporary one and
then updating the files from the worktree. Crucially, it calls `git
diff-index` to update also those files that are in the HEAD but have
been unstaged in the index.

However, when the temporary index is updated via `git update-index --add
--remove`, skip-worktree entries mark the files as deleted by mistake.

Let's use the newly-introduced `--ignore-skip-worktree-entries` option
of `git update-index` to prevent exactly this from happening.

Note that the regression test case deliberately avoids replicating the
scenario described above and instead tries to recreate just the symptom.

Reported by Dan Thompson.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-02 15:22:45 +09:00
Johannes Schindelin
8dfb04ae96 update-index: optionally leave skip-worktree entries alone
While `git update-index` mostly ignores paths referring to index entries
whose skip-worktree bit is set, in b4d1690df1 (Teach Git to respect
skip-worktree bit (reading part), 2009-08-20), for reasons that are not
entirely obvious, the `--remove` option was made special: it _does_
remove index entries even if their skip-worktree bit is set.

Seeing as this behavior has been in place for a decade now, it does not
make sense to change it.

However, in preparation for fixing a bug in `git stash` where it
pretends that skip-worktree entries have actually been removed, we need
a mode where `git update-index` leaves all skip-worktree entries alone,
even if the `--remove` option was passed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-02 15:22:00 +09:00
Ralf Thielow
391c7e40b5 fetch.c: fix typo in a warning message
Signed-off-by: Ralf Thielow <ralf.thielow@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-02 14:49:18 +09:00
Philippe Blain
4782cf2ab6 worktree: teach "add" to ignore submodule.recurse config
"worktree add" internally calls "reset --hard", but if
submodule.recurse is set, reset tries to recurse into
initialized submodules, which makes start_command try to
cd into non-existing submodule paths and die.

Fix that by making sure that the call to reset in "worktree add"
does not recurse.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-30 09:57:15 +09:00
SZEDER Gábor
44ae131e38 builtin/blame.c: remove '--indent-heuristic' from usage string
The indent heuristic is our default diff heuristic since 33de716387
(diff: enable indent heuristic by default, 2017-05-08), but the usage
string of 'git blame' still mentions it as "experimental heuristic".

We could simply update the short help associated with the option, but
according to the comment above the option's declaration it was "only
included here to get included in the "-h" output".  That made sense
while the feature was still experimental and we wanted to give it more
exposure, but nowadays it's unnecessary.

So let's rather remove the '--indent-heuristic' option from 'git
blame's usage string.  Note that 'git blame' will still accept this
option, as it is parsed in parse_revision_opt().

Astute readers may notice that this patch removes a comment mentioning
"the following two options", but it only removes one option.  The
reason is that the comment is outdated: that other options was
'--compaction-heuristic', and it has already been removed in
3cde4e02ee (diff: retire "compaction" heuristics, 2016-12-23), but
that commit forgot to update this comment.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-29 12:28:56 +09:00
Miriam Rubio
6c02042139 clone: rename static function dir_exists().
builtin/clone.c has a static function dir_exists() that
checks if a given path exists on the filesystem.  It returns
true (and it is correct for it to return true) when the
given path exists as a non-directory (e.g. a regular file).

This is confusing.  What the caller wants to check, and what
this function wants to return, is if the path exists, so
rename it to path_exists().

Signed-off-by: Miriam Rubio <mirucam@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-29 11:54:23 +09:00
Jeff King
5afc4b1dc6 fsck: only provide oid/type in fsck_error callback
None of the callbacks actually care about having a "struct object";
they're happy with just the oid and type information. So let's give
ourselves more flexibility to avoid having a "struct object" by just
passing the broken-down fields.

Note that the callback already takes a "type" field for the fsck message
type. We'll rename that to "msg_type" (and use "object_type" for the
object type) to make the distinction explicit.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-28 14:05:18 +09:00
Jeff King
82ef89b318 fsck: don't require object structs for display functions
Our printable_type() and describe_object() functions take whole object
structs, but they really only care about the oid and type. Let's take
those individually in order to give our callers more flexibility.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-28 14:05:18 +09:00
Jeff King
733902905d fsck: use oids rather than objects for object_name API
We don't actually care about having object structs; we only need to look
up decorations by oid. Let's accept this more limited form, which will
give our callers more flexibility.

Note that the decoration API we rely on uses object structs itself (even
though it only looks at their oids). We can solve this by switching to
a kh_oid_map (we could also use the hashmap oidmap, but it's more
awkward for the simple case of just storing a void pointer).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-28 14:05:18 +09:00
Jeff King
a59cfb3230 fsck: unify object-name code
Commit 90cf590f53 (fsck: optionally show more helpful info for broken
links, 2016-07-17) added a system for decorating objects with names. The
code is split across builtin/fsck.c (which gives the initial names) and
fsck.c (which adds to the names as it traverses the object graph). This
leads to some duplication, where both sites have near-identical
describe_object() functions (the difference being that the one in
builtin/fsck.c uses a circular array of buffers to allow multiple calls
in a single printf).

Let's provide a unified object_name API for fsck. That lets us drop the
duplication, as well as making the interface boundaries more clear
(which will let us refactor the implementation more in a future patch).

We'll leave describe_object() in builtin/fsck.c as a thin wrapper around
the new API, as it relies on a static global to make its many callers a
bit shorter.

We'll also convert the bare add_decoration() calls in builtin/fsck.c to
put_object_name(). This fixes two minor bugs:

  1. We leak many small strings. add_decoration() has a last-one-wins
     approach: it updates the decoration to the new string and returns
     the old one. But we ignore the return value, leaking the old
     string. This is quite common to trigger, since we look at reflogs:
     the tip of any ref will be described both by looking at the actual
     ref, as well as the latest reflog entry. So we'd always end up
     leaking one of those strings.

  2. The last-one-wins approach gives us lousy names. For instance, we
     first look at all of the refs, and then all of the reflogs. So
     rather than seeing "refs/heads/master", we're likely to overwrite
     it with "HEAD@{12345678}". We're generally better off using the
     first name we find.

     And indeed, the test in t1450 expects this ugly HEAD@{} name. After
     this patch, we've switched to using fsck_put_object_name()'s
     first-one-wins semantics, and we output the more human-friendly
     "refs/tags/julius" (and the test is updated accordingly).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-28 14:05:17 +09:00
SZEDER Gábor
8b656572ca builtin/commit-graph.c: remove subcommand-less usage string
The first line in 'git commit-graph's usage string indicates that this
command can be invoked without specifying a subcommand.  However, this
is not the case:

  $ git commit-graph
  usage: git commit-graph [--object-dir <objdir>]
     or: git commit-graph read [--object-dir <objdir>]
  [...]
  $ echo $?
  129

Remove this line from the usage string.

The synopsis in the manpage doesn't contain this line.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-28 12:24:17 +09:00
brian m. carlson
2eabd38313 rev-parse: add a --show-object-format option
Add an option to print the object format used for input, output, or
storage. This allows shell scripts to discover the hash algorithm in
use.

Since the transition plan allows for multiple input algorithms, document
that we may provide multiple results for input, and the format that the
results may take. While we don't support this now, documenting it early
means that script authors can future-proof their scripts for when we do.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-28 11:34:57 +09:00
Jonathan Tan
c7aadccba0 fetch: delay fetch_if_missing=0 until after config
Suppose, from a repository that has ".gitmodules", we clone with
--filter=blob:none:

  git clone --filter=blob:none --no-checkout \
    https://kernel.googlesource.com/pub/scm/git/git

Then we fetch:

  git -C git fetch

This will cause a "unable to load config blob object", because the
fetch_config_from_gitmodules() invocation in cmd_fetch() will attempt to
load ".gitmodules" (which Git knows to exist because the client has the
tree of HEAD) while fetch_if_missing is set to 0.

fetch_if_missing is set to 0 too early - ".gitmodules" here should be
lazily fetched.  Git must set fetch_if_missing to 0 before the fetch
because as part of the fetch, packfile negotiation happens (and we do
not want to fetch any missing objects when checking existence of
objects), but we do not need to set it so early. Move the setting of
fetch_if_missing to the earliest possible point in cmd_fetch(), right
before any fetching happens.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-24 11:34:44 +09:00
Junio C Hamano
e0ff2d4c7e Merge branch 'cb/pcre2-chartables-leakfix'
Leakfix.

* cb/pcre2-chartables-leakfix:
  grep: avoid leak of chartables in PCRE2
  grep: make PCRE2 aware of custom allocator
  grep: make PCRE1 aware of custom allocator
2019-10-23 14:43:11 +09:00
William Baker
680cba2c2b multi-pack-index: add [--[no-]progress] option.
Add the --[no-]progress option to git multi-pack-index.
Pass the MIDX_PROGRESS flag to the subcommand functions
when progress should be displayed by multi-pack-index.
The progress feature was added to 'verify' in 144d703
("multi-pack-index: report progress during 'verify'", 2018-09-13)
but some subcommands were not updated to display progress, and
the ability to opt-out was overlooked.

Signed-off-by: William Baker <William.Baker@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-23 12:05:06 +09:00
William Baker
efbc3aee08 midx: add MIDX_PROGRESS flag
Add the MIDX_PROGRESS flag and update the
write|verify|expire|repack functions in midx.h
to accept a flags parameter.  The MIDX_PROGRESS
flag indicates whether the caller of the function
would like progress information to be displayed.
This patch only changes the method prototypes
and does not change the functionality. The
functionality change will be handled by a later patch.

Signed-off-by: William Baker <William.Baker@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-23 12:05:05 +09:00
Hariom Verma
86795774bb builtin/blame.c: constants into bit shift format
We are looking at bitfield constants, and elsewhere in the Git source
code, such cases are handled via bit shift operators rather than octal
numbers, which also makes it easier to spot holes in the range
(if, say, 1<<5 was missing, it is easier to spot it between 1<<4
and 1<<6 than it is to spot a missing 040 between a 020 and a 0100).

Signed-off-by: Hariom Verma <hariom18599@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-21 12:45:51 +09:00
Denton Liu
feebd2d256 rebase: hide --preserve-merges option
Since --preserve-merges has been deprecated in favour of
--rebase-merges, mark this option as hidden so it no longer shows up in
the usage and completions.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-21 12:03:49 +09:00
Junio C Hamano
bb52def6da Merge branch 'jj/stash-reset-only-toplevel'
"git stash save" lost local changes to submodules, which has been
corrected.

* jj/stash-reset-only-toplevel:
  stash: avoid recursive hard reset on submodules
2019-10-18 11:40:49 +09:00
Junio C Hamano
f1afbb063f Merge branch 'bw/format-patch-o-create-leading-dirs'
"git format-patch -o <outdir>" did an equivalent of "mkdir <outdir>"
not "mkdir -p <outdir>", which is being corrected.

* bw/format-patch-o-create-leading-dirs:
  format-patch: create leading components of output directory
2019-10-18 11:40:48 +09:00
Carlo Marcelo Arenas Belón
513f2b0bbd grep: make PCRE2 aware of custom allocator
94da9193a6 (grep: add support for PCRE v2, 2017-06-01) didn't include
a way to override the system allocator, and so it is incompatible with
custom allocators (e.g. nedmalloc). This problem became obvious when we
tried to plug a memory leak by `free()`ing a data structure allocated by
PCRE2, triggering a segfault in Windows (where we use nedmalloc by
default).

PCRE2 requires the use of a general context to override the allocator
and therefore, there is a lot more code needed than in PCRE1, including
a couple of wrapper functions.

Extend the grep API with a "destructor" that could be called to cleanup
any objects that were created and used globally.

Update `builtin/grep.c` to use that new API, but any other future users
should make sure to have matching `grep_init()`/`grep_destroy()` calls
if they are using the pattern matching functionality.

Move some of the logic that was before done per thread (in the workers)
into an earlier phase to avoid degrading performance, but as the use of
PCRE2 with custom allocators is better understood it is expected more of
its functions will be instructed to use the custom allocator as well as
was done in the original code[1] this work was based on.

[1] https://public-inbox.org/git/3397e6797f872aedd18c6d795f4976e1c579514b.1565005867.git.gitgitgadget@gmail.com/

Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-18 10:33:18 +09:00
Doan Tran Cong Danh
d58deb9c4e notes: fix minimum number of parameters to "copy" subcommand
The builtin/notes.c::copy() function is prepared to handle either
one or two arguments given from the command line; when one argument
is given, to-obj defaults to HEAD.

bbb1b8a3 ("notes: check number of parameters to "git notes copy"",
2010-06-28) tried to make sure "git notes copy" (with *no* other
argument) does not dereference NULL by checking the number of
parameters, but it incorrectly insisted that we need two arguments,
instead of either one or two.  This disabled the defaulting to-obj
to HEAD.

Correct it.

Signed-off-by: Doan Tran Cong Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-18 09:43:10 +09:00
Jonathan Tan
5374a290aa fetch-pack: write fetched refs to .promisor
The specification of promisor packfiles (in partial-clone.txt) states
that the .promisor files that accompany packfiles do not matter (just
like .keep files), so whenever a packfile is fetched from the promisor
remote, Git has been writing empty .promisor files. But these files
could contain more useful information.

So instead of writing empty files, write the refs fetched to these
files. This makes it easier to debug issues with partial clones, as we
can identify what refs (and their associated hashes) were fetched at the
time the packfile was downloaded, and if necessary, compare those hashes
against what the promisor remote reports now.

This is implemented by teaching fetch-pack to write its own non-empty
.promisor file whenever it knows the name of the pack's lockfile. This
covers the case wherein the user runs "git fetch" with an internal
protocol or HTTP protocol v2 (fetch_refs_via_pack() in transport.c sets
lock_pack) and with HTTP protocol v0/v1 (fetch_git() in remote-curl.c
passes "--lock-pack" to "fetch-pack").

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Acked-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-16 11:07:51 +09:00
Phillip Wood
49697cb721 move run_commit_hook() to libgit and use it there
This function was declared in commit.h but was implemented in
builtin/commit.c so was not part of libgit. Move it to libgit so we can
use it in the sequencer. This simplifies the implementation of
run_prepare_commit_msg_hook() and will be used in the next commit.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-16 10:30:51 +09:00
Denton Liu
bf8e65b30b format-patch: teach --cover-from-description option
Before, when format-patch generated a cover letter, only the body would
be populated with a branch's description while the subject would be
populated with placeholder text. However, users may want to have the
subject of their cover letter automatically populated in the same way.

Teach format-patch to accept the `--cover-from-description` option and
corresponding `format.coverFromDescription` config, allowing users to
populate different parts of the cover letter (including the subject
now).

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-16 10:26:45 +09:00
Denton Liu
a92331df18 format-patch: use enum variables
Before, `thread` and `config_cover_letter` were defined as ints even
though they behaved as enums. Define actual enums and change these
variables to use these new definitions.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-16 10:26:44 +09:00
Denton Liu
46273df7bf format-patch: replace erroneous and condition
Commit 30984ed2e9 (format-patch: support deep threading, 2009-02-19),
introduced the following lines:

	#define THREAD_SHALLOW 1

	[...]

	thread = git_config_bool(var, value) && THREAD_SHALLOW;

Since git_config_bool() returns a bool, the trailing `&& THREAD_SHALLOW`
is a no-op. Replace this errorneous and condition with a ternary
statement so that it is clear what the configured value is when a
boolean is given.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-16 10:26:42 +09:00
Junio C Hamano
3b9ec27919 Merge branch 'js/trace2-fetch-push'
Dev support.

* js/trace2-fetch-push:
  transport: push codepath can take arbitrary repository
  push: add trace2 instrumentation
  fetch: add trace2 instrumentation
2019-10-15 13:48:03 +09:00
Junio C Hamano
5efabc7ed9 Merge branch 'ew/hashmap'
Code clean-up of the hashmap API, both users and implementation.

* ew/hashmap:
  hashmap_entry: remove first member requirement from docs
  hashmap: remove type arg from hashmap_{get,put,remove}_entry
  OFFSETOF_VAR macro to simplify hashmap iterators
  hashmap: introduce hashmap_free_entries
  hashmap: hashmap_{put,remove} return hashmap_entry *
  hashmap: use *_entry APIs for iteration
  hashmap_cmp_fn takes hashmap_entry params
  hashmap_get{,_from_hash} return "struct hashmap_entry *"
  hashmap: use *_entry APIs to wrap container_of
  hashmap_get_next returns "struct hashmap_entry *"
  introduce container_of macro
  hashmap_put takes "struct hashmap_entry *"
  hashmap_remove takes "const struct hashmap_entry *"
  hashmap_get takes "const struct hashmap_entry *"
  hashmap_add takes "struct hashmap_entry *"
  hashmap_get_next takes "const struct hashmap_entry *"
  hashmap_entry_init takes "struct hashmap_entry *"
  packfile: use hashmap_entry in delta_base_cache_entry
  coccicheck: detect hashmap_entry.hash assignment
  diff: use hashmap_entry_init on moved_entry.ent
2019-10-15 13:48:02 +09:00
Junio C Hamano
16d9d7184b Merge branch 'en/fast-imexport-nested-tags'
Updates to fast-import/export.

* en/fast-imexport-nested-tags:
  fast-export: handle nested tags
  t9350: add tests for tags of things other than a commit
  fast-export: allow user to request tags be marked with --mark-tags
  fast-export: add support for --import-marks-if-exists
  fast-import: add support for new 'alias' command
  fast-import: allow tags to be identified by mark labels
  fast-import: fix handling of deleted tags
  fast-export: fix exporting a tag and nothing else
2019-10-15 13:48:00 +09:00
Junio C Hamano
6d5291be45 Merge branch 'js/azure-pipelines-msvc'
CI updates.

* js/azure-pipelines-msvc:
  ci: also build and test with MS Visual Studio on Azure Pipelines
  ci: really use shallow clones on Azure Pipelines
  tests: let --immediate and --write-junit-xml play well together
  test-tool run-command: learn to run (parts of) the testsuite
  vcxproj: include more generated files
  vcxproj: only copy `git-remote-http.exe` once it was built
  msvc: work around a bug in GetEnvironmentVariable()
  msvc: handle DEVELOPER=1
  msvc: ignore some libraries when linking
  compat/win32/path-utils.h: add #include guards
  winansi: use FLEX_ARRAY to avoid compiler warning
  msvc: avoid using minus operator on unsigned types
  push: do not pretend to return `int` from `die_push_simple()`
2019-10-15 13:48:00 +09:00
Junio C Hamano
ccc289915a Merge branch 'gs/commit-graph-trace-with-cmd'
Dev support.

* gs/commit-graph-trace-with-cmd:
  commit-graph: emit trace2 cmd_mode for each sub-command
2019-10-15 13:48:00 +09:00
Junio C Hamano
d96e31e390 Merge branch 'js/fetch-jobs'
"git fetch --jobs=<n>" allowed <n> parallel jobs when fetching
submodules, but this did not apply to "git fetch --multiple" that
fetches from multiple remote repositories.  It now does.

* js/fetch-jobs:
  fetch: let --jobs=<n> parallelize --multiple, too
2019-10-15 13:48:00 +09:00
Junio C Hamano
280bd44551 Merge branch 'en/merge-recursive-cleanup'
The merge-recursive machiery is one of the most complex parts of
the system that accumulated cruft over time.  This large series
cleans up the implementation quite a bit.

* en/merge-recursive-cleanup: (26 commits)
  merge-recursive: fix the fix to the diff3 common ancestor label
  merge-recursive: fix the diff3 common ancestor label for virtual commits
  merge-recursive: alphabetize include list
  merge-recursive: add sanity checks for relevant merge_options
  merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
  merge-recursive: split internal fields into a separate struct
  merge-recursive: avoid losing output and leaking memory holding that output
  merge-recursive: comment and reorder the merge_options fields
  merge-recursive: consolidate unnecessary fields in merge_options
  merge-recursive: move some definitions around to clean up the header
  merge-recursive: rename merge_options argument to opt in header
  merge-recursive: rename 'mrtree' to 'result_tree', for clarity
  merge-recursive: use common name for ancestors/common/base_list
  merge-recursive: fix some overly long lines
  cache-tree: share code between functions writing an index as a tree
  merge-recursive: don't force external callers to do our logging
  merge-recursive: remove useless parameter in merge_trees()
  merge-recursive: exit early if index != head
  Ensure index matches head before invoking merge machinery, round N
  merge-recursive: remove another implicit dependency on the_repository
  ...
2019-10-15 13:47:59 +09:00
Jakob Jarmar
556895d0c8 stash: avoid recursive hard reset on submodules
git stash push does not recursively stash submodules, but if
submodule.recurse is set, it may recursively reset --hard them. Having
only the destructive action recurse is likely to be surprising
behaviour, and unlikely to be desirable, so the easiest fix should be to
ensure that the call to git reset --hard never recurses into submodules.

This matches the behavior of check_changes_tracked_files, which ignores
submodules.

Signed-off-by: Jakob Jarmar <jakob@jarmar.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-15 10:34:44 +09:00
Bert Wesarg
edefc31873 format-patch: create leading components of output directory
'git format-patch -o <outdir>' did an equivalent of 'mkdir <outdir>'
not 'mkdir -p <outdir>', which is being corrected.

Avoid the usage of 'adjust_shared_perm' on the leading directories which
may have security implications. Achieved by temporarily disabling of
'config.sharedRepository' like 'git init' does.

Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-12 11:51:20 +09:00
Junio C Hamano
66102cfad8 Merge branch 'js/stash-apply-in-secondary-worktree'
"git stash apply" in a subdirectory of a secondary worktree failed
to access the worktree correctly, which has been corrected.

* js/stash-apply-in-secondary-worktree:
  stash apply: report status correctly even in a worktree's subdirectory
2019-10-11 14:24:48 +09:00
Junio C Hamano
a4c5d9f66e Merge branch 'rs/dedup-includes'
Code cleanup.

* rs/dedup-includes:
  treewide: remove duplicate #include directives
2019-10-11 14:24:48 +09:00
Junio C Hamano
676278f8ea Merge branch 'bc/object-id-part17'
Preparation for SHA-256 upgrade continues.

* bc/object-id-part17: (26 commits)
  midx: switch to using the_hash_algo
  builtin/show-index: replace sha1_to_hex
  rerere: replace sha1_to_hex
  builtin/receive-pack: replace sha1_to_hex
  builtin/index-pack: replace sha1_to_hex
  packfile: replace sha1_to_hex
  wt-status: convert struct wt_status to object_id
  cache: remove null_sha1
  builtin/worktree: switch null_sha1 to null_oid
  builtin/repack: write object IDs of the proper length
  pack-write: use hash_to_hex when writing checksums
  sequencer: convert to use the_hash_algo
  bisect: switch to using the_hash_algo
  sha1-lookup: switch hard-coded constants to the_hash_algo
  config: use the_hash_algo in abbrev comparison
  combine-diff: replace GIT_SHA1_HEXSZ with the_hash_algo
  bundle: switch to use the_hash_algo
  connected: switch GIT_SHA1_HEXSZ to the_hash_algo
  show-index: switch hard-coded constants to the_hash_algo
  blame: remove needless comparison with GIT_SHA1_HEXSZ
  ...
2019-10-11 14:24:46 +09:00
Junio C Hamano
aafb75452b Merge branch 'en/clean-nested-with-ignored'
"git clean" fixes.

* en/clean-nested-with-ignored:
  dir: special case check for the possibility that pathspec is NULL
  clean: fix theoretical path corruption
  clean: rewrap overly long line
  clean: avoid removing untracked files in a nested git repository
  clean: disambiguate the definition of -d
  git-clean.txt: do not claim we will delete files with -n/--dry-run
  dir: add commentary explaining match_pathspec_item's return value
  dir: if our pathspec might match files under a dir, recurse into it
  dir: make the DO_MATCH_SUBMODULE code reusable for a non-submodule case
  dir: also check directories for matching pathspecs
  dir: fix off-by-one error in match_pathspec_item
  dir: fix typo in comment
  t7300: add testcases showing failure to clean specified pathspecs
2019-10-11 14:24:46 +09:00
Junio C Hamano
0b4fae553c Merge branch 'sg/name-rev-cutoff-underflow-fix'
Integer arithmetic fix.

* sg/name-rev-cutoff-underflow-fix:
  name-rev: avoid cutoff timestamp underflow
2019-10-09 14:00:58 +09:00
Junio C Hamano
417056578a Merge branch 'bw/submodule-helper-usage-fix'
Typofix.

* bw/submodule-helper-usage-fix:
  builtin/submodule--helper: fix usage string for 'update-clone'
2019-10-07 11:33:03 +09:00