Commit Graph

14068 Commits

Author SHA1 Message Date
Victoria Dye
fd56fba97f reset: introduce --[no-]refresh option to --mixed
Add a new --[no-]refresh option that is intended to explicitly determine
whether a mixed reset should end in an index refresh.

Starting at 9ac8125d1a (reset: don't compute unstaged changes after reset
when --quiet, 2018-10-23), using the '--quiet' option results in skipping
the call to 'refresh_index(...)' at the end of a mixed reset with the goal
of improving performance. However, by coupling behavior that modifies the
index with the option that silences logs, there is no way for users to have
one without the other (i.e., silenced logs with a refreshed index) without
incurring the overhead of a separate call to 'git update-index --refresh'.
Furthermore, there is minimal user-facing documentation indicating that
--quiet skips the index refresh, potentially leading to unexpected issues
executing commands after 'git reset --quiet' that do not themselves refresh
the index (e.g., internals of 'git stash', 'git read-tree').

To mitigate these issues, '--[no-]refresh' and 'reset.refresh' are
introduced to provide a dedicated mechanism for refreshing the index. When
either is set, '--quiet' and 'reset.quiet' revert to controlling only
whether logs are silenced and do not affect index refresh.

Helped-by: Derrick Stolee <derrickstolee@github.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-14 18:51:56 -07:00
Victoria Dye
e86ec71d20 reset: revise index refresh advice
Update the advice describing index refresh from "enumerate unstaged changes"
to "refresh the index." Describing 'refresh_index(...)' as "enumerating
unstaged changes" is not fully representative of what an index refresh is
doing; more generally, it updates the properties of index entries that are
affected by outside-of-index state, e.g. CE_UPTODATE, which is affected by
the file contents on-disk. This distinction is relevant to operations that
read the index but do not refresh first - e.g., 'git read-tree' - where a
stale index may cause incorrect behavior.

In addition to changing the advice message, use the "advise" function to
print advice.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-14 18:51:56 -07:00
Patrick Steinhardt
a2565c48e4 repack: add config to skip updating server info
By default, git-repack(1) will update server info that is required by
the dumb HTTP transport. This can be skipped by passing the `-n` flag,
but what we're noticably missing is a config option to permanently
disable updating this information.

Add a new option "repack.updateServerInfo" which can be used to disable
the logic. Most hosting providers have turned off the dumb HTTP protocol
anyway, and on the client-side it woudln't typically be useful either.
Giving a persistent way to disable this feature thus makes quite some
sense to avoid wasting compute cycles and storage.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-14 22:25:13 +00:00
Junio C Hamano
b896f729e2 The eleventh batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-13 22:56:18 +00:00
Junio C Hamano
4eb845ac0a Merge branch 'nj/read-tree-doc-reffix'
Documentation mark-up fix.

* nj/read-tree-doc-reffix:
  Documentation: git-read-tree: separate links using commas
2022-03-13 22:56:18 +00:00
Junio C Hamano
f62106d750 Merge branch 'ab/make-optim-noop'
Makefile refactoring with a bit of suffixes rule stripping to
optimize the runtime overhead.

* ab/make-optim-noop:
  Makefiles: add and use wildcard "mkdir -p" template
  Makefile: add "$(QUIET)" boilerplate to shared.mak
  Makefile: move $(comma), $(empty) and $(space) to shared.mak
  Makefile: move ".SUFFIXES" rule to shared.mak
  Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
  Makefile: disable GNU make built-in wildcard rules
  Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
  scalar Makefile: use "The default target of..." pattern
2022-03-13 22:56:17 +00:00
Jaydeep P Das
09188ed930 userdiff: add builtin diff driver for kotlin language.
The xfuncname pattern finds func/class declarations
in diffs to display as a hunk header. The word_regex
pattern finds individual tokens in Kotlin code to generate
appropriate diffs.

This patch adds xfuncname regex and word_regex for Kotlin
language.

Signed-off-by: Jaydeep P Das <jaydeepjd.8914@gmail.com>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-12 18:15:47 -08:00
Neeraj Singh
844a8ad4f8 core.fsync: add configuration parsing
This change introduces code to parse the core.fsync setting and
configure the fsync_components variable.

core.fsync is configured as a comma-separated list of component names to
sync. Each time a core.fsync variable is encountered in the
configuration heirarchy, we start off with a clean state with the
platform default value. Passing 'none' resets the value to indicate
nothing will be synced. We gather all negative and positive entries from
the comma separated list and then compute the new value by removing all
the negative entries and adding all of the positive entries.

We issue a warning for components that are not recognized so that the
configuration code is compatible with configs from future versions of
Git with more repo components.

Complete documentation for the new setting is included in a later patch
in the series so that it can be reviewed once in final form.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-10 15:10:22 -08:00
Neeraj Singh
abf38abec2 core.fsyncmethod: add writeout-only mode
This commit introduces the `core.fsyncMethod` configuration
knob, which can currently be set to `fsync` or `writeout-only`.

The new writeout-only mode attempts to tell the operating system to
flush its in-memory page cache to the storage hardware without issuing a
CACHE_FLUSH command to the storage controller.

Writeout-only fsync is significantly faster than a vanilla fsync on
common hardware, since data is written to a disk-side cache rather than
all the way to a durable medium. Later changes in this patch series will
take advantage of this primitive to implement batching of hardware
flushes.

When git_fsync is called with FSYNC_WRITEOUT_ONLY, it may fail and the
caller is expected to do an ordinary fsync as needed.

On Apple platforms, the fsync system call does not issue a CACHE_FLUSH
directive to the storage controller. This change updates fsync to do
fcntl(F_FULLFSYNC) to make fsync actually durable. We maintain parity
with existing behavior on Apple platforms by setting the default value
of the new core.fsyncMethod option.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-10 15:10:22 -08:00
Junio C Hamano
1a4874565f The tenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09 13:38:46 -08:00
Junio C Hamano
1f3c5f39e0 Merge branch 'ab/help-fixes'
Updates to how command line options to "git help" are handled.

* ab/help-fixes:
  help: don't print "\n" before single-section output
  help: add --no-[external-commands|aliases] for use with --all
  help: error if [-a|-g|-c] and [-i|-m|-w] are combined
  help: correct usage & behavior of "git help --all"
  help: note the option name on option incompatibility
  help.c: split up list_all_cmds_help() function
  help tests: test "git" and "git help [-a|-g] spacing
  help.c: use puts() instead of printf{,_ln}() for consistency
  help doc: add missing "]" to "[-a|--all]"
2022-03-09 13:38:24 -08:00
Junio C Hamano
69a3b75fa6 Merge branch 'ab/c99-variadic-macros'
Remove the escape hatch we added when we introduced the weather
balloon to use variadic macros unconditionally, to make it official
that we now have a hard dependency on the feature.

* ab/c99-variadic-macros:
  C99: remove hardcoded-out !HAVE_VARIADIC_MACROS code
  git-compat-util.h: clarify GCC v.s. C99-specific in comment
2022-03-09 13:38:24 -08:00
Junio C Hamano
4763ccd7f4 Merge branch 'hn/reftable-no-empty-keys'
General clean-up in reftable implementation, including
clarification of the API documentation, tightening the code to
honor documented length limit, etc.

* hn/reftable-no-empty-keys:
  reftable: rename writer_stats to reftable_writer_stats
  reftable: add test for length of disambiguating prefix
  reftable: ensure that obj_id_len is >= 2 on writing
  reftable: avoid writing empty keys at the block layer
  reftable: add a test that verifies that writing empty keys fails
  reftable: reject 0 object_id_len
  Documentation: object_id_len goes up to 31
2022-03-09 13:38:24 -08:00
Junio C Hamano
d169d51504 Merge branch 'jc/cat-file-batch-commands'
"git cat-file" learns "--batch-command" mode, which is a more
flexible interface than the existing "--batch" or "--batch-check"
modes, to allow different kinds of inquiries made.

* jc/cat-file-batch-commands:
  cat-file: add --batch-command mode
  cat-file: add remove_timestamp helper
  cat-file: introduce batch_mode enum to replace print_contents
  cat-file: rename cmdmode to transform_mode
2022-03-09 13:38:24 -08:00
Junio C Hamano
82386b4496 Merge branch 'en/present-despite-skipped'
In sparse-checkouts, files mis-marked as missing from the working tree
could lead to later problems.  Such files were hard to discover, and
harder to correct.  Automatically detecting and correcting the marking
of such files has been added to avoid these problems.

* en/present-despite-skipped:
  repo_read_index: add config to expect files outside sparse patterns
  Accelerate clear_skip_worktree_from_present_files() by caching
  Update documentation related to sparsity and the skip-worktree bit
  repo_read_index: clear SKIP_WORKTREE bit from files present in worktree
  unpack-trees: fix accidental loss of user changes
  t1011: add testcase demonstrating accidental loss of user modifications
2022-03-09 13:38:23 -08:00
Derrick Stolee
105c6f14ad bundle: parse filter capability
The v3 bundle format has capabilities, allowing newer versions of Git to
create bundles with newer features. Older versions that do not
understand these new capabilities will fail with a helpful warning.

Create a new capability allowing Git to understand that the contained
pack-file is filtered according to some object filter. Typically, this
filter will be "blob:none" for a blobless partial clone.

This change teaches Git to parse this capability, place its value in the
bundle header, and demonstrate this understanding by adding a message to
'git bundle verify'.

Since we will use gently_parse_list_objects_filter() outside of
list-objects-filter-options.c, make it an external method and move its
API documentation to before its declaration.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09 10:25:27 -08:00
Derrick Stolee
f0d2f84919 MyFirstObjectWalk: update recommended usage
The previous change consolidated traverse_commit_list() and
traverse_commit_list_filtered(). This allows us to simplify the
recommended usage in MyFirstObjectWalk.txt to use this new set of
values.

While here, add some clarification on the difference between the two
methods.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09 10:25:27 -08:00
Derrick Stolee
1f52cdfacb index-pack: document and test the --promisor option
The --promisor option of 'git index-pack' was created in 88e2f9e
(introduce fetch-object: fetch one promisor object, 2017-12-05) but was
untested. It is currently unused within the Git codebase, but that will
change in an upcoming change to 'git bundle unbundle' when there is a
filter capability.

For now, add documentation about the option and add a test to ensure it
is working as expected.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09 10:25:26 -08:00
Derrick Stolee
6dbf4b8172 commit-graph: declare bankruptcy on GDAT chunks
The Generation Data (GDAT) and Generation Data Overflow (GDOV) chunks
store corrected commit date offsets, used for generation number v2.
Recent changes have demonstrated that previous versions of Git were
incorrectly parsing data from these chunks, but might have also been
writing them incorrectly.

I asserted [1] that the previous fixes were sufficient because the known
reasons for incorrectly writing generation number v2 data relied on
parsing the information incorrectly out of a commit-graph file, but the
previous versions of Git were not reading the generation number v2 data.

However, Patrick demonstrated [2] a case where in split commit-graphs
across an alternate boundary (and possibly some other special
conditions) it was possible to have a commit-graph that was generated by
a previous version of Git have incorrect generation number v2 data which
results in errors like the following:

  commit-graph generation for commit <oid> is 1623273624 < 1623273710

[1] https://lore.kernel.org/git/f50e74f0-9ffa-f4f2-4663-269801495ed3@github.com/
[2] https://lore.kernel.org/git/Yh93vOkt2DkrGPh2@ncase/

Clearly, there is something else going on. The situation is not
completely understood, but the errors do not reproduce if the
commit-graphs are all generated by a Git version including these recent
fixes.

If we cannot trust the existing data in the GDAT and GDOV chunks, then
we can alter the format to change the chunk IDs for these chunks. This
causes the new version of Git to silently ignore the older chunks (and
disabling generation number v2 in the process) while writing new
commit-graph files with correct data in the GDA2 and GDO2 chunks.

Update commit-graph-format.txt including a historical note about these
deprecated chunks.

Reported-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-07 09:17:03 -08:00
Junio C Hamano
c2162907e9 The ninth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-06 21:25:33 -08:00
Junio C Hamano
7a4e06c42a Merge branch 'jt/ls-files-stage-recurse'
Many output modes of "ls-files" do not work with its
"--recurse-submodules" option, but the "-s" mode has been taught to
work with it.

* jt/ls-files-stage-recurse:
  ls-files: support --recurse-submodules --stage
2022-03-06 21:25:33 -08:00
Junio C Hamano
061fd5727d Merge branch 'ah/advice-switch-requires-detach-to-detach'
The error message given by "git switch HEAD~4" has been clarified
to suggest the "--detach" option that is required.

* ah/advice-switch-requires-detach-to-detach:
  switch: mention the --detach option when dying due to lack of a branch
2022-03-06 21:25:32 -08:00
Junio C Hamano
aae90a156d Merge branch 'ds/worktree-docs'
Tighten the language around "working tree" and "worktree" in the
docs.

* ds/worktree-docs:
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: extract checkout_worktree()
  worktree: extract copy_sparse_checkout()
  worktree: extract copy_filtered_worktree_config()
  worktree: combine two translatable messages
2022-03-06 21:25:31 -08:00
Taylor Blau
56710a7ae0 builtin/remote.c: show progress when renaming remote references
When renaming a remote, Git needs to rename all remote tracking
references to the remote's new name (e.g., renaming
"refs/remotes/old/foo" to "refs/remotes/new/foo" when renaming a remote
from "old" to "new").

This can be somewhat slow when there are many references to rename,
since each rename is done in a separate call to rename_ref() as opposed
to grouping all renames together into the same transaction. It would be
nice to execute all renames as a single transaction, but there is a
snag: the reference transaction backend doesn't support renames during a
transaction (only individually, via rename_ref()).

The reasons there are described in more detail in [1], but the main
problem is that in order to preserve the existing reflog, it must be
moved while holding both locks (i.e., on "oldname" and "newname"), and
the ref transaction code doesn't support inserting arbitrary actions
into the middle of a transaction like that.

As an aside, adding support for this to the ref transaction code is
less straightforward than inserting both a ref_update() and ref_delete()
call into the same transaction. rename_ref()'s special handling to
detect D/F conflicts would need to be rewritten for the transaction code
if we wanted to proactively catch D/F conflicts when renaming a
reference during a transaction. The reftable backend could support this
much more readily because of its lack of D/F conflicts.

Instead of a more complex modification to the ref transaction code,
display a progress meter when running verbosely in order to convince the
user that Git is doing work while renaming a remote.

This is mostly done as-expected, with the minor caveat that we
intentionally count symrefs renames twice, since renaming a symref takes
place over two separate calls (one to delete the old one, and another to
create the new one).

[1]: https://lore.kernel.org/git/572367B4.4050207@alum.mit.edu/

Suggested-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:44:05 -08:00
Nihal Jere
63a36017fe Documentation: git-read-tree: separate links using commas
This makes it consistent with the rest of the documentation.

Signed-off-by: Nihal Jere <nihal@nihaljere.xyz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:25:17 -08:00
Ævar Arnfjörð Bjarmason
0b6d0bc924 Makefiles: add and use wildcard "mkdir -p" template
Add a template to do the "mkdir -p" of $(@D) (the parent dir of $@)
for us, and use it for the "make lint-docs" targets I added in
8650c6298c (doc lint: make "lint-docs" non-.PHONY, 2021-10-15).

As seen in 4c64fb5aad (Documentation/Makefile: fix lint-docs mkdir
dependency, 2021-10-26) maintaining these manual lists of parent
directory dependencies is fragile, in addition to being obviously
verbose.

I used this pattern at the time because I couldn't find another method
than "order-only" prerequisites to avoid doing a "mkdir -p $(@D)" for
every file being created, which as noted in [1] would be significantly
slower.

But as it turns out we can use this neat trick of only doing a "mkdir
-p" if the $(wildcard) macro tells us the path doesn't exist. A re-run
of a performance test similar to that noted downthread of [1] in [2]
shows that this is faster, in addition to being less verbose and more
reliable (this uses my "git-hyperfine" thin wrapper for "hyperfine"[3]):

    $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make -C Documentation lint-docs' -p 'rm -rf Documentation/.build' 'make -C Documentation -j1 lint-docs'
    Benchmark 1: make -C Documentation -j1 lint-docs' in 'HEAD~1
      Time (mean ± σ):      2.914 s ±  0.062 s    [User: 2.449 s, System: 0.489 s]
      Range (min … max):    2.834 s …  3.020 s    10 runs

    Benchmark 2: make -C Documentation -j1 lint-docs' in 'HEAD~0
      Time (mean ± σ):      2.315 s ±  0.062 s    [User: 1.950 s, System: 0.386 s]
      Range (min … max):    2.229 s …  2.397 s    10 runs

    Summary
      'make -C Documentation -j1 lint-docs' in 'HEAD~0' ran
        1.26 ± 0.04 times faster than 'make -C Documentation -j1 lint-docs' in 'HEAD~1'

So let's use that pattern both for the "lint-docs" target, and a few
miscellaneous other targets.

This method of creating parent directories is explicitly racy in that
we don't know if we're going to say always create a "foo" followed by
a "foo/bar" under parallelism, or skip the "foo" because we created
"foo/bar" first. In this case it doesn't matter for anything except
that we aren't guaranteed to get the same number of rules firing when
running make in parallel.

1. https://lore.kernel.org/git/211028.861r45y3pt.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/211028.86o879vvtp.gmgdl@evledraar.gmail.com/
3. https://gitlab.com/avar/git-hyperfine/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:14:55 -08:00
Ævar Arnfjörð Bjarmason
a9fda017f4 Makefile: add "$(QUIET)" boilerplate to shared.mak
The $(QUIET) variables we define are largely duplicated between our
various Makefiles, let's define them in the new "shared.mak" instead.

Since we're not using the environment to pass these around we don't
need to export the "QUIET_GEN" and "QUIET_BUILT_IN" variables
anymore. The "QUIET_GEN" variable is used in "git-gui/Makefile" and
"gitweb/Makefile", but they've got their own definition for those. The
"QUIET_BUILT_IN" variable is only used in the top-level "Makefile". We
still need to export the "V" variable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:14:55 -08:00
Ævar Arnfjörð Bjarmason
8df786d298 Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
We have various behavior that's shared across our Makefiles, or that
really should be (e.g. via defined templates). Let's create a
top-level "shared.mak" to house those sorts of things, and start by
adding the ".DELETE_ON_ERROR" flag to it.

See my own 7b76d6bf22 (Makefile: add and use the ".DELETE_ON_ERROR"
flag, 2021-06-29) and db10fc6c09 (doc: simplify Makefile using
.DELETE_ON_ERROR, 2021-05-21) for the addition and use of the
".DELETE_ON_ERROR" flag.

I.e. this changes the behavior of existing rules in the altered
Makefiles (except "Makefile" & "Documentation/Makefile"). I'm
confident that this is safe having read the relevant rules in those
Makfiles, and as the GNU make manual notes that it isn't the default
behavior is out of an abundance of backwards compatibility
caution. From edition 0.75 of its manual, covering GNU make 4.3:

    [Enabling '.DELETE_ON_ERROR' is] almost always what you want
    'make' to do, but it is not historical practice; so for
    compatibility, you must explicitly request it.

This doesn't introduce a bug by e.g. having this
".DELETE_ON_ERROR" flag only apply to this new shared.mak, Makefiles
have no such scoping semantics.

It does increase the danger that any Makefile without an explicit "The
default target of this Makefile is..." snippet to define the default
target as "all" could have its default rule changed if our new
shared.mak ever defines a "real" rule. In subsequent commits we'll be
careful not to do that, and such breakage would be obvious e.g. in the
case of "make -C t".

We might want to make that less fragile still (e.g. by using
".DEFAULT_GOAL" as noted in the preceding commit), but for now let's
simply include "shared.mak" without adding that boilerplate to all the
Makefiles that don't have it already. Most of those are already
exposed to that potential caveat e.g. due to including "config.mak*".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:14:55 -08:00
Elijah Newren
ecc7c8841d repo_read_index: add config to expect files outside sparse patterns
Typically with sparse checkouts, we expect files outside the sparsity
patterns to be marked as SKIP_WORKTREE and be missing from the working
tree.  Sometimes this expectation would be violated however; including
in cases such as:
  * users grabbing files from elsewhere and writing them to the worktree
    (perhaps by editing a cached copy in an editor, copying/renaming, or
     even untarring)
  * various git commands having incomplete or no support for the
    SKIP_WORKTREE bit[1,2]
  * users attempting to "abort" a sparse-checkout operation with a
    not-so-early Ctrl+C (updating $GIT_DIR/info/sparse-checkout and the
    working tree is not atomic)[3].
When the SKIP_WORKTREE bit in the index did not reflect the presence of
the file in the working tree, it traditionally caused confusion and was
difficult to detect and recover from.  So, in a sparse checkout, since
af6a51875a (repo_read_index: clear SKIP_WORKTREE bit from files present
in worktree, 2022-01-14), Git automatically clears the SKIP_WORKTREE
bit at index read time for entries corresponding to files that are
present in the working tree.

There is another workflow, however, where it is expected that paths
outside the sparsity patterns appear to exist in the working tree and
that they do not lose the SKIP_WORKTREE bit, at least until they get
modified.  A Git-aware virtual file system[4] takes advantage of its
position as a file system driver to expose all files in the working
tree, fetch them on demand using partial clone on access, and tell Git
to pay attention to them on demand by updating the sparse checkout
pattern on writes.  This means that commands like "git status" only have
to examine files that have potentially been modified, whereas commands
like "ls" are able to show the entire codebase without requiring manual
updates to the sparse checkout pattern.

Thus since af6a51875a, Git with such Git-aware virtual file systems
unsets the SKIP_WORKTREE bit for all files and commands like "git
status" have to fetch and examine them all.

Introduce a configuration setting sparse.expectFilesOutsideOfPatterns to
allow limiting the tracked set of files to a small set once again.  A
Git-aware virtual file system or other application that wants to
maintain files outside of the sparse checkout can set this in a
repository to instruct Git not to check for the presence of
SKIP_WORKTREE files.  The setting defaults to false, so most users of
sparse checkout will still get the benefit of an automatically updating
index to recover from the variety of difficult issues detailed in
af6a51875a for paths with SKIP_WORKTREE set despite the path being
present.

[1] https://lore.kernel.org/git/xmqqbmb1a7ga.fsf@gitster-ct.c.googlers.com/
[2] The three long paragraphs in the middle of
    https://lore.kernel.org/git/CABPp-BH9tju7WVm=QZDOvaMDdZbpNXrVWQdN-jmfN8wC6YVhmw@mail.gmail.com/
[3] https://lore.kernel.org/git/CABPp-BFnFpzwGC11TLoLs8YK5yiisA5D5-fFjXnJsbESVDwZsA@mail.gmail.com/
[4] such as the vfsd described in
https://lore.kernel.org/git/20220207190320.2960362-1-jonathantanmy@google.com/

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-01 23:37:48 -08:00
Alex Henrie
808213ba36 switch: mention the --detach option when dying due to lack of a branch
Users who are accustomed to doing `git checkout <tag>` assume that
`git switch <tag>` will do the same thing. Inform them of the --detach
option so they aren't left wondering why `git switch` doesn't work but
`git checkout` does.

Signed-off-by: Alex Henrie <alexhenrie24@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-25 22:21:48 -08:00
Junio C Hamano
715d08a9e5 The eighth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-25 15:47:38 -08:00
Junio C Hamano
2e65591ed6 Merge branch 'js/apply-partial-clone-filters-recursively'
"git clone --filter=... --recurse-submodules" only makes the
top-level a partial clone, while submodules are fully cloned.  This
behaviour is changed to pass the same filter down to the submodules.

* js/apply-partial-clone-filters-recursively:
  clone, submodule: pass partial clone filters to submodules
2022-02-25 15:47:35 -08:00
Junio C Hamano
6249ce2d1b Merge branch 'ds/sparse-checkout-requires-per-worktree-config'
"git sparse-checkout" wants to work with per-worktree configuration,
but did not work well in a worktree attached to a bare repository.

* ds/sparse-checkout-requires-per-worktree-config:
  config: make git_configset_get_string_tmp() private
  worktree: copy sparse-checkout patterns and config on add
  sparse-checkout: set worktree-config correctly
  config: add repo_config_set_worktree_gently()
  worktree: create init_worktree_config()
  Documentation: add extensions.worktreeConfig details
2022-02-25 15:47:33 -08:00
Junio C Hamano
dab1b7905d The seventh batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 16:58:13 -08:00
Junio C Hamano
66633f25c6 Merge branch 'bc/clarify-eol-attr'
Documentation update

* bc/clarify-eol-attr:
  doc: clarify interaction between 'eol' and text=auto
2022-02-23 16:58:04 -08:00
Jonathan Tan
290eada0ac ls-files: support --recurse-submodules --stage
e77aa336f1 ("ls-files: optionally recurse into submodules", 2016-10-10)
taught ls-files the --recurse-submodules argument, but only in a limited
set of circumstances. In particular, --stage was unsupported, perhaps
because there was no repo_find_unique_abbrev(), which was only
introduced in 8bb95572b0 ("sha1-name.c: add
repo_find_unique_abbrev_r()", 2019-04-16). This function is needed for
using --recurse-submodules with --stage.

Now that we have repo_find_unique_abbrev(), teach support for this
combination of arguments.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 16:41:55 -08:00
Ævar Arnfjörð Bjarmason
1ce590133b help: add --no-[external-commands|aliases] for use with --all
Add the ability to only emit git's own usage information under
--all. This also allows us to extend the "test_section_spacing" tests
added in a preceding commit to test "git help --all"
output.

Previously we could not do that, as the tests might find a git-*
command in the "$PATH", which would make the output differ from one
setup to another.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 13:41:37 -08:00
Ævar Arnfjörð Bjarmason
5e8068b74d help: correct usage & behavior of "git help --all"
Do the same for the "--all" option that I did for "--guides" in
9856ea6785 (help: correct usage & behavior of "git help --guides",
2021-09-22). I.e. we've documented it as ignoring non-option
arguments, let's have it error out instead.

As with other changes made in 62f035aee3 (Merge branch
'ab/help-config-vars', 2021-10-13) this is technically a change in
behavior, but in practice it's just a bug fix. We were ignoring this
before, but by erroring we can simplify our documentation and
synopsis, as well as avoid user confusion as they wonder what the
difference between e.g. "git help --all" and "git help --all status"
is (there wasn't any difference).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 13:41:37 -08:00
Ævar Arnfjörð Bjarmason
4bf5cdab8e help doc: add missing "]" to "[-a|--all]"
Add a missing "]" to documentation added in 63eae83f8f (help: add "-a
--verbose" to list all commands with synopsis, 2018-05-20). This made
it seem as though "--[no-]verbose" can only be provided with "--all",
not "-a". The corresponding usage information in the C
code ("builtin_help_usage") does not have the same problem.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 13:41:36 -08:00
Han-Wen Nienhuys
fa48de62ac Documentation: object_id_len goes up to 31
The value is stored in a 5-bit field, so we can't support more without
a format version upgrade.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 13:36:26 -08:00
Derrick Stolee
07d85380b2 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the last of multiple changes to git-worktree.txt, starting at
the LIST OUTPUT FORMAT section.

The EXAMPLES section has an instance of "working tree" that must stay as
it is, because it is not talking about a worktree, but an example of why
a user might want to create a worktree.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
f13a146c81 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the sixth of multiple changes to git-worktree.txt, restricted to
the DETAILS section.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
7b215826f3 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the fifth of multiple changes to git-worktree.txt, restricted to
the CONFIGURATION FILE section.

While here, clear up some language to improve readability.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
a777d4c750 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the fourth of multiple changes to git-worktree.txt, restricted
to the REFS section.

This section previously described "per working tree" refs but they are
now replaced with "per-worktree" refs, which matches the definition in
glossary-content.txt.

The first paragraph of this section was also a bit confusing, so it is
cleaned up to make it easier to understand.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
6036be1458 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the third of multiple changes to git-worktree.txt, restricted to
the OPTIONS section.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
599701441e worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the second of multiple changes to git-worktree.txt, restricted
to the COMMANDS section.

There is some language around the movement of "the working tree of a
linked worktree" which is used once, but the remaining uses are left as
just moving "a linked worktree" for brevity.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:41 -08:00
Derrick Stolee
c57bf8ce9e worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the first of multiple changes to git-worktree.txt, restricted to
the DESCRIPTION section.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:41 -08:00
Ævar Arnfjörð Bjarmason
56a29d2c97 C99: remove hardcoded-out !HAVE_VARIADIC_MACROS code
Remove the "else" branches of the HAVE_VARIADIC_MACROS macro, which
have been unconditionally omitted since 765dc16888 (git-compat-util:
always enable variadic macros, 2021-01-28).

Since were always omitted, anyone trying to use a compiler without
variadic macro support to compile a git since version
git v2.31.0 or later would have had a compilation error. 10 months
across a few releases since then should have been enough time for
anyone who cared to run into that and report the issue.

In addition to that, for anyone unsetting HAVE_VARIADIC_MACROS we've
been emitting extremely verbose warnings since at least
ee4512ed48 (trace2: create new combined trace facility,
2019-02-22). That's because there is no such thing as a
"region_enter_printf" or "region_leave_printf" format, so at least
under GCC and Clang everything that includes trace.h (almost every
file) emits a couple of warnings about that.

There's a large benefit to being able to have a hard dependency rely
on variadic macros, the code surrounding usage.c is hard to maintain
if we need to write two implementations of everything, and by relying
on "__FILE__" and "__LINE__" along with "__VA_ARGS__" we can in the
future make error(), die() etc. log where they were called from. We've
also recently merged d67fc4bf0b (Merge branch 'bc/require-c99',
2021-12-10) which further cements our hard dependency on C99.

So let's delete the fallback code, and update our CodingGuidelines to
note that we depend on this. The added bullet-point starts with
lower-case for consistency with other bullet-points in that section.

The diff in "trace.h" is relatively hard to read, since we need to
retain the existing API docs, which were comments on the code used if
HAVE_VARIADIC_MACROS was not defined.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-21 19:14:19 -08:00
Junio C Hamano
e6ebfd0e8c The sixth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-18 13:53:30 -08:00
Junio C Hamano
9a1d16989f Merge branch 'jc/glossary-worktree'
"working tree" and "per-worktree ref" were in glossary, but
"worktree" itself wasn't, which has been corrected.

* jc/glossary-worktree:
  glossary: describe "worktree"
2022-02-18 13:53:30 -08:00
Junio C Hamano
5cc9522b15 Merge branch 'gc/branch-recurse-submodules'
"git branch" learned the "--recurse-submodules" option.

* gc/branch-recurse-submodules:
  branch.c: use 'goto cleanup' in setup_tracking() to fix memory leaks
  branch: add --recurse-submodules option for branch creation
  builtin/branch: consolidate action-picking logic in cmd_branch()
  branch: add a dry_run parameter to create_branch()
  branch: make create_branch() always create a branch
  branch: move --set-upstream-to behavior to dwim_and_setup_tracking()
2022-02-18 13:53:29 -08:00
John Cai
440c705ea6 cat-file: add --batch-command mode
Add a new flag --batch-command that accepts commands and arguments
from stdin, similar to git-update-ref --stdin.

At GitLab, we use a pair of long running cat-file processes when
accessing object content. One for iterating over object metadata with
--batch-check, and the other to grab object contents with --batch.

However, if we had --batch-command, we wouldn't need to keep both
processes around, and instead just have one --batch-command process
where we can flip between getting object info, and getting object
contents. Since we have a pair of cat-file processes per repository,
this means we can get rid of roughly half of long lived git cat-file
processes. Given there are many repositories being accessed at any given
time, this can lead to huge savings.

git cat-file --batch-command

will enter an interactive command mode whereby the user can enter in
commands and their arguments that get queued in memory:

<command1> [arg1] [arg2] LF
<command2> [arg1] [arg2] LF

When --buffer mode is used, commands will be queued in memory until a
flush command is issued that execute them:

flush LF

The reason for a flush command is that when a consumer process (A)
talks to a git cat-file process (B) and interactively writes to and
reads from it in --buffer mode, (A) needs to be able to control when
the buffer is flushed to stdout.

Currently, from (A)'s perspective, the only way is to either

1. kill (B)'s process
2. send an invalid object to stdin.

1. is not ideal from a performance perspective as it will require
spawning a new cat-file process each time, and 2. is hacky and not a
good long term solution.

With this mechanism of queueing up commands and letting (A) issue a
flush command, process (A) can control when the buffer is flushed and
can guarantee it will receive all of the output when in --buffer mode.
--batch-command also will not allow (B) to flush to stdout until a flush
is received.

This patch adds the basic structure for adding command which can be
extended in the future to add more commands. It also adds the following
two commands (on top of the flush command):

contents <object> LF
info <object> LF

The contents command takes an <object> argument and prints out the object
contents.

The info command takes an <object> argument and prints out the object
metadata.

These can be used in the following way with --buffer:

info <object> LF
contents <object> LF
contents <object> LF
info <object> LF
flush LF
info <object> LF
flush LF

When used without --buffer:

info <object> LF
contents <object> LF
contents <object> LF
info <object> LF
info <object> LF

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: John Cai <johncai86@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-18 11:21:46 -08:00
Junio C Hamano
e2ac9141e6 The fifth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-17 16:25:06 -08:00
Junio C Hamano
a4ec347888 Merge branch 'po/doc-check-ignore-markup-fix'
Typofix.

* po/doc-check-ignore-markup-fix:
  doc: check-ignore: code-quote an exclamation mark
2022-02-17 16:25:05 -08:00
Junio C Hamano
2f45f3e2bc Merge branch 'vd/sparse-clean-etc'
"git update-index", "git checkout-index", and "git clean" are
taught to work better with the sparse checkout feature.

* vd/sparse-clean-etc:
  update-index: reduce scope of index expansion in do_reupdate
  update-index: integrate with sparse index
  update-index: add tests for sparse-checkout compatibility
  checkout-index: integrate with sparse index
  checkout-index: add --ignore-skip-worktree-bits option
  checkout-index: expand sparse checkout compatibility tests
  clean: integrate with sparse index
  reset: reorder wildcard pathspec conditions
  reset: fix validation in sparse index test
2022-02-17 16:25:05 -08:00
Junio C Hamano
708cbef33a Merge branch 'jz/rev-list-exclude-first-parent-only'
"git log" and friends learned an option --exclude-first-parent-only
to propagate UNINTERESTING bit down only along the first-parent
chain, just like --first-parent option shows commits that lack the
UNINTERESTING bit only along the first-parent chain.

* jz/rev-list-exclude-first-parent-only:
  git-rev-list: add --exclude-first-parent-only flag
2022-02-17 16:25:05 -08:00
Junio C Hamano
45fe28c951 The fourth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-16 15:14:30 -08:00
Junio C Hamano
b9f791aee6 Merge branch 'js/no-more-legacy-stash'
Removal of unused code and doc.

* js/no-more-legacy-stash:
  stash: stop warning about the obsolete `stash.useBuiltin` config setting
  stash: remove documentation for `stash.useBuiltin`
  add: remove support for `git-legacy-stash`
  git-sh-setup: remove remnant bits referring to `git-legacy-stash`
2022-02-16 15:14:30 -08:00
Junio C Hamano
9a160990ef Merge branch 'js/diff-filter-negation-fix'
"git diff --diff-filter=aR" is now parsed correctly.

* js/diff-filter-negation-fix:
  diff-filter: be more careful when looking for negative bits
  diff.c: move the diff filter bits definitions up a bit
  docs(diff): lose incorrect claim about `diff-files --diff-filter=A`
2022-02-16 15:14:30 -08:00
Junio C Hamano
70ff41ffcf Merge branch 'en/fetch-negotiation-default-fix'
Interaction between fetch.negotiationAlgorithm and
feature.experimental configuration variables has been corrected.

* en/fetch-negotiation-default-fix:
  repo-settings: rename the traditional default fetch.negotiationAlgorithm
  repo-settings: fix error handling for unknown values
  repo-settings: fix checking for fetch.negotiationAlgorithm=default
2022-02-16 15:14:30 -08:00
Junio C Hamano
f2cb46a6b3 Merge branch 'tb/midx-bitmap-corruption-fix'
A bug that made multi-pack bitmap and the object order out-of-sync,
making the .midx data corrupt, has been fixed.

* tb/midx-bitmap-corruption-fix:
  pack-bitmap.c: gracefully fallback after opening pack/MIDX
  midx: read `RIDX` chunk when present
  t/lib-bitmap.sh: parameterize tests over reverse index source
  t5326: move tests to t/lib-bitmap.sh
  t5326: extract `test_rev_exists`
  t5326: drop unnecessary setup
  pack-revindex.c: instrument loading on-disk reverse index
  midx.c: make changing the preferred pack safe
  t5326: demonstrate bitmap corruption after permutation
2022-02-16 15:14:29 -08:00
Junio C Hamano
90b7153806 Merge branch 'en/remerge-diff'
"git log --remerge-diff" shows the difference from mechanical merge
result and the result that is actually recorded in a merge commit.

* en/remerge-diff:
  diff-merges: avoid history simplifications when diffing merges
  merge-ort: mark conflict/warning messages from inner merges as omittable
  show, log: include conflict/warning messages in --remerge-diff headers
  diff: add ability to insert additional headers for paths
  merge-ort: format messages slightly different for use in headers
  merge-ort: mark a few more conflict messages as omittable
  merge-ort: capture and print ll-merge warnings in our preferred fashion
  ll-merge: make callers responsible for showing warnings
  log: clean unneeded objects during `log --remerge-diff`
  show, log: provide a --remerge-diff capability
2022-02-16 15:14:29 -08:00
Junio C Hamano
dd77ff8181 Merge branch 'll/doc-mktree-typofix'
Typofix.

* ll/doc-mktree-typofix:
  fix typo in git-mktree.txt
2022-02-16 15:14:26 -08:00
brian m. carlson
6a5678f257 doc: clarify interaction between 'eol' and text=auto
The `eol` takes effect on text files only when the index has the
contents in LF line endings.  Paths with contents in CRLF line
endings in the index may become dirty unless text=auto.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-14 13:01:25 -08:00
Junio C Hamano
b80121027d The third batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-11 16:56:01 -08:00
Junio C Hamano
83760938bd Merge branch 'jc/doc-log-messages'
Update the contributor-facing documents on proposed log messages.

* jc/doc-log-messages:
  SubmittingPatches: explain why we care about log messages
  CodingGuidelines: hint why we value clearly written log messages
  SubmittingPatches: write problem statement in the log in the present tense
2022-02-11 16:55:58 -08:00
Junio C Hamano
c46452eb98 Merge branch 'gh/doc-typos'
Typofix.

* gh/doc-typos:
  Documentation/config/pgp.txt: add missing apostrophe
  Documentation/config/pgp.txt: replace stray <TAB> character with <SPC>
2022-02-11 16:55:58 -08:00
Junio C Hamano
8db2f665e1 Merge branch 'bc/clarify-eol-attr'
Doc and test update around the eol attribute.

* bc/clarify-eol-attr:
  docs: correct documentation about eol attribute
  t0027: add tests for eol without text in .gitattributes
2022-02-11 16:55:57 -08:00
Junio C Hamano
2df5387ed0 glossary: describe "worktree"
We have description on "per worktree ref", but "worktree" is not
described in the glossary.  We do have "working tree", though.

Casually put, a "working tree" is what your editor and compiler
interacts with.  "worktree" is a mechanism to allow one or more
"working tree"s to be attached to a repository and used to check out
different commits and branches independently, which includes not
just a "working tree" but also repository metadata like HEAD, the
index to support simultaneous use of them.  Historically, we used
these terms interchangeably but we have been trying to use "working
tree" when we mean it, instead of "worktree".

Most of the existing references to "working tree" in the glossary do
refer primarily to the working tree portion, except for one that
said refs like HEAD and refs/bisect/* are per "working tree", but it
is more precise to say they are per "worktree".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-09 18:34:41 -08:00
Josh Steadmon
f05da2b48b clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.

To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.

This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.

Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.

[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
	2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
	2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-09 15:38:36 -08:00
Junio C Hamano
2b9c120970 The second batch for 2.36
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-09 14:21:18 -08:00
Junio C Hamano
c70bc338e9 Merge branch 'ab/config-based-hooks-2'
More "config-based hooks".

* ab/config-based-hooks-2:
  run-command: remove old run_hook_{le,ve}() hook API
  receive-pack: convert push-to-checkout hook to hook.h
  read-cache: convert post-index-change to use hook.h
  commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
  git-p4: use 'git hook' to run hooks
  send-email: use 'git hook run' for 'sendemail-validate'
  git hook run: add an --ignore-missing flag
  hooks: convert worktree 'post-checkout' hook to hook library
  hooks: convert non-worktree 'post-checkout' hook to hook library
  merge: convert post-merge to use hook.h
  am: convert applypatch-msg to use hook.h
  rebase: convert pre-rebase to use hook.h
  hook API: add a run_hooks_l() wrapper
  am: convert {pre,post}-applypatch to use hook.h
  gc: use hook library for pre-auto-gc hook
  hook API: add a run_hooks() wrapper
  hook: add 'run' subcommand
2022-02-09 14:21:00 -08:00
Junio C Hamano
d9976b1845 Merge branch 'jc/name-rev-stdin'
"git name-rev --stdin" does not behave like usual "--stdin" at
all.  Start the process of renaming it to "--annotate-stdin".

* jc/name-rev-stdin:
  name-rev.c: use strbuf_getline instead of limited size buffer
  name-rev: deprecate --stdin in favor of --annotate-stdin
2022-02-09 14:21:00 -08:00
Junio C Hamano
472a219f8d Merge branch 'gc/fetch-negotiate-only-early-return'
"git fetch --negotiate-only" is an internal command used by "git
push" to figure out which part of our history is missing from the
other side.  It should never recurse into submodules even when
fetch.recursesubmodules configuration variable is set, nor it
should trigger "gc".  The code has been tightened up to ensure it
only does common ancestry discovery and nothing else.

* gc/fetch-negotiate-only-early-return:
  fetch: help translators by reusing the same message template
  fetch --negotiate-only: do not update submodules
  fetch: skip tasks related to fetching objects
  fetch: use goto cleanup in cmd_fetch()
2022-02-09 14:20:59 -08:00
Junio C Hamano
e704a4486e Merge branch 'tl/doc-cli-options-first'
We explain that revs come first before the pathspec among command
line arguments, but did not spell out that dashed options come
before other args, which has been corrected.

* tl/doc-cli-options-first:
  git-cli.txt: clarify "options first and then args"
2022-02-09 14:20:59 -08:00
Junio C Hamano
13ce8f9f14 Merge branch 'jt/conditional-config-on-remote-url'
The conditional inclusion mechanism of configuration files using
"[includeIf <condition>]" learns to base its decision on the
URL of the remote repository the repository interacts with.

* jt/conditional-config-on-remote-url:
  config: include file if remote URL matches a glob
  config: make git_config_include() static
2022-02-09 14:20:59 -08:00
Derrick Stolee
7316dc5f6f sparse-checkout: set worktree-config correctly
`git sparse-checkout set/init` enables worktree-specific
configuration[*] by setting extensions.worktreeConfig=true, but neglects
to perform the additional necessary bookkeeping of relocating
`core.bare=true` and `core.worktree` from $GIT_COMMON_DIR/config to
$GIT_COMMON_DIR/config.worktree, as documented in git-worktree.txt. As a
result of this oversight, these settings, which are nonsensical for
secondary worktrees, can cause Git commands to incorrectly consider a
worktree bare (in the case of `core.bare`) or operate on the wrong
worktree (in the case of `core.worktree`). Fix this problem by taking
advantage of the recently-added init_worktree_config() which enables
`extensions.worktreeConfig` and takes care of necessary bookkeeping.

While at it, for backward-compatibility reasons, also stop upgrading the
repository format to "1" since doing so is (unintentionally) not
required to take advantage of `extensions.worktreeConfig`, as explained
by 11664196ac ("Revert "check_repository_format_gently(): refuse
extensions for old repositories"", 2020-07-15).

[*] The main reason to use worktree-specific config for the
sparse-checkout builtin was to avoid enabling sparse-checkout patterns
in one and causing a loss of files in another. If a worktree does not
have a sparse-checkout patterns file, then the sparse-checkout logic
will not kick in on that worktree.

Reported-by: Sean Allred <allred.sean@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-08 09:49:20 -08:00
Derrick Stolee
5c11c0d52c Documentation: add extensions.worktreeConfig details
The extensions.worktreeConfig extension was added in 58b284a (worktree:
add per-worktree config files, 2018-10-21) and was somewhat documented
in Documentation/git-config.txt. However, the extensions.worktreeConfig
value was not specified further in the list of possible config keys. The
location of the config.worktree file is not specified, and there are
some precautions that should be mentioned clearly, but are only
mentioned in git-worktree.txt.

Expand the documentation to help users discover the complexities of
extensions.worktreeConfig by adding details and cross links in these
locations (relative to Documentation/):

- config/extensions.txt
- git-config.txt
- git-worktree.txt

The updates focus on items such as

* $GIT_DIR/config.worktree takes precedence over $GIT_COMMON_DIR/config.

* The core.worktree and core.bare=true settings are incorrect to have in
  the common config file when extensions.worktreeConfig is enabled.

* The sparse-checkout settings core.sparseCheckout[Cone] are recommended
  to be set in the worktree config.

As documented in 11664196ac ("Revert "check_repository_format_gently():
refuse extensions for old repositories"", 2020-07-15), this extension
must be considered regardless of the repository format version for
historical reasons.

A future change will update references to extensions.worktreeConfig
within git-sparse-checkout.txt, but a behavior change is needed before
making those updates.

Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-08 09:49:20 -08:00
Junio C Hamano
38062e73e0 The first batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 09:43:36 -08:00
Junio C Hamano
008028a910 Merge branch 'ab/cat-file'
Assorted updates to "git cat-file", especially "-h".

* ab/cat-file:
  cat-file: s/_/-/ in typo'd usage_msg_optf() message
  cat-file: don't whitespace-pad "(...)" in SYNOPSIS and usage output
  cat-file: use GET_OID_ONLY_TO_DIE in --(textconv|filters)
  object-name.c: don't have GET_OID_ONLY_TO_DIE imply *_QUIETLY
  cat-file: correct and improve usage information
  cat-file: fix remaining usage bugs
  cat-file: make --batch-all-objects a CMDMODE
  cat-file: move "usage" variable to cmd_cat_file()
  cat-file docs: fix SYNOPSIS and "-h" output
  parse-options API: add a usage_msg_optf()
  cat-file tests: test messaging on bad objects/paths
  cat-file tests: test bad usage
2022-02-05 09:42:31 -08:00
Glen Choo
961b130d20 branch: add --recurse-submodules option for branch creation
To improve the submodules UX, we would like to teach Git to handle
branches in submodules. Start this process by teaching "git branch" the
--recurse-submodules option so that "git branch --recurse-submodules
topic" will create the `topic` branch in the superproject and its
submodules.

Although this commit does not introduce breaking changes, it does not
work well with existing --recurse-submodules commands because "git
branch --recurse-submodules" writes to the submodule ref store, but most
commands only consider the superproject gitlink and ignore the submodule
ref store. For example, "git checkout --recurse-submodules" will check
out the commits in the superproject gitlinks (and put the submodules in
detached HEAD) instead of checking out the submodule branches.

Because of this, this commit introduces a new configuration value,
`submodule.propagateBranches`. The plan is for Git commands to
prioritize submodule ref store information over superproject gitlinks if
this value is true. Because "git branch --recurse-submodules" writes to
submodule ref stores, for the sake of clarity, it will not function
unless this configuration value is set.

This commit also includes changes that support working with submodules
from a superproject commit because "branch --recurse-submodules" (and
future commands) need to read .gitmodules and gitlinks from the
superproject commit, but submodules are typically read from the
filesystem's .gitmodules and the index's gitlinks. These changes are:

* add a submodules_of_tree() helper that gives the relevant
  information of an in-tree submodule (e.g. path and oid) and
  initializes the repository
* add is_tree_submodule_active() by adding a treeish_name parameter to
  is_submodule_active()
* add the "submoduleNotUpdated" advice to advise users to update the
  submodules in their trees

Incidentally, fix an incorrect usage string that combined the 'list'
usage of git branch (-l) with the 'create' usage; this string has been
incorrect since its inception, a8dfd5eac4 (Make builtin-branch.c use
parse_options., 2007-10-07).

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Glen Choo <chooglen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-04 08:16:39 -08:00
Philip Oakley
9325285df4 doc: check-ignore: code-quote an exclamation mark
The plain quoted exclamation mark renders as italics in the
Windows pdf help manual.

Fix this with back-tick quoting and surrounding double quotes
as exemplified by the gitignore.txt guide.

While at it, fix  the surrounding double quotes for the other
special characters usages.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-03 11:13:49 -08:00
Liginity Lee
45d0212a71 fix typo in git-mktree.txt
fix a typo: change "as" to "a".

Signed-off-by: Liginity Lee <liginity@outlook.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-02 14:50:09 -08:00
Elijah Newren
db757e8b8d show, log: provide a --remerge-diff capability
When this option is specified, we remerge all (two parent) merge commits
and diff the actual merge commit to the automatically created version,
in order to show how users removed conflict markers, resolved the
different conflict versions, and potentially added new changes outside
of conflict regions in order to resolve semantic merge problems (or,
possibly, just to hide other random changes).

This capability works by creating a temporary object directory and
marking it as the primary object store.  This makes it so that any blobs
or trees created during the automatic merge are easily removable
afterwards by just deleting all objects from the temporary object
directory.

There are a few ways that this implementation is suboptimal:
  * `log --remerge-diff` becomes slow, because the temporary object
    directory can fill with many loose objects while running
  * the log output can be muddied with misplaced "warning: cannot merge
    binary files" messages, since ll-merge.c unconditionally writes those
    messages to stderr while running instead of allowing callers to
    manage them.
  * important conflict and warning messages are simply dropped; thus for
    conflicts like modify/delete or rename/rename or file/directory which
    are not representable with content conflict markers, there may be no
    way for a user of --remerge-diff to know that there had been a
    conflict which was resolved (and which possibly motivated other
    changes in the merge commit).
  * when fixing the previous issue, note that some unimportant conflict
    and warning messages might start being included.  We should instead
    make sure these remain dropped.
Subsequent commits will address these issues.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-02 10:02:27 -08:00
Elijah Newren
714edc620c repo-settings: rename the traditional default fetch.negotiationAlgorithm
Give the traditional default fetch.negotiationAlgorithm the name
'consecutive'.  Also allow a choice of 'default' to have Git decide
between the choices (currently, picking 'skipping' if
feature.experimental is true and 'consecutive' otherwise).  Update the
documentation accordingly.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-02 09:36:17 -08:00
Junio C Hamano
09e0be130d Merge branch 'js/branch-track-inherit' into gc/branch-recurse-submodules
* js/branch-track-inherit:
  branch,checkout: fix --track documentation
  branch,checkout: fix --track usage strings
  config: require lowercase for branch.*.autosetupmerge
  branch: add flags and config to inherit tracking
  branch: accept multiple upstream branches for tracking
2022-01-31 10:37:44 -08:00
Junio C Hamano
5d01301f2b Git 2.35.1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4fA2sf7nIh/HeOzvsLXohpav5ssFAmH0jvsACgkQsLXohpav
 5ssJ8w//aAEXTNfDH15U+PwZ95OxAGV0mT4JCSK7fjB7kjqAMRLziVGPrlXzGTjc
 wxKVd5nvQAjB86P+sKSC/6AUx38QXlsurtUb+yhRMcogXm2ESdHTQ8IsofCo4yRq
 zDAQyf8RPKJFygMdVDjfXOIRP5NXTqVTAa67QihZOuqy5PvcEh7IRScd0f5pIsyD
 RO55G7eC9KZZJH5yrz0OpEj3xHxqZL+tO22ue777YUagUxS8Kt+183Fyc73H/eBz
 69TxEc76t54AjNNFgghTiF00zirl7qv/lRCYldqFfpoutxtAHdjuiy7E+h/NhY9r
 fkrVVGA3ms//ln32SU26Jm3zxEu31PSrCqP2KzdTdVO/I97jWnWytEZSWvVCZB6m
 BZDG1yKrLSlH48SqgxnPLGMeeqA1kldqI9e3lgpTMPUJO7xjXi0i10lFhL4N+pPX
 VMsjxlcaV6A1ey0CMCMtBZ2g8sJkKUuSA/HJrhhXBJkl88oWUAvPC6zIBG2xrDHy
 1qc4cAFNw4zwJylnyUYXbkeRn1/Z6lg8JeUfdWdZktjmL0ZbgLTM+CWKJhx/1WN2
 hu5DEFa7yH4IS4GW6W6N14mxTrMC2IehqKLhHd/0L8cqEVsnUb0xWLa0zChAdM7s
 G1XpNQSJxyBj74tJ8lMB+0P1cpqQF4ZkVk2Tx8e0xyVEaa5Vljg=
 =/DjY
 -----END PGP SIGNATURE-----

Sync with Git 2.35.1
2022-01-28 16:58:04 -08:00
Junio C Hamano
90fb70e458 Name the next one 2.36 to prepare for 2.35.1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-28 16:57:10 -08:00
Junio C Hamano
4c53a8c20f Git 2.35.1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-28 16:48:42 -08:00
Johannes Schindelin
d843e319f8 docs(diff): lose incorrect claim about diff-files --diff-filter=A
Originally, before we had `--intent-to-add`, there was no way that `git
diff-files` could see added files: if a file did not exist in the index,
`git diff-files` would not show it because it looks only at worktree
files when there is an index entry at the same path.

We used this example in the documentation of the diff options to explain
that not every `--diff-filter=<option>` has an effect in all scenarios.

Even when we added `--intent-to-add`, the comment was still correct,
because initially we showed such files as modified instead of added.

However, when that bug was fixed in feea6946a5 (diff-files: treat
"i-t-a" files as "not-in-index", 2020-06-20), the comment in the
documentation became incorrect.

Let's just remove it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-28 10:18:17 -08:00
Johannes Schindelin
deeaf5ee07 stash: remove documentation for stash.useBuiltin
In 8a2cd3f512 (stash: remove the stash.useBuiltin setting, 2020-03-03),
we removed the setting, and for a couple of major versions, we still
documented the setting, telling users that it is gone.

We can now safely remove even the documentation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 18:00:37 -08:00
Junio C Hamano
cdba0295b0 SubmittingPatches: explain why we care about log messages
Extend the "describe your changes well" section to cover whom we are
trying to help by doing so in the first place.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 17:50:04 -08:00
Junio C Hamano
607817a3c8 CodingGuidelines: hint why we value clearly written log messages
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 17:50:04 -08:00
Junio C Hamano
fa1101afb6 SubmittingPatches: write problem statement in the log in the present tense
We give a guidance for proposed log message to write problem
statement first, followed by the reasoning behind, and recipe for,
the solution.  Clarify that we describe the situation _before_ the
proposed patch is applied in the present tense (not in the past
tense e.g. "we used to do X, but thanks to this commit we now do Y")
for consistency.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 17:50:04 -08:00
Taylor Blau
95e8383bac midx.c: make changing the preferred pack safe
The previous patch demonstrates a bug where a MIDX's auxiliary object
order can become out of sync with a MIDX bitmap.

This is because of two confounding factors:

  - First, the object order is stored in a file which is named according
    to the multi-pack index's checksum, and the MIDX does not store the
    object order. This means that the object order can change without
    altering the checksum.

  - But the .rev file is moved into place with finalize_object_file(),
    which link(2)'s the file into place instead of renaming it. For us,
    that means that a modified .rev file will not be moved into place if
    MIDX's checksum was unchanged.

This fix is to force the MIDX's checksum to change when the preferred
pack changes but the set of packs contained in the MIDX does not. In
other words, when the object order changes, the MIDX's checksum needs to
change with it (regardless of whether the MIDX is tracking the same or
different packs).

This prevents a race whereby changing the object order (but not the
packs themselves) enables a reader to see the new .rev file with the old
MIDX, or similarly seeing the new bitmap with the old object order.

But why can't we just stop hardlinking the .rev into place instead
adding additional data to the MIDX? Suppose that's what we did. Then
when we go to generate the new bitmap, we'll load the old MIDX bitmap,
along with the MIDX that it references. That's fine, since the new MIDX
isn't moved into place until after the new bitmap is generated. But the
new object order *has* been moved into place. So we'll read the old
bitmaps in the new order when generating the new bitmap file, meaning
that without this secondary change, bitmap generation itself would
become a victim of the race described here.

This can all be prevented by forcing the MIDX's checksum to change when
the object order does. By embedding the entire object order into the
MIDX, we do just that. That is, the MIDX's checksum will change in
response to any perturbation of the underlying object order. In t5326,
this will cause the MIDX's checksum to update (even without changing the
set of packs in the MIDX), preventing the stale read problem.

Note that this makes it safe to continue to link(2) the MIDX .rev file
into place, since it is now impossible to have a .rev file that is
out-of-sync with the MIDX whose checksum it references. (But we will do
away with MIDX .rev files later in this series anyway, so this is
somewhat of a moot point).

In theory, it is possible to store a "fingerprint" of the full object
order here, so long as that fingerprint changes at least as often as the
full object order does. Some possibilities here include storing the
identity of the preferred pack, along with the mtimes of the
non-preferred packs in a consistent order. But storing a limited part of
the information makes it difficult to reason about whether or not there
are gaps between the two that would cause us to get bitten by this bug
again.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Reviewed-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 12:07:52 -08:00
Greg Hurrell
cbac0076ef Documentation/config/pgp.txt: add missing apostrophe
Add an apostrophe to "signatures" to indicate the possessive
relationship in "the signature's creation".

Signed-off-by: Greg Hurrell <greg@hurrell.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-26 18:31:59 -08:00
Greg Hurrell
7838d9c2a9 Documentation/config/pgp.txt: replace stray <TAB> character with <SPC>
Specifically, replace the tab between "the" and "first" with a space.

Signed-off-by: Greg Hurrell <greg@hurrell.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-26 18:31:59 -08:00
Junio C Hamano
89bece5c8c Git 2.35
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-24 09:25:25 -08:00
Junio C Hamano
297ca895a2 Merge branch 'js/branch-track-inherit'
"git branch -h" incorrectly said "--track[=direct|inherit]",
implying that "--trackinherit" is a valid option, which has been
corrected.
source: <3de40324bea6a1dd9bca2654721471e3809e87d8.1642538935.git.steadmon@google.com>
source: <c3c26192-aee9-185a-e559-b8735139e49c@web.de>

* js/branch-track-inherit:
  branch,checkout: fix --track documentation
2022-01-20 15:25:38 -08:00
René Scharfe
6327f0efed branch,checkout: fix --track documentation
Document that the accepted variants of the --track option are --track,
--track=direct, and --track=inherit.  The equal sign in the latter two
cannot be replaced with whitespace; in general optional arguments need
to be attached firmly to their option.

Put "direct" consistently before "inherit", if only for the reasons
that the former is the default, explained first in the documentation,
and comes before the latter alphabetically.

Mention both modes in the short help so that readers don't have to look
them up in the full documentation.  They are literal strings and thus
untranslatable.  PARSE_OPT_LITERAL_ARGHELP is inferred due to the pipe
and parenthesis characters, so we don't have to provide that flag
explicitly.

Mention that -t has the same effect as --track and --track=direct.
There is no way to specify inherit mode using the short option, because
short options generally don't accept optional arguments.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-20 11:07:51 -08:00