Commit Graph

14361 Commits

Author SHA1 Message Date
Taylor Blau
f9825d1cf7 builtin/repack.c: support generating a cruft pack
Expose a way to split the contents of a repository into a main and cruft
pack when doing an all-into-one repack with `git repack --cruft -d`, and
a complementary configuration variable.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26 15:48:26 -07:00
Taylor Blau
b757353676 builtin/pack-objects.c: --cruft without expiration
Teach `pack-objects` how to generate a cruft pack when no objects are
dropped (i.e., `--cruft-expiration=never`). Later patches will teach
`pack-objects` how to generate a cruft pack that prunes objects.

When generating a cruft pack which does not prune objects, we want to
collect all unreachable objects into a single pack (noting and updating
their mtimes as we accumulate them). Ordinary use will pass the result
of a `git repack -A` as a kept pack, so when this patch says "kept
pack", readers should think "reachable objects".

Generating a non-expiring cruft packs works as follows:

  - Callers provide a list of every pack they know about, and indicate
    which packs are about to be removed.

  - All packs which are going to be removed (we'll call these the
    redundant ones) are marked as kept in-core.

    Any packs the caller did not mention (but are known to the
    `pack-objects` process) are also marked as kept in-core. Packs not
    mentioned by the caller are assumed to be unknown to them, i.e.,
    they entered the repository after the caller decided which packs
    should be kept and which should be discarded.

    Since we do not want to include objects in these "unknown" packs
    (because we don't know which of their objects are or aren't
    reachable), these are also marked as kept in-core.

  - Then, we enumerate all objects in the repository, and add them to
    our packing list if they do not appear in an in-core kept pack.

This results in a new cruft pack which contains all known objects that
aren't included in the kept packs. When the kept pack is the result of
`git repack -A`, the resulting pack contains all unreachable objects.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26 15:48:26 -07:00
Taylor Blau
94cd775a6c pack-mtimes: support reading .mtimes files
To store the individual mtimes of objects in a cruft pack, introduce a
new `.mtimes` format that can optionally accompany a single pack in the
repository.

The format is defined in Documentation/technical/pack-format.txt, and
stores a 4-byte network order timestamp for each object in name (index)
order.

This patch prepares for cruft packs by defining the `.mtimes` format,
and introducing a basic API that callers can use to read out individual
mtimes.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26 15:48:26 -07:00
Junio C Hamano
8ddf593a25 Fourth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26 14:51:40 -07:00
Junio C Hamano
2785b71ef9 Merge branch 'ac/remote-v-with-object-list-filters'
"git remote -v" now shows the list-objects-filter used during
fetching from the remote, if available.

* ac/remote-v-with-object-list-filters:
  builtin/remote.c: teach `-v` to list filters for promisor remotes
2022-05-26 14:51:32 -07:00
Junio C Hamano
2088a0c0cd Merge branch 'cb/path-owner-check-with-sudo'
With a recent update to refuse access to repositories of other
people by default, "sudo make install" and "sudo git describe"
stopped working.  This series intends to loosen it while keeping
the safety.

* cb/path-owner-check-with-sudo:
  t0034: add negative tests and allow git init to mostly work under sudo
  git-compat-util: avoid failing dir ownership checks if running privileged
  t: regression git needs safe.directory when using sudo
2022-05-26 14:51:32 -07:00
Junio C Hamano
7ec4a9e74f Merge branch 'cg/tools-for-git-doc'
A new doc that lists tips for tools to work with Git's codebase.

* cg/tools-for-git-doc:
  Documentation/ToolsForGit.txt: Tools for developing Git
2022-05-26 14:51:31 -07:00
Junio C Hamano
f49c478f62 Merge branch 'tk/simple-autosetupmerge'
"git -c branch.autosetupmerge=simple branch $A $B" will set the $B
as $A's upstream only when $A and $B shares the same name, and "git
-c push.default=simple" on branch $A would push to update the
branch $A at the remote $B came from.  Also more places use the
sole remote, if exists, before defaulting to 'origin'.

* tk/simple-autosetupmerge:
  push: new config option "push.autoSetupRemote" supports "simple" push
  push: default to single remote even when not named origin
  branch: new autosetupmerge option 'simple' for matching branches
2022-05-26 14:51:30 -07:00
Junio C Hamano
6afdb07b7b Third batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-25 16:42:49 -07:00
Junio C Hamano
3ce9483c1a Merge branch 'pb/ggg-in-mfc-doc'
Documentation update.

* pb/ggg-in-mfc-doc:
  MyFirstContribution: drop PR description for GGG single-patch contributions
  MyFirstContribution: reference "The cover letter" in GitGitGadget section
  MyFirstContribution: reference "The cover letter" in "Preparing Email"
  MyFirstContribution: add standalone section on cover letter
  MyFirstContribution: add "Anatomy of a Patch Series" section
2022-05-25 16:42:48 -07:00
Junio C Hamano
7a3eb28697 Second batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-23 14:39:55 -07:00
Taylor Blau
3d89a8c118 Documentation/technical: add cruft-packs.txt
Create a technical document to explain cruft packs. It contains a brief
overview of the problem, some background, details on the implementation,
and a couple of alternative approaches not considered here.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-20 22:31:21 -07:00
Junio C Hamano
f9b95943b6 First batch for 2.37
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-20 15:27:00 -07:00
Junio C Hamano
3af1df0415 Merge branch 'tk/p4-metadata-coding-strategies'
"git p4" updates.

* tk/p4-metadata-coding-strategies:
  git-p4: improve encoding handling to support inconsistent encodings
2022-05-20 15:27:00 -07:00
Junio C Hamano
9a7176d9fb Merge branch 'jc/update-ozlabs-url'
* jc/update-ozlabs-url:
  SubmittingPatches: use more stable git.ozlabs.org URL
2022-05-20 15:26:58 -07:00
Junio C Hamano
3ab732864a Merge branch 'js/trace2-doc-fixes'
Trace2 documentation updates.

* js/trace2-doc-fixes:
  trace2 docs: add missing full stop
  trace2 docs: clarify what `varargs` is all about
  trace2 docs: fix a JSON formatted example
  trace2 docs: surround more terms in backticks
  trace2 docs: "printf" is not an English word
  trace2 docs: a couple of grammar fixes
2022-05-20 15:26:56 -07:00
Junio C Hamano
6f24da652c Merge branch 'mv/log-since-as-filter'
"git log --since=X" will stop traversal upon seeing a commit that
is older than X, but there may be commits behind it that is younger
than X when the commit was created with a faulty clock.  A new
option is added to keep digging without stopping, and instead
filter out commits with timestamp older than X.

* mv/log-since-as-filter:
  log: "--since-as-filter" option is a non-terminating "--since" variant
2022-05-20 15:26:56 -07:00
Junio C Hamano
4976f244f3 Merge branch 'gf/shorthand-version-and-help'
"git -v" and "git -h" are now understood as "git --version" and
"git --help".

* gf/shorthand-version-and-help:
  cli: add -v and -h shorthands
2022-05-20 15:26:53 -07:00
Junio C Hamano
1256a25ecd Merge branch 'sg/safe-directory-tests-and-docs'
New tests for the safe.directory mechanism.

* sg/safe-directory-tests-and-docs:
  safe.directory: document and check that it's ignored in the environment
  t0033-safe-directory: check when 'safe.directory' is ignored
  t0033-safe-directory: check the error message without matching the trash dir
2022-05-20 15:26:52 -07:00
Christian Couder
511cfd3bff http: add custom hostname to IP address resolutions
Libcurl has a CURLOPT_RESOLVE easy option that allows
the result of hostname resolution in the following
format to be passed:

	[+]HOST:PORT:ADDRESS[,ADDRESS]

This way, redirects and everything operating against the
HOST+PORT will use the provided ADDRESS(s).

The following format is also allowed to stop using
hostname resolutions that have already been passed:

	-HOST:PORT

See https://curl.se/libcurl/c/CURLOPT_RESOLVE.html for
more details.

Let's add a corresponding "http.curloptResolve" config
option that takes advantage of CURLOPT_RESOLVE.

Each value configured for the "http.curloptResolve" key
is passed "as is" to libcurl through CURLOPT_RESOLVE, so
it should be in one of the above 2 formats. This keeps
the implementation simple and makes us consistent with
libcurl's CURLOPT_RESOLVE, and with curl's corresponding
`--resolve` command line option.

The implementation uses CURLOPT_RESOLVE only in
get_active_slot() which is called by all the HTTP
request sending functions.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-16 09:46:52 -07:00
Carlo Marcelo Arenas Belón
ae9abbb63e git-compat-util: avoid failing dir ownership checks if running privileged
bdc77d1d68 (Add a function to determine whether a path is owned by the
current user, 2022-03-02) checks for the effective uid of the running
process using geteuid() but didn't account for cases where that user was
root (because git was invoked through sudo or a compatible tool) and the
original uid that repository trusted for its config was no longer known,
therefore failing the following otherwise safe call:

  guy@renard ~/Software/uncrustify $ sudo git describe --always --dirty
  [sudo] password for guy:
  fatal: unsafe repository ('/home/guy/Software/uncrustify' is owned by someone else)

Attempt to detect those cases by using the environment variables that
those tools create to keep track of the original user id, and do the
ownership check using that instead.

This assumes the environment the user is running on after going
privileged can't be tampered with, and also adds code to restrict that
the new behavior only applies if running as root, therefore keeping the
most common case, which runs unprivileged, from changing, but because of
that, it will miss cases where sudo (or an equivalent) was used to change
to another unprivileged user or where the equivalent tool used to raise
privileges didn't track the original id in a sudo compatible way.

Because of compatibility with sudo, the code assumes that uid_t is an
unsigned integer type (which is not required by the standard) but is used
that way in their codebase to generate SUDO_UID.  In systems where uid_t
is signed, sudo might be also patched to NOT be unsigned and that might
be able to trigger an edge case and a bug (as described in the code), but
it is considered unlikely to happen and even if it does, the code would
just mostly fail safely, so there was no attempt either to detect it or
prevent it by the code, which is something that might change in the future,
based on expected user feedback.

Reported-by: Guy Maurel <guy.j@maurel.de>
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Randall Becker <rsbecker@nexbridge.com>
Helped-by: Phillip Wood <phillip.wood123@gmail.com>
Suggested-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-12 18:12:23 -07:00
Philippe Blain
4ec5008062 MyFirstContribution: drop PR description for GGG single-patch contributions
By default, GitHub prefills the PR description using the commit message
for single-commit PRs. This results in a duplicate commit message below
the three-dash line if the contributor does not empty out the PR
description before submitting, which adds noise for reviewers.

Add a note to that effect in MyFirstContribution.txt.

This partly addresses:
https://github.com/gitgitgadget/gitgitgadget/issues/340

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-12 18:10:00 -07:00
Philippe Blain
c2cd4b592f MyFirstContribution: reference "The cover letter" in GitGitGadget section
The "Sending Patches via GitGitGadget" section mentions that the PR
title and description will be used as the cover letter, but does not
explain what is a cover letter or what should be included in it.

Refer readers to the new "The cover letter" section added in a previous
commit.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-12 18:10:00 -07:00
Philippe Blain
e97d474c7a MyFirstContribution: reference "The cover letter" in "Preparing Email"
The previous commit added a standalone section on the purpose of the
cover letter, drawing inspiration from the existing content of the
"Preparing Email" section.

Adjust "Preparing Email" to reference "The cover letter", to avoid
content duplication.

Also, use the imperative mode for the cover letter subject, as is done
in "The cover letter".

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-12 18:09:59 -07:00
Philippe Blain
afc8c92535 MyFirstContribution: add standalone section on cover letter
An explanation of the purpose of the cover letter is included in the
"Sending Patches with git send-email" / "Preparing Email" section but is
missing from the "Sending Patches via GitGitGadget" section.

Add a standalone section "The cover letter" under the "Getting Started:
Anatomy of a Patch Series" header to explain what the cover letter is
used for and to draft the cover letter of the 'psuh' topic used in the
tutorial.

For now we mostly copy content from the "Sending Patches with git
send-email" section but do not adjust that section, nor the GGG section,
to reference the new section. This is done in following commits.

Also, adjust the "Preparing Email" Asciidoc anchor to avoid conflicts.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-12 18:09:59 -07:00
Philippe Blain
489ef3ba57 MyFirstContribution: add "Anatomy of a Patch Series" section
Before describing how to send patches to the mailing list either with
GitGitGadget or 'git send-email', the MyFirstContribution tutorial
includes a small "Getting Ready to Share" section where the two
different methods are briefly introduced.

Use this section to also describe what a patch series looks like once
submitted, so that readers get an understanding of the end result before
diving into how to accomplish that end result.

Start by copying the "thread overview" section of a recent contribution
from the public-inbox web UI and explaining how each commit is a
separate mail, and point out the cover letter.

Subsequent commits will move the existing description of the purpose of
the cover letter from the 'git send-email' section to this "anatomy"
section.

Also, change the wording in the introductory paragraph to use
"contributions" instead of "patches", since this makes more sense when
talking about GitHub pull requests.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-12 18:09:59 -07:00
Junio C Hamano
277cf0bc36 second 0th batch of topics from the previous cycle
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-11 13:56:23 -07:00
Junio C Hamano
202161fa8d Merge branch 'ah/rebase-keep-base-fix'
"git rebase --keep-base <upstream> <branch-to-rebase>" computed the
commit to rebase onto incorrectly, which has been corrected.

* ah/rebase-keep-base-fix:
  rebase: use correct base for --keep-base when a branch is given
2022-05-11 13:56:21 -07:00
Junio C Hamano
b014cee8de SubmittingPatches: use more stable git.ozlabs.org URL
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-11 08:19:08 -07:00
Junio C Hamano
b9de974d38 0th batch for topics from the previous cycle
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-10 17:41:11 -07:00
Junio C Hamano
123dfdff0d Merge branch 'fr/vimdiff-layout'
Reimplement "vimdiff[123]" mergetool drivers with a more generic
layout mechanism.

* fr/vimdiff-layout:
  mergetools: add description to all diff/merge tools
  vimdiff: add tool documentation
  vimdiff: integrate layout tests in the unit tests framework ('t' folder)
  vimdiff: new implementation with layout support
2022-05-10 17:41:11 -07:00
Abhradeep Chakraborty
ef6d15ca53 builtin/remote.c: teach -v to list filters for promisor remotes
`git remote -v` (`--verbose`) lists down the names of remotes along with
their URLs. It would be beneficial for users to also specify the filter
types for promisor remotes. Something like this -

	origin	remote-url (fetch) [blob:none]
	origin	remote-url (push)

Teach `git remote -v` to also specify the filters for promisor remotes.

Closes: https://github.com/gitgitgadget/git/issues/1211
Signed-off-by: Abhradeep Chakraborty <chakrabortyabhradeep79@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-09 10:53:58 -07:00
Junio C Hamano
e8005e4871 Git 2.36.1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4fA2sf7nIh/HeOzvsLXohpav5ssFAmJ0Q4gACgkQsLXohpav
 5ssw9xAA5qxcmcCjugS49OTAEMa6pQnHcF6pvucy71N1c7DbbPjEFFwA9bIGQ/tB
 Lf5vtYouKbnYzoPqzdeNx8y+/DmAph61QOQx0kPb01YnG23sR3dDYMtKJWydsMlb
 eAs0D9GaWW7fsUdvY6Rlpvd6HKYZVbxhCCcCH1IBYDw9S4F8C2zRJ/hdlON1TD1i
 DIaakVKLjbff2zrbK0XWIaLnlN14tKqTmJDpxDF5VlE8CExzsiZrCrSc0ynY28qm
 UwTg0+WGf/wUbFpTkUn+YsP5U6CGNEwXPkdykjqT96wwkbxQNDF4kFfLEbrC3vqv
 yAP36gJnaWQ1UWwwS9pVaOHHYdfUJdQt0fr+3ibmMtY6BfVGsei96dBU5iYPkcV0
 Lf/9qAXaljVmQeMXTJMOYBC0ViXXdoqJcGtflPZCFYXQnd3THJJjan44VfhUVFHr
 wgj1V/BedE9E12uH5mr88cYHQRP+8Kg1EairsMcuyOzAleZbq/vqUY0kc8y0YoyC
 +tRbEfD7p9F8NcIto2kBRsZt3OHLtJI8cYiTLt1Wnav5zCJqWbmps6g3JOsCyLnx
 EyR2EaPoHZajYM2ifAR0QFE59o2ZNa175Ba4vWM7Upmt0LwHA2+bSJTRCvXCGn/D
 uwNOH3HGwFT8hacJB2rreKGTTFO0EdxPgWA9Nnfk3S/9LzSQWCM=
 =QmOj
 -----END PGP SIGNATURE-----

Sync with v2.36.1
2022-05-05 14:39:03 -07:00
Junio C Hamano
e54793a95a Git 2.36.1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-05 14:36:37 -07:00
Johannes Schindelin
a6c80c313c trace2 docs: add missing full stop
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:31:05 -07:00
Johannes Schindelin
236d1177ba trace2 docs: clarify what varargs is all about
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:31:05 -07:00
Johannes Schindelin
c4667b17fb trace2 docs: fix a JSON formatted example
The example was not in valid JSON format due to a duplicate key "sid".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:31:04 -07:00
Johannes Schindelin
1c538b951f trace2 docs: surround more terms in backticks
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:31:04 -07:00
Johannes Schindelin
c191b9188b trace2 docs: "printf" is not an English word
We append an ellipsis and enclose it in backticks to indicate that it is
a function elsewhere, let's also use that here.

While at it, ensure the same for `waitpid()`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:31:04 -07:00
Johannes Schindelin
3188d3c0b9 trace2 docs: a couple of grammar fixes
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:31:04 -07:00
Tao Klerks
f7b5ff607f git-p4: improve encoding handling to support inconsistent encodings
git-p4 is designed to run correctly under python2.7 and python3, but
its functional behavior wrt importing user-entered text differs across
these environments:

Under python2, git-p4 "naively" writes the Perforce bytestream into git
metadata (and does not set an "encoding" header on the commits); this
means that any non-utf-8 byte sequences end up creating invalidly-encoded
commit metadata in git.

Under python3, git-p4 attempts to decode the Perforce bytestream as utf-8
data, and fails badly (with an unhelpful error) when non-utf-8 data is
encountered.

Perforce clients (especially p4v) encourage user entry of changelist
descriptions (and user full names) in OS-local encoding, and store the
resulting bytestream to the server unmodified - such that different
clients can end up creating mutually-unintelligible messages. The most
common inconsistency, in many Perforce environments, is likely to be utf-8
(typical in linux) vs cp-1252 (typical in windows).

Make the changelist-description- and user-fullname-handling code
python-runtime-agnostic, introducing three "strategies" selectable via
config:
- 'passthrough', behaving as previously under python2,
- 'strict', behaving as previously under python3, and
- 'fallback', favoring utf-8 but supporting a secondary encoding when
utf-8 decoding fails, and finally escaping high-range bytes if the
decoding with the secondary encoding also fails.

Keep the python2 default behavior as-is ('legacy' strategy), but switch
the python3 default strategy to 'fallback' with default fallback encoding
'cp1252'.

Also include tests exercising these encoding strategies, documentation for
the new config, and improve the user-facing error messages when decoding
does fail.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 10:30:01 -07:00
Junio C Hamano
f5aaf72f1b A bit more regression fixes for 2.36
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-04 09:51:39 -07:00
Tao Klerks
05d57750c6 push: new config option "push.autoSetupRemote" supports "simple" push
In some "simple" centralized workflows, users expect remote tracking
branch names to match local branch names. "git push" pushes to the
remote version/instance of the branch, and "git pull" pulls any changes
to the remote branch (changes made by the same user in another place, or
by other users).

This expectation is supported by the push.default default option "simple"
which refuses a default push for a mismatching tracking branch name, and
by the new branch.autosetupmerge option, "simple", which only sets up
remote tracking for same-name remote branches.

When a new branch has been created by the user and has not yet been
pushed (and push.default is not set to "current"), the user is prompted
with a "The current branch %s has no upstream branch" error, and
instructions on how to push and add tracking.

This error is helpful in that following the advice once per branch
"resolves" the issue for that branch forever, but inconvenient in that
for the "simple" centralized workflow, this is always the right thing to
do, so it would be better to just do it.

Support this workflow with a new config setting, push.autoSetupRemote,
which will cause a default push, when there is no remote tracking branch
configured, to push to the same-name on the remote and --set-upstream.

Also add a hint offering this new option when the "The current branch %s
has no upstream branch" error is encountered, and add corresponding tests.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-29 11:20:55 -07:00
Tao Klerks
8a649be7e8 push: default to single remote even when not named origin
With "push.default=current" configured, a simple "git push" will push to
the same-name branch on the current branch's branch.<name>.pushRemote, or
remote.pushDefault, or origin. If none of these are defined, the push will
fail with error "fatal: No configured push destination".

The same "default to origin if no config" behavior applies with
"push.default=matching".

Other commands use "origin" as a default when there are multiple options,
but default to the single remote when there is only one - for example,
"git checkout <something>". This "assume the single remote if there is
only one" behavior is more friendly/useful than a defaulting behavior
that only uses the name "origin" no matter what.

Update "git push" to also default to the single remote (and finally fall
back to "origin" as default if there are several), for
"push.default=current" and for other current and future remote-defaulting
push behaviors.

This change also modifies the behavior of ls-remote in a consistent way,
so defaulting not only supplies 'origin', but any single configured remote
also.

Document the change in behavior, correct incorrect assumptions in related
tests, and add test cases reflecting this new single-remote-defaulting
behavior.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-29 11:20:55 -07:00
Tao Klerks
bdaf1dfae7 branch: new autosetupmerge option 'simple' for matching branches
With the default push.default option, "simple", beginners are
protected from accidentally pushing to the "wrong" branch in
centralized workflows: if the remote tracking branch they would push
to does not have the same name as the local branch, and they try to do
a "default push", they get an error and explanation with options.

There is a particular centralized workflow where this often happens:
a user branches to a new local topic branch from an existing
remote branch, eg with "checkout -b feature1 origin/master". With
the default branch.autosetupmerge configuration (value "true"), git
will automatically add origin/master as the upstream tracking branch.

When the user pushes with a default "git push", with the intention of
pushing their (new) topic branch to the remote, they get an error, and
(amongst other things) a suggestion to run "git push origin HEAD".

If they follow this suggestion the push succeeds, but on subsequent
default pushes they continue to get an error - so eventually they
figure out to add "-u" to change the tracking branch, or they spelunk
the push.default config doc as proposed and set it to "current", or
some GUI tooling does one or the other of these things for them.

When one of their coworkers later works on the same topic branch,
they don't get any of that "weirdness". They just "git checkout
feature1" and everything works exactly as they expect, with the shared
remote branch set up as remote tracking branch, and push and pull
working out of the box.

The "stable state" for this way of working is that local branches have
the same-name remote tracking branch (origin/feature1 in this
example), and multiple people can work on that remote feature branch
at the same time, trusting "git pull" to merge or rebase as required
for them to be able to push their interim changes to that same feature
branch on that same remote.

(merging from the upstream "master" branch, and merging back to it,
are separate more involved processes in this flow).

There is a problem in this flow/way of working, however, which is that
the first user, when they first branched from origin/master, ended up
with the "wrong" remote tracking branch (different from the stable
state). For a while, before they pushed (and maybe longer, if they
don't use -u/--set-upstream), their "git pull" wasn't getting other
users' changes to the feature branch - it was getting any changes from
the remote "master" branch instead (a completely different class of
changes!)

An experienced git user might say "well yeah, that's what it means to
have the remote tracking branch set to origin/master!" - but the
original user above didn't *ask* to have the remote master branch
added as remote tracking branch - that just happened automatically
when they branched their feature branch. They didn't necessarily even
notice or understand the meaning of the "set up to track 'origin/master'"
message when they created the branch - especially if they are using a
GUI.

Looking at how to fix this, you might think "OK, so disable auto setup
of remote tracking - set branch.autosetupmerge to false" - but that
will inconvenience the *second* user in this story - the one who just
wanted to start working on the topic branch. The first and second
users swap roles at different points in time of course - they should
both have a sane configuration that does the right thing in both
situations.

Make this "branches have the same name locally as on the remote"
workflow less painful / more obvious by introducing a new
branch.autosetupmerge option called "simple", to match the same-name
"push.default" option that makes similar assumptions.

This new option automatically sets up tracking in a *subset* of the
current default situations: when the original ref is a remote tracking
branch *and* has the same branch name on the remote (as the new local
branch name).

Update the error displayed when the 'push.default=simple' configuration
rejects a mismatching-upstream-name default push, to offer this new
branch.autosetupmerge option that will prevent this class of error.

With this new configuration, in the example situation above, the first
user does *not* get origin/master set up as the tracking branch for
the new local branch. If they "git pull" in their new local-only
branch, they get an error explaining there is no upstream branch -
which makes sense and is helpful. If they "git push", they get an
error explaining how to push *and* suggesting they specify
--set-upstream - which is exactly the right thing to do for them.

This new option is likely not appropriate for users intentionally
implementing a "triangular workflow" with a shared upstream tracking
branch, that they "git pull" in and a "private" feature branch that
they push/force-push to just for remote safe-keeping until they are
ready to push up to the shared branch explicitly/separately. Such
users are likely to prefer keeping the current default
merge.autosetupmerge=true behavior, and change their push.default to
"current".

Also extend the existing branch tests with three new cases testing
this option - the obvious matching-name and non-matching-name cases,
and also a non-matching-ref-type case. The matching-name case needs to
temporarily create an independent repo to fetch from, as the general
strategy of using the local repo as the remote in these tests
precludes locally branching with the same name as in the "remote".

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-29 11:20:55 -07:00
Junio C Hamano
0f828332d5 Some regression fixes for 2.36
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-28 10:46:04 -07:00
SZEDER Gábor
756d15923b safe.directory: document and check that it's ignored in the environment
The description of 'safe.directory' mentions that it's respected in
the system and global configs, and ignored in the repository config
and on the command line, but it doesn't mention whether it's respected
or ignored when specified via environment variables (nor does the
commit message adding 'safe.directory' [1]).

Clarify that 'safe.directory' is ignored when specified in the
environment, and add tests to make sure that it remains so.

[1] 8959555cee (setup_git_directory(): add an owner check for the
                top-level directory, 2022-03-02)

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-27 13:30:56 -07:00
Miklos Vajna
96697781e0 log: "--since-as-filter" option is a non-terminating "--since" variant
The "--since=<time>" option of "git log" limits the commits displayed by
the command by stopping the traversal once it sees a commit whose
timestamp is older than the given time and not digging further into its
parents.

This is OK in a history where a commit always has a newer timestamp than
any of its parents'.  Once you see a commit older than the given <time>,
all ancestor commits of it are even older than the time anyway.  It
poses, however, a problem when there is a commit with a wrong timestamp
that makes it appear older than its parents.  Stopping traversal at the
"incorrectly old" commit will hide its ancestors that are newer than
that wrong commit and are newer than the cut-off time given with the
--since option.  --max-age and --after being the synonyms to --since,
they share the same issue.

Add a new "--since-as-filter" option that is a variant of
"--since=<time>".  Instead of stopping the traversal to hide an old
enough commit and its all ancestors, exclude commits with an old
timestamp from the output but still keep digging the history.

Without other traversal stopping options, this will force the command in
"git log" family to dig down the history to the root.  It may be an
acceptable cost for a small project with short history and many commits
with screwy timestamps.

It is quite unlikely for us to add traversal stopper other than since,
so have this as a --since-as-filter option, rather than a separate
--as-filter, that would be probably more confusing.

Signed-off-by: Miklos Vajna <vmiklos@vmiklos.hu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-23 09:36:07 -07:00
Elijah Newren
5d4b293340 Documentation: some sparsity wording clarifications
Improve the wording for a couple paragraphs in two different manuals
relating to sparse behavior.

Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:39 -07:00
Elijah Newren
a8defed07c git-sparse-checkout.txt: mark non-cone mode as deprecated
While we have no current plans to actually remove --no-cone mode, we
think users would be better off not using it.  Update the documentation
accordingly, including explaining why we think non-cone mode is
problematic for users.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:39 -07:00
Elijah Newren
72fa58ef50 git-sparse-checkout.txt: flesh out pattern set sections a bit
The "Internals -- Cone Pattern Set" section starts off discussing
patterns, despite the fact that cone mode is about avoiding the
patterns.  This made sense back when non-cone mode was the default and
we started by discussing the full pattern set, but now that we are
changing the default, it makes more sense to discuss cone-mode first and
avoid the full discussion of patterns.  Split this section into two, the
first with details about how cone mode operates, and the second
following the full pattern set section and discussing how the cone mode
patterns used under the hood relate to the full pattern set.

While at it, flesh out the "Internals -- Full Pattern Set" section a bit
to include more examples as well.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
5d295dc396 git-sparse-checkout.txt: add a new EXAMPLES section
Since many users like to learn from examples, provide a section in the
manual with example commands that would be used and a brief explanation
of what each does.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
0d86f59a3c git-sparse-checkout.txt: shuffle some sections and mark as internal
With cone mode as the default, it makes sense to discuss it before
non-cone mode.  Also, the new default means we can just use directories
in most cases and users do not need to understand patterns or their
meanings.  Let's take advantage of this to mark several sections as
"INTERNALS", notifying the user that they do not need to know all those
details in order to make use of the sparse-checkout command.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
71ceb816b3 git-sparse-checkout.txt: update docs for deprecation of 'init'
The 'init' subcommand of sparse-checkout was deprecated in ba2f3f58ac
("git-sparse-checkout.txt: update to document init/set/reapply changes",
2021-12-14), but a couple places in the manual still assumed it was the
primary way to use sparse-checkout.  Correct them.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
f69dfef355 git-sparse-checkout.txt: wording updates for the cone mode default
Now that cone mode is the default, we'd like to focus on the arguments
to set/add being directories rather than patterns, and it probably makes
sense to provide an earlier heads up that files from leading directories
get included as well.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
2d95707a02 sparse-checkout: make --cone the default
Make cone mode the default, and update the documentation accordingly.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
COGONI Guillaume
7a06a854ee Documentation/ToolsForGit.txt: Tools for developing Git
This document gathers tips, scripts and configuration file to help
people working on Git’s codebase use their favorite tools while
following Git’s coding style.

Move the part about Emacs configuration from CodingGuidelines to
ToolsForGit.txt because it's the purpose of the new file centralize the
information about tools.

But, add a mention to Documentation/ToolsForGit.txt in CodingGuidelines
because there is also information about the coding style in it.

Helped-by: Matthieu Moy <Matthieu.Moy@univ-lyon1.fr>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: COGONI Guillaume <cogoni.guillaume@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 09:50:25 -07:00
Alex Henrie
9e5ebe9668 rebase: use correct base for --keep-base when a branch is given
--keep-base rebases onto the merge base of the given upstream and the
current HEAD regardless of whether a branch is given. This is contrary
to the documentation and to the option's intended purpose. Instead,
rebase onto the merge base of the given upstream and the given branch.

Signed-off-by: Alex Henrie <alexhenrie24@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 09:35:45 -07:00
Junio C Hamano
4027e30c53 Merge branch 'jc/revert-ref-transaction-hook-changes'
Revert the "deletion of a ref should not trigger transaction events
for loose and packed ref backends separately" that regresses the
behaviour when a ref is not modified since it was packed.

* jc/revert-ref-transaction-hook-changes:
  RelNotes: revert the description on the reverted topics
  Revert "fetch: increase test coverage of fetches"
  Revert "Merge branch 'ps/avoid-unnecessary-hook-invocation-with-packed-refs'"
2022-04-14 14:17:12 -07:00
Junio C Hamano
43159864b6 RelNotes: revert the description on the reverted topics
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 16:55:36 -07:00
Junio C Hamano
255ede9980 RelNotes: mention safe.directory
Helped-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 16:51:41 -07:00
Junio C Hamano
26ff9be6e7 RelNotes: clarify "bisect run unexecutable" tweak
We do not have to guess how common the mistake the change targets is
when describing it.  Such an argument may be good while proposing a
change, but does not quite belong in the record of what has already
happened, i.e. a release note.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 16:42:33 -07:00
Junio C Hamano
1ac7422e39 Git 2.35.3
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4fA2sf7nIh/HeOzvsLXohpav5ssFAmJXTO4ACgkQsLXohpav
 5ss62xAAzwmmKlJkcgdKRcVimfMF+hPNvBFsnKBZZRtAV+4vCOFa2EN2bgWJexZh
 SfuDzdJrFf+A4Emb0Z2nd9ZmSJJznybwYJCkHatfEnH+qy/H+5ju3NwgD84DOCad
 DauretQn2zhwosJDsF82MbogQrOTYQjfftalFZZwYyD5AoSbsiR/diIrjjP6q+Qo
 RlKXagPM8hxZLrdOjMir75Wr/OrFDXMlO2kE2+5IgR/EO8KmjltFZgeciLnFXllN
 qQ77Klu1B9xsUjypK0/Vxbg389pqSHRCR28MaKwHbPQsXz8+ZTeCfgv7u500BWa+
 Yl3Cye1GtZtD3zCu4Ik/D++Bu53P8NmHXzAst6hhMnyZZUQ8meeVoLdZH5eZscc6
 vlv+wyLiyqILWknWIEibATavqjBWeFAqRXC//RPdZbUjoeE7fAVA8u+LZvOBCKna
 altnI497uJAL15eWU8878X8y1rmZJfXpx0euwYZbmo6Hj/GHY/1w3RYanJ+shOkk
 f8Qu4AUWNYAyHUANbxczSVoV3VR9xLdKgqbuGZsNZDRPUMo6POBNSxnjExsAnr6b
 SRmpmQQsbZr2vO9i12dPQJbqRCo5++rrmM/qi+ozmM1xGCDyeSiHgsnDUQV7AGkZ
 0/hwg+mhykvLEnMIbDLZirI1uNecomO83Q/YhcWdBFlsDXb0IJw=
 =AeAR
 -----END PGP SIGNATURE-----

Sync with Git 2.35.3
2022-04-13 15:26:32 -07:00
Junio C Hamano
d516b2db0a Git 2.35.3
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 15:21:34 -07:00
Junio C Hamano
2f0dde7852 Git 2.34.3
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 15:21:31 -07:00
Junio C Hamano
1f65dd6ae6 Git 2.33.3
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 15:21:28 -07:00
Junio C Hamano
1530434434 Git 2.32.2
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 15:21:26 -07:00
Junio C Hamano
09f66d65f8 Git 2.31.3
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 15:21:08 -07:00
Junio C Hamano
17083c79ae Git 2.30.4
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 13:31:29 -07:00
Derrick Stolee
0f85c4a30b setup: opt-out of check with safe.directory=*
With the addition of the safe.directory in 8959555ce
(setup_git_directory(): add an owner check for the top-level directory,
2022-03-02) released in v2.35.2, we are receiving feedback from a
variety of users about the feature.

Some users have a very large list of shared repositories and find it
cumbersome to add this config for every one of them.

In a more difficult case, certain workflows involve running Git commands
within containers. The container boundary prevents any global or system
config from communicating `safe.directory` values from the host into the
container. Further, the container almost always runs as a different user
than the owner of the directory in the host.

To simplify the reactions necessary for these users, extend the
definition of the safe.directory config value to include a possible '*'
value. This value implies that all directories are safe, providing a
single setting to opt-out of this protection.

Note that an empty assignment of safe.directory clears all previous
values, and this is already the case with the "if (!value || !*value)"
condition.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-13 12:42:51 -07:00
Junio C Hamano
11cfe55261 Git 2.36-rc2
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-11 21:27:02 -07:00
Junio C Hamano
f1b50ec6f8 Git 2.35.2
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4fA2sf7nIh/HeOzvsLXohpav5ssFAmI8+U4ACgkQsLXohpav
 5ss7GA//YwNiPt8TY2bghKyt2qNiKKNl6sEq1MC0r4w68q5hGgmDjByvWA1K/4W5
 JLfhauBZB6ucx9zrcr6v6nr+a9/y4MC/bEq0Mbw16asyipyrZ0/e4FLsY4A8lVcw
 5vX81LMj5vV7NwHVJiHQ7Qxcyu5ZBCV0UyqIJiIwqXquXMF6UE22dLsraCopIJ3Z
 lLhqf2XgDVSfICvi55e6xgzeVHknJ0CtN8+nOvDmeZmTkjpGK9xPUlHoD9zn8/kN
 Fyfn8fdAwn3+0Yw9HF1i78+WL/btuHebIpCFt0DNHOX0SkBTMpwwMIv0hu83yVb2
 mMfqhDwzkdUWeZsne5gtF2ZunF1hWa0e9a9bZ3IgHojZ1BFMzGusIPR6K//IWKrJ
 PQUdqb7i1lD4IZePrPseN6dPKQQskbBSsw0zSLOBYIhFc4AK5VoZIDHDkVUtMbLH
 Y/eAViGGSfX6WfRTTiZvyZOqJg06fS2z/aQBfO6oKw1J9iTJDUW+5R/IZHqZcLo1
 xe+P1r4mJzsRLspOODJvhJxIpE3aoW0H3/88nUiA3FMz7Qt9aPsgDwtl7p3WyZwu
 bP+FLuoRNEvb1mgz1Y7qXz5/agz/8CxfQFR7oJLi/qGjX6xXVLd1ZIVKiy04awbw
 AEWEWsm64uSOMH3tOzH2J7dfpykSADxNMEzt2SVrRH/UIVvlRa4=
 =f9iS
 -----END PGP SIGNATURE-----

Merge tag 'v2.35.2'
2022-04-11 16:44:45 -07:00
Junio C Hamano
c42977afe0 Merge branch 'tl/ls-tree-oid-only'
Docfix.

* tl/ls-tree-oid-only:
  ls-tree doc: document interaction with submodules
2022-04-08 13:53:48 -07:00
Ævar Arnfjörð Bjarmason
acd34fd5f6 ls-tree doc: document interaction with submodules
The ls-tree documentation had never been updated after it learned to
interact with submodules to explicitly mention them. The initial
support was added in f35a6d3bce (Teach core object handling functions
about gitlinks, 2007-04-09). E.g. the discussion of --long added in
f35a6d3bce (Teach core object handling functions about gitlinks,
2007-04-09) didn't explicitly mention them.

But this documentation added in 455923e0a1 (ls-tree: introduce
"--format" option, 2022-03-23) had no such excuse, and was actively
misleading by providing an exhaustive but incomplete list of object
types we'd emit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-08 11:21:11 -07:00
Junio C Hamano
bf23fe5c37 Merge branch 'jc/cat-file-batch-commands'
Doc fix.

* jc/cat-file-batch-commands:
  Documentation: add --batch-command to cat-file synopsis
2022-04-07 15:04:19 -07:00
Ævar Arnfjörð Bjarmason
473fa2df08 Documentation: add --batch-command to cat-file synopsis
440c705ea6 (cat-file: add --batch-command mode, 2022-02-18) added
the new option and operating mode without listing it to the synopsis
section.  Fix it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-07 13:31:54 -07:00
Junio C Hamano
07135d6be7 Merge branch 'tz/doc-litdd-fixes'
Documentation markup fix.

* tz/doc-litdd-fixes:
  doc: replace "--" with {litdd} in credential-cache/fsmonitor
2022-04-07 12:23:47 -07:00
Junio C Hamano
77da6db57f Merge branch 'tl/ls-tree-oid-only'
* tl/ls-tree-oid-only:
  git-ls-tree.txt: fix the name of "%(objectsize:padded)"
2022-04-07 12:23:23 -07:00
Martin Ågren
5e65dac9c8 git-ls-tree.txt: fix the name of "%(objectsize:padded)"
Commit 455923e0a1 ("ls-tree: introduce "--format" option", 2022-03-23)
introduced `--format` and the various placeholders it can take, such as
%(objectname) and %(objectsize).

At some point when that patch was being developed, those placeholders
had shorter names, e.g., %(name) and %(size), which can be seen in the
commit message of 455923e0a1. One instance of "%(size:padded)" also
managed to enter the documentation in the final version of the patch.
Correct it to "%(objectsize:padded)".

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-07 10:11:20 -07:00
Todd Zullinger
f3ea4bed2a doc: replace "--" with {litdd} in credential-cache/fsmonitor
Asciidoc renders `--` as em-dash.  This is not appropriate for command
names.  It also breaks linkgit links to these commands.

Fix git-credential-cache--daemon and git-fsmonitor--daemon.  The latter
was added 3248486920 (fsmonitor: document builtin fsmonitor, 2022-03-25)
and included several links.  A check for broken links in the HTML docs
turned this up.

Manually inspecting the other Documentation/git-*--*.txt files turned up
the issue in git-credential-cache--daemon.

While here, quote `git credential-cache--daemon` in the synopsis to
match the vast majority of our other documentation.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-06 16:06:06 -07:00
Junio C Hamano
0f0303a479 Merge branch 'ab/make-optim-noop'
A micro fix to a topic earlier merged to 'master'
source: <patch-1.1-05949221e3f-20220319T002715Z-avarab@gmail.com>

* ab/make-optim-noop:
  contrib/scalar: fix 'all' target in Makefile
  Documentation/Makefile: fix "make info" regression in dad9cd7d51
2022-04-06 15:21:58 -07:00
Neeraj Singh
c0f4752ed2 core.fsyncmethod: batched disk flushes for loose-objects
When adding many objects to a repo with `core.fsync=loose-object`,
the cost of fsync'ing each object file can become prohibitive.

One major source of the cost of fsync is the implied flush of the
hardware writeback cache within the disk drive. This commit introduces
a new `core.fsyncMethod=batch` option that batches up hardware flushes.
It hooks into the bulk-checkin odb-transaction functionality, takes
advantage of tmp-objdir, and uses the writeout-only support code.

When the new mode is enabled, we do the following for each new object:
1a. Create the object in a tmp-objdir.
2a. Issue a pagecache writeback request and wait for it to complete.

At the end of the entire transaction when unplugging bulk checkin:
1b. Issue an fsync against a dummy file to flush the log and hardware
   writeback cache, which should by now have seen the tmp-objdir writes.
2b. Rename all of the tmp-objdir files to their final names.
3b. When updating the index and/or refs, we assume that Git will issue
   another fsync internal to that operation. This is not the default
   today, but the user now has the option of syncing the index and there
   is a separate patch series to implement syncing of refs.

On a filesystem with a singular journal that is updated during name
operations (e.g. create, link, rename, etc), such as NTFS, HFS+, or XFS
we would expect the fsync to trigger a journal writeout so that this
sequence is enough to ensure that the user's data is durable by the time
the git command returns. This sequence also ensures that no object files
appear in the main object store unless they are fsync-durable.

Batch mode is only enabled if core.fsync includes loose-objects. If
the legacy core.fsyncObjectFiles setting is enabled, but core.fsync does
not include loose-objects, we will use file-by-file fsyncing.

In step (1a) of the sequence, the tmp-objdir is created lazily to avoid
work if no loose objects are ever added to the ODB. We use a tmp-objdir
to maintain the invariant that no loose-objects are visible in the main
ODB unless they are properly fsync-durable. This is important since
future ODB operations that try to create an object with specific
contents will silently drop the new data if an object with the target
hash exists without checking that the loose-object contents match the
hash. Only a full git-fsck would restore the ODB to a functional state
where dataloss doesn't occur.

In step (1b) of the sequence, we issue a fsync against a dummy file
created specifically for the purpose. This method has a little higher
cost than using one of the input object files, but makes adding new
callers of this mechanism easier, since we don't need to figure out
which object file is "last" or risk sharing violations by caching the fd
of the last object file.

_Performance numbers_:

Linux - Hyper-V VM running Kernel 5.11 (Ubuntu 20.04) on a fast SSD.
Mac - macOS 11.5.1 running on a Mac mini on a 1TB Apple SSD.
Windows - Same host as Linux, a preview version of Windows 11.

Adding 500 files to the repo with 'git add' Times reported in seconds.

object file syncing | Linux | Mac   | Windows
--------------------|-------|-------|--------
           disabled | 0.06  |  0.35 | 0.61
              fsync | 1.88  | 11.18 | 2.47
              batch | 0.15  |  0.41 | 1.53

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-06 13:13:01 -07:00
Junio C Hamano
fca85986bb Merge branch 'ns/core-fsyncmethod' into ns/batch-fsync
* ns/core-fsyncmethod:
  configure.ac: fix HAVE_SYNC_FILE_RANGE definition
  core.fsyncmethod: correctly camel-case warning message
  core.fsync: fix incorrect expression for default configuration
  core.fsync: documentation and user-friendly aggregate options
  core.fsync: new option to harden the index
  core.fsync: add configuration parsing
  core.fsync: introduce granular fsync control infrastructure
  core.fsyncmethod: add writeout-only mode
  wrapper: make inclusion of Windows csprng header tightly scoped
2022-04-06 13:01:54 -07:00
Ævar Arnfjörð Bjarmason
8af0699b7a Documentation/Makefile: fix "make info" regression in dad9cd7d51
Fix a regression in my dad9cd7d51 (Makefile: move ".SUFFIXES" rule to
shared.mak, 2022-03-03). As explained in the GNU make documentation
for the $* variable, available at:

	info make --index-search='$*'

This rule relied on ".texi" being in the default list of suffixes, as
seen at:

	make -f/dev/null -p | grep -v -e ^# -e ^$|grep -F .SUFFIXES

The documentation explains what was going on here:

	In an explicit rule, there is no stem; so '$*' cannot be determined
	in that way.  Instead, if the target name ends with a recognized
	suffix (*note Old-Fashioned Suffix Rules: Suffix Rules.), '$*' is
	set to the target name minus the suffix.  For example, if the
	target name is 'foo.c', then '$*' is set to 'foo', since '.c' is a
	suffix.  GNU 'make' does this bizarre thing only for compatibility
	with other implementations of 'make'.  You should generally avoid
	using '$*' except in implicit rules or static pattern rules.

	If the target name in an explicit rule does not end with a
	recognized suffix, '$*' is set to the empty string for that rule.

I.e. this rule added back in 5cefc33bff (Documentation: add
gitman.info target, 2007-12-10) was resolving gitman.texi from
gitman.info. We can instead just use the more obvious $< variable
referring to the prerequisite.

This was the only use of $* in our Makefiles in an explicit rule, the
three remaining ones are all implicit rules, and therefore didn't
depend on the ".SUFFIXES" list.

Reported-by: Adam Dinwoodie <adam@dinwoodie.org>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Tested-by: Adam Dinwoodie <adam@dinwoodie.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-06 09:43:48 -07:00
Junio C Hamano
faa21c10d4 Git 2.36-rc0
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-04 10:56:25 -07:00
Junio C Hamano
7c6d8ee8fa Merge branch 'pw/worktree-list-with-z'
"git worktree list --porcelain" did not c-quote pathnames and lock
reasons with unsafe bytes correctly, which is worked around by
introducing NUL terminated output format with "-z".

* pw/worktree-list-with-z:
  worktree: add -z option for list subcommand
2022-04-04 10:56:25 -07:00
Junio C Hamano
77ceb11342 Merge branch 'jc/coding-guidelines-decl-in-for-loop'
Coding Guidelines clarification.

* jc/coding-guidelines-decl-in-for-loop:
  CodingGuidelines: give deadline for "for (int i = 0; ..."
2022-04-04 10:56:24 -07:00
Junio C Hamano
439c1e6d5d Merge branch 'jh/builtin-fsmonitor-part2'
Built-in fsmonitor (part 2).

* jh/builtin-fsmonitor-part2: (30 commits)
  t7527: test status with untracked-cache and fsmonitor--daemon
  fsmonitor: force update index after large responses
  fsmonitor--daemon: use a cookie file to sync with file system
  fsmonitor--daemon: periodically truncate list of modified files
  t/perf/p7519: add fsmonitor--daemon test cases
  t/perf/p7519: speed up test on Windows
  t/perf/p7519: fix coding style
  t/helper/test-chmtime: skip directories on Windows
  t/perf: avoid copying builtin fsmonitor files into test repo
  t7527: create test for fsmonitor--daemon
  t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon
  help: include fsmonitor--daemon feature flag in version info
  fsmonitor--daemon: implement handle_client callback
  compat/fsmonitor/fsm-listen-darwin: implement FSEvent listener on MacOS
  compat/fsmonitor/fsm-listen-darwin: add MacOS header files for FSEvent
  compat/fsmonitor/fsm-listen-win32: implement FSMonitor backend on Windows
  fsmonitor--daemon: create token-based changed path cache
  fsmonitor--daemon: define token-ids
  fsmonitor--daemon: add pathname classification
  fsmonitor--daemon: implement 'start' command
  ...
2022-04-04 10:56:24 -07:00
Junio C Hamano
ba2452b247 Merge branch 'tk/ambiguous-fetch-refspec'
Give hint when branch tracking cannot be established because fetch
refspecs from multiple remote repositories overlap.

* tk/ambiguous-fetch-refspec:
  tracking branches: add advice to ambiguous refspec error
2022-04-04 10:56:24 -07:00
Junio C Hamano
0f5e885173 Merge branch 'rc/fetch-refetch'
"git fetch --refetch" learned to fetch everything without telling
the other side what we already have, which is useful when you
cannot trust what you have in the local object store.

* rc/fetch-refetch:
  docs: mention --refetch fetch option
  fetch: after refetch, encourage auto gc repacking
  t5615-partial-clone: add test for fetch --refetch
  fetch: add --refetch option
  builtin/fetch-pack: add --refetch option
  fetch-pack: add refetch
  fetch-negotiator: add specific noop initializer
2022-04-04 10:56:23 -07:00
Junio C Hamano
3928e902e3 Merge branch 'ds/partial-bundle-more'
Code clean-up.

* ds/partial-bundle-more:
  pack-objects: lazily set up "struct rev_info", don't leak
  bundle: output hash information in 'verify'
  bundle: move capabilities to end of 'verify'
  pack-objects: parse --filter directly into revs.filter
  pack-objects: move revs out of get_object_list()
  list-objects-filter: remove CL_ARG__FILTER
2022-04-04 10:56:21 -07:00
Junio C Hamano
1041d58b4d Merge branch 'tl/ls-tree-oid-only'
"git ls-tree" learns "--oid-only" option, similar to "--name-only",
and more generalized "--format" option.

* tl/ls-tree-oid-only:
  ls-tree: split up "fast path" callbacks
  ls-tree: detect and error on --name-only --name-status
  ls-tree: support --object-only option for "git-ls-tree"
  ls-tree: introduce "--format" option
  cocci: allow padding with `strbuf_addf()`
  ls-tree: introduce struct "show_tree_data"
  ls-tree: slightly refactor `show_tree()`
  ls-tree: fix "--name-only" and "--long" combined use bug
  ls-tree: simplify nesting if/else logic in "show_tree()"
  ls-tree: rename "retval" to "recurse" in "show_tree()"
  ls-tree: use "size_t", not "int" for "struct strbuf"'s "len"
  ls-tree: use "enum object_type", not {blob,tree,commit}_type
  ls-tree: add missing braces to "else" arms
  ls-tree: remove commented-out code
  ls-tree tests: add tests for --name-status
2022-04-04 10:56:21 -07:00
Fernando Ramos
7b5cf8be18 vimdiff: add tool documentation
Running 'git {merge,diff}tool --tool-help' now also prints usage
information about the vimdiff tool (and its variants) instead of just
its name.

Two new functions ('diff_cmd_help()' and 'merge_cmd_help()') have been
added to the set of functions that each merge tool (ie. scripts found
inside "mergetools/") can overwrite to provided tool specific
information.

Right now, only 'mergetools/vimdiff' implements these functions, but
other tools are encouraged to do so in the future, specially if they
take configuration options not explained anywhere else (as it is the
case with the 'vimdiff' tool and the new 'layout' option)

Note that the function 'show_tool_names', used in the implementation of
'git mergetool --tool-help', is also used in Documentation/Makefile to
generate the list of allowed values for the configuration variables
'{diff,merge}.{gui,}tool'. Adjust the rule so its output is an Asciidoc
"description list" instead of a plain list, with the tool name as the
item and the newly added tool description as the description.

In addition, a section has been added to
"Documentation/git-mergetool.txt" to explain the new "layout"
configuration option with examples.

Helped-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Fernando Ramos <greenfoo@u92.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-03 15:09:52 -07:00
Tao Klerks
e4921d877a tracking branches: add advice to ambiguous refspec error
The error "not tracking: ambiguous information for ref" is raised
when we are evaluating what tracking information to set on a branch,
and find that the ref to be added as tracking branch is mapped
under multiple remotes' fetch refspecs.

This can easily happen when a user copy-pastes a remote definition
in their git config, and forgets to change the tracking path.

Add advice in this situation, explicitly highlighting which remotes
are involved and suggesting how to correct the situation. Also
update a test to explicitly expect that advice.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-01 10:09:18 -07:00
Garrit Franke
6b52f48b8f cli: add -v and -h shorthands
Change the behavior of "git -v" to be synonymous with "--version" /
"version", and "git -h" to be synonymous with "--help", but not "help".

These shorthands both display the "unknown option" message. Following
this change, "-v" displays the version, and "-h" displays the help text
of the "git" command.

It should be noted that the "-v" shorthand could be misinterpreted by
the user to mean "verbose" instead of "version", since some sub-commands
make use of it in this context. The top-level "git" command does not
have a "verbose" flag, so it's safe to introduce this shorthand
unambiguously.

Signed-off-by: Garrit Franke <garrit@slashdev.space>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-31 15:57:10 -07:00
Phillip Wood
d97eb302ea worktree: add -z option for list subcommand
Add a -z option to be used in conjunction with --porcelain that gives
NUL-terminated output. As 'worktree list --porcelain' does not quote
worktree paths this enables it to handle worktree paths that contain
newlines.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-31 13:28:55 -07:00
Junio C Hamano
6563706568 CodingGuidelines: give deadline for "for (int i = 0; ..."
We raised the weather balloon to see if we can allow the construct
in 44ba10d6 (revision: use C99 declaration of variable in for()
loop, 2021-11-14), which was shipped as a part of Git v2.35.
Document that fact in the coding guidelines, and more importantly,
give ourselves a deadline to revisit and update.

Let's declare that we will officially adopt the variable declaration
in the initializaiton part of "for ()" statement this winter, unless
we find that a platform we care about does not grok it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-31 13:12:26 -07:00
Junio C Hamano
4b6846d9dc The 17th batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-30 18:01:11 -07:00
Junio C Hamano
6d51217467 Merge branch 'vd/stash-silence-reset'
"git stash" does not allow subcommands it internally runs as its
implementation detail, except for "git reset", to emit messages;
now "git reset" part has also been squelched.

* vd/stash-silence-reset:
  reset: show --no-refresh in the short-help
  reset: remove 'reset.refresh' config option
  reset: remove 'reset.quiet' config option
  reset: do not make '--quiet' disable index refresh
  stash: make internal resets quiet and refresh index
  reset: suppress '--no-refresh' advice if logging is silenced
  reset: replace '--quiet' with '--no-refresh' in performance advice
  reset: introduce --[no-]refresh option to --mixed
  reset: revise index refresh advice
2022-03-30 18:01:10 -07:00
Junio C Hamano
805e0a6808 The 16th batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-29 12:22:03 -07:00
Robert Coup
4963d3e41f docs: mention --refetch fetch option
Document it for partial clones as a means to apply a new filter, and
reference it from the remote.<name>.partialclonefilter config parameter.

Signed-off-by: Robert Coup <robert@coup.net.nz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-28 10:25:53 -07:00
Robert Coup
7390f05a3c fetch: after refetch, encourage auto gc repacking
After invoking `fetch --refetch`, the object db will likely contain many
duplicate objects. If auto-maintenance is enabled, invoke it with
appropriate settings to encourage repacking/consolidation.

* gc.autoPackLimit: unless this is set to 0 (disabled), override the
  value to 1 to force pack consolidation.
* maintenance.incremental-repack.auto: unless this is set to 0, override
  the value to -1 to force incremental repacking.

Signed-off-by: Robert Coup <robert@coup.net.nz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-28 10:25:53 -07:00
Robert Coup
3c7bab06e1 fetch: add --refetch option
Teach fetch and transports the --refetch option to force a full fetch
without negotiating common commits with the remote. Use when applying a
new partial clone filter to refetch all matching objects.

Signed-off-by: Robert Coup <robert@coup.net.nz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-28 10:25:52 -07:00
Robert Coup
869a0eb4eb builtin/fetch-pack: add --refetch option
Add a refetch option to fetch-pack to force a full fetch. Use when
applying a new partial clone filter to refetch all matching objects.

Signed-off-by: Robert Coup <robert@coup.net.nz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-28 10:25:52 -07:00
Junio C Hamano
abf474a5dd The 15th batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-25 16:38:25 -07:00
Junio C Hamano
dd9ff30dff Merge branch 'gc/recursive-fetch-with-unused-submodules'
When "git fetch --recurse-submodules" grabbed submodule commits
that would be needed to recursively check out newly fetched commits
in the superproject, it only paid attention to submodules that are
in the current checkout of the superproject.  We now do so for all
submodules that have been run "git submodule init" on.

* gc/recursive-fetch-with-unused-submodules:
  submodule: fix latent check_has_commit() bug
  fetch: fetch unpopulated, changed submodules
  submodule: move logic into fetch_task_create()
  submodule: extract get_fetch_task()
  submodule: store new submodule commits oid_array in a struct
  submodule: inline submodule_commits() into caller
  submodule: make static functions read submodules from commits
  t5526: create superproject commits with test helper
  t5526: stop asserting on stderr literally
  t5526: introduce test helper to assert on fetches
2022-03-25 16:38:25 -07:00
Junio C Hamano
6e1a8952e9 Merge branch 'ps/fsync-refs'
Updates to refs traditionally weren't fsync'ed, but we can
configure using core.fsync variable to do so.

* ps/fsync-refs:
  core.fsync: new option to harden references
2022-03-25 16:38:25 -07:00
Junio C Hamano
eb804cd405 Merge branch 'ns/core-fsyncmethod'
Replace core.fsyncObjectFiles with two new configuration variables,
core.fsync and core.fsyncMethod.

* ns/core-fsyncmethod:
  core.fsync: documentation and user-friendly aggregate options
  core.fsync: new option to harden the index
  core.fsync: add configuration parsing
  core.fsync: introduce granular fsync control infrastructure
  core.fsyncmethod: add writeout-only mode
  wrapper: make inclusion of Windows csprng header tightly scoped
2022-03-25 16:38:24 -07:00
Jeff Hostetler
3248486920 fsmonitor: document builtin fsmonitor
Document how `core.fsmonitor` can be set to a boolean to enable
or disable the builtin FSMonitor.

Update references to `core.fsmonitor` and `core.fsmonitorHookVersion` and
pointers to `Watchman` to refer to it.

Create `git-fsmonitor--daemon` manual page and describe its features.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-25 16:04:15 -07:00
Johannes Schindelin
53ef17d3ee Git 2.35.2
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-03-24 00:31:43 +01:00
Johannes Schindelin
1f480d5127 Sync with 2.34.2
* maint-2.34:
  Git 2.34.2
  Git 2.33.2
  Git 2.32.1
  Git 2.31.2
  GIT-VERSION-GEN: bump to v2.33.1
  Git 2.30.3
  setup_git_directory(): add an owner check for the top-level directory
  Add a function to determine whether a path is owned by the current user
2022-03-24 00:31:42 +01:00
Johannes Schindelin
4d0b43aa76 Git 2.34.2
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-03-24 00:31:36 +01:00
Johannes Schindelin
93fbff09eb Sync with 2.33.2
* maint-2.33:
  Git 2.33.2
  Git 2.32.1
  Git 2.31.2
  GIT-VERSION-GEN: bump to v2.33.1
  Git 2.30.3
  setup_git_directory(): add an owner check for the top-level directory
  Add a function to determine whether a path is owned by the current user
2022-03-24 00:31:36 +01:00
Johannes Schindelin
87ed4fc046 Git 2.33.2
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-03-24 00:31:32 +01:00
Johannes Schindelin
303b876f76 Sync with 2.32.1
* maint-2.32:
  Git 2.32.1
  Git 2.31.2
  Git 2.30.3
  setup_git_directory(): add an owner check for the top-level directory
  Add a function to determine whether a path is owned by the current user
2022-03-24 00:31:32 +01:00
Johannes Schindelin
9bcd7a8eca Git 2.32.1
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-03-24 00:31:29 +01:00
Johannes Schindelin
201b0c7af6 Sync with 2.31.2
* maint-2.31:
  Git 2.31.2
  Git 2.30.3
  setup_git_directory(): add an owner check for the top-level directory
  Add a function to determine whether a path is owned by the current user
2022-03-24 00:31:28 +01:00
Johannes Schindelin
44de39c45c Git 2.31.2
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-03-24 00:24:29 +01:00
Johannes Schindelin
6a2381a3e5 Sync with 2.30.3
* maint-2.30:
  Git 2.30.3
  setup_git_directory(): add an owner check for the top-level directory
  Add a function to determine whether a path is owned by the current user
2022-03-24 00:24:29 +01:00
Johannes Schindelin
cb95038137 Git 2.30.3
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-03-24 00:22:17 +01:00
Victoria Dye
7cff6765fe reset: remove 'reset.refresh' config option
Remove the 'reset.refresh' option, requiring that users explicitly specify
'--no-refresh' if they want to skip refreshing the index.

The 'reset.refresh' option was introduced in 101cee42dd (reset: introduce
--[no-]refresh option to --mixed, 2022-03-11) as a replacement for the
refresh-skipping behavior originally controlled by 'reset.quiet'.

Although 'reset.refresh=false' functionally served the same purpose as
'reset.quiet=true', it exposed [1] the fact that the existence of a global
"skip refresh" option could potentially cause problems for users. Allowing a
global config option to avoid refreshing the index forces scripts using 'git
reset --mixed' to defensively use '--refresh' if index refresh is expected;
if that option is missing, behavior of a script could vary from user-to-user
without explanation.

Furthermore, globally disabling index refresh in 'reset --mixed' was
initially devised as a passive performance improvement; since the
introduction of the option, other changes have been made to Git (e.g., the
sparse index) with a greater potential performance impact without
sacrificing index correctness. Therefore, we can more aggressively err on
the side of correctness and limit the cases of skipping index refresh to
only when a user specifies the '--no-refresh' option.

[1] https://lore.kernel.org/git/xmqqy2179o3c.fsf@gitster.g/

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23 14:39:45 -07:00
Victoria Dye
2efc9b84e5 reset: remove 'reset.quiet' config option
Remove the 'reset.quiet' config option, remove '--no-quiet' documentation in
'Documentation/git-reset.txt'. In 4c3abd0551 (reset: add new reset.quiet
config setting, 2018-10-23), 'reset.quiet' was introduced as a way to
globally change the default behavior of 'git reset --mixed' to skip index
refresh.

However, now that '--quiet' does not affect index refresh, 'reset.quiet'
would only serve to globally silence logging. This was not the original
intention of the config setting, and there's no precedent for such a setting
in other commands with a '--quiet' option, so it appears to be obsolete.

In addition to the options & its documentation, remove 'reset.quiet' from
the recommended config for 'scalar'.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23 14:39:45 -07:00
Victoria Dye
45bf76284b reset: do not make '--quiet' disable index refresh
Update '--quiet' to no longer implicitly skip refreshing the index in a
mixed reset. Users now have the ability to explicitly disable refreshing the
index with the '--no-refresh' option, so they no longer need to use
'--quiet' to do so. Moreover, we explicitly remove the refresh-skipping
behavior from '--quiet' because it is completely unrelated to the stated
purpose of the option: "Be quiet, only report errors."

Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23 14:39:44 -07:00
Junio C Hamano
a68dfadae5 The 14th batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23 14:09:31 -07:00
Junio C Hamano
bfce3e7b92 Merge branch 'ps/repack-with-server-info'
"git repack" learned a new configuration to disable triggering of
age-old "update-server-info" command, which is rarely useful these
days.

* ps/repack-with-server-info:
  repack: add config to skip updating server info
  repack: refactor to avoid double-negation of update-server-info
2022-03-23 14:09:30 -07:00
Junio C Hamano
ecb939a9ce Merge branch 'ds/doc-maintenance-synopsis-fix'
Doc update.

* ds/doc-maintenance-synopsis-fix:
  maintenance: fix synopsis in documentation
2022-03-23 14:09:30 -07:00
Junio C Hamano
3ece3cb865 Merge branch 'jd/userdiff-kotlin'
A new built-in userdiff driver for kotlin.

* jd/userdiff-kotlin:
  userdiff: add builtin diff driver for kotlin language.
2022-03-23 14:09:29 -07:00
Derrick Stolee
017303eb48 bundle: move capabilities to end of 'verify'
The 'filter' capability was added in 105c6f14a (bundle: parse filter
capability, 2022-03-09), but was added in a strange place in the 'git
bundle verify' output.

The tests for this show output like the following:

	The bundle contains these 2 refs:
	<COMMIT1> <REF1>
	<COMMIT2> <REF2>
	The bundle uses this filter: blob:none
	The bundle records a complete history.

This looks very odd if we have a thin bundle that contains boundary
commits instead of a complete history:

	The bundle contains these 2 refs:
	<COMMIT1> <REF1>
	<COMMIT2> <REF2>
	The bundle uses this filter: blob:none
	The bundle requires these 2 refs:
	<COMMIT3>
	<COMMIT4>

This separation between tip refs and boundary refs is unfortunate. Move
the filter capability output to the end of the output. Update the
documentation to match.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23 13:13:59 -07:00
Teng Long
cab851c2f8 ls-tree: support --object-only option for "git-ls-tree"
'--object-only' is an alias for '--format=%(objectname)'. It cannot
be used together other format-altering options like '--name-only',
'--long' or '--format', they are mutually exclusive.

The "--name-only" option outputs <filepath> only. Likewise, <objectName>
is another high frequency used field, so implement '--object-only' option
will bring intuitive and clear semantics for this scenario. Using
'--format=%(objectname)' we can achieve a similar effect, but the former
is with a lower learning cost(without knowing the format requirement
of '--format' option).

Even so, if a user is prefer to use "--format=%(objectname)", this is entirely
welcome because they are not only equivalent in function, but also have almost
identical performance. The reason is this commit also add the specific of
"--format=%(objectname)" to the current fast-pathes (builtin formats) to
avoid running unnecessary parsing mechanisms.

The following performance benchmarks are based on torvalds/linux.git:

  When hit the fast-path:

      Benchmark 1: /opt/git/ls-tree-oid-only/bin/git ls-tree -r --object-only HEAD
        Time (mean ± σ):      83.6 ms ±   2.0 ms    [User: 59.4 ms, System: 24.1 ms]
        Range (min … max):    80.4 ms …  87.2 ms    35 runs

      Benchmark 1: /opt/git/ls-tree-oid-only/bin/git ls-tree -r --format='%(objectname)' HEAD
        Time (mean ± σ):      84.1 ms ±   1.8 ms    [User: 61.7 ms, System: 22.3 ms]
        Range (min … max):    80.9 ms …  87.5 ms    35 runs

  But for a customized format, it will be slower:

       Benchmark 1: /opt/git/ls-tree-oid-only/bin/git ls-tree -r --format='oid: %(objectname)' HEAD
         Time (mean ± σ):      96.5 ms ±   2.5 ms    [User: 72.9 ms, System: 23.5 ms]
  	 Range (min … max):    93.1 ms … 104.1 ms    31 runs

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Teng Long <dyroneteng@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23 11:38:40 -07:00
Ævar Arnfjörð Bjarmason
455923e0a1 ls-tree: introduce "--format" option
Add a --format option to ls-tree. It has an existing default output,
and then --long and --name-only options to emit the default output
along with the objectsize and, or to only emit object paths.

Rather than add --type-only, --object-only etc. we can just support a
--format using a strbuf_expand() similar to "for-each-ref
--format". We might still add such options in the future for
convenience.

The --format implementation is slower than the existing code, but this
change does not cause any performance regressions. We'll leave the
existing show_tree() unchanged, and only run show_tree_fmt() in if
a --format different than the hardcoded built-in ones corresponding to
the existing modes is provided.

I.e. something like the "--long" output would be much slower with
this, mainly due to how we need to allocate various things to do with
quote.c instead of spewing the output directly to stdout.

The new option of '--format' comes from Ævar Arnfjörð Bjarmasonn's
idea and suggestion, this commit makes modifications in terms of the
original discussion on community [1].

In [1] there was a "GIT_TEST_LS_TREE_FORMAT_BACKEND" variable to
ensure that we had test coverage for passing tests that would
otherwise use show_tree() through show_tree_fmt(), and thus that the
formatting mechanism could handle all the same cases as the
non-formatting options.

Somewhere in subsequent re-rolls of that we seem to have drifted away
from what the goal of these tests should be. We're trying to ensure
correctness of show_tree_fmt(). We can't tell if we "hit [the]
fast-path" here, and instead of having an explicit test for that, we
can just add it to something our "test_ls_tree_format" tests for.

Here is the statistics about performance tests:

1. Default format (hitten the builtin formats):

    "git ls-tree <tree-ish>" vs "--format='%(mode) %(type) %(object)%x09%(file)'"

    $hyperfine --warmup=10 "/opt/git/master/bin/git ls-tree -r HEAD"
    Benchmark 1: /opt/git/master/bin/git ls-tree -r HEAD
    Time (mean ± σ):     105.2 ms ±   3.3 ms    [User: 84.3 ms, System: 20.8 ms]
    Range (min … max):    99.2 ms … 113.2 ms    28 runs

    $hyperfine --warmup=10 "/opt/git/ls-tree-oid-only/bin/git ls-tree -r --format='%(mode) %(type) %(object)%x09%(file)'  HEAD"
    Benchmark 1: /opt/git/ls-tree-oid-only/bin/git ls-tree -r --format='%(mode) %(type) %(object)%x09%(file)'  HEAD
    Time (mean ± σ):     106.4 ms ±   2.7 ms    [User: 86.1 ms, System: 20.2 ms]
    Range (min … max):   100.2 ms … 110.5 ms    29 runs

2. Default format includes object size (hitten the builtin formats):

    "git ls-tree -l <tree-ish>" vs "--format='%(mode) %(type) %(object) %(size:padded)%x09%(file)'"

    $hyperfine --warmup=10 "/opt/git/master/bin/git ls-tree -r -l HEAD"
    Benchmark 1: /opt/git/master/bin/git ls-tree -r -l HEAD
    Time (mean ± σ):     335.1 ms ±   6.5 ms    [User: 304.6 ms, System: 30.4 ms]
    Range (min … max):   327.5 ms … 348.4 ms    10 runs

    $hyperfine --warmup=10 "/opt/git/ls-tree-oid-only/bin/git ls-tree -r --format='%(mode) %(type) %(object) %(size:padded)%x09%(file)'  HEAD"
    Benchmark 1: /opt/git/ls-tree-oid-only/bin/git ls-tree -r --format='%(mode) %(type) %(object) %(size:padded)%x09%(file)'  HEAD
    Time (mean ± σ):     337.2 ms ±   8.2 ms    [User: 309.2 ms, System: 27.9 ms]
    Range (min … max):   328.8 ms … 349.4 ms    10 runs

Links:
	[1] https://public-inbox.org/git/RFC-patch-6.7-eac299f06ff-20211217T131635Z-avarab@gmail.com/
	[2] https://lore.kernel.org/git/cb717d08be87e3239117c6c667cb32caabaad33d.1646390152.git.dyroneteng@gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Teng Long <dyroneteng@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23 11:38:40 -07:00
Junio C Hamano
f01e51a7cf The thirteenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-21 15:14:24 -07:00
Junio C Hamano
7391ecd338 Merge branch 'ds/partial-bundles'
Bundle file format gets extended to allow a partial bundle,
filtered by similar criteria you would give when making a
partial/lazy clone.

* ds/partial-bundles:
  clone: fail gracefully when cloning filtered bundle
  bundle: unbundle promisor packs
  bundle: create filtered bundles
  rev-list: move --filter parsing into revision.c
  bundle: parse filter capability
  list-objects: handle NULL function pointers
  MyFirstObjectWalk: update recommended usage
  list-objects: consolidate traverse_commit_list[_filtered]
  pack-bitmap: drop filter in prepare_bitmap_walk()
  pack-objects: use rev.filter when possible
  revision: put object filter into struct rev_info
  list-objects-filter-options: create copy helper
  index-pack: document and test the --promisor option
2022-03-21 15:14:24 -07:00
Johannes Schindelin
8959555cee setup_git_directory(): add an owner check for the top-level directory
It poses a security risk to search for a git directory outside of the
directories owned by the current user.

For example, it is common e.g. in computer pools of educational
institutes to have a "scratch" space: a mounted disk with plenty of
space that is regularly swiped where any authenticated user can create
a directory to do their work. Merely navigating to such a space with a
Git-enabled `PS1` when there is a maliciously-crafted `/scratch/.git/`
can lead to a compromised account.

The same holds true in multi-user setups running Windows, as `C:\` is
writable to every authenticated user by default.

To plug this vulnerability, we stop Git from accepting top-level
directories owned by someone other than the current user. We avoid
looking at the ownership of each and every directories between the
current and the top-level one (if there are any between) to avoid
introducing a performance bottleneck.

This new default behavior is obviously incompatible with the concept of
shared repositories, where we expect the top-level directory to be owned
by only one of its legitimate users. To re-enable that use case, we add
support for adding exceptions from the new default behavior via the
config setting `safe.directory`.

The `safe.directory` config setting is only respected in the system and
global configs, not from repository configs or via the command-line, and
can have multiple values to allow for multiple shared repositories.

We are particularly careful to provide a helpful message to any user
trying to use a shared repository.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2022-03-21 13:16:26 +01:00
Junio C Hamano
74cc1aa55f The twelfth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16 17:53:09 -07:00
Junio C Hamano
a54cc523ad Merge branch 'ds/commit-graph-gen-v2-fixes'
Fixes to the way generation number v2 in the commit-graph files are
(not) handled.

* ds/commit-graph-gen-v2-fixes:
  commit-graph: declare bankruptcy on GDAT chunks
  commit-graph: fix generation number v2 overflow values
  commit-graph: start parsing generation v2 (again)
  commit-graph: fix ordering bug in generation numbers
  t5318: extract helpers to lib-commit-graph.sh
  test-read-graph: include extra post-parse info
2022-03-16 17:53:09 -07:00
Junio C Hamano
47c52b2dad Merge branch 'tb/rename-remote-progress'
"git remote rename A B", depending on the number of remote-tracking
refs involved, takes long time renaming them.  The command has been
taught to show progress bar while making the user wait.

* tb/rename-remote-progress:
  builtin/remote.c: show progress when renaming remote references
  builtin/remote.c: parse options in 'rename'
2022-03-16 17:53:08 -07:00
Glen Choo
b90d9f7632 fetch: fetch unpopulated, changed submodules
"git fetch --recurse-submodules" only considers populated
submodules (i.e. submodules that can be found by iterating the index),
which makes "git fetch" behave differently based on which commit is
checked out. As a result, even if the user has initialized all submodules
correctly, they may not fetch the necessary submodule commits, and
commands like "git checkout --recurse-submodules" might fail.

Teach "git fetch" to fetch cloned, changed submodules regardless of
whether they are populated. This is in addition to the current behavior
of fetching populated submodules (which is always attempted regardless
of what was fetched in the superproject, or even if nothing was fetched
in the superproject).

A submodule may be encountered multiple times (via the list of
populated submodules or via the list of changed submodules). When this
happens, "git fetch" only reads the 'populated copy' and ignores the
'changed copy'. Amend the verify_fetch_result() test helper so that we
can assert on which 'copy' is being read.

Signed-off-by: Glen Choo <chooglen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-16 16:08:59 -07:00
Patrick Steinhardt
bc22d845c4 core.fsync: new option to harden references
When writing both loose and packed references to disk we first create a
lockfile, write the updated values into that lockfile, and on commit we
rename the file into place. According to filesystem developers, this
behaviour is broken because applications should always sync data to disk
before doing the final rename to ensure data consistency [1][2][3]. If
applications fail to do this correctly, a hard crash of the machine can
easily result in corrupted on-disk data.

This kind of corruption can in fact be easily observed with Git when the
machine hard-resets shortly after writing references to disk. On
machines with ext4, this will likely lead to the "empty files" problem:
the file has been renamed, but its data has not been synced to disk. The
result is that the reference is corrupt, and in the worst case this can
lead to data loss.

Implement a new option to harden references so that users and admins can
avoid this scenario by syncing locked loose and packed references to
disk before we rename them into place.

[1]: https://thunk.org/tytso/blog/2009/03/15/dont-fear-the-fsync/
[2]: https://btrfs.wiki.kernel.org/index.php/FAQ (What are the crash guarantees of overwrite-by-rename)
[3]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/ext4.rst (see auto_da_alloc)

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-15 13:30:58 -07:00
Junio C Hamano
0099792400 Merge branch 'ns/core-fsyncmethod' into ps/fsync-refs
* ns/core-fsyncmethod:
  core.fsync: documentation and user-friendly aggregate options
  core.fsync: new option to harden the index
  core.fsync: add configuration parsing
  core.fsync: introduce granular fsync control infrastructure
  core.fsyncmethod: add writeout-only mode
  wrapper: make inclusion of Windows csprng header tightly scoped
2022-03-15 13:30:37 -07:00
Neeraj Singh
b9f5d0358d core.fsync: documentation and user-friendly aggregate options
This commit adds aggregate options for the core.fsync setting that are
more user-friendly. These options are specified in terms of 'levels of
safety', indicating which Git operations are considered to be sync
points for durability.

The new documentation is also included here in its entirety for ease of
review.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-15 12:32:55 -07:00
Derrick Stolee
f4976ef739 maintenance: fix synopsis in documentation
The synopsis for 'git maintenance' did not include the commands other
than the 'run' command. Update this to include the others. The 'start'
command is the only one of these that parses additional options, and
then only the --scheduler option.

Also move the 'register' command down after 'stop' and before
'unregister' for a logical grouping of the commands instead of an
alphabetical one. The diff makes it look as three other commands are
moved up.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-15 10:52:43 -07:00
Victoria Dye
9396251b37 reset: replace '--quiet' with '--no-refresh' in performance advice
Replace references to '--quiet' with '--no-refresh' in the advice on how to
skip refreshing the index. When the advice was introduced, '--quiet' was the
only way to avoid the expensive 'refresh_index(...)' at the end of a mixed
reset. After introducing '--no-refresh', however, '--quiet' became only a
fallback option for determining refresh behavior, overridden by
'--[no-]refresh' or 'reset.refresh' if either is set. To ensure users are
advised to use the most reliable option for avoiding 'refresh_index(...)',
replace recommendation of '--quiet' with '--[no-]refresh'.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-14 18:51:56 -07:00
Victoria Dye
fd56fba97f reset: introduce --[no-]refresh option to --mixed
Add a new --[no-]refresh option that is intended to explicitly determine
whether a mixed reset should end in an index refresh.

Starting at 9ac8125d1a (reset: don't compute unstaged changes after reset
when --quiet, 2018-10-23), using the '--quiet' option results in skipping
the call to 'refresh_index(...)' at the end of a mixed reset with the goal
of improving performance. However, by coupling behavior that modifies the
index with the option that silences logs, there is no way for users to have
one without the other (i.e., silenced logs with a refreshed index) without
incurring the overhead of a separate call to 'git update-index --refresh'.
Furthermore, there is minimal user-facing documentation indicating that
--quiet skips the index refresh, potentially leading to unexpected issues
executing commands after 'git reset --quiet' that do not themselves refresh
the index (e.g., internals of 'git stash', 'git read-tree').

To mitigate these issues, '--[no-]refresh' and 'reset.refresh' are
introduced to provide a dedicated mechanism for refreshing the index. When
either is set, '--quiet' and 'reset.quiet' revert to controlling only
whether logs are silenced and do not affect index refresh.

Helped-by: Derrick Stolee <derrickstolee@github.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-14 18:51:56 -07:00
Victoria Dye
e86ec71d20 reset: revise index refresh advice
Update the advice describing index refresh from "enumerate unstaged changes"
to "refresh the index." Describing 'refresh_index(...)' as "enumerating
unstaged changes" is not fully representative of what an index refresh is
doing; more generally, it updates the properties of index entries that are
affected by outside-of-index state, e.g. CE_UPTODATE, which is affected by
the file contents on-disk. This distinction is relevant to operations that
read the index but do not refresh first - e.g., 'git read-tree' - where a
stale index may cause incorrect behavior.

In addition to changing the advice message, use the "advise" function to
print advice.

Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-14 18:51:56 -07:00
Patrick Steinhardt
a2565c48e4 repack: add config to skip updating server info
By default, git-repack(1) will update server info that is required by
the dumb HTTP transport. This can be skipped by passing the `-n` flag,
but what we're noticably missing is a config option to permanently
disable updating this information.

Add a new option "repack.updateServerInfo" which can be used to disable
the logic. Most hosting providers have turned off the dumb HTTP protocol
anyway, and on the client-side it woudln't typically be useful either.
Giving a persistent way to disable this feature thus makes quite some
sense to avoid wasting compute cycles and storage.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-14 22:25:13 +00:00
Junio C Hamano
b896f729e2 The eleventh batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-13 22:56:18 +00:00
Junio C Hamano
4eb845ac0a Merge branch 'nj/read-tree-doc-reffix'
Documentation mark-up fix.

* nj/read-tree-doc-reffix:
  Documentation: git-read-tree: separate links using commas
2022-03-13 22:56:18 +00:00
Junio C Hamano
f62106d750 Merge branch 'ab/make-optim-noop'
Makefile refactoring with a bit of suffixes rule stripping to
optimize the runtime overhead.

* ab/make-optim-noop:
  Makefiles: add and use wildcard "mkdir -p" template
  Makefile: add "$(QUIET)" boilerplate to shared.mak
  Makefile: move $(comma), $(empty) and $(space) to shared.mak
  Makefile: move ".SUFFIXES" rule to shared.mak
  Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
  Makefile: disable GNU make built-in wildcard rules
  Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
  scalar Makefile: use "The default target of..." pattern
2022-03-13 22:56:17 +00:00
Jaydeep P Das
09188ed930 userdiff: add builtin diff driver for kotlin language.
The xfuncname pattern finds func/class declarations
in diffs to display as a hunk header. The word_regex
pattern finds individual tokens in Kotlin code to generate
appropriate diffs.

This patch adds xfuncname regex and word_regex for Kotlin
language.

Signed-off-by: Jaydeep P Das <jaydeepjd.8914@gmail.com>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-12 18:15:47 -08:00
Neeraj Singh
844a8ad4f8 core.fsync: add configuration parsing
This change introduces code to parse the core.fsync setting and
configure the fsync_components variable.

core.fsync is configured as a comma-separated list of component names to
sync. Each time a core.fsync variable is encountered in the
configuration heirarchy, we start off with a clean state with the
platform default value. Passing 'none' resets the value to indicate
nothing will be synced. We gather all negative and positive entries from
the comma separated list and then compute the new value by removing all
the negative entries and adding all of the positive entries.

We issue a warning for components that are not recognized so that the
configuration code is compatible with configs from future versions of
Git with more repo components.

Complete documentation for the new setting is included in a later patch
in the series so that it can be reviewed once in final form.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-10 15:10:22 -08:00
Neeraj Singh
abf38abec2 core.fsyncmethod: add writeout-only mode
This commit introduces the `core.fsyncMethod` configuration
knob, which can currently be set to `fsync` or `writeout-only`.

The new writeout-only mode attempts to tell the operating system to
flush its in-memory page cache to the storage hardware without issuing a
CACHE_FLUSH command to the storage controller.

Writeout-only fsync is significantly faster than a vanilla fsync on
common hardware, since data is written to a disk-side cache rather than
all the way to a durable medium. Later changes in this patch series will
take advantage of this primitive to implement batching of hardware
flushes.

When git_fsync is called with FSYNC_WRITEOUT_ONLY, it may fail and the
caller is expected to do an ordinary fsync as needed.

On Apple platforms, the fsync system call does not issue a CACHE_FLUSH
directive to the storage controller. This change updates fsync to do
fcntl(F_FULLFSYNC) to make fsync actually durable. We maintain parity
with existing behavior on Apple platforms by setting the default value
of the new core.fsyncMethod option.

Signed-off-by: Neeraj Singh <neerajsi@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-10 15:10:22 -08:00
Junio C Hamano
1a4874565f The tenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09 13:38:46 -08:00
Junio C Hamano
1f3c5f39e0 Merge branch 'ab/help-fixes'
Updates to how command line options to "git help" are handled.

* ab/help-fixes:
  help: don't print "\n" before single-section output
  help: add --no-[external-commands|aliases] for use with --all
  help: error if [-a|-g|-c] and [-i|-m|-w] are combined
  help: correct usage & behavior of "git help --all"
  help: note the option name on option incompatibility
  help.c: split up list_all_cmds_help() function
  help tests: test "git" and "git help [-a|-g] spacing
  help.c: use puts() instead of printf{,_ln}() for consistency
  help doc: add missing "]" to "[-a|--all]"
2022-03-09 13:38:24 -08:00
Junio C Hamano
69a3b75fa6 Merge branch 'ab/c99-variadic-macros'
Remove the escape hatch we added when we introduced the weather
balloon to use variadic macros unconditionally, to make it official
that we now have a hard dependency on the feature.

* ab/c99-variadic-macros:
  C99: remove hardcoded-out !HAVE_VARIADIC_MACROS code
  git-compat-util.h: clarify GCC v.s. C99-specific in comment
2022-03-09 13:38:24 -08:00
Junio C Hamano
4763ccd7f4 Merge branch 'hn/reftable-no-empty-keys'
General clean-up in reftable implementation, including
clarification of the API documentation, tightening the code to
honor documented length limit, etc.

* hn/reftable-no-empty-keys:
  reftable: rename writer_stats to reftable_writer_stats
  reftable: add test for length of disambiguating prefix
  reftable: ensure that obj_id_len is >= 2 on writing
  reftable: avoid writing empty keys at the block layer
  reftable: add a test that verifies that writing empty keys fails
  reftable: reject 0 object_id_len
  Documentation: object_id_len goes up to 31
2022-03-09 13:38:24 -08:00
Junio C Hamano
d169d51504 Merge branch 'jc/cat-file-batch-commands'
"git cat-file" learns "--batch-command" mode, which is a more
flexible interface than the existing "--batch" or "--batch-check"
modes, to allow different kinds of inquiries made.

* jc/cat-file-batch-commands:
  cat-file: add --batch-command mode
  cat-file: add remove_timestamp helper
  cat-file: introduce batch_mode enum to replace print_contents
  cat-file: rename cmdmode to transform_mode
2022-03-09 13:38:24 -08:00
Junio C Hamano
82386b4496 Merge branch 'en/present-despite-skipped'
In sparse-checkouts, files mis-marked as missing from the working tree
could lead to later problems.  Such files were hard to discover, and
harder to correct.  Automatically detecting and correcting the marking
of such files has been added to avoid these problems.

* en/present-despite-skipped:
  repo_read_index: add config to expect files outside sparse patterns
  Accelerate clear_skip_worktree_from_present_files() by caching
  Update documentation related to sparsity and the skip-worktree bit
  repo_read_index: clear SKIP_WORKTREE bit from files present in worktree
  unpack-trees: fix accidental loss of user changes
  t1011: add testcase demonstrating accidental loss of user modifications
2022-03-09 13:38:23 -08:00
Derrick Stolee
105c6f14ad bundle: parse filter capability
The v3 bundle format has capabilities, allowing newer versions of Git to
create bundles with newer features. Older versions that do not
understand these new capabilities will fail with a helpful warning.

Create a new capability allowing Git to understand that the contained
pack-file is filtered according to some object filter. Typically, this
filter will be "blob:none" for a blobless partial clone.

This change teaches Git to parse this capability, place its value in the
bundle header, and demonstrate this understanding by adding a message to
'git bundle verify'.

Since we will use gently_parse_list_objects_filter() outside of
list-objects-filter-options.c, make it an external method and move its
API documentation to before its declaration.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09 10:25:27 -08:00
Derrick Stolee
f0d2f84919 MyFirstObjectWalk: update recommended usage
The previous change consolidated traverse_commit_list() and
traverse_commit_list_filtered(). This allows us to simplify the
recommended usage in MyFirstObjectWalk.txt to use this new set of
values.

While here, add some clarification on the difference between the two
methods.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09 10:25:27 -08:00
Derrick Stolee
1f52cdfacb index-pack: document and test the --promisor option
The --promisor option of 'git index-pack' was created in 88e2f9e
(introduce fetch-object: fetch one promisor object, 2017-12-05) but was
untested. It is currently unused within the Git codebase, but that will
change in an upcoming change to 'git bundle unbundle' when there is a
filter capability.

For now, add documentation about the option and add a test to ensure it
is working as expected.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-09 10:25:26 -08:00
Derrick Stolee
6dbf4b8172 commit-graph: declare bankruptcy on GDAT chunks
The Generation Data (GDAT) and Generation Data Overflow (GDOV) chunks
store corrected commit date offsets, used for generation number v2.
Recent changes have demonstrated that previous versions of Git were
incorrectly parsing data from these chunks, but might have also been
writing them incorrectly.

I asserted [1] that the previous fixes were sufficient because the known
reasons for incorrectly writing generation number v2 data relied on
parsing the information incorrectly out of a commit-graph file, but the
previous versions of Git were not reading the generation number v2 data.

However, Patrick demonstrated [2] a case where in split commit-graphs
across an alternate boundary (and possibly some other special
conditions) it was possible to have a commit-graph that was generated by
a previous version of Git have incorrect generation number v2 data which
results in errors like the following:

  commit-graph generation for commit <oid> is 1623273624 < 1623273710

[1] https://lore.kernel.org/git/f50e74f0-9ffa-f4f2-4663-269801495ed3@github.com/
[2] https://lore.kernel.org/git/Yh93vOkt2DkrGPh2@ncase/

Clearly, there is something else going on. The situation is not
completely understood, but the errors do not reproduce if the
commit-graphs are all generated by a Git version including these recent
fixes.

If we cannot trust the existing data in the GDAT and GDOV chunks, then
we can alter the format to change the chunk IDs for these chunks. This
causes the new version of Git to silently ignore the older chunks (and
disabling generation number v2 in the process) while writing new
commit-graph files with correct data in the GDA2 and GDO2 chunks.

Update commit-graph-format.txt including a historical note about these
deprecated chunks.

Reported-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-07 09:17:03 -08:00
Junio C Hamano
c2162907e9 The ninth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-06 21:25:33 -08:00
Junio C Hamano
7a4e06c42a Merge branch 'jt/ls-files-stage-recurse'
Many output modes of "ls-files" do not work with its
"--recurse-submodules" option, but the "-s" mode has been taught to
work with it.

* jt/ls-files-stage-recurse:
  ls-files: support --recurse-submodules --stage
2022-03-06 21:25:33 -08:00
Junio C Hamano
061fd5727d Merge branch 'ah/advice-switch-requires-detach-to-detach'
The error message given by "git switch HEAD~4" has been clarified
to suggest the "--detach" option that is required.

* ah/advice-switch-requires-detach-to-detach:
  switch: mention the --detach option when dying due to lack of a branch
2022-03-06 21:25:32 -08:00
Junio C Hamano
aae90a156d Merge branch 'ds/worktree-docs'
Tighten the language around "working tree" and "worktree" in the
docs.

* ds/worktree-docs:
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: use 'worktree' over 'working tree'
  worktree: extract checkout_worktree()
  worktree: extract copy_sparse_checkout()
  worktree: extract copy_filtered_worktree_config()
  worktree: combine two translatable messages
2022-03-06 21:25:31 -08:00
Taylor Blau
56710a7ae0 builtin/remote.c: show progress when renaming remote references
When renaming a remote, Git needs to rename all remote tracking
references to the remote's new name (e.g., renaming
"refs/remotes/old/foo" to "refs/remotes/new/foo" when renaming a remote
from "old" to "new").

This can be somewhat slow when there are many references to rename,
since each rename is done in a separate call to rename_ref() as opposed
to grouping all renames together into the same transaction. It would be
nice to execute all renames as a single transaction, but there is a
snag: the reference transaction backend doesn't support renames during a
transaction (only individually, via rename_ref()).

The reasons there are described in more detail in [1], but the main
problem is that in order to preserve the existing reflog, it must be
moved while holding both locks (i.e., on "oldname" and "newname"), and
the ref transaction code doesn't support inserting arbitrary actions
into the middle of a transaction like that.

As an aside, adding support for this to the ref transaction code is
less straightforward than inserting both a ref_update() and ref_delete()
call into the same transaction. rename_ref()'s special handling to
detect D/F conflicts would need to be rewritten for the transaction code
if we wanted to proactively catch D/F conflicts when renaming a
reference during a transaction. The reftable backend could support this
much more readily because of its lack of D/F conflicts.

Instead of a more complex modification to the ref transaction code,
display a progress meter when running verbosely in order to convince the
user that Git is doing work while renaming a remote.

This is mostly done as-expected, with the minor caveat that we
intentionally count symrefs renames twice, since renaming a symref takes
place over two separate calls (one to delete the old one, and another to
create the new one).

[1]: https://lore.kernel.org/git/572367B4.4050207@alum.mit.edu/

Suggested-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:44:05 -08:00
Nihal Jere
63a36017fe Documentation: git-read-tree: separate links using commas
This makes it consistent with the rest of the documentation.

Signed-off-by: Nihal Jere <nihal@nihaljere.xyz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:25:17 -08:00
Ævar Arnfjörð Bjarmason
0b6d0bc924 Makefiles: add and use wildcard "mkdir -p" template
Add a template to do the "mkdir -p" of $(@D) (the parent dir of $@)
for us, and use it for the "make lint-docs" targets I added in
8650c6298c (doc lint: make "lint-docs" non-.PHONY, 2021-10-15).

As seen in 4c64fb5aad (Documentation/Makefile: fix lint-docs mkdir
dependency, 2021-10-26) maintaining these manual lists of parent
directory dependencies is fragile, in addition to being obviously
verbose.

I used this pattern at the time because I couldn't find another method
than "order-only" prerequisites to avoid doing a "mkdir -p $(@D)" for
every file being created, which as noted in [1] would be significantly
slower.

But as it turns out we can use this neat trick of only doing a "mkdir
-p" if the $(wildcard) macro tells us the path doesn't exist. A re-run
of a performance test similar to that noted downthread of [1] in [2]
shows that this is faster, in addition to being less verbose and more
reliable (this uses my "git-hyperfine" thin wrapper for "hyperfine"[3]):

    $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make -C Documentation lint-docs' -p 'rm -rf Documentation/.build' 'make -C Documentation -j1 lint-docs'
    Benchmark 1: make -C Documentation -j1 lint-docs' in 'HEAD~1
      Time (mean ± σ):      2.914 s ±  0.062 s    [User: 2.449 s, System: 0.489 s]
      Range (min … max):    2.834 s …  3.020 s    10 runs

    Benchmark 2: make -C Documentation -j1 lint-docs' in 'HEAD~0
      Time (mean ± σ):      2.315 s ±  0.062 s    [User: 1.950 s, System: 0.386 s]
      Range (min … max):    2.229 s …  2.397 s    10 runs

    Summary
      'make -C Documentation -j1 lint-docs' in 'HEAD~0' ran
        1.26 ± 0.04 times faster than 'make -C Documentation -j1 lint-docs' in 'HEAD~1'

So let's use that pattern both for the "lint-docs" target, and a few
miscellaneous other targets.

This method of creating parent directories is explicitly racy in that
we don't know if we're going to say always create a "foo" followed by
a "foo/bar" under parallelism, or skip the "foo" because we created
"foo/bar" first. In this case it doesn't matter for anything except
that we aren't guaranteed to get the same number of rules firing when
running make in parallel.

1. https://lore.kernel.org/git/211028.861r45y3pt.gmgdl@evledraar.gmail.com/
2. https://lore.kernel.org/git/211028.86o879vvtp.gmgdl@evledraar.gmail.com/
3. https://gitlab.com/avar/git-hyperfine/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:14:55 -08:00
Ævar Arnfjörð Bjarmason
a9fda017f4 Makefile: add "$(QUIET)" boilerplate to shared.mak
The $(QUIET) variables we define are largely duplicated between our
various Makefiles, let's define them in the new "shared.mak" instead.

Since we're not using the environment to pass these around we don't
need to export the "QUIET_GEN" and "QUIET_BUILT_IN" variables
anymore. The "QUIET_GEN" variable is used in "git-gui/Makefile" and
"gitweb/Makefile", but they've got their own definition for those. The
"QUIET_BUILT_IN" variable is only used in the top-level "Makefile". We
still need to export the "V" variable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:14:55 -08:00
Ævar Arnfjörð Bjarmason
8df786d298 Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
We have various behavior that's shared across our Makefiles, or that
really should be (e.g. via defined templates). Let's create a
top-level "shared.mak" to house those sorts of things, and start by
adding the ".DELETE_ON_ERROR" flag to it.

See my own 7b76d6bf22 (Makefile: add and use the ".DELETE_ON_ERROR"
flag, 2021-06-29) and db10fc6c09 (doc: simplify Makefile using
.DELETE_ON_ERROR, 2021-05-21) for the addition and use of the
".DELETE_ON_ERROR" flag.

I.e. this changes the behavior of existing rules in the altered
Makefiles (except "Makefile" & "Documentation/Makefile"). I'm
confident that this is safe having read the relevant rules in those
Makfiles, and as the GNU make manual notes that it isn't the default
behavior is out of an abundance of backwards compatibility
caution. From edition 0.75 of its manual, covering GNU make 4.3:

    [Enabling '.DELETE_ON_ERROR' is] almost always what you want
    'make' to do, but it is not historical practice; so for
    compatibility, you must explicitly request it.

This doesn't introduce a bug by e.g. having this
".DELETE_ON_ERROR" flag only apply to this new shared.mak, Makefiles
have no such scoping semantics.

It does increase the danger that any Makefile without an explicit "The
default target of this Makefile is..." snippet to define the default
target as "all" could have its default rule changed if our new
shared.mak ever defines a "real" rule. In subsequent commits we'll be
careful not to do that, and such breakage would be obvious e.g. in the
case of "make -C t".

We might want to make that less fragile still (e.g. by using
".DEFAULT_GOAL" as noted in the preceding commit), but for now let's
simply include "shared.mak" without adding that boilerplate to all the
Makefiles that don't have it already. Most of those are already
exposed to that potential caveat e.g. due to including "config.mak*".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03 14:14:55 -08:00
Elijah Newren
ecc7c8841d repo_read_index: add config to expect files outside sparse patterns
Typically with sparse checkouts, we expect files outside the sparsity
patterns to be marked as SKIP_WORKTREE and be missing from the working
tree.  Sometimes this expectation would be violated however; including
in cases such as:
  * users grabbing files from elsewhere and writing them to the worktree
    (perhaps by editing a cached copy in an editor, copying/renaming, or
     even untarring)
  * various git commands having incomplete or no support for the
    SKIP_WORKTREE bit[1,2]
  * users attempting to "abort" a sparse-checkout operation with a
    not-so-early Ctrl+C (updating $GIT_DIR/info/sparse-checkout and the
    working tree is not atomic)[3].
When the SKIP_WORKTREE bit in the index did not reflect the presence of
the file in the working tree, it traditionally caused confusion and was
difficult to detect and recover from.  So, in a sparse checkout, since
af6a51875a (repo_read_index: clear SKIP_WORKTREE bit from files present
in worktree, 2022-01-14), Git automatically clears the SKIP_WORKTREE
bit at index read time for entries corresponding to files that are
present in the working tree.

There is another workflow, however, where it is expected that paths
outside the sparsity patterns appear to exist in the working tree and
that they do not lose the SKIP_WORKTREE bit, at least until they get
modified.  A Git-aware virtual file system[4] takes advantage of its
position as a file system driver to expose all files in the working
tree, fetch them on demand using partial clone on access, and tell Git
to pay attention to them on demand by updating the sparse checkout
pattern on writes.  This means that commands like "git status" only have
to examine files that have potentially been modified, whereas commands
like "ls" are able to show the entire codebase without requiring manual
updates to the sparse checkout pattern.

Thus since af6a51875a, Git with such Git-aware virtual file systems
unsets the SKIP_WORKTREE bit for all files and commands like "git
status" have to fetch and examine them all.

Introduce a configuration setting sparse.expectFilesOutsideOfPatterns to
allow limiting the tracked set of files to a small set once again.  A
Git-aware virtual file system or other application that wants to
maintain files outside of the sparse checkout can set this in a
repository to instruct Git not to check for the presence of
SKIP_WORKTREE files.  The setting defaults to false, so most users of
sparse checkout will still get the benefit of an automatically updating
index to recover from the variety of difficult issues detailed in
af6a51875a for paths with SKIP_WORKTREE set despite the path being
present.

[1] https://lore.kernel.org/git/xmqqbmb1a7ga.fsf@gitster-ct.c.googlers.com/
[2] The three long paragraphs in the middle of
    https://lore.kernel.org/git/CABPp-BH9tju7WVm=QZDOvaMDdZbpNXrVWQdN-jmfN8wC6YVhmw@mail.gmail.com/
[3] https://lore.kernel.org/git/CABPp-BFnFpzwGC11TLoLs8YK5yiisA5D5-fFjXnJsbESVDwZsA@mail.gmail.com/
[4] such as the vfsd described in
https://lore.kernel.org/git/20220207190320.2960362-1-jonathantanmy@google.com/

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-01 23:37:48 -08:00
Alex Henrie
808213ba36 switch: mention the --detach option when dying due to lack of a branch
Users who are accustomed to doing `git checkout <tag>` assume that
`git switch <tag>` will do the same thing. Inform them of the --detach
option so they aren't left wondering why `git switch` doesn't work but
`git checkout` does.

Signed-off-by: Alex Henrie <alexhenrie24@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-25 22:21:48 -08:00
Junio C Hamano
715d08a9e5 The eighth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-25 15:47:38 -08:00
Junio C Hamano
2e65591ed6 Merge branch 'js/apply-partial-clone-filters-recursively'
"git clone --filter=... --recurse-submodules" only makes the
top-level a partial clone, while submodules are fully cloned.  This
behaviour is changed to pass the same filter down to the submodules.

* js/apply-partial-clone-filters-recursively:
  clone, submodule: pass partial clone filters to submodules
2022-02-25 15:47:35 -08:00
Junio C Hamano
6249ce2d1b Merge branch 'ds/sparse-checkout-requires-per-worktree-config'
"git sparse-checkout" wants to work with per-worktree configuration,
but did not work well in a worktree attached to a bare repository.

* ds/sparse-checkout-requires-per-worktree-config:
  config: make git_configset_get_string_tmp() private
  worktree: copy sparse-checkout patterns and config on add
  sparse-checkout: set worktree-config correctly
  config: add repo_config_set_worktree_gently()
  worktree: create init_worktree_config()
  Documentation: add extensions.worktreeConfig details
2022-02-25 15:47:33 -08:00
Junio C Hamano
dab1b7905d The seventh batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 16:58:13 -08:00
Junio C Hamano
66633f25c6 Merge branch 'bc/clarify-eol-attr'
Documentation update

* bc/clarify-eol-attr:
  doc: clarify interaction between 'eol' and text=auto
2022-02-23 16:58:04 -08:00
Jonathan Tan
290eada0ac ls-files: support --recurse-submodules --stage
e77aa336f1 ("ls-files: optionally recurse into submodules", 2016-10-10)
taught ls-files the --recurse-submodules argument, but only in a limited
set of circumstances. In particular, --stage was unsupported, perhaps
because there was no repo_find_unique_abbrev(), which was only
introduced in 8bb95572b0 ("sha1-name.c: add
repo_find_unique_abbrev_r()", 2019-04-16). This function is needed for
using --recurse-submodules with --stage.

Now that we have repo_find_unique_abbrev(), teach support for this
combination of arguments.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 16:41:55 -08:00
Ævar Arnfjörð Bjarmason
1ce590133b help: add --no-[external-commands|aliases] for use with --all
Add the ability to only emit git's own usage information under
--all. This also allows us to extend the "test_section_spacing" tests
added in a preceding commit to test "git help --all"
output.

Previously we could not do that, as the tests might find a git-*
command in the "$PATH", which would make the output differ from one
setup to another.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 13:41:37 -08:00
Ævar Arnfjörð Bjarmason
5e8068b74d help: correct usage & behavior of "git help --all"
Do the same for the "--all" option that I did for "--guides" in
9856ea6785 (help: correct usage & behavior of "git help --guides",
2021-09-22). I.e. we've documented it as ignoring non-option
arguments, let's have it error out instead.

As with other changes made in 62f035aee3 (Merge branch
'ab/help-config-vars', 2021-10-13) this is technically a change in
behavior, but in practice it's just a bug fix. We were ignoring this
before, but by erroring we can simplify our documentation and
synopsis, as well as avoid user confusion as they wonder what the
difference between e.g. "git help --all" and "git help --all status"
is (there wasn't any difference).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 13:41:37 -08:00
Ævar Arnfjörð Bjarmason
4bf5cdab8e help doc: add missing "]" to "[-a|--all]"
Add a missing "]" to documentation added in 63eae83f8f (help: add "-a
--verbose" to list all commands with synopsis, 2018-05-20). This made
it seem as though "--[no-]verbose" can only be provided with "--all",
not "-a". The corresponding usage information in the C
code ("builtin_help_usage") does not have the same problem.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 13:41:36 -08:00
Han-Wen Nienhuys
fa48de62ac Documentation: object_id_len goes up to 31
The value is stored in a 5-bit field, so we can't support more without
a format version upgrade.

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 13:36:26 -08:00
Derrick Stolee
07d85380b2 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the last of multiple changes to git-worktree.txt, starting at
the LIST OUTPUT FORMAT section.

The EXAMPLES section has an instance of "working tree" that must stay as
it is, because it is not talking about a worktree, but an example of why
a user might want to create a worktree.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
f13a146c81 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the sixth of multiple changes to git-worktree.txt, restricted to
the DETAILS section.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
7b215826f3 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the fifth of multiple changes to git-worktree.txt, restricted to
the CONFIGURATION FILE section.

While here, clear up some language to improve readability.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
a777d4c750 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the fourth of multiple changes to git-worktree.txt, restricted
to the REFS section.

This section previously described "per working tree" refs but they are
now replaced with "per-worktree" refs, which matches the definition in
glossary-content.txt.

The first paragraph of this section was also a bit confusing, so it is
cleaned up to make it easier to understand.

Helped-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
6036be1458 worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the third of multiple changes to git-worktree.txt, restricted to
the OPTIONS section.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:42 -08:00
Derrick Stolee
599701441e worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the second of multiple changes to git-worktree.txt, restricted
to the COMMANDS section.

There is some language around the movement of "the working tree of a
linked worktree" which is used once, but the remaining uses are left as
just moving "a linked worktree" for brevity.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:41 -08:00
Derrick Stolee
c57bf8ce9e worktree: use 'worktree' over 'working tree'
It is helpful to distinguish between a 'working tree' and a 'worktree'.
A worktree contains a working tree plus additional metadata. This
metadata includes per-worktree refs and worktree-specific config.

This is the first of multiple changes to git-worktree.txt, restricted to
the DESCRIPTION section.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-23 12:24:41 -08:00
Ævar Arnfjörð Bjarmason
56a29d2c97 C99: remove hardcoded-out !HAVE_VARIADIC_MACROS code
Remove the "else" branches of the HAVE_VARIADIC_MACROS macro, which
have been unconditionally omitted since 765dc16888 (git-compat-util:
always enable variadic macros, 2021-01-28).

Since were always omitted, anyone trying to use a compiler without
variadic macro support to compile a git since version
git v2.31.0 or later would have had a compilation error. 10 months
across a few releases since then should have been enough time for
anyone who cared to run into that and report the issue.

In addition to that, for anyone unsetting HAVE_VARIADIC_MACROS we've
been emitting extremely verbose warnings since at least
ee4512ed48 (trace2: create new combined trace facility,
2019-02-22). That's because there is no such thing as a
"region_enter_printf" or "region_leave_printf" format, so at least
under GCC and Clang everything that includes trace.h (almost every
file) emits a couple of warnings about that.

There's a large benefit to being able to have a hard dependency rely
on variadic macros, the code surrounding usage.c is hard to maintain
if we need to write two implementations of everything, and by relying
on "__FILE__" and "__LINE__" along with "__VA_ARGS__" we can in the
future make error(), die() etc. log where they were called from. We've
also recently merged d67fc4bf0b (Merge branch 'bc/require-c99',
2021-12-10) which further cements our hard dependency on C99.

So let's delete the fallback code, and update our CodingGuidelines to
note that we depend on this. The added bullet-point starts with
lower-case for consistency with other bullet-points in that section.

The diff in "trace.h" is relatively hard to read, since we need to
retain the existing API docs, which were comments on the code used if
HAVE_VARIADIC_MACROS was not defined.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-21 19:14:19 -08:00
Junio C Hamano
e6ebfd0e8c The sixth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-18 13:53:30 -08:00
Junio C Hamano
9a1d16989f Merge branch 'jc/glossary-worktree'
"working tree" and "per-worktree ref" were in glossary, but
"worktree" itself wasn't, which has been corrected.

* jc/glossary-worktree:
  glossary: describe "worktree"
2022-02-18 13:53:30 -08:00
Junio C Hamano
5cc9522b15 Merge branch 'gc/branch-recurse-submodules'
"git branch" learned the "--recurse-submodules" option.

* gc/branch-recurse-submodules:
  branch.c: use 'goto cleanup' in setup_tracking() to fix memory leaks
  branch: add --recurse-submodules option for branch creation
  builtin/branch: consolidate action-picking logic in cmd_branch()
  branch: add a dry_run parameter to create_branch()
  branch: make create_branch() always create a branch
  branch: move --set-upstream-to behavior to dwim_and_setup_tracking()
2022-02-18 13:53:29 -08:00
John Cai
440c705ea6 cat-file: add --batch-command mode
Add a new flag --batch-command that accepts commands and arguments
from stdin, similar to git-update-ref --stdin.

At GitLab, we use a pair of long running cat-file processes when
accessing object content. One for iterating over object metadata with
--batch-check, and the other to grab object contents with --batch.

However, if we had --batch-command, we wouldn't need to keep both
processes around, and instead just have one --batch-command process
where we can flip between getting object info, and getting object
contents. Since we have a pair of cat-file processes per repository,
this means we can get rid of roughly half of long lived git cat-file
processes. Given there are many repositories being accessed at any given
time, this can lead to huge savings.

git cat-file --batch-command

will enter an interactive command mode whereby the user can enter in
commands and their arguments that get queued in memory:

<command1> [arg1] [arg2] LF
<command2> [arg1] [arg2] LF

When --buffer mode is used, commands will be queued in memory until a
flush command is issued that execute them:

flush LF

The reason for a flush command is that when a consumer process (A)
talks to a git cat-file process (B) and interactively writes to and
reads from it in --buffer mode, (A) needs to be able to control when
the buffer is flushed to stdout.

Currently, from (A)'s perspective, the only way is to either

1. kill (B)'s process
2. send an invalid object to stdin.

1. is not ideal from a performance perspective as it will require
spawning a new cat-file process each time, and 2. is hacky and not a
good long term solution.

With this mechanism of queueing up commands and letting (A) issue a
flush command, process (A) can control when the buffer is flushed and
can guarantee it will receive all of the output when in --buffer mode.
--batch-command also will not allow (B) to flush to stdout until a flush
is received.

This patch adds the basic structure for adding command which can be
extended in the future to add more commands. It also adds the following
two commands (on top of the flush command):

contents <object> LF
info <object> LF

The contents command takes an <object> argument and prints out the object
contents.

The info command takes an <object> argument and prints out the object
metadata.

These can be used in the following way with --buffer:

info <object> LF
contents <object> LF
contents <object> LF
info <object> LF
flush LF
info <object> LF
flush LF

When used without --buffer:

info <object> LF
contents <object> LF
contents <object> LF
info <object> LF
info <object> LF

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: John Cai <johncai86@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-18 11:21:46 -08:00
Junio C Hamano
e2ac9141e6 The fifth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-17 16:25:06 -08:00
Junio C Hamano
a4ec347888 Merge branch 'po/doc-check-ignore-markup-fix'
Typofix.

* po/doc-check-ignore-markup-fix:
  doc: check-ignore: code-quote an exclamation mark
2022-02-17 16:25:05 -08:00
Junio C Hamano
2f45f3e2bc Merge branch 'vd/sparse-clean-etc'
"git update-index", "git checkout-index", and "git clean" are
taught to work better with the sparse checkout feature.

* vd/sparse-clean-etc:
  update-index: reduce scope of index expansion in do_reupdate
  update-index: integrate with sparse index
  update-index: add tests for sparse-checkout compatibility
  checkout-index: integrate with sparse index
  checkout-index: add --ignore-skip-worktree-bits option
  checkout-index: expand sparse checkout compatibility tests
  clean: integrate with sparse index
  reset: reorder wildcard pathspec conditions
  reset: fix validation in sparse index test
2022-02-17 16:25:05 -08:00
Junio C Hamano
708cbef33a Merge branch 'jz/rev-list-exclude-first-parent-only'
"git log" and friends learned an option --exclude-first-parent-only
to propagate UNINTERESTING bit down only along the first-parent
chain, just like --first-parent option shows commits that lack the
UNINTERESTING bit only along the first-parent chain.

* jz/rev-list-exclude-first-parent-only:
  git-rev-list: add --exclude-first-parent-only flag
2022-02-17 16:25:05 -08:00
Junio C Hamano
45fe28c951 The fourth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-16 15:14:30 -08:00
Junio C Hamano
b9f791aee6 Merge branch 'js/no-more-legacy-stash'
Removal of unused code and doc.

* js/no-more-legacy-stash:
  stash: stop warning about the obsolete `stash.useBuiltin` config setting
  stash: remove documentation for `stash.useBuiltin`
  add: remove support for `git-legacy-stash`
  git-sh-setup: remove remnant bits referring to `git-legacy-stash`
2022-02-16 15:14:30 -08:00
Junio C Hamano
9a160990ef Merge branch 'js/diff-filter-negation-fix'
"git diff --diff-filter=aR" is now parsed correctly.

* js/diff-filter-negation-fix:
  diff-filter: be more careful when looking for negative bits
  diff.c: move the diff filter bits definitions up a bit
  docs(diff): lose incorrect claim about `diff-files --diff-filter=A`
2022-02-16 15:14:30 -08:00
Junio C Hamano
70ff41ffcf Merge branch 'en/fetch-negotiation-default-fix'
Interaction between fetch.negotiationAlgorithm and
feature.experimental configuration variables has been corrected.

* en/fetch-negotiation-default-fix:
  repo-settings: rename the traditional default fetch.negotiationAlgorithm
  repo-settings: fix error handling for unknown values
  repo-settings: fix checking for fetch.negotiationAlgorithm=default
2022-02-16 15:14:30 -08:00
Junio C Hamano
f2cb46a6b3 Merge branch 'tb/midx-bitmap-corruption-fix'
A bug that made multi-pack bitmap and the object order out-of-sync,
making the .midx data corrupt, has been fixed.

* tb/midx-bitmap-corruption-fix:
  pack-bitmap.c: gracefully fallback after opening pack/MIDX
  midx: read `RIDX` chunk when present
  t/lib-bitmap.sh: parameterize tests over reverse index source
  t5326: move tests to t/lib-bitmap.sh
  t5326: extract `test_rev_exists`
  t5326: drop unnecessary setup
  pack-revindex.c: instrument loading on-disk reverse index
  midx.c: make changing the preferred pack safe
  t5326: demonstrate bitmap corruption after permutation
2022-02-16 15:14:29 -08:00
Junio C Hamano
90b7153806 Merge branch 'en/remerge-diff'
"git log --remerge-diff" shows the difference from mechanical merge
result and the result that is actually recorded in a merge commit.

* en/remerge-diff:
  diff-merges: avoid history simplifications when diffing merges
  merge-ort: mark conflict/warning messages from inner merges as omittable
  show, log: include conflict/warning messages in --remerge-diff headers
  diff: add ability to insert additional headers for paths
  merge-ort: format messages slightly different for use in headers
  merge-ort: mark a few more conflict messages as omittable
  merge-ort: capture and print ll-merge warnings in our preferred fashion
  ll-merge: make callers responsible for showing warnings
  log: clean unneeded objects during `log --remerge-diff`
  show, log: provide a --remerge-diff capability
2022-02-16 15:14:29 -08:00
Junio C Hamano
dd77ff8181 Merge branch 'll/doc-mktree-typofix'
Typofix.

* ll/doc-mktree-typofix:
  fix typo in git-mktree.txt
2022-02-16 15:14:26 -08:00
brian m. carlson
6a5678f257 doc: clarify interaction between 'eol' and text=auto
The `eol` takes effect on text files only when the index has the
contents in LF line endings.  Paths with contents in CRLF line
endings in the index may become dirty unless text=auto.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-14 13:01:25 -08:00
Junio C Hamano
b80121027d The third batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-11 16:56:01 -08:00
Junio C Hamano
83760938bd Merge branch 'jc/doc-log-messages'
Update the contributor-facing documents on proposed log messages.

* jc/doc-log-messages:
  SubmittingPatches: explain why we care about log messages
  CodingGuidelines: hint why we value clearly written log messages
  SubmittingPatches: write problem statement in the log in the present tense
2022-02-11 16:55:58 -08:00
Junio C Hamano
c46452eb98 Merge branch 'gh/doc-typos'
Typofix.

* gh/doc-typos:
  Documentation/config/pgp.txt: add missing apostrophe
  Documentation/config/pgp.txt: replace stray <TAB> character with <SPC>
2022-02-11 16:55:58 -08:00
Junio C Hamano
8db2f665e1 Merge branch 'bc/clarify-eol-attr'
Doc and test update around the eol attribute.

* bc/clarify-eol-attr:
  docs: correct documentation about eol attribute
  t0027: add tests for eol without text in .gitattributes
2022-02-11 16:55:57 -08:00
Junio C Hamano
2df5387ed0 glossary: describe "worktree"
We have description on "per worktree ref", but "worktree" is not
described in the glossary.  We do have "working tree", though.

Casually put, a "working tree" is what your editor and compiler
interacts with.  "worktree" is a mechanism to allow one or more
"working tree"s to be attached to a repository and used to check out
different commits and branches independently, which includes not
just a "working tree" but also repository metadata like HEAD, the
index to support simultaneous use of them.  Historically, we used
these terms interchangeably but we have been trying to use "working
tree" when we mean it, instead of "worktree".

Most of the existing references to "working tree" in the glossary do
refer primarily to the working tree portion, except for one that
said refs like HEAD and refs/bisect/* are per "working tree", but it
is more precise to say they are per "worktree".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-09 18:34:41 -08:00
Josh Steadmon
f05da2b48b clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.

To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.

This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.

Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.

[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
	2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
	2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-09 15:38:36 -08:00
Junio C Hamano
2b9c120970 The second batch for 2.36
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-09 14:21:18 -08:00
Junio C Hamano
c70bc338e9 Merge branch 'ab/config-based-hooks-2'
More "config-based hooks".

* ab/config-based-hooks-2:
  run-command: remove old run_hook_{le,ve}() hook API
  receive-pack: convert push-to-checkout hook to hook.h
  read-cache: convert post-index-change to use hook.h
  commit: convert {pre-commit,prepare-commit-msg} hook to hook.h
  git-p4: use 'git hook' to run hooks
  send-email: use 'git hook run' for 'sendemail-validate'
  git hook run: add an --ignore-missing flag
  hooks: convert worktree 'post-checkout' hook to hook library
  hooks: convert non-worktree 'post-checkout' hook to hook library
  merge: convert post-merge to use hook.h
  am: convert applypatch-msg to use hook.h
  rebase: convert pre-rebase to use hook.h
  hook API: add a run_hooks_l() wrapper
  am: convert {pre,post}-applypatch to use hook.h
  gc: use hook library for pre-auto-gc hook
  hook API: add a run_hooks() wrapper
  hook: add 'run' subcommand
2022-02-09 14:21:00 -08:00
Junio C Hamano
d9976b1845 Merge branch 'jc/name-rev-stdin'
"git name-rev --stdin" does not behave like usual "--stdin" at
all.  Start the process of renaming it to "--annotate-stdin".

* jc/name-rev-stdin:
  name-rev.c: use strbuf_getline instead of limited size buffer
  name-rev: deprecate --stdin in favor of --annotate-stdin
2022-02-09 14:21:00 -08:00
Junio C Hamano
472a219f8d Merge branch 'gc/fetch-negotiate-only-early-return'
"git fetch --negotiate-only" is an internal command used by "git
push" to figure out which part of our history is missing from the
other side.  It should never recurse into submodules even when
fetch.recursesubmodules configuration variable is set, nor it
should trigger "gc".  The code has been tightened up to ensure it
only does common ancestry discovery and nothing else.

* gc/fetch-negotiate-only-early-return:
  fetch: help translators by reusing the same message template
  fetch --negotiate-only: do not update submodules
  fetch: skip tasks related to fetching objects
  fetch: use goto cleanup in cmd_fetch()
2022-02-09 14:20:59 -08:00
Junio C Hamano
e704a4486e Merge branch 'tl/doc-cli-options-first'
We explain that revs come first before the pathspec among command
line arguments, but did not spell out that dashed options come
before other args, which has been corrected.

* tl/doc-cli-options-first:
  git-cli.txt: clarify "options first and then args"
2022-02-09 14:20:59 -08:00
Junio C Hamano
13ce8f9f14 Merge branch 'jt/conditional-config-on-remote-url'
The conditional inclusion mechanism of configuration files using
"[includeIf <condition>]" learns to base its decision on the
URL of the remote repository the repository interacts with.

* jt/conditional-config-on-remote-url:
  config: include file if remote URL matches a glob
  config: make git_config_include() static
2022-02-09 14:20:59 -08:00
Derrick Stolee
7316dc5f6f sparse-checkout: set worktree-config correctly
`git sparse-checkout set/init` enables worktree-specific
configuration[*] by setting extensions.worktreeConfig=true, but neglects
to perform the additional necessary bookkeeping of relocating
`core.bare=true` and `core.worktree` from $GIT_COMMON_DIR/config to
$GIT_COMMON_DIR/config.worktree, as documented in git-worktree.txt. As a
result of this oversight, these settings, which are nonsensical for
secondary worktrees, can cause Git commands to incorrectly consider a
worktree bare (in the case of `core.bare`) or operate on the wrong
worktree (in the case of `core.worktree`). Fix this problem by taking
advantage of the recently-added init_worktree_config() which enables
`extensions.worktreeConfig` and takes care of necessary bookkeeping.

While at it, for backward-compatibility reasons, also stop upgrading the
repository format to "1" since doing so is (unintentionally) not
required to take advantage of `extensions.worktreeConfig`, as explained
by 11664196ac ("Revert "check_repository_format_gently(): refuse
extensions for old repositories"", 2020-07-15).

[*] The main reason to use worktree-specific config for the
sparse-checkout builtin was to avoid enabling sparse-checkout patterns
in one and causing a loss of files in another. If a worktree does not
have a sparse-checkout patterns file, then the sparse-checkout logic
will not kick in on that worktree.

Reported-by: Sean Allred <allred.sean@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-08 09:49:20 -08:00
Derrick Stolee
5c11c0d52c Documentation: add extensions.worktreeConfig details
The extensions.worktreeConfig extension was added in 58b284a (worktree:
add per-worktree config files, 2018-10-21) and was somewhat documented
in Documentation/git-config.txt. However, the extensions.worktreeConfig
value was not specified further in the list of possible config keys. The
location of the config.worktree file is not specified, and there are
some precautions that should be mentioned clearly, but are only
mentioned in git-worktree.txt.

Expand the documentation to help users discover the complexities of
extensions.worktreeConfig by adding details and cross links in these
locations (relative to Documentation/):

- config/extensions.txt
- git-config.txt
- git-worktree.txt

The updates focus on items such as

* $GIT_DIR/config.worktree takes precedence over $GIT_COMMON_DIR/config.

* The core.worktree and core.bare=true settings are incorrect to have in
  the common config file when extensions.worktreeConfig is enabled.

* The sparse-checkout settings core.sparseCheckout[Cone] are recommended
  to be set in the worktree config.

As documented in 11664196ac ("Revert "check_repository_format_gently():
refuse extensions for old repositories"", 2020-07-15), this extension
must be considered regardless of the repository format version for
historical reasons.

A future change will update references to extensions.worktreeConfig
within git-sparse-checkout.txt, but a behavior change is needed before
making those updates.

Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-08 09:49:20 -08:00
Junio C Hamano
38062e73e0 The first batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 09:43:36 -08:00
Junio C Hamano
008028a910 Merge branch 'ab/cat-file'
Assorted updates to "git cat-file", especially "-h".

* ab/cat-file:
  cat-file: s/_/-/ in typo'd usage_msg_optf() message
  cat-file: don't whitespace-pad "(...)" in SYNOPSIS and usage output
  cat-file: use GET_OID_ONLY_TO_DIE in --(textconv|filters)
  object-name.c: don't have GET_OID_ONLY_TO_DIE imply *_QUIETLY
  cat-file: correct and improve usage information
  cat-file: fix remaining usage bugs
  cat-file: make --batch-all-objects a CMDMODE
  cat-file: move "usage" variable to cmd_cat_file()
  cat-file docs: fix SYNOPSIS and "-h" output
  parse-options API: add a usage_msg_optf()
  cat-file tests: test messaging on bad objects/paths
  cat-file tests: test bad usage
2022-02-05 09:42:31 -08:00
Glen Choo
961b130d20 branch: add --recurse-submodules option for branch creation
To improve the submodules UX, we would like to teach Git to handle
branches in submodules. Start this process by teaching "git branch" the
--recurse-submodules option so that "git branch --recurse-submodules
topic" will create the `topic` branch in the superproject and its
submodules.

Although this commit does not introduce breaking changes, it does not
work well with existing --recurse-submodules commands because "git
branch --recurse-submodules" writes to the submodule ref store, but most
commands only consider the superproject gitlink and ignore the submodule
ref store. For example, "git checkout --recurse-submodules" will check
out the commits in the superproject gitlinks (and put the submodules in
detached HEAD) instead of checking out the submodule branches.

Because of this, this commit introduces a new configuration value,
`submodule.propagateBranches`. The plan is for Git commands to
prioritize submodule ref store information over superproject gitlinks if
this value is true. Because "git branch --recurse-submodules" writes to
submodule ref stores, for the sake of clarity, it will not function
unless this configuration value is set.

This commit also includes changes that support working with submodules
from a superproject commit because "branch --recurse-submodules" (and
future commands) need to read .gitmodules and gitlinks from the
superproject commit, but submodules are typically read from the
filesystem's .gitmodules and the index's gitlinks. These changes are:

* add a submodules_of_tree() helper that gives the relevant
  information of an in-tree submodule (e.g. path and oid) and
  initializes the repository
* add is_tree_submodule_active() by adding a treeish_name parameter to
  is_submodule_active()
* add the "submoduleNotUpdated" advice to advise users to update the
  submodules in their trees

Incidentally, fix an incorrect usage string that combined the 'list'
usage of git branch (-l) with the 'create' usage; this string has been
incorrect since its inception, a8dfd5eac4 (Make builtin-branch.c use
parse_options., 2007-10-07).

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Glen Choo <chooglen@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-04 08:16:39 -08:00
Philip Oakley
9325285df4 doc: check-ignore: code-quote an exclamation mark
The plain quoted exclamation mark renders as italics in the
Windows pdf help manual.

Fix this with back-tick quoting and surrounding double quotes
as exemplified by the gitignore.txt guide.

While at it, fix  the surrounding double quotes for the other
special characters usages.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-03 11:13:49 -08:00
Liginity Lee
45d0212a71 fix typo in git-mktree.txt
fix a typo: change "as" to "a".

Signed-off-by: Liginity Lee <liginity@outlook.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-02 14:50:09 -08:00
Elijah Newren
db757e8b8d show, log: provide a --remerge-diff capability
When this option is specified, we remerge all (two parent) merge commits
and diff the actual merge commit to the automatically created version,
in order to show how users removed conflict markers, resolved the
different conflict versions, and potentially added new changes outside
of conflict regions in order to resolve semantic merge problems (or,
possibly, just to hide other random changes).

This capability works by creating a temporary object directory and
marking it as the primary object store.  This makes it so that any blobs
or trees created during the automatic merge are easily removable
afterwards by just deleting all objects from the temporary object
directory.

There are a few ways that this implementation is suboptimal:
  * `log --remerge-diff` becomes slow, because the temporary object
    directory can fill with many loose objects while running
  * the log output can be muddied with misplaced "warning: cannot merge
    binary files" messages, since ll-merge.c unconditionally writes those
    messages to stderr while running instead of allowing callers to
    manage them.
  * important conflict and warning messages are simply dropped; thus for
    conflicts like modify/delete or rename/rename or file/directory which
    are not representable with content conflict markers, there may be no
    way for a user of --remerge-diff to know that there had been a
    conflict which was resolved (and which possibly motivated other
    changes in the merge commit).
  * when fixing the previous issue, note that some unimportant conflict
    and warning messages might start being included.  We should instead
    make sure these remain dropped.
Subsequent commits will address these issues.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-02 10:02:27 -08:00
Elijah Newren
714edc620c repo-settings: rename the traditional default fetch.negotiationAlgorithm
Give the traditional default fetch.negotiationAlgorithm the name
'consecutive'.  Also allow a choice of 'default' to have Git decide
between the choices (currently, picking 'skipping' if
feature.experimental is true and 'consecutive' otherwise).  Update the
documentation accordingly.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-02 09:36:17 -08:00
Junio C Hamano
09e0be130d Merge branch 'js/branch-track-inherit' into gc/branch-recurse-submodules
* js/branch-track-inherit:
  branch,checkout: fix --track documentation
  branch,checkout: fix --track usage strings
  config: require lowercase for branch.*.autosetupmerge
  branch: add flags and config to inherit tracking
  branch: accept multiple upstream branches for tracking
2022-01-31 10:37:44 -08:00
Junio C Hamano
5d01301f2b Git 2.35.1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4fA2sf7nIh/HeOzvsLXohpav5ssFAmH0jvsACgkQsLXohpav
 5ssJ8w//aAEXTNfDH15U+PwZ95OxAGV0mT4JCSK7fjB7kjqAMRLziVGPrlXzGTjc
 wxKVd5nvQAjB86P+sKSC/6AUx38QXlsurtUb+yhRMcogXm2ESdHTQ8IsofCo4yRq
 zDAQyf8RPKJFygMdVDjfXOIRP5NXTqVTAa67QihZOuqy5PvcEh7IRScd0f5pIsyD
 RO55G7eC9KZZJH5yrz0OpEj3xHxqZL+tO22ue777YUagUxS8Kt+183Fyc73H/eBz
 69TxEc76t54AjNNFgghTiF00zirl7qv/lRCYldqFfpoutxtAHdjuiy7E+h/NhY9r
 fkrVVGA3ms//ln32SU26Jm3zxEu31PSrCqP2KzdTdVO/I97jWnWytEZSWvVCZB6m
 BZDG1yKrLSlH48SqgxnPLGMeeqA1kldqI9e3lgpTMPUJO7xjXi0i10lFhL4N+pPX
 VMsjxlcaV6A1ey0CMCMtBZ2g8sJkKUuSA/HJrhhXBJkl88oWUAvPC6zIBG2xrDHy
 1qc4cAFNw4zwJylnyUYXbkeRn1/Z6lg8JeUfdWdZktjmL0ZbgLTM+CWKJhx/1WN2
 hu5DEFa7yH4IS4GW6W6N14mxTrMC2IehqKLhHd/0L8cqEVsnUb0xWLa0zChAdM7s
 G1XpNQSJxyBj74tJ8lMB+0P1cpqQF4ZkVk2Tx8e0xyVEaa5Vljg=
 =/DjY
 -----END PGP SIGNATURE-----

Sync with Git 2.35.1
2022-01-28 16:58:04 -08:00
Junio C Hamano
90fb70e458 Name the next one 2.36 to prepare for 2.35.1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-28 16:57:10 -08:00
Junio C Hamano
4c53a8c20f Git 2.35.1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-28 16:48:42 -08:00
Johannes Schindelin
d843e319f8 docs(diff): lose incorrect claim about diff-files --diff-filter=A
Originally, before we had `--intent-to-add`, there was no way that `git
diff-files` could see added files: if a file did not exist in the index,
`git diff-files` would not show it because it looks only at worktree
files when there is an index entry at the same path.

We used this example in the documentation of the diff options to explain
that not every `--diff-filter=<option>` has an effect in all scenarios.

Even when we added `--intent-to-add`, the comment was still correct,
because initially we showed such files as modified instead of added.

However, when that bug was fixed in feea6946a5 (diff-files: treat
"i-t-a" files as "not-in-index", 2020-06-20), the comment in the
documentation became incorrect.

Let's just remove it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-28 10:18:17 -08:00
Johannes Schindelin
deeaf5ee07 stash: remove documentation for stash.useBuiltin
In 8a2cd3f512 (stash: remove the stash.useBuiltin setting, 2020-03-03),
we removed the setting, and for a couple of major versions, we still
documented the setting, telling users that it is gone.

We can now safely remove even the documentation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 18:00:37 -08:00
Junio C Hamano
cdba0295b0 SubmittingPatches: explain why we care about log messages
Extend the "describe your changes well" section to cover whom we are
trying to help by doing so in the first place.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 17:50:04 -08:00
Junio C Hamano
607817a3c8 CodingGuidelines: hint why we value clearly written log messages
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 17:50:04 -08:00
Junio C Hamano
fa1101afb6 SubmittingPatches: write problem statement in the log in the present tense
We give a guidance for proposed log message to write problem
statement first, followed by the reasoning behind, and recipe for,
the solution.  Clarify that we describe the situation _before_ the
proposed patch is applied in the present tense (not in the past
tense e.g. "we used to do X, but thanks to this commit we now do Y")
for consistency.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 17:50:04 -08:00
Taylor Blau
95e8383bac midx.c: make changing the preferred pack safe
The previous patch demonstrates a bug where a MIDX's auxiliary object
order can become out of sync with a MIDX bitmap.

This is because of two confounding factors:

  - First, the object order is stored in a file which is named according
    to the multi-pack index's checksum, and the MIDX does not store the
    object order. This means that the object order can change without
    altering the checksum.

  - But the .rev file is moved into place with finalize_object_file(),
    which link(2)'s the file into place instead of renaming it. For us,
    that means that a modified .rev file will not be moved into place if
    MIDX's checksum was unchanged.

This fix is to force the MIDX's checksum to change when the preferred
pack changes but the set of packs contained in the MIDX does not. In
other words, when the object order changes, the MIDX's checksum needs to
change with it (regardless of whether the MIDX is tracking the same or
different packs).

This prevents a race whereby changing the object order (but not the
packs themselves) enables a reader to see the new .rev file with the old
MIDX, or similarly seeing the new bitmap with the old object order.

But why can't we just stop hardlinking the .rev into place instead
adding additional data to the MIDX? Suppose that's what we did. Then
when we go to generate the new bitmap, we'll load the old MIDX bitmap,
along with the MIDX that it references. That's fine, since the new MIDX
isn't moved into place until after the new bitmap is generated. But the
new object order *has* been moved into place. So we'll read the old
bitmaps in the new order when generating the new bitmap file, meaning
that without this secondary change, bitmap generation itself would
become a victim of the race described here.

This can all be prevented by forcing the MIDX's checksum to change when
the object order does. By embedding the entire object order into the
MIDX, we do just that. That is, the MIDX's checksum will change in
response to any perturbation of the underlying object order. In t5326,
this will cause the MIDX's checksum to update (even without changing the
set of packs in the MIDX), preventing the stale read problem.

Note that this makes it safe to continue to link(2) the MIDX .rev file
into place, since it is now impossible to have a .rev file that is
out-of-sync with the MIDX whose checksum it references. (But we will do
away with MIDX .rev files later in this series anyway, so this is
somewhat of a moot point).

In theory, it is possible to store a "fingerprint" of the full object
order here, so long as that fingerprint changes at least as often as the
full object order does. Some possibilities here include storing the
identity of the preferred pack, along with the mtimes of the
non-preferred packs in a consistent order. But storing a limited part of
the information makes it difficult to reason about whether or not there
are gaps between the two that would cause us to get bitten by this bug
again.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Reviewed-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-27 12:07:52 -08:00
Greg Hurrell
cbac0076ef Documentation/config/pgp.txt: add missing apostrophe
Add an apostrophe to "signatures" to indicate the possessive
relationship in "the signature's creation".

Signed-off-by: Greg Hurrell <greg@hurrell.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-26 18:31:59 -08:00
Greg Hurrell
7838d9c2a9 Documentation/config/pgp.txt: replace stray <TAB> character with <SPC>
Specifically, replace the tab between "the" and "first" with a space.

Signed-off-by: Greg Hurrell <greg@hurrell.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-26 18:31:59 -08:00
Junio C Hamano
89bece5c8c Git 2.35
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-24 09:25:25 -08:00
Junio C Hamano
297ca895a2 Merge branch 'js/branch-track-inherit'
"git branch -h" incorrectly said "--track[=direct|inherit]",
implying that "--trackinherit" is a valid option, which has been
corrected.
source: <3de40324bea6a1dd9bca2654721471e3809e87d8.1642538935.git.steadmon@google.com>
source: <c3c26192-aee9-185a-e559-b8735139e49c@web.de>

* js/branch-track-inherit:
  branch,checkout: fix --track documentation
2022-01-20 15:25:38 -08:00
René Scharfe
6327f0efed branch,checkout: fix --track documentation
Document that the accepted variants of the --track option are --track,
--track=direct, and --track=inherit.  The equal sign in the latter two
cannot be replaced with whitespace; in general optional arguments need
to be attached firmly to their option.

Put "direct" consistently before "inherit", if only for the reasons
that the former is the default, explained first in the documentation,
and comes before the latter alphabetically.

Mention both modes in the short help so that readers don't have to look
them up in the full documentation.  They are literal strings and thus
untranslatable.  PARSE_OPT_LITERAL_ARGHELP is inferred due to the pipe
and parenthesis characters, so we don't have to provide that flag
explicitly.

Mention that -t has the same effect as --track and --track=direct.
There is no way to specify inherit mode using the short option, because
short options generally don't accept optional arguments.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-20 11:07:51 -08:00
Glen Choo
386c076a86 fetch --negotiate-only: do not update submodules
`git fetch --negotiate-only` is an implementation detail of push
negotiation and, unlike most `git fetch` invocations, does not actually
update the main repository. Thus it should not update submodules even
if submodule recursion is enabled.

This is not just slow, it is wrong e.g. push negotiation with
"submodule.recurse=true" will cause submodules to be updated because it
invokes `git fetch --negotiate-only`.

Fix this by disabling submodule recursion if --negotiate-only was given.
Since this makes --negotiate-only and --recurse-submodules incompatible,
check for this invalid combination and die.

This does not use the "goto cleanup" introduced in the previous commit
because we want to recurse through submodules whenever a ref is fetched,
and this can happen without introducing new objects.

Signed-off-by: Glen Choo <chooglen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-18 16:22:58 -08:00
Jonathan Tan
399b198489 config: include file if remote URL matches a glob
This is a feature that supports config file inclusion conditional on
whether the repo has a remote with a URL that matches a glob.

Similar to my previous work on remote-suggested hooks [1], the main
motivation is to allow remote repo administrators to provide recommended
configs in a way that can be consumed more easily (e.g. through a
package installable by a package manager - it could, for example,
contain a file to be included conditionally and a post-install script
that adds the include directive to the system-wide config file).

In order to do this, Git reruns the config parsing mechanism upon
noticing the first URL-conditional include in order to find all remote
URLs, and these remote URLs are then used to determine if that first and
all subsequent includes are executed. Remote URLs are not allowed to be
configued in any URL-conditionally-included file.

[1] https://lore.kernel.org/git/cover.1623881977.git.jonathantanmy@google.com/

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Acked-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-18 13:55:53 -08:00
Teng Long
c11f95010c git-cli.txt: clarify "options first and then args"
There are some commands permit the user whether to provide options
first before args, or the reverse order. For example:

    git push --dry-run <remote> <ref>

And:

    git push <remote> <ref> --dry-run

Both of them is supported, but some commands do not, for instance:

     git ls-remote --heads <remote>

And:

     git ls-remote <remote> --heads

If <remote> only has one ref and it's name is "refs/heads/--heads", you
will get the same result, otherwise will not.This is because the former
in the second example will parse "--heads" as an "option" which means
to limit to only "refs/heads" when listing the remote references, the
latter treat "--heads" as an argument which means to filter the result
list with the given pattern.

Therefore, we want to specify a bit more in "gitcli.txt" about the way
we recommend and help to resolve the ambiguity around some git command
usage. The related disscussions locate at [1].

By the way, there are some issues with lowercase letters in the document,
which have been modified together.

[1] https://public-inbox.org/git/cover.1642129840.git.dyroneteng@gmail.com/

Signed-off-by: Teng Long <dyroneteng@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-17 11:42:25 -08:00
Elijah Newren
9023535bd3 Update documentation related to sparsity and the skip-worktree bit
Make several small updates, to address a few documentation issues
I spotted:
  * sparse-checkout focused on "patterns" even though the inputs (and
    outputs in the case of `list`) are directories in cone-mode
  * The description section of the sparse-checkout documentation
    was a bit sparse (no pun intended), and focused more on internal
    mechanics rather than end user usage.  This made sense in the
    early days when the command was even more experimental, but let's
    adjust a bit to try to make it more approachable to end users who
    may want to consider using it.  Keep the scary backward
    compatibility warning, though; we're still hard at work trying to
    fix up commands to behave reasonably in sparse checkouts.
  * both read-tree and update-index tried to describe how to use the
    skip-worktree bit, but both predated the sparse-checkout command.
    The sparse-checkout command is a far easier mechanism to use and
    for users trying to reduce the size of their working tree, we
    should recommend users to look at it instead.
  * The update-index documentation pointed out that assume-unchanged
    and skip-worktree sounded similar but had different purposes.
    However, it made no attempt to explain the differences, only to
    point out that they were different.  Explain the differences.
  * The update-index documentation focused much more on (internal?)
    implementation details than on end-user usage.  Try to explain
    its purpose better for users of update-index, rather than
    fellow developers trying to work with the SKIP_WORKTREE bit.
  * Clarify that when core.sparseCheckout=true, we treat a file's
    presence in the working tree as being an override to the
    SKIP_WORKTREE bit (i.e. in sparse checkouts when the file is
    present we ignore the SKIP_WORKTREE bit).

Note that this commit, like many touching documentation, is best viewed
with the `--color-words` option to diff/log.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-14 14:44:07 -08:00
Junio C Hamano
48609de3bf Merge branch 'vd/sparse-clean-etc' into en/present-despite-skipped
* vd/sparse-clean-etc:
  update-index: reduce scope of index expansion in do_reupdate
  update-index: integrate with sparse index
  update-index: add tests for sparse-checkout compatibility
  checkout-index: integrate with sparse index
  checkout-index: add --ignore-skip-worktree-bits option
  checkout-index: expand sparse checkout compatibility tests
  clean: integrate with sparse index
  reset: reorder wildcard pathspec conditions
  reset: fix validation in sparse index test
2022-01-13 13:50:44 -08:00
Victoria Dye
88078f543b checkout-index: add --ignore-skip-worktree-bits option
Update `checkout-index` to no longer refresh files that have the
`skip-worktree` bit set, exiting with an error if `skip-worktree` filenames
are directly provided to `checkout-index`. The newly-added
`--ignore-skip-worktree-bits` option provides a mechanism to replicate the
old behavior, checking out *all* files specified (even those with
`skip-worktree` enabled).

The ability to toggle whether files should be checked-out based on
`skip-worktree` already exists in `git checkout` and `git restore` (both of
which have an `--ignore-skip-worktree-bits` option). The change to, by
default, ignore `skip-worktree` files is especially helpful for
sparse-checkout; it prevents inadvertent creation of files outside the
sparse definition on disk and eliminates the need to expand a sparse index
when using the `--all` option.

Internal usage of `checkout-index` in `git stash` and `git filter-branch` do
not make explicit use of files with `skip-worktree` enabled, so
`--ignore-skip-worktree-bits` is not added to them.

Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Victoria Dye <vdye@github.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-13 13:49:45 -08:00
Junio C Hamano
1ffcbaa1a5 Last minute fixes before -rc1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-12 16:27:08 -08:00
Jerry Zhang
9d505b7b49 git-rev-list: add --exclude-first-parent-only flag
It is useful to know when a branch first diverged in history
from some integration branch in order to be able to enumerate
the user's local changes. However, these local changes can
include arbitrary merges, so it is necessary to ignore this
merge structure when finding the divergence point.

In order to do this, teach the "rev-list" family to accept
"--exclude-first-parent-only", which restricts the traversal
of excluded commits to only follow first parent links.

   -A-----E-F-G--main
     \   / /
      B-C-D--topic

In this example, the goal is to return the set {B, C, D} which
represents a topic branch that has been merged into main branch.
`git rev-list topic ^main` will end up returning no commits
since excluding main will end up traversing the commits on topic
as well. `git rev-list --exclude-first-parent-only topic ^main`
however will return {B, C, D} as desired.

Add docs for the new flag, and clarify the doc for --first-parent
to indicate that it applies to traversing the set of included
commits only.

Signed-off-by: Jerry Zhang <jerry@skydio.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-12 11:08:42 -08:00