Commit Graph

14872 Commits

Author SHA1 Message Date
Junio C Hamano
57a3b971e9 Merge branch 'fc/doc-checkout-markup-updates'
Doc mark-up update.

* fc/doc-checkout-markup-updates:
  doc: git-checkout: reorganize examples
  doc: git-checkout: trivial callout cleanup
2023-04-27 16:00:59 -07:00
Junio C Hamano
d6661e6843 Merge branch 'fc/doc-use-datestamp-in-commit'
Instead of the time the formatter was run, show the timestamp
recorded in the commit in the documentation.

* fc/doc-use-datestamp-in-commit:
  doc: set actual revdate for manpages
2023-04-27 16:00:59 -07:00
Junio C Hamano
849c8b3dbf Merge branch 'tb/pack-revindex-on-disk'
The on-disk reverse index that allows mapping from the pack offset
to the object name for the object stored at the offset has been
enabled by default.

* tb/pack-revindex-on-disk:
  t: invert `GIT_TEST_WRITE_REV_INDEX`
  config: enable `pack.writeReverseIndex` by default
  pack-revindex: introduce `pack.readReverseIndex`
  pack-revindex: introduce GIT_TEST_REV_INDEX_DIE_ON_DISK
  pack-revindex: make `load_pack_revindex` take a repository
  t5325: mark as leak-free
  pack-write.c: plug a leak in stage_tmp_packfiles()
2023-04-27 16:00:59 -07:00
Junio C Hamano
2807bd2c10 The thirteenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-25 13:56:20 -07:00
Junio C Hamano
c4c9d5586f Merge branch 'rj/send-email-validate-hook-count-messages'
The sendemail-validate validate hook learned to pass the total
number of input files and where in the sequence each invocation is
via environment variables.

* rj/send-email-validate-hook-count-messages:
  send-email: export patch counters in validate environment
2023-04-25 13:56:20 -07:00
Junio C Hamano
9ce9dea4e1 Sync with Git 2.40.1 2023-04-24 22:31:32 -07:00
Junio C Hamano
7580f92ffa The twelfth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-21 15:35:09 -07:00
Junio C Hamano
b64894c206 Merge branch 'ow/ref-filter-omit-empty'
"git branch --format=..." and "git format-patch --format=..."
learns "--omit-empty" to hide refs that whose formatting result
becomes an empty string from the output.

* ow/ref-filter-omit-empty:
  branch, for-each-ref, tag: add option to omit empty lines
2023-04-21 15:35:05 -07:00
Junio C Hamano
9e0d1aa495 Merge branch 'ah/format-patch-thread-doc'
Doc update.

* ah/format-patch-thread-doc:
  format-patch: correct documentation of --thread without an argument
2023-04-21 15:35:05 -07:00
Junio C Hamano
09a7b61c1d Merge branch 'fc/doc-stop-using-manversion'
Doc build simplification.

* fc/doc-stop-using-manversion:
  doc: simplify man version
2023-04-21 15:35:04 -07:00
Junio C Hamano
9c6990cca2 The eleventh batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-20 14:33:36 -07:00
Junio C Hamano
a4a4db8cf7 Merge branch 'gc/better-error-when-local-clone-fails-with-symlink'
"git clone --local" stops copying from an original repository that
has symbolic links inside its $GIT_DIR; an error message when that
happens has been updated.

* gc/better-error-when-local-clone-fails-with-symlink:
  clone: error specifically with --local and symlinked objects
2023-04-20 14:33:36 -07:00
Felipe Contreras
8dda6c3de2 doc: git-checkout: reorganize examples
The examples are an ordered list, however, they are complex enough that
a callout is inside example 1, and that confuses the parsers as the list
continuation (`+`) is unclear (are we continuing the previous list item,
or the previous callout?).

We could use an open block as the asciidoctor documentation suggests,
but that has a tiny formatting issue (a newline is missing).

To simplify things for everyone (the reader, the writer, and the parser)
let's use subsections.

After this change, the HTML documentation generated with asciidoc has
the right indentation.

Cc: Jeff King <peff@peff.net>
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-18 15:47:13 -07:00
Felipe Contreras
f8bc75a55e doc: git-checkout: trivial callout cleanup
The callouts are directly tied to the listing above, remove spaces to
make it clear they are one and the same.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-18 15:36:36 -07:00
Junio C Hamano
667fcf4e15 The tenth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-17 18:05:13 -07:00
Junio C Hamano
c232ebacb2 Merge branch 'fc/remove-header-workarounds-for-asciidoc'
Doc toolchain update to remove old workaround for AsciiDoc.

* fc/remove-header-workarounds-for-asciidoc:
  doc: asciidoc: remove custom header macro
2023-04-17 18:05:12 -07:00
Junio C Hamano
953823fcbf Merge branch 'la/mfc-markup-fix'
Documentation mark-up fix.

* la/mfc-markup-fix:
  MyFirstContribution: render literal *
2023-04-17 18:05:12 -07:00
Junio C Hamano
9d8370d445 Merge branch 'tk/mergetool-gui-default-config'
"git mergetool" and "git difftool" learns a new configuration
guiDefault to optionally favor configured guitool over non-gui-tool
automatically when $DISPLAY is set.

* tk/mergetool-gui-default-config:
  mergetool: new config guiDefault supports auto-toggling gui by DISPLAY
2023-04-17 18:05:11 -07:00
Johannes Schindelin
0d1bd1dfb3 Git 2.40.1
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:16:10 +02:00
Johannes Schindelin
3d3c11852c Sync with 2.39.3
* maint-2.39: (34 commits)
  Git 2.39.3
  Git 2.38.5
  Git 2.37.7
  Git 2.36.6
  Git 2.35.8
  Makefile: force -O0 when compiling with SANITIZE=leak
  Git 2.34.8
  Git 2.33.8
  Git 2.32.7
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  range-diff: use ssize_t for parsed "len" in read_patches()
  ...
2023-04-17 21:16:10 +02:00
Johannes Schindelin
9bbde12fee Git 2.39.3
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:16:08 +02:00
Johannes Schindelin
15628975cf Sync with 2.38.5
* maint-2.38: (32 commits)
  Git 2.38.5
  Git 2.37.7
  Git 2.36.6
  Git 2.35.8
  Git 2.34.8
  Git 2.33.8
  Git 2.32.7
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  ...
2023-04-17 21:16:08 +02:00
Johannes Schindelin
ec58344906 Git 2.38.5
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:16:07 +02:00
Johannes Schindelin
c96ecfe6a5 Sync with 2.37.7
* maint-2.37: (31 commits)
  Git 2.37.7
  Git 2.36.6
  Git 2.35.8
  Git 2.34.8
  Git 2.33.8
  Git 2.32.7
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  t0003: GETTEXT_POISON fix, part 1
  ...
2023-04-17 21:16:06 +02:00
Johannes Schindelin
d27ae36bbb Git 2.37.7
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:16:05 +02:00
Johannes Schindelin
1df551ce5c Sync with 2.36.6
* maint-2.36: (30 commits)
  Git 2.36.6
  Git 2.35.8
  Git 2.34.8
  Git 2.33.8
  Git 2.32.7
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  t0003: GETTEXT_POISON fix, part 1
  t0033: GETTEXT_POISON fix
  ...
2023-04-17 21:16:04 +02:00
Johannes Schindelin
ecaa3db171 Git 2.36.6
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:16:03 +02:00
Johannes Schindelin
62298def14 Sync with 2.35.8
* maint-2.35: (29 commits)
  Git 2.35.8
  Git 2.34.8
  Git 2.33.8
  Git 2.32.7
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  t0003: GETTEXT_POISON fix, part 1
  t0033: GETTEXT_POISON fix
  http: support CURLOPT_PROTOCOLS_STR
  ...
2023-04-17 21:16:02 +02:00
Johannes Schindelin
7380a72f6b Git 2.35.8
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:16:00 +02:00
Johannes Schindelin
8cd052ea53 Sync with 2.34.8
* maint-2.34: (28 commits)
  Git 2.34.8
  Git 2.33.8
  Git 2.32.7
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  t0003: GETTEXT_POISON fix, part 1
  t0033: GETTEXT_POISON fix
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  ...
2023-04-17 21:15:59 +02:00
Johannes Schindelin
abcb63fb70 Git 2.34.8
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:15:57 +02:00
Johannes Schindelin
d6e9f67a8e Sync with 2.33.8
* maint-2.33: (27 commits)
  Git 2.33.8
  Git 2.32.7
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  t0003: GETTEXT_POISON fix, part 1
  t0033: GETTEXT_POISON fix
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
  ...
2023-04-17 21:15:56 +02:00
Johannes Schindelin
3a19048ce4 Git 2.33.8
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:15:54 +02:00
Johannes Schindelin
bcd874d50f Sync with 2.32.7
* maint-2.32: (26 commits)
  Git 2.32.7
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  t0003: GETTEXT_POISON fix, part 1
  t0033: GETTEXT_POISON fix
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
  ci: install python on ubuntu
  ...
2023-04-17 21:15:52 +02:00
Johannes Schindelin
b8787a98db Git 2.32.7
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:15:51 +02:00
Johannes Schindelin
31f7fe5e34 Sync with 2.31.8
* maint-2.31: (25 commits)
  Git 2.31.8
  tests: avoid using `test_i18ncmp`
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  t0003: GETTEXT_POISON fix, part 1
  t0033: GETTEXT_POISON fix
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
  ci: install python on ubuntu
  ci: use the same version of p4 on both Linux and macOS
  ...
2023-04-17 21:15:49 +02:00
Johannes Schindelin
ea56f91275 Git 2.31.8
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:15:47 +02:00
Johannes Schindelin
b524e896b6 Sync with 2.30.9
* maint-2.30: (23 commits)
  Git 2.30.9
  gettext: avoid using gettext if the locale dir is not present
  apply --reject: overwrite existing `.rej` symlink if it exists
  http.c: clear the 'finished' member once we are done with it
  clone.c: avoid "exceeds maximum object size" error with GCC v12.x
  range-diff: use ssize_t for parsed "len" in read_patches()
  range-diff: handle unterminated lines in read_patches()
  range-diff: drop useless "offset" variable from read_patches()
  t5604: GETTEXT_POISON fix, conclusion
  t5604: GETTEXT_POISON fix, part 1
  t5619: GETTEXT_POISON fix
  t0003: GETTEXT_POISON fix, conclusion
  t0003: GETTEXT_POISON fix, part 1
  t0033: GETTEXT_POISON fix
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
  ci: install python on ubuntu
  ci: use the same version of p4 on both Linux and macOS
  ci: remove the pipe after "p4 -V" to catch errors
  github-actions: run gcc-8 on ubuntu-20.04 image
  ...
2023-04-17 21:15:44 +02:00
Taylor Blau
668f2d5361 Git 2.30.9
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-04-17 21:15:43 +02:00
Robin Jarry
3c8d3adeae send-email: export patch counters in validate environment
When sending patch series (with a cover-letter or not)
sendemail-validate is called with every email/patch file independently
from the others. When one of the patches depends on a previous one, it
may not be possible to use this hook in a meaningful way. A hook that
wants to check some property of the whole series needs to know which
patch is the final one.

Expose the current and total number of patches to the hook via the
GIT_SENDEMAIL_PATCH_COUNTER and GIT_SENDEMAIL_PATCH_TOTAL environment
variables so that both incremental and global validation is possible.

Sharing any other state between successive invocations of the validate
hook must be done via external means. For example, by storing it in
a git config sendemail.validateWorktree entry.

Add a sample script with placeholder validations and update tests to
check that the counters are properly exported.

Suggested-by: Phillip Wood <phillip.wood123@gmail.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-14 10:41:15 -07:00
Felipe Contreras
28fde3a1f4 doc: set actual revdate for manpages
manpages expect the date of the last revision, if that is not found
DocBook Stylesheets go through a series of hacks to generate one with
the format `%d/%d/%Y` which is not ideal.

In addition to this format not being standard, different tools generate
dates with different formats.

There's no need for any confusion if we specify the revision date, so
let's do so.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-14 10:37:41 -07:00
Junio C Hamano
df113b5560 Merge branch 'fc/doc-stop-using-manversion' into fc/doc-use-datestamp-in-commit
* fc/doc-stop-using-manversion:
  doc: simplify man version
2023-04-14 10:33:32 -07:00
Junio C Hamano
276699360d Merge branch 'fc/remove-header-workarounds-for-asciidoc' into fc/doc-use-datestamp-in-commit
* fc/remove-header-workarounds-for-asciidoc:
  doc: asciidoc: remove custom header macro
2023-04-14 10:33:15 -07:00
Øystein Walle
aabfdc9514 branch, for-each-ref, tag: add option to omit empty lines
If the given format string expands to the empty string, a newline is
still printed. This makes using the output linewise more tedious. For
example, git update-ref --stdin does not accept empty lines.

Add options to "git branch", "git for-each-ref", and "git tag" to
not print these empty lines.  The default behavior remains the same.

Signed-off-by: Øystein Walle <oystwa@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-13 08:07:45 -07:00
Taylor Blau
a8dd7e05b1 config: enable pack.writeReverseIndex by default
Back in e37d0b8730 (builtin/index-pack.c: write reverse indexes,
2021-01-25), Git learned how to read and write a pack's reverse index
from a file instead of in-memory.

A pack's reverse index is a mapping from pack position (that is, the
order that objects appear together in a ".pack")  to their position in
lexical order (that is, the order that objects are listed in an ".idx"
file).

Reverse indexes are consulted often during pack-objects, as well as
during auxiliary operations that require mapping between pack offsets,
pack order, and index index.

They are useful in GitHub's infrastructure, where we have seen a
dramatic increase in performance when writing ".rev" files[1]. In
particular:

  - an ~80% reduction in the time it takes to serve fetches on a popular
    repository, Homebrew/homebrew-core.

  - a ~60% reduction in the peak memory usage to serve fetches on that
    same repository.

  - a collective savings of ~35% in CPU time across all pack-objects
    invocations serving fetches across all repositories in a single
    datacenter.

Reverse indexes are also beneficial to end-users as well as forges. For
example, the time it takes to generate a pack containing the objects for
the 10 most recent commits in linux.git (representing a typical push) is
significantly faster when on-disk reverse indexes are available:

    $ { git rev-parse HEAD && printf '^' && git rev-parse HEAD~10 } >in
    $ hyperfine -L v false,true 'git.compile -c pack.readReverseIndex={v} pack-objects --delta-base-offset --revs --stdout <in >/dev/null'
    Benchmark 1: git.compile -c pack.readReverseIndex=false pack-objects --delta-base-offset --revs --stdout <in >/dev/null
      Time (mean ± σ):     543.0 ms ±  20.3 ms    [User: 616.2 ms, System: 58.8 ms]
      Range (min … max):   521.0 ms … 577.9 ms    10 runs

    Benchmark 2: git.compile -c pack.readReverseIndex=true pack-objects --delta-base-offset --revs --stdout <in >/dev/null
      Time (mean ± σ):     245.0 ms ±  11.4 ms    [User: 335.6 ms, System: 31.3 ms]
      Range (min … max):   226.0 ms … 259.6 ms    13 runs

    Summary
      'git.compile -c pack.readReverseIndex=true pack-objects --delta-base-offset --revs --stdout <in >/dev/null' ran
	2.22 ± 0.13 times faster than 'git.compile -c pack.readReverseIndex=false pack-objects --delta-base-offset --revs --stdout <in >/dev/null'

The same is true of writing a pack containing the objects for the 30
most-recent commits:

    $ { git rev-parse HEAD && printf '^' && git rev-parse HEAD~30 } >in
    $ hyperfine -L v false,true 'git.compile -c pack.readReverseIndex={v} pack-objects --delta-base-offset --revs --stdout <in >/dev/null'
    Benchmark 1: git.compile -c pack.readReverseIndex=false pack-objects --delta-base-offset --revs --stdout <in >/dev/null
      Time (mean ± σ):     866.5 ms ±  16.2 ms    [User: 1414.5 ms, System: 97.0 ms]
      Range (min … max):   839.3 ms … 886.9 ms    10 runs

    Benchmark 2: git.compile -c pack.readReverseIndex=true pack-objects --delta-base-offset --revs --stdout <in >/dev/null
      Time (mean ± σ):     581.6 ms ±  10.2 ms    [User: 1181.7 ms, System: 62.6 ms]
      Range (min … max):   567.5 ms … 599.3 ms    10 runs

    Summary
      'git.compile -c pack.readReverseIndex=true pack-objects --delta-base-offset --revs --stdout <in >/dev/null' ran
	1.49 ± 0.04 times faster than 'git.compile -c pack.readReverseIndex=false pack-objects --delta-base-offset --revs --stdout <in >/dev/null'

...and savings on trivial operations like computing the on-disk size of
a single (packed) object are even more dramatic:

    $ git rev-parse HEAD >in
    $ hyperfine -L v false,true 'git.compile -c pack.readReverseIndex={v} cat-file --batch-check="%(objectsize:disk)" <in'
    Benchmark 1: git.compile -c pack.readReverseIndex=false cat-file --batch-check="%(objectsize:disk)" <in
      Time (mean ± σ):     305.8 ms ±  11.4 ms    [User: 264.2 ms, System: 41.4 ms]
      Range (min … max):   290.3 ms … 331.1 ms    10 runs

    Benchmark 2: git.compile -c pack.readReverseIndex=true cat-file --batch-check="%(objectsize:disk)" <in
      Time (mean ± σ):       4.0 ms ±   0.3 ms    [User: 1.7 ms, System: 2.3 ms]
      Range (min … max):     1.6 ms …   4.6 ms    1155 runs

    Summary
      'git.compile -c pack.readReverseIndex=true cat-file --batch-check="%(objectsize:disk)" <in' ran
       76.96 ± 6.25 times faster than 'git.compile -c pack.readReverseIndex=false cat-file --batch-check="%(objectsize:disk)" <in'

In the more than two years since e37d0b8730 was merged, Git's
implementation of on-disk reverse indexes has been thoroughly tested,
both from users enabling `pack.writeReverseIndexes`, and from GitHub's
deployment of the feature. The latter has been running without incident
for more than two years.

This patch changes Git's behavior to write on-disk reverse indexes by
default when indexing a pack, which should make the above operations
faster for everybody's Git installation after a repack.

(The previous commit explains some potential drawbacks of using on-disk
reverse indexes in certain limited circumstances, that essentially boil
down to a trade-off between time to generate, and time to access. For
those limited cases, the `pack.readReverseIndex` escape hatch can be
used).

[1]: https://github.blog/2021-04-29-scaling-monorepo-maintenance/#reverse-indexes

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Acked-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-13 07:55:46 -07:00
Taylor Blau
dbcf611617 pack-revindex: introduce pack.readReverseIndex
Since 1615c567b8 (Documentation/config/pack.txt: advertise
'pack.writeReverseIndex', 2021-01-25), we have had the
`pack.writeReverseIndex` configuration option, which tells Git whether
or not it is allowed to write a ".rev" file when indexing a pack.

Introduce a complementary configuration knob, `pack.readReverseIndex` to
control whether or not Git will read any ".rev" file(s) that may be
available on disk.

This option is useful for debugging, as well as disabling the effect of
".rev" files in certain instances.

This is useful because of the trade-off[^1] between the time it takes to
generate a reverse index (slow from scratch, fast when reading an
existing ".rev" file), and the time it takes to access a record (the
opposite).

For example, even though it is faster to use the on-disk reverse index
when computing the on-disk size of a packed object, it is slower to
enumerate the same value for all objects.

Here are a couple of examples from linux.git. When computing the above
for a single object, using the on-disk reverse index is significantly
faster:

    $ git rev-parse HEAD >in
    $ hyperfine -L v false,true 'git.compile -c pack.readReverseIndex={v} cat-file --batch-check="%(objectsize:disk)" <in'
    Benchmark 1: git.compile -c pack.readReverseIndex=false cat-file --batch-check="%(objectsize:disk)" <in
      Time (mean ± σ):     302.5 ms ±  12.5 ms    [User: 258.7 ms, System: 43.6 ms]
      Range (min … max):   291.1 ms … 328.1 ms    10 runs

    Benchmark 2: git.compile -c pack.readReverseIndex=true cat-file --batch-check="%(objectsize:disk)" <in
      Time (mean ± σ):       3.9 ms ±   0.3 ms    [User: 1.6 ms, System: 2.4 ms]
      Range (min … max):     2.0 ms …   4.4 ms    801 runs

    Summary
      'git.compile -c pack.readReverseIndex=true cat-file --batch-check="%(objectsize:disk)" <in' ran
       77.29 ± 7.14 times faster than 'git.compile -c pack.readReverseIndex=false cat-file --batch-check="%(objectsize:disk)" <in'

, but when instead trying to compute the on-disk object size for all
objects in the repository, using the ".rev" file is a disadvantage over
creating the reverse index from scratch:

    $ hyperfine -L v false,true 'git.compile -c pack.readReverseIndex={v} cat-file --batch-check="%(objectsize:disk)" --batch-all-objects'
    Benchmark 1: git.compile -c pack.readReverseIndex=false cat-file --batch-check="%(objectsize:disk)" --batch-all-objects
      Time (mean ± σ):      8.258 s ±  0.035 s    [User: 7.949 s, System: 0.308 s]
      Range (min … max):    8.199 s …  8.293 s    10 runs

    Benchmark 2: git.compile -c pack.readReverseIndex=true cat-file --batch-check="%(objectsize:disk)" --batch-all-objects
      Time (mean ± σ):     16.976 s ±  0.107 s    [User: 16.706 s, System: 0.268 s]
      Range (min … max):   16.839 s … 17.105 s    10 runs

    Summary
      'git.compile -c pack.readReverseIndex=false cat-file --batch-check="%(objectsize:disk)" --batch-all-objects' ran
	2.06 ± 0.02 times faster than 'git.compile -c pack.readReverseIndex=true cat-file --batch-check="%(objectsize:disk)" --batch-all-objects'

Luckily, the results when running `git cat-file` with `--unordered` are
closer together:

    $ hyperfine -L v false,true 'git.compile -c pack.readReverseIndex={v} cat-file --unordered --batch-check="%(objectsize:disk)" --batch-all-objects'
    Benchmark 1: git.compile -c pack.readReverseIndex=false cat-file --unordered --batch-check="%(objectsize:disk)" --batch-all-objects
      Time (mean ± σ):      5.066 s ±  0.105 s    [User: 4.792 s, System: 0.274 s]
      Range (min … max):    4.943 s …  5.220 s    10 runs

    Benchmark 2: git.compile -c pack.readReverseIndex=true cat-file --unordered --batch-check="%(objectsize:disk)" --batch-all-objects
      Time (mean ± σ):      6.193 s ±  0.069 s    [User: 5.937 s, System: 0.255 s]
      Range (min … max):    6.145 s …  6.356 s    10 runs

    Summary
      'git.compile -c pack.readReverseIndex=false cat-file --unordered --batch-check="%(objectsize:disk)" --batch-all-objects' ran
        1.22 ± 0.03 times faster than 'git.compile -c pack.readReverseIndex=true cat-file --unordered --batch-check="%(objectsize:disk)" --batch-all-objects'

Because the equilibrium point between these two is highly machine- and
repository-dependent, allow users to configure whether or not they will
read any ".rev" file(s) with this configuration knob.

[^1]: Generating a reverse index in memory takes O(N) time (where N is
  the number of objects in the repository), since we use a radix sort.
  Reading an entry from an on-disk ".rev" file is slower since each
  operation is bound by disk I/O instead of memory I/O.

  In order to compute the on-disk size of a packed object, we need to
  find the offset of our object, and the adjacent object (the on-disk
  size difference of these two). Finding the first offset requires a
  binary search. Finding the latter involves a single .rev lookup.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Acked-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-13 07:55:46 -07:00
Junio C Hamano
9857273be0 The ninth batch
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-04-11 13:49:13 -07:00
Junio C Hamano
a86083e25f Merge branch 'fc/doc-manpage-base-url-fix'
Modernize manpage generation toolchain.

* fc/doc-manpage-base-url-fix:
  doc: remove manpage-base-url workaround
2023-04-11 13:49:13 -07:00
Junio C Hamano
95e6111e7c Merge branch 'dw/doc-submittingpatches-grammofix'
Grammofix.

* dw/doc-submittingpatches-grammofix:
  SubmittingPatches: clarify MUA discussion with "the"
2023-04-11 13:49:13 -07:00
Junio C Hamano
647a2bb3ff Merge branch 'jc/spell-id-in-both-caps-in-message-id'
Consistently spell "Message-ID" as such, not "Message-Id".

* jc/spell-id-in-both-caps-in-message-id:
  e-mail workflow: Message-ID is spelled with ID in both capital letters
2023-04-11 13:49:12 -07:00