Commit Graph

18780 Commits

Author SHA1 Message Date
Junio C Hamano
192a3fa31d Merge branch 'ab/plug-random-leaks'
Leakfix.

* ab/plug-random-leaks:
  reflog: free() ref given to us by dwim_log()
  submodule--helper: fix small memory leaks
  clone: fix a memory leak of the "git_dir" variable
  grep: fix a "path_list" memory leak
  grep: use object_array_clear() in cmd_grep()
  grep: prefer "struct grep_opt" over its "void *" equivalent
2021-10-29 15:43:13 -07:00
Junio C Hamano
55b99febc4 Merge branch 'ab/plug-handle-path-exclude-leak'
Leakfix.

* ab/plug-handle-path-exclude-leak:
  config.c: don't leak memory in handle_path_include()
2021-10-29 15:43:13 -07:00
Junio C Hamano
735907bde1 Merge branch 'jk/http-push-status-fix'
"git push" client talking to an HTTP server did not diagnose the
lack of the final status report from the other side correctly,
which has been corrected.

* jk/http-push-status-fix:
  transport-helper: recognize "expecting report" error from send-pack
  send-pack: complain about "expecting report" with --helper-status
2021-10-29 15:43:12 -07:00
Junio C Hamano
8ba651b56e Merge branch 'ab/test-bail'
A new feature has been added to abort early in the test framework.

* ab/test-bail:
  test-lib.sh: use "Bail out!" syntax on bad SANITIZE=leak use
  test-lib.sh: de-duplicate error() teardown code
2021-10-29 15:43:12 -07:00
Junio C Hamano
0988e665e9 Revert "logmsg_reencode(): warn when iconv() fails"
This reverts commit fd680bc5 (logmsg_reencode(): warn when iconv()
fails, 2021-08-27).  Throwing a warning for each and every commit
that gets reencoded, without allowing a way to squelch, would make
it unpleasant for folks who have to deal with an ancient part of the
history in an old project that used wrong encoding in the commits.
2021-10-29 13:48:58 -07:00
Alex Riesen
fa21296b58 Document positive variant of commit and merge option "--no-verify"
This documents "--verify" option of the commands. It can be used to re-enable
the hooks disabled by an earlier "--no-verify" in command-line.

Signed-off-by: Alexander Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-29 11:22:56 -07:00
Junio C Hamano
361cb52383 pull: --ff-only should make it a noop when already-up-to-date
Earlier, we made sure that "git pull --ff-only" (and "git -c
pull.ff=only pull") errors out when our current HEAD is not an
ancestor of the tip of the history we are merging, but the condition
to trigger the error was implemented incorrectly.

Imagine you forked from a remote branch, built your history on top
of it, and then attempted to pull from them again.  If they have not
made any update in the meantime, our current HEAD is obviously not
their ancestor, and this new error triggers.

Without the --ff-only option, we just report that there is no need
to pull; we did the same historically with --ff-only, too.

Make sure we do not fail with the recently added check to restore
the historical behaviour.

Reported-by: Kenneth Arnold <ka37@calvin.edu>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-29 00:15:39 -07:00
Jeff King
559664c792 t5310: drop lib-bundle.sh include
Commit ddfe900612 (test-lib-functions: move function to lib-bitmap.sh,
2021-02-09) meant to include lib-bitmap.sh in t5310, but also includes
lib-bundle.sh. Yet we don't use any of its functions, nor have anything
to do with bundles. This is probably just a typo/copy-paste error, as
lib-bundle.sh was added (correctly) to other scripts in the same series.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-29 00:06:12 -07:00
Alex Riesen
47bfdfb3fd pull: honor --no-verify and do not call the commit-msg hook
The option was incorrectly auto-translated to "--no-verify-signatures",
which causes the unexpected effect of the hook being called.
And an even more unexpected effect of disabling verification of signatures.

The manual page describes the option to behave same as the similarly
named option of "git merge", which seems to be the original intention
of this option in the "pull" command.

Signed-off-by: Alexander Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-28 09:52:09 -07:00
Thiago Perrotta
2b7b75850c send-email: programmatically generate bash completions
"git send-email --git-completion-helper" only prints "format-patch"
flags. Make it print "send-email" flags as well, extracting them
programmatically from its three existing "GetOptions".

Introduce a "uniq" subroutine, otherwise --cc-cover, --to-cover and
other flags would show up twice. In addition, deduplicate flags common
to both "send-email" and "format-patch", like --from.

Remove extraneous flags: --h and --git-completion-helper.

Add trailing "=" to options that expect an argument, inline with
the format-patch implementation.

Add a completion test for "send-email --validate", a send-email flag.

Signed-off-by: Thiago Perrotta <tbperrotta@gmail.com>
Based-on-patch-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-28 09:04:24 -07:00
Matheus Tavares
20141e322c add, rm, mv: fix bug that prevents the update of non-sparse dirs
These three commands recently learned to avoid updating paths outside
the sparse checkout even if they are missing the SKIP_WORKTREE bit. This
is done using path_in_sparse_checkout(), which checks whether a given
path matches the current list of sparsity rules, similar to what
clear_ce_flags() does when we run "git sparse checkout init" or "git
sparse-checkout reapply". However, clear_ce_flags() uses a recursive
approach, applying the match results from parent directories on paths
that get the UNDECIDED result, whereas path_in_sparse_checkout() only
attempts to match the full path and immediately considers UNDECIDED as
NOT_MATCHED. This makes the function miss matches with leading
directories. For example, if the user has the sparsity patterns "!/a"
and "b/", add, rm, and mv will fail to update the path "a/b/c" and end
up displaying a warning about it being outside the sparse checkout even
though it isn't. This problem only occurs in full pattern mode as the
pattern matching functions never return UNDECIDED for cone mode.

To fix this, replicate the recursive behavior of clear_ce_flags() in
path_in_sparse_checkout(), falling back to the parent directory match
when a path gets the UNDECIDED result. (If this turns out to be too
expensive in some cases, we may want to later add some form of caching
to accelerate multiple queries within the same directory. This is not
implemented in this patch, though.) Also add two tests for each affected
command (add, rm, and mv) to check that they behave correctly with the
recursive pattern matching. The first test would previously fail without
this patch while the second already succeeded. It is added mostly to
make sure that we are not breaking the existing pattern matching for
directories that are really sparse, and also as a protection against any
future regressions.

Two other existing tests had to be changed as well: one test in t3602
checks that "git rm -r <dir>" won't remove sparse entries, but it didn't
allow the non-sparse entries inside <dir> to be removed. The other one,
in t7002, tested that "git mv" would correctly display a warning message
for sparse paths, but it accidentally expected the message to include
two non-sparse paths as well.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
Acked-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-28 08:46:07 -07:00
Taylor Blau
7f4c3508c0 t/helper/test-read-midx.c: free MIDX within read_midx_file()
When calling `read_midx_file()` to show information about a MIDX or list
the objects contained within it we fail to call `close_midx()`, leaking
the memory allocated to store that MIDX.

Fix this by calling `close_midx()` before exiting the function. We can
drop the "early" return when `show_objects` is non-zero, since the next
instruction is also a return.

(We could just as easily put a `cleanup` label here as with previous
patches. But the only other time we terminate the function early is
when we fail to load a MIDX in the first place. `close_midx()` does
handle a NULL argument, but the extra complexity is likely not
warranted).

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-27 16:26:37 -07:00
Junio C Hamano
c6fc44e9bf Merge branch 'ab/test-lib-diff-cleanup'
Test clean-up.

* ab/test-lib-diff-cleanup:
  tests: stop using top-level "README" and "COPYING" files
  "lib-diff" tests: make "README" and "COPYING" test data smaller
2021-10-25 16:07:01 -07:00
Junio C Hamano
67f310e1ad Merge branch 'ab/test-cleanly-recreate-trash-directory'
Improve test framework around unwritable directories.

* ab/test-cleanly-recreate-trash-directory:
  test-lib.sh: try to re-chmod & retry on failed trash removal
2021-10-25 16:07:00 -07:00
Junio C Hamano
54c4f8ce52 Merge branch 'ab/mark-leak-free-tests-more'
Bunch of tests are marked as "passing leak check".

* ab/mark-leak-free-tests-more:
  merge: add missing strbuf_release()
  ls-files: add missing string_list_clear()
  ls-files: fix a trivial dir_clear() leak
  tests: fix test-oid-array leak, test in SANITIZE=leak
  tests: fix a memory leak in test-oidtree.c
  tests: fix a memory leak in test-parse-options.c
  tests: fix a memory leak in test-prio-queue.c
2021-10-25 16:06:59 -07:00
Junio C Hamano
5a4f8381b6 Merge branch 'ab/mark-leak-free-tests'
Bunch of tests are marked as "passing leak check".

* ab/mark-leak-free-tests:
  leak tests: mark some misc tests as passing with SANITIZE=leak
  leak tests: mark various "generic" tests as passing with SANITIZE=leak
  leak tests: mark some read-tree tests as passing with SANITIZE=leak
  leak tests: mark some ls-files tests as passing with SANITIZE=leak
  leak tests: mark all checkout-index tests as passing with SANITIZE=leak
  leak tests: mark all trace2 tests as passing with SANITIZE=leak
  leak tests: mark all ls-tree tests as passing with SANITIZE=leak
  leak tests: run various "test-tool" tests in t00*.sh SANITIZE=leak
  leak tests: run various built-in tests in t00*.sh SANITIZE=leak
2021-10-25 16:06:59 -07:00
Junio C Hamano
65ca3245f9 Merge branch 'ab/parse-options-cleanup'
Random changes to parse-options implementation.

* ab/parse-options-cleanup:
  parse-options: change OPT_{SHORT,UNSET} to an enum
  parse-options tests: test optname() output
  parse-options.[ch]: make opt{bug,name}() "static"
  commit-graph: stop using optname()
  parse-options.c: move optname() earlier in the file
  parse-options.h: make the "flags" in "struct option" an enum
  parse-options.c: use exhaustive "case" arms for "enum parse_opt_result"
  parse-options.[ch]: consistently use "enum parse_opt_result"
  parse-options.[ch]: consistently use "enum parse_opt_flags"
  parse-options.h: move PARSE_OPT_SHELL_EVAL between enums
2021-10-25 16:06:59 -07:00
Junio C Hamano
f3f157ff27 Merge branch 'js/userdiff-cpp'
Userdiff patterns for the C++ language has been updated.

* js/userdiff-cpp:
  userdiff-cpp: back out the digit-separators in numbers
  userdiff-cpp: learn the C++ spaceship operator
  userdiff-cpp: permit the digit-separating single-quote in numbers
  userdiff-cpp: prepare test cases with yet unsupported features
  userdiff-cpp: tighten word regex
  t4034: add tests showing problematic cpp tokenizations
  t4034/cpp: actually test that operator tokens are not split
2021-10-25 16:06:59 -07:00
Junio C Hamano
ef1639145d Merge branch 'fs/ssh-signing-fix'
Fix-up for the other topic already in 'next'.

* fs/ssh-signing-fix:
  gpg-interface: fix leak of strbufs in get_ssh_key_fingerprint()
  gpg-interface: fix leak of "line" in parse_ssh_output()
  ssh signing: clarify trustlevel usage in docs
  ssh signing: fmt-merge-msg tests & config parse
2021-10-25 16:06:58 -07:00
Junio C Hamano
18c6653da0 Merge branch 'fs/ssh-signing'
Use ssh public crypto for object and push-cert signing.

* fs/ssh-signing:
  ssh signing: test that gpg fails for unknown keys
  ssh signing: tests for logs, tags & push certs
  ssh signing: duplicate t7510 tests for commits
  ssh signing: verify signatures using ssh-keygen
  ssh signing: provide a textual signing_key_id
  ssh signing: retrieve a default key from ssh-agent
  ssh signing: add ssh key format and signing code
  ssh signing: add test prereqs
  ssh signing: preliminary refactoring and clean-up
2021-10-25 16:06:58 -07:00
Junio C Hamano
e058b1846c Merge branch 'pw/sparse-cache-tree-verify-fix'
Recent sparse-index addition, namely any use of index_name_pos(),
can expand sparse index entries and breaks any code that walks
cache-tree or existing index entries.  One such instance of such a
breakage has been corrected.

* pw/sparse-cache-tree-verify-fix:
  t1092: run "rebase --apply" without "-q" in testing
  sparse index: fix use-after-free bug in cache_tree_verify()
2021-10-25 16:06:57 -07:00
Junio C Hamano
2c428e4205 Merge branch 'ab/fix-commit-error-message-upon-unwritable-object-store'
"git commit" gave duplicated error message when the object store
was unwritable, which has been corrected.

* ab/fix-commit-error-message-upon-unwritable-object-store:
  commit: fix duplication regression in permission error output
  unwritable tests: assert exact error output
2021-10-25 16:06:57 -07:00
Junio C Hamano
6ffb5fc069 Merge branch 'rs/add-dry-run-without-objects'
Stop "git add --dry-run" from creating new blob and tree objects.

* rs/add-dry-run-without-objects:
  add: don't write objects with --dry-run
2021-10-25 16:06:57 -07:00
Junio C Hamano
525705a0a2 Merge branch 'rs/disable-gc-during-perf-tests'
Avoid performance measurements from getting ruined by gc and other
housekeeping pauses interfering in the middle.

* rs/disable-gc-during-perf-tests:
  perf: disable automatic housekeeping
2021-10-25 16:06:57 -07:00
Junio C Hamano
162a13b855 Merge branch 'jt/no-abuse-alternate-odb-for-submodules'
Follow through the work to use the repo interface to access
submodule objects in-process, instead of abusing the alternate
object database interface.

* jt/no-abuse-alternate-odb-for-submodules:
  submodule: trace adding submodule ODB as alternate
  submodule: pass repo to check_has_commit()
  object-file: only register submodule ODB if needed
  merge-{ort,recursive}: remove add_submodule_odb()
  refs: peeling non-the_repository iterators is BUG
  refs: teach arbitrary repo support to iterators
  refs: plumb repo into ref stores
2021-10-25 16:06:56 -07:00
Junio C Hamano
bfa646c2cb Merge branch 'ab/unpack-trees-leakfix'
Leakfix.

* ab/unpack-trees-leakfix:
  sequencer: fix a memory leak in do_reset()
  sequencer: add a "goto cleanup" to do_reset()
  unpack-trees: don't leak memory in verify_clean_subdirectory()
2021-10-25 16:06:56 -07:00
Junio C Hamano
91016984db Merge branch 'jh/perf-remove-test-times'
Perf test fix.

* jh/perf-remove-test-times:
  t/perf/perf-lib.sh: remove test_times.* at the end test_perf_()
2021-10-25 16:06:56 -07:00
Junio C Hamano
061a21d36d Merge branch 'ab/fsck-unexpected-type'
"git fsck" has been taught to report mismatch between expected and
actual types of an object better.

* ab/fsck-unexpected-type:
  fsck: report invalid object type-path combinations
  fsck: don't hard die on invalid object types
  object-file.c: stop dying in parse_loose_header()
  object-file.c: return ULHR_TOO_LONG on "header too long"
  object-file.c: use "enum" return type for unpack_loose_header()
  object-file.c: simplify unpack_loose_short_header()
  object-file.c: make parse_loose_header_extended() public
  object-file.c: return -1, not "status" from unpack_loose_header()
  object-file.c: don't set "typep" when returning non-zero
  cat-file tests: test for current --allow-unknown-type behavior
  cat-file tests: add corrupt loose object test
  cat-file tests: test for missing/bogus object with -t, -s and -p
  cat-file tests: move bogus_* variable declarations earlier
  fsck tests: test for garbage appended to a loose object
  fsck tests: test current hash/type mismatch behavior
  fsck tests: refactor one test to use a sub-repo
  fsck tests: add test for fsck-ing an unknown type
2021-10-25 16:06:56 -07:00
Johannes Sixt
386076ec92 userdiff-cpp: back out the digit-separators in numbers
The implementation of digit-separating single-quotes introduced a
note-worthy regression: the change of a character literal with a
digit would splice the digit and the closing single-quote. For
example, the change from 'a' to '2' is now tokenized as
'[-a'-]{+2'+} instead of '[-a-]{+2+}'.

The options to fix the regression are:

- Tighten the regular expression such that the single-quote can only
  occur between digits (that would match the official syntax).

- Remove support for digit separators.

I chose to remove support, because

- I have not seen a lot of code make use of digit separators.

- If code does use digit separators, then the numbers are typically
  long. If a change in one of the segments occurs, it is actually
  better visible if only that segment is highlighted as the word
  that changed instead of the whole long number.

This choice does introduce another minor regression, though, which
is highlighted in the test case: when a change occurs in the second
or later segment of a hexadecimal number where the segment begins
with a digit, but also has letters, the segment is mistaken as
consisting of a number and an identifier. I can live with that.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-25 08:47:44 -07:00
Ævar Arnfjörð Bjarmason
b202e51b15 grep: fix a "path_list" memory leak
Free the "path_list" used in builtin/grep.c, it was declared as
STRING_LIST_INIT_NODUP, let's change it to a STRING_LIST_INIT_DUP
since an early user in cmd_grep() appends a string passed via
parse-options.c to it, which needs to be duplicated.

Let's then convert the remaining callers to use
string_list_append_nodup() instead, allowing us to free the list.

This makes all the tests in t7811-grep-open.sh pass, 6/10 would fail
before this change. The only remaining failure would have been due to
a stray "git checkout" (which still leaks memory). In this case we can
use a "git reset --hard" instead, so let's do that, and move the
test_when_finished() above the code that would modify the relevant
file.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-23 10:45:25 -07:00
Øystein Walle
2e59e78096 status: print stash info with --porcelain=v2 --show-stash
The v2 porcelain format is very convenient for obtaining a lot of
information about the current state of the repo, but does not contain
any info about the stash. git status already accepts --show-stash but
it's silently ignored when --porcelain=v2 is given.

Let's add a simple line to print the number of stash entries but in a
format similar in style to the rest of the format.

Signed-off-by: Øystein Walle <oystwa@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-21 17:24:30 -07:00
Ævar Arnfjörð Bjarmason
25ad722126 config.c: don't leak memory in handle_path_include()
Fix a memory leak in the error() path in handle_path_include(), this
allows us to run t1305-config-include.sh under SANITIZE=leak,
previously 4 tests there would fail. This fixes up a leak in
9b25a0b52e (config: add include directive, 2012-02-06).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-21 16:26:45 -07:00
Junio C Hamano
98e7ab6d42 for-each-ref: delay parsing of --sort=<atom> options
The for-each-ref family of commands invoke parsers immediately when
it sees each --sort=<atom> option, and die before even seeing the
other options on the command line when the <atom> is unrecognised.

Instead, accumulate them in a string list, and have them parsed into
a ref_sorting structure after the command line parsing is done.  As
a consequence, "git branch --sort=bogus -h" used to fail to give the
brief help, which arguably may have been a feature, now does so,
which is more consistent with how other options work.

The patch is smaller than the actual extent of the "damage" to the
codebase, thanks to the fact that the original code consistently
used OPT_REF_SORT() macro to handle command line options.  We only
needed to replace the variable used for the list, and implementation
of the callback function used in the macro.

The old rule was for the users of the API to:

 - Declare ref_sorting and ref_sorting_tail variables;

 - OPT_REF_SORT() macro will instantiate ref_sorting instance (which
   may barf and die) and append it to the tail;

 - Append to the tail each ref_sorting read from the configuration
   by parsing in the config callback (which may barf and die);

 - See if ref_sorting is null and use ref_sorting_default() instead.

Now the rule is not all that different but is simpler:

 - Declare ref_sorting_options string list.

 - OPT_REF_SORT() macro will append it to the string list;

 - Append to the string list the sort key read from the
   configuration;

 - call ref_sorting_options() to turn the string list to ref_sorting
   structure (which also deals with the default value).

As side effects, this change also cleans up a few issues:

 - 95be717c (parse_opt_ref_sorting: always use with NONEG flag,
   2019-03-20) muses that "git for-each-ref --no-sort" should simply
   clear the sort keys accumulated so far; it now does.

 - The implementation detail of "struct ref_sorting" and the helper
   function parse_ref_sorting() can now be private to the ref-filter
   API implementation.

 - If you set branch.sort to a bogus value, the any "git branch"
   invocation, not only the listing mode, would abort with the
   original code; now it doesn't

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-20 14:33:07 -07:00
Junio C Hamano
a86ed75f32 Merge branch 'rs/make-verify-path-really-verify-again'
Recent sparse-index work broke safety against attempts to add paths
with trailing slashes to the index, which has been corrected.

* rs/make-verify-path-really-verify-again:
  read-cache: let verify_path() reject trailing dir separators again
  read-cache: add verify_path_internal()
  t3905: show failure to ignore sub-repo
2021-10-18 15:47:58 -07:00
Junio C Hamano
092228ee5c Merge branch 'jk/cat-file-batch-all-wo-replace'
"git cat-file --batch" with the "--batch-all-objects" option is
supposed to iterate over all the objects found in a repository, but
it used to translate these object names using the replace mechanism,
which defeats the point of enumerating all objects in the repository.
This has been corrected.

* jk/cat-file-batch-all-wo-replace:
  cat-file: use packed_object_info() for --batch-all-objects
  cat-file: split ordered/unordered batch-all-objects callbacks
  cat-file: disable refs/replace with --batch-all-objects
  cat-file: mention --unordered along with --batch-all-objects
  t1006: clean up broken objects
2021-10-18 15:47:57 -07:00
Junio C Hamano
0b69bb0fb1 Merge branch 'tb/repack-write-midx'
"git repack" has been taught to generate multi-pack reachability
bitmaps.

* tb/repack-write-midx:
  test-read-midx: fix leak of bitmap_index struct
  builtin/repack.c: pass `--refs-snapshot` when writing bitmaps
  builtin/repack.c: make largest pack preferred
  builtin/repack.c: support writing a MIDX while repacking
  builtin/repack.c: extract showing progress to a variable
  builtin/repack.c: rename variables that deal with non-kept packs
  builtin/repack.c: keep track of existing packs unconditionally
  midx: preliminary support for `--refs-snapshot`
  builtin/multi-pack-index.c: support `--stdin-packs` mode
  midx: expose `write_midx_file_only()` publicly
2021-10-18 15:47:57 -07:00
Junio C Hamano
223a1bfb58 Merge branch 'js/retire-preserve-merges'
The "--preserve-merges" option of "git rebase" has been removed.

* js/retire-preserve-merges:
  sequencer: restrict scope of a formerly public function
  rebase: remove a no-longer-used function
  rebase: stop mentioning the -p option in comments
  rebase: remove obsolete code comment
  rebase: drop the internal `rebase--interactive` command
  git-svn: drop support for `--preserve-merges`
  rebase: drop support for `--preserve-merges`
  pull: remove support for `--rebase=preserve`
  tests: stop testing `git rebase --preserve-merges`
  remote: warn about unhandled branch.<name>.rebase values
  t5520: do not use `pull.rebase=preserve`
2021-10-18 15:47:56 -07:00
Junio C Hamano
0ef08090d2 Merge branch 'rs/mergesort'
The mergesort implementation used to sort linked list has been
optimized.

* rs/mergesort:
  test-mergesort: use repeatable random numbers
  mergesort: use ranks stack
  p0071: test performance of llist_mergesort()
  p0071: measure sorting of already sorted and reversed files
  test-mergesort: add unriffle_skewed mode
  test-mergesort: add unriffle mode
  test-mergesort: add generate subcommand
  test-mergesort: add test subcommand
  test-mergesort: add sort subcommand
  test-mergesort: use strbuf_getline()
2021-10-18 15:47:56 -07:00
Jeff King
c5c3486f38 transport-helper: recognize "expecting report" error from send-pack
When a transport helper pushes via send-pack, it passes --helper-status
to get a machine-readable status back for each ref. The previous commit
taught the send-pack code to hand back "error expecting report" if the
server did not send us the proper ref-status. And that's enough to cause
us to recognize that an error occurred for the ref and print something
sensible in our final status table.

But we do interpret these messages on the remote-helper side to turn
them back into REF_STATUS_* enum values.  Recognizing this token to turn
it back into REF_STATUS_EXPECTING_REPORT has two advantages:

  1. We now print exactly the same message in the human-readable (and
     machine-readable --porcelain) output for this situation whether the
     transport went through a helper (e.g., http) or not (e.g., ssh).

  2. If any code in the helper really cares about distinguishing
     EXPECT_REPORT from more generic error conditions, it could now do
     so. I didn't find any, so this is mostly future-proofing.

So this is mostly cosmetic for now, but it seems like the
least-surprising thing for the transport-helper code to be doing.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-18 13:27:36 -07:00
Jeff King
e4c9538a9c send-pack: complain about "expecting report" with --helper-status
When pushing to a server which erroneously omits the final ref-status
report, the client side should complain about the refs for which we
didn't receive the status (because we can't just assume they were
updated). This works over most transports like ssh, but for http we'll
print a very misleading "Everything up-to-date".

It works for ssh because send-pack internally sets the status of each
ref to REF_STATUS_EXPECTING_REPORT, and then if the server doesn't tell
us about a particular ref, it will stay at that value. When we print the
final status table, we'll see that we're still on EXPECTING_REPORT and
complain then.

But for http, we go through remote-curl, which invokes send-pack with
"--stateless-rpc --helper-status". The latter option causes send-pack to
return a machine-readable list of ref statuses to the remote helper. But
ever since its inception in de1a2fdd38 (Smart push over HTTP: client
side, 2009-10-30), the send-pack code has simply omitted mention of any
ref which ended up in EXPECTING_REPORT.

In the remote helper, we then take the absence of any status report
from send-pack to mean that the ref was not even something we tried to
send, and thus it prints "Everything up-to-date". Fortunately it does
detect the eventual non-zero exit from send-pack, and propagates that in
its own non-zero exit code. So at least a careful script invoking "git
push" would notice the failure.  But sending the misleading message on
stderr is certainly confusing for humans (not to mention the
machine-readable "push --porcelain" output, though again, any careful
script should be checking the exit code from push, too).

Nobody seems to have noticed because the server in this instance has to
be misbehaving: it has promised to support the ref-status capability
(otherwise the client will not set EXPECTING_REPORT at all), but didn't
send us any. If the connection were simply cut, then send-pack would
complain about getting EOF while trying to read the status. But if the
server actually sends a flush packet (i.e., saying "now you have all of
the ref statuses" without actually sending any), then the client ends up
in this confused situation.

The fix is simple: we should return an error message from "send-pack
--helper-status", just like we would for any other error per-ref error
condition (in the test I included, the server simply omits all ref
status responses, but a more insidious version of this would skip only
some of them).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-18 13:26:52 -07:00
Sergey Organov
41a28eb6c1 stash: implement '--staged' option for 'push' and 'save'
Stash only the changes that are staged.

This mode allows to easily stash-out for later reuse some changes
unrelated to the current work in progress.

Unlike 'stash push --patch', --staged supports use of any tool to
select the changes to stash-out, including, but not limited to 'git
add --interactive'.

Signed-off-by: Sergey Organov <sorganov@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-18 13:09:21 -07:00
Phillip Wood
5e311edfd3 t1092: run "rebase --apply" without "-q" in testing
We run a few operations and make sure they produce identical results
with and without sparse-index; the version we merged to the "next"
branch used the "-q" option to work around a breakage caused by a
version used at Microsoft with some unreleased changes, but since
we would want to make sure the commands produce identical results,
including reports given to the output that lists which commits were
picked, use of "-q" loses too much interesting information.

Let's drop "-q" from the command invocation and revisit the issue
when the problematic changes are upstreamed.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-18 09:24:51 -07:00
Ævar Arnfjörð Bjarmason
f1da24ca5e refs API: post-migration API renaming [2/2]
Rename the transitory refs_werrres_ref_unsafe() function to
refs_resolve_ref_unsafe(), now that all callers of the old function
have learned to pass in a "failure_errno" parameter.

The coccinelle semantic patch added in the preceding commit works, but
I couldn't figure out how to get spatch(1) to re-flow these argument
lists (and sometimes make lines way too long), so this rename was done
with:

    perl -pi -e 's/refs_werrres_ref_unsafe/refs_resolve_ref_unsafe/g' \
    $(git grep -l refs_werrres_ref_unsafe -- '*.c')

But after that "make contrib/coccinelle/refs.cocci.patch" comes up
empty, so the result would have been the same. Let's remove that
transitory semantic patch file, we won't need to retain it for any
other in-flight changes, refs_werrres_ref_unsafe() only existed within
this patch series.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16 11:17:04 -07:00
Ævar Arnfjörð Bjarmason
6846f7248d refs tests: ignore ignore errno in test-ref-store helper
The cmd_resolve_ref() function has always ignored errno on failure,
but let's do so explicitly when using the refs_resolve_ref_unsafe()
function.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16 11:17:03 -07:00
Ævar Arnfjörð Bjarmason
5ac15ad250 reflog tests: add --updateref tests
Add tests that cover blindspots in "git reflog delete --updateref"
behavior. Before this change removing the "type & REF_ISSYMREF" check
added in 5e6f003ca8 (reflog_expire(): ignore --updateref for symbolic
references, 2015-03-03) would not fail any tests.

The "--updateref" option was added in 55f1056537 (git-reflog: add
option --updateref to write the last reflog sha1 into the ref,
2008-02-22) for use in git-stash.sh, see e25d5f9c82 (git-stash: add
new 'drop' subcommand, 2008-02-22).

Even though the regression test I need is just the "C" case here,
let's test all these combinations for good measure. I started out
doing these as a for-loop, but I think this is more readable.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16 11:17:02 -07:00
Han-Wen Nienhuys
aa30fe1481 branch tests: test for errno propagating on failing read
Add a test for "git branch" to cover the case where .git/refs is
symlinked. To check availability, refs_verify_refname_available() will
run refs_read_raw_ref() on each prefix, leading to a read() from
.git/refs (which is a directory).

It would probably be more robust to re-issue the lstat() as a normal
stat(), in which case, we would fall back to the directory case, but
for now let's just test for the existing behavior as-is. This test
covers a regression in a commit that only ever made it to "next", see
[1].

1. http://lore.kernel.org/git/pull.1068.git.git.1629203489546.gitgitgadget@gmail.com

Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16 11:17:02 -07:00
Glen Choo
a897ab7ed1 gc: perform incremental repack when implictly enabled
builtin/gc.c has two ways of checking if multi-pack-index is enabled:
- git_config_get_bool() in incremental_repack_auto_condition()
- the_repository->settings.core_multi_pack_index in
  maintenance_task_incremental_repack()

The two implementations have existed since the incremental-repack task
was introduced in e841a79a13 (maintenance: add incremental-repack auto
condition, 2020-09-25). These two values can diverge because
prepare_repo_settings() enables the feature in the_repository->settings
by default.

In the case where core.multiPackIndex is not set in the config, the auto
condition would fail, causing the incremental-repack task to not be
run. Because we always want to consider the default values, we should
always use the_repository->settings.

Standardize on using the_repository->settings.core_multi_pack_index to
check if multi-pack-index is enabled.

Signed-off-by: Glen Choo <chooglen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 14:30:10 -07:00
Glen Choo
dc5570872f fsck: verify multi-pack-index when implictly enabled
Like the previous commit, change fsck to check the
"core_multi_pack_index" variable set in "repo-settings.c" instead of
reading the "core.multiPackIndex" config variable. This fixes a bug
where we wouldn't verify midx if the config key was missing. This bug
was introduced in 18e449f86b (midx: enable core.multiPackIndex by
default, 2020-09-25) where core.multiPackIndex was turned on by default.

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Glen Choo <chooglen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 14:30:08 -07:00
Glen Choo
f30e4d854b fsck: verify commit graph when implicitly enabled
Change fsck to check the "core_commit_graph" variable set in
"repo-settings.c" instead of reading the "core.commitGraph" variable.
This fixes a bug where we wouldn't verify the commit-graph if the
config key was missing. This bug was introduced in
31b1de6a09 (commit-graph: turn on commit-graph by default, 2019-08-13),
where core.commitGraph was turned on by default.

Add tests to "t5318-commit-graph.sh" to verify that fsck checks the
commit-graph as expected for the 3 values of core.commitGraph. Also,
disable GIT_TEST_COMMIT_GRAPH in t/t0410-partial-clone.sh because some
test cases use fsck in ways that assume that commit-graph checking is
disabled.

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Glen Choo <chooglen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 14:30:07 -07:00
Junio C Hamano
ed41385ad6 Merge branch 'ab/ignore-replace-while-working-on-commit-graph' into gc/use-repo-settings
* ab/ignore-replace-while-working-on-commit-graph:
  commit-graph: don't consider "replace" objects with "verify"
  commit-graph tests: fix another graph_git_two_modes() helper
  commit-graph tests: fix error-hiding graph_git_two_modes() helper
2021-10-15 14:30:00 -07:00
Hamza Mahfooz
ae39ba431a grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data
If we attempt to grep non-ascii log message text with an ascii pattern, we
run into the following issue:

    $ git log --color --author='.var.*Bjar' -1 origin/master | grep ^Author
    grep: (standard input): binary file matches

So, to fix this teach the grep code to use PCRE2_UTF, as long as the log
output is encoded in UTF-8.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Hamza Mahfooz <someguy@effective-light.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 12:45:39 -07:00
Ævar Arnfjörð Bjarmason
1c720357ce tests: stop using top-level "README" and "COPYING" files
In 459b8d22e5 (tests: do not borrow from COPYING and README from the
real source, 2015-02-15) tests that used "lib-diff.sh" (called
"diff-lib.sh" then) were made to stop relying on the top-level COPYING
file, but we still had other tests that referenced it.

Let's move them over to use the "COPYING_test_data" utility function
introduced in the preceding commit, and in the case of the one test
that needed the "README" file use a ROT 13 version of that "COPYING"
test data. That test added in afd222967c (Extend testing git-mv for
renaming of subdirectories, 2006-07-26) just needs more test data that's not the same as the "COPYING" test data, so a ROT 13 version will do.

This change removes the last references to ../{README,COPYING} in the
test suite.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 09:36:48 -07:00
Ævar Arnfjörð Bjarmason
15b808da74 "lib-diff" tests: make "README" and "COPYING" test data smaller
Follow-up the change in 459b8d22e5 (tests: do not borrow from COPYING
and README from the real source, 2015-02-15) by not shipping a full
copy of older versions of the top-level "COPYING" and "README" files.

The tests that use them just need the small blurb at the top of
"COPYING" as test data, or mock data that's dissimilar. Let's provide
that with a "COPYING_test_data" function instead.

We're not replacing this with some other generic test
data (e.g. "lorum ipsum") because these tests require test file header
to be the old "COPYING" file. See e.g. "t4003-diff-rename-1.sh" which
changes the file, and then does full "test_cmp" comparisons on the
resulting "git diff" output.

This change only changes tests that used the "lib-diff.sh" library,
but splits up what they need into a new "lib-diff-data.sh". A
subsequent commit will change related tests that were missed in
459b8d22e5.

For the test in "t4008-diff-break-rewrite.sh" the "README" file can go
away in favor of echoing the line "some dissimilar content" to a file
in the one test that needed it.

The point of that test is to start with files "A" and "B", and then
have A be more similar to the state of "B" than to its old version (by
copying over the content from the "COPYING" file). Just comparing the
pre-image of "some dissimilar content" and later a munged version of
the "COPYING" output serves that purpose.

While we're at it get rid of a stray "echo $tree" debugging line added
in 15d061b435 ([PATCH] Fix the way diffcore-rename records unremoved
source., 2005-05-27), and stop calling "hash-object" to get the hash
of an object we've just added to the index. We can instead extract
that information from the index itself with "rev-parse".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 09:36:46 -07:00
Ævar Arnfjörð Bjarmason
095d112f8c commit-graph: don't consider "replace" objects with "verify"
Extend the code added in d6538246d3 (commit-graph: not compatible
with replace objects, 2018-08-20) which ignored replace objects in the
"write" command to ignore it in the "verify" command too.

We can just move this assignment to the cmd_commit_graph(), it
dispatches to "write" and "verify", and we're unlikely to ever get a
sub-command that would like to consider replace refs.

This will make tests added in eddc1f556c (mktag tests: test
update-ref and reachable fsck, 2021-06-17) pass in combination with
the "GIT_TEST_COMMIT_GRAPH" mode added in 859fdc0c3c (commit-graph:
define GIT_TEST_COMMIT_GRAPH, 2018-08-29), except that mode is
currently broken (but is being fixed concurrently). See the discussion
starting at [1].

1. https://lore.kernel.org/git/87wnmihswp.fsf@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 09:21:30 -07:00
Ævar Arnfjörð Bjarmason
a046aa38ca commit-graph tests: fix another graph_git_two_modes() helper
In 135a712375 (commit-graph: add --split option to builtin,
2019-06-18) this function was copy/pasted to the split commit-graph
tests, as in the preceding commit we need to fix this to use
&&-chaining, so it won't be hiding errors.

Unlike its sister function in "t5318-commit-graph.sh", which we got
lucky with, this one was hiding a real test failure. A tests added in
c523035cbd (commit-graph: allow cross-alternate chains, 2019-06-18)
has never worked as intended. Unlike most other graph_git_behavior
uses in this file it clones the repository into a sub-directory, so
we'll need to refer to "commits/6" as "origin/commits/6".

It's not easy to simply move the "graph_git_behavior" to the test
above it, since it itself spawns a "test_expect_success". Let's
instead add support to "graph_git_behavior()" and
"graph_git_two_modes()" to pass a "-C" argument to git.

We also need to add a "test -d fork" here, because otherwise we'll
fail on e.g.:

    GIT_SKIP_TESTS=t5324.13 ./t5324-split-commit-graph.sh

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 09:21:30 -07:00
Ævar Arnfjörð Bjarmason
3247919a75 commit-graph tests: fix error-hiding graph_git_two_modes() helper
The graph_git_two_modes() helper added in 177722b344 (commit:
integrate commit graph with commit parsing, 2018-04-10) didn't
&&-chain its "git commit-graph" invocations, which as can be seen with
SANITIZE=leak will happily mark tests as passing if both of these
commands die, since test_cmp() will be comparing two empty files.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 09:21:30 -07:00
Ævar Arnfjörð Bjarmason
5d22e18965 test-lib.sh: try to re-chmod & retry on failed trash removal
Try to re-chmod the trash directory on startup if we fail to "rm -rf"
it. This fixes problems where the test leaves the trash directory
behind in a bad permission state for whatever reason.

This fixes an interaction between [1] where t0004-unwritable.sh was
made to use "test_when_finished" for cleanup, and [2] which added the
"--immediate" mode. If a test in this file failed when running with
"--immediate" we wouldn't run the "test_when_finished" block, which
re-chmods the ".git/objects" directory (see [1]).

This can be demonstrated as e.g. (output snipped for less verbosity):

    $ ./t0004-unwritable.sh --run=3 --immediate
    ok 1 # skip setup (--run)
    ok 2 # skip write-tree should notice unwritable repository (--run)
    not ok 3 - commit should notice unwritable repository
    [...]
    $ ./t0004-unwritable.sh --run=3 --immediate
    rm: cannot remove '[...]/trash directory.t0004-unwritable/.git/objects/info': Permission denied
    FATAL: Cannot prepare test area
    [...]

Instead of some version of reverting [1] let's make the test-lib.sh
resilient to this edge-case, it will happen due to [1], but also
e.g. if the relevant "test-lib.sh" process is kill -9'd during the
test run. We should try harder to recover in this case. If we fail to
remove the test directory let's retry after (re-)chmod-ing it.

This doesn't need to be guarded by something that's equivalent to
"POSIXPERM" since if we don't support "chmod" we were about to fail
anyway.

Let's also discard any error output from (a possibly nonexisting)
"chmod", we'll fail on the subsequent "rm -rf" anyway, likewise for
the first "rm -rf" invocation, we don't want to get the "cannot
remove" output if we can get around it with the "chmod", but we do
want any error output from the second "rm -rf", in case that doesn't
fix the issue.

The lack of &&-chaining between the "chmod" and "rm -rf" is
intentional, if we fail the first "rm -rf", can't chmod, but then
succeed the second time around that's what we were hoping for. We just
want to nuke the directory, not carry forward every possible error
code or error message.

1. dbda967684 (t0004 (unwritable files): simplify error handling,
   2010-09-06)
2. b586744a86 (test: skip clean-up when running under --immediate
   mode, 2011-06-27)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15 08:54:33 -07:00
Ævar Arnfjörð Bjarmason
234383cd40 test-lib.sh: use "Bail out!" syntax on bad SANITIZE=leak use
Improve the "GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode added in
956d2e4639 (tests: add a test mode for SANITIZE=leak, run it in CI,
2021-09-23) to use a TAP "Bail out!" message when exiting. This will
cause the test run to exit immediately under a TAP consumer like
"prove(1)".

See 614fe01521 (test-lib: bail out when "-v" used under "prove",
2016-10-22) for the initial introduction of "Bail out!" to the
--verbose being amended here.

Before this compiling with "SANITIZE=" and running the tests with
"prove(1)" would cause all the tests to be run to the end (output
trimmed for fewer columns):

    $ GIT_TEST_PASSING_SANITIZE_LEAK=true make
    rm -f -r 'test-results'
    *** prove ***
    t0000-basic.sh ......... Dubious, test returned 1 (wstat 256, 0x100)
    No subtests run
    t0001-init.sh .......... Dubious, test returned 1 (wstat 256, 0x100)
    No subtests run
    [...output where we list every single t[0-9]*.sh file as failing snipped]

Whereas now we'll fail early, like this ("->" line wrapping added):

    $ GIT_TEST_PASSING_SANITIZE_LEAK=true make
    [...]

    t0000-basic.sh ..................................... Bailout called.  Further testing stopped:
    -> GIT_TEST_PASSING_SANITIZE_LEAK=true has no effect except when compiled with SANITIZE=leak
    FAILED--Further testing stopped: GIT_TEST_PASSING_SANITIZE_LEAK=true has no effect except
    -> when compiled with SANITIZE=leak
    make: *** [Makefile:53: prove] Error 1

This change also adds a red color to the "Bailout called" line, as
we're now using "say_color error". That improves existing output in
the case of e.g.:

    $ prove -j8 t[0-9]*.sh :: -v
    Bailout called.  Further testing stopped:  verbose mode forbidden under TAP harness; try --verbose-log
    FAILED--Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log

We don't need to have a "Bail out! " prefix when we're not running
under a TAP consumer (i.e. if test -n "$HARNESS_ACTIVE"), but let's
not make the output conditional on that. Showing it under e.g.:

    $ GIT_TEST_PASSING_SANITIZE_LEAK=true ./t0095-bloom.sh
    Bail out! GIT_TEST_PASSING_SANITIZE_LEAK=true has no effect except when compiled with SANITIZE=leak

Doesn't harm anything, and I don't think the (small) complexity of
only adding this if we're under "$HARNESS_ACTIVE" is worth it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-14 09:54:54 -07:00
Ævar Arnfjörð Bjarmason
8583bf7559 test-lib.sh: de-duplicate error() teardown code
De-duplicate the "finalize_junit_xml; GIT_EXIT_OK=t; exit 1" code
shared between the "error()" and "--immediate on failure" code paths,
in preparation for adding a third user in a subsequent commit.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-14 09:54:54 -07:00
Junio C Hamano
d7bc852151 Merge branch 'ab/align-parse-options-help'
When "git cmd -h" shows more than one line of usage text (e.g.
the cmd subcommand may take sub-sub-command), parse-options API
learned to align these lines, even across i18n/l10n.

* ab/align-parse-options-help:
  parse-options: properly align continued usage output
  git rev-parse --parseopt tests: add more usagestr tests
  send-pack: properly use parse_options() API for usage string
  parse-options API users: align usage output in C-strings
2021-10-13 15:15:58 -07:00
Junio C Hamano
62f035aee3 Merge branch 'ab/help-config-vars'
Teach "git help -c" into helping the command line completion of
configuration variables.

* ab/help-config-vars:
  help: move column config discovery to help.c library
  help / completion: make "git help" do the hard work
  help tests: test --config-for-completion option & output
  help: simplify by moving to OPT_CMDMODE()
  help: correct logic error in combining --all and --guides
  help: correct logic error in combining --all and --config
  help tests: add test for --config output
  help: correct usage & behavior of "git help --guides"
  help: correct the usage string in -h and documentation
2021-10-13 15:15:58 -07:00
Junio C Hamano
af303ee392 Merge branch 'jh/builtin-fsmonitor-part1'
Built-in fsmonitor (part 1).

* jh/builtin-fsmonitor-part1:
  t/helper/simple-ipc: convert test-simple-ipc to use start_bg_command
  run-command: create start_bg_command
  simple-ipc/ipc-win32: add Windows ACL to named pipe
  simple-ipc/ipc-win32: add trace2 debugging
  simple-ipc: move definition of ipc_active_state outside of ifdef
  simple-ipc: preparations for supporting binary messages.
  trace2: add trace2_child_ready() to report on background children
2021-10-13 15:15:58 -07:00
Junio C Hamano
cf006037bf Merge branch 'ab/lib-subtest'
Updates to the tests in t0000 to test the test framework.

* ab/lib-subtest:
  test-lib tests: get rid of copy/pasted mock test code
  test-lib tests: assert 1 exit code, not non-zero
  test-lib tests: refactor common part of check_sub_test_lib_test*()
  test-lib tests: avoid subshell for "test_cmp" for readability
  test-lib tests: don't provide a description for the sub-tests
  test-lib tests: split up "write and run" into two functions
  test-lib tests: move "run_sub_test" to a new lib-subtest.sh
2021-10-13 15:15:57 -07:00
Junio C Hamano
a7c2daa06d Merge branch 'en/removing-untracked-fixes'
Various fixes in code paths that move untracked files away to make room.

* en/removing-untracked-fixes:
  Documentation: call out commands that nuke untracked files/directories
  Comment important codepaths regarding nuking untracked files/dirs
  unpack-trees: avoid nuking untracked dir in way of locally deleted file
  unpack-trees: avoid nuking untracked dir in way of unmerged file
  Change unpack_trees' 'reset' flag into an enum
  Remove ignored files by default when they are in the way
  unpack-trees: make dir an internal-only struct
  unpack-trees: introduce preserve_ignored to unpack_trees_options
  read-tree, merge-recursive: overwrite ignored files by default
  checkout, read-tree: fix leak of unpack_trees_options.dir
  t2500: add various tests for nuking untracked files
2021-10-13 15:15:57 -07:00
Junio C Hamano
1fdfb774aa Merge branch 'mt/grep-submodule-textconv'
"git grep --recurse-submodules" takes trees and blobs from the
submodule repository, but the textconv settings when processing a
blob from the submodule is not taken from the submodule repository.
A test is added to demonstrate the issue, without fixing it.

* mt/grep-submodule-textconv:
  grep: demonstrate bug with textconv attributes and submodules
2021-10-13 15:15:56 -07:00
Junio C Hamano
2d498a7c89 Merge branch 'ds/add-rm-with-sparse-index'
"git add", "git mv", and "git rm" have been adjusted to avoid
updating paths outside of the sparse-checkout definition unless
the user specifies a "--sparse" option.

* ds/add-rm-with-sparse-index:
  advice: update message to suggest '--sparse'
  mv: refuse to move sparse paths
  rm: skip sparse paths with missing SKIP_WORKTREE
  rm: add --sparse option
  add: update --renormalize to skip sparse paths
  add: update --chmod to skip sparse paths
  add: implement the --sparse option
  add: skip tracked paths outside sparse-checkout cone
  add: fail when adding an untracked sparse file
  dir: fix pattern matching on dirs
  dir: select directories correctly
  t1092: behavior for adding sparse files
  t3705: test that 'sparse_entry' is unstaged
2021-10-13 15:15:56 -07:00
Jeff King
e4c497a194 urlmatch: add underscore to URL_HOST_CHARS
When parsing a URL to normalize it, we allow hostnames to contain only
dot (".") or dash ("-"), plus brackets and colons for IPv6 literals.
This matches the old URL standard in RFC 1738, which says:

  host           = hostname | hostnumber
  hostname       = *[ domainlabel "." ] toplabel
  domainlabel    = alphadigit | alphadigit *[ alphadigit | "-" ] alphadigit

But this was later updated by RFC 3986, which is more liberal:

  host        = IP-literal / IPv4address / reg-name
  reg-name    = *( unreserved / pct-encoded / sub-delims )
  unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"

While names with underscore in them are not common and possibly violate
some DNS rules, they do work in practice, and we will happily contact
them over http://, git://, or ssh://. It seems odd to ignore them for
purposes of URL matching, especially when the URL RFC seems to allow
them.

There shouldn't be any downside here. It's not a syntactically
significant character in a URL, so we won't be confused about parsing;
we'd have simply rejected such a URL previously (the test here checks
the url code directly, but the obvious user-visible effect would be
failing to match credential.http://foo_bar.example.com.helper, or
similar config in http.<url>.*).

Arguably we'd want to allow tilde ("~") here, too. There's likewise
probably no downside, but I didn't add it simply because it seems like
an even less likely character to appear in a hostname.

Reported-by: Alex Waite <alex@waite.eu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:29:25 -07:00
Ævar Arnfjörð Bjarmason
7ff24785cb leak tests: mark some misc tests as passing with SANITIZE=leak
Mark some tests that match "*{mktree,commit,diff,grep,rm,merge,hunk}*"
as passing when git is compiled with SANITIZE=leak. They'll now be
listed as running under the "GIT_TEST_PASSING_SANITIZE_LEAK=true" test
mode (the "linux-leaks" CI target).

These were picked because we still have a lot of failures in adjacent
areas, and we didn't have much if any coverage of e.g. grep and diff
before this change, we could still whitelist a lot more tests, but
let's stop for now.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Ævar Arnfjörð Bjarmason
288a480621 leak tests: mark various "generic" tests as passing with SANITIZE=leak
Mark various "generic" tests as passing when git is compiled with
SANITIZE=leak. These tests were subjectively picked from the lists of
passing tests since they're all small, and test some generic feature
such as wildmatch(), commonly used environment variables, ident
parsing etc.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Ævar Arnfjörð Bjarmason
98300c8157 leak tests: mark some read-tree tests as passing with SANITIZE=leak
Mark some tests that match "*read-tree*" as passing when git is
compiled with SANITIZE=leak. They'll now be listed as running under
the "GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode (the "linux-leaks"
CI target). We still have around half the tests that match
"*read-tree*" failing, but let's whitelist those that don't.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Ævar Arnfjörð Bjarmason
0b3481c9ab leak tests: mark some ls-files tests as passing with SANITIZE=leak
Mark some tests that match "*ls-files*" as passing when git is
compiled with SANITIZE=leak. They'll now be listed as running under
the "GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode (the "linux-leaks"
CI target). We still have others that match '*ls-files*" that fail
under SANITIZE=leak.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Ævar Arnfjörð Bjarmason
b7bcdbdb48 leak tests: mark all checkout-index tests as passing with SANITIZE=leak
Mark some tests that match "*{checkout,switch}*" as passing when git
is compiled with SANITIZE=leak. They'll now be listed as running under
the "GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode (the "linux-leaks"
CI target).

Unfortunately almost all of those tests fail when compiled with
SANITIZE=leak, these only pass because they run "checkout-index", not
the main "checkout" command.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Ævar Arnfjörð Bjarmason
0033ab3f59 leak tests: mark all trace2 tests as passing with SANITIZE=leak
Mark all tests that match "*trace2*" as passing when git is compiled
with SANITIZE=leak. They'll now be listed as running under the
"GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode (the "linux-leaks" CI
target).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Ævar Arnfjörð Bjarmason
809aeedb0e leak tests: mark all ls-tree tests as passing with SANITIZE=leak
Mark those tests that match "*ls-tree*" as passing when git is
compiled with SANITIZE=leak. They'll now be listed as running under
the "GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode (the "linux-leaks"
CI target).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Ævar Arnfjörð Bjarmason
fdc8f79f1f leak tests: run various "test-tool" tests in t00*.sh SANITIZE=leak
Mark various existing tests in t00*.sh that invoke a "test-tool" with
as passing when git is compiled with SANITIZE=leak.

They'll now be listed as running under the
"GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode (the "linux-leaks" CI
target).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Ævar Arnfjörð Bjarmason
c150064dbe leak tests: run various built-in tests in t00*.sh SANITIZE=leak
Mark various existing tests in t00*.sh that invoke git built-ins with
TEST_PASSES_SANITIZE_LEAK=true as passing when git is compiled with
SANITIZE=leak.

They'll now be listed as running under the
"GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode (the "linux-leaks" CI
target).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12 18:23:24 -07:00
Junio C Hamano
9e25a2e85a Merge branch 'ah/connect-parse-feature-v0-fix' into maint
Protocol v0 clients can get stuck parsing a malformed feature line.

* ah/connect-parse-feature-v0-fix:
  connect: also update offset for features without values
2021-10-12 13:51:49 -07:00
Junio C Hamano
77edbde474 Merge branch 'jk/http-redact-fix' into maint
Sensitive data in the HTTP trace were supposed to be redacted, but
we failed to do so in HTTP/2 requests.

* jk/http-redact-fix:
  http: match headers case-insensitively when redacting
2021-10-12 13:51:48 -07:00
Junio C Hamano
ef09a7fbbe Merge branch 'da/difftool-dir-diff-symlink-fix' into maint
"git difftool --dir-diff" mishandled symbolic links.

* da/difftool-dir-diff-symlink-fix:
  difftool: fix symlink-file writing in dir-diff mode
2021-10-12 13:51:48 -07:00
Junio C Hamano
b59c06092f Merge branch 'cb/cvsserver' into maint
"git cvsserver" had a long-standing bug in its authentication code,
which has finally been corrected (it is unclear and is a separate
question if anybody is seriously using it, though).

* cb/cvsserver:
  Documentation: cleanup git-cvsserver
  git-cvsserver: protect against NULL in crypt(3)
  git-cvsserver: use crypt correctly to compare password hashes
2021-10-12 13:51:48 -07:00
Junio C Hamano
c365967f21 Merge branch 'jk/clone-unborn-head-in-bare' into maint
"git clone" from a repository whose HEAD is unborn into a bare
repository didn't follow the branch name the other side used, which
is corrected.

* jk/clone-unborn-head-in-bare:
  clone: handle unborn branch in bare repos
2021-10-12 13:51:47 -07:00
Junio C Hamano
e61304f21d Merge branch 'en/stash-df-fix' into maint
"git stash", where the tentative change involves changing a
directory to a file (or vice versa), was confused, which has been
corrected.

* en/stash-df-fix:
  stash: restore untracked files AFTER restoring tracked files
  stash: avoid feeding directories to update-index
  t3903: document a pair of directory/file bugs
2021-10-12 13:51:47 -07:00
Junio C Hamano
b809c3d900 Merge branch 'en/am-abort-fix' into maint
When "git am --abort" fails to abort correctly, it still exited
with exit status of 0, which has been corrected.

* en/am-abort-fix:
  am: fix incorrect exit status on am fail to abort
  t4151: add a few am --abort tests
  git-am.txt: clarify --abort behavior
2021-10-12 13:51:46 -07:00
Junio C Hamano
b5f309dc7f Merge branch 'ps/update-ref-batch-flush' into maint
"git update-ref --stdin" failed to flush its output as needed,
which potentially led the conversation to a deadlock.

* ps/update-ref-batch-flush:
  t1400: avoid SIGPIPE race condition on fifo
  update-ref: fix streaming of status updates
2021-10-12 13:51:46 -07:00
Junio C Hamano
62a7648a5e Merge branch 'jc/trivial-threeway-binary-merge' into maint
The "git apply -3" code path learned not to bother the lower level
merge machinery when the three-way merge can be trivially resolved
without the content level merge.

* jc/trivial-threeway-binary-merge:
  apply: resolve trivial merge without hitting ll-merge with "--3way"
2021-10-12 13:51:45 -07:00
Junio C Hamano
1af632444b Merge branch 'ab/send-email-config-fix' into maint
Regression fix.

* ab/send-email-config-fix:
  send-email: fix a "first config key wins" regression in v2.33.0
2021-10-12 13:51:44 -07:00
Junio C Hamano
d9e2677559 Merge branch 'jk/log-warn-on-bogus-encoding' into maint
Doc update plus improved error reporting.

* jk/log-warn-on-bogus-encoding:
  docs: use "character encoding" to refer to commit-object encoding
  logmsg_reencode(): warn when iconv() fails
2021-10-12 13:51:43 -07:00
Junio C Hamano
48939c572c Merge branch 'tk/fast-export-anonymized-tag-fix' into maint
The output from "git fast-export", when its anonymization feature
is in use, showed an annotated tag incorrectly.

* tk/fast-export-anonymized-tag-fix:
  fast-export: fix anonymized tag using original length
2021-10-12 13:51:43 -07:00
Junio C Hamano
f11b01bdd5 Merge branch 'mh/send-email-reset-in-reply-to' into maint
Even when running "git send-email" without its own threaded
discussion support, a threading related header in one message is
carried over to the subsequent message to result in an unwanted
threading, which has been corrected.

* mh/send-email-reset-in-reply-to:
  send-email: avoid incorrect header propagation
2021-10-12 13:51:43 -07:00
Junio C Hamano
e4bea4a67d Merge branch 'sg/set-ceiling-during-tests' into maint
Buggy tests could damage repositories outside the throw-away test
area we created.  We now by default export GIT_CEILING_DIRECTORIES
to limit the damage from such a stray test.

* sg/set-ceiling-during-tests:
  test-lib: set GIT_CEILING_DIRECTORIES to protect the surrounding repository
2021-10-12 13:51:42 -07:00
Junio C Hamano
0a5af02acb Merge branch 'ka/want-ref-in-namespace' into maint
"git upload-pack" which runs on the other side of "git fetch"
forgot to take the ref namespaces into account when handling
want-ref requests.

* ka/want-ref-in-namespace:
  docs: clarify the interaction of transfer.hideRefs and namespaces
  upload-pack.c: treat want-ref relative to namespace
  t5730: introduce fetch command helper
2021-10-12 13:51:42 -07:00
Junio C Hamano
69247e283c Merge branch 'sg/column-nl' into maint
The parser for the "--nl" option of "git column" has been
corrected.

* sg/column-nl:
  column: fix parsing of the '--nl' option
2021-10-12 13:51:41 -07:00
Junio C Hamano
474e4f9b55 Merge branch 'rs/branch-allow-deleting-dangling' into maint
"git branch -D <branch>" used to refuse to remove a broken branch
ref that points at a missing commit, which has been corrected.

* rs/branch-allow-deleting-dangling:
  branch: allow deleting dangling branches with --force
2021-10-12 13:51:41 -07:00
Junio C Hamano
cd9a57f6a0 Merge branch 'mt/quiet-with-delayed-checkout' into maint
The delayed checkout code path in "git checkout" etc. were chatty
even when --quiet and/or --no-progress options were given.

* mt/quiet-with-delayed-checkout:
  checkout: make delayed checkout respect --quiet and --no-progress
2021-10-12 13:51:40 -07:00
Junio C Hamano
872c9e67ec Merge branch 'dd/diff-files-unmerged-fix' into maint
"git diff --relative" segfaulted and/or produced incorrect result
when there are unmerged paths.

* dd/diff-files-unmerged-fix:
  diff-lib: ignore paths that are outside $cwd if --relative asked
2021-10-12 13:51:40 -07:00
Junio C Hamano
3fb1d4da5e Merge branch 'pw/rebase-r-fixes' into maint
Various bugs in "git rebase -r" have been fixed.

* pw/rebase-r-fixes:
  rebase -r: fix merge -c with a merge strategy
  rebase -r: don't write .git/MERGE_MSG when fast-forwarding
  rebase -i: add another reword test
  rebase -r: make 'merge -c' behave like reword
2021-10-12 13:51:39 -07:00
Junio C Hamano
8b02ffee3f Merge branch 'pw/rebase-skip-final-fix' into maint
Checking out all the paths from HEAD during the last conflicted
step in "git rebase" and continuing would cause the step to be
skipped (which is expected), but leaves MERGE_MSG file behind in
$GIT_DIR and confuses the next "git commit", which has been
corrected.

* pw/rebase-skip-final-fix:
  rebase --continue: remove .git/MERGE_MSG
  rebase --apply: restore some tests
  t3403: fix commit authorship
2021-10-12 13:51:39 -07:00
Junio C Hamano
444b8548b1 Merge branch 'jk/commit-edit-fixup-fix' into maint
"git commit --fixup" now works with "--edit" again, after it was
broken in v2.32.

* jk/commit-edit-fixup-fix:
  commit: restore --edit when combined with --fixup
2021-10-12 13:51:38 -07:00
Junio C Hamano
1725c4c64b Merge branch 'jk/apply-binary-hunk-parsing-fix' into maint
"git apply" miscounted the bytes and failed to read to the end of
binary hunks.

* jk/apply-binary-hunk-parsing-fix:
  apply: keep buffer/size pair in sync when parsing binary hunks
2021-10-12 13:51:37 -07:00
Junio C Hamano
b20f67a659 Merge branch 'en/pull-conflicting-options' into maint
"git pull" had various corner cases that were not well thought out
around its --rebase backend, e.g. "git pull --ff-only" did not stop
but went ahead and rebased when the history on other side is not a
descendant of our history.  The series tries to fix them up.

* en/pull-conflicting-options:
  pull: fix handling of multiple heads
  pull: update docs & code for option compatibility with rebasing
  pull: abort by default when fast-forwarding is not possible
  pull: make --rebase and --no-rebase override pull.ff=only
  pull: since --ff-only overrides, handle it first
  pull: abort if --ff-only is given and fast-forwarding is impossible
  t7601: add tests of interactions with multiple merge heads and config
  t7601: test interaction of merge/rebase/fast-forward flags and options
2021-10-12 13:51:36 -07:00