Commit Graph

19763 Commits

Author SHA1 Message Date
Ævar Arnfjörð Bjarmason
fee65b194d t/Makefile: don't remove test-results in "clean-except-prove-cache"
When "make test" is run with the default of "DEFAULT_TEST_TARGET=test"
we'll leave the "test-results" directory in-place, but don't do so for
the "prove" target.

The reason for this is that when 28d836c815 (test: allow running the
tests under "prove", 2010-10-14) allowed for running the tests under
"prove" there was no point in leaving the "test-results" in place.

The "prove" target provides its own summary, so we don't need to run
"aggregate-results", which is the reason we have "test-results" in the
first place. See 2d84e9fb6d (Modify test-lib.sh to output stats to
t/test-results/*, 2008-06-08).

But in a subsequent commit test-lib.sh will start emitting reports of
memory leaks in test-results/*, and it will be useful to analyze these
after the fact.

This wouldn't be a problem as failing tests will halt the removal of
the files (we'll never reach "clean-except-prove-cache" from the
"prove" target), but will be subsequently as we'll want to report a
successful run, but might still have e.g. logs of known memory leaks
in test-results/*.

So let's stop removing this, it's sufficient that "make clean" removes
it, and that "pre-clean" (which both "test" and "prove" depend on)
will remove it, i.e. we'll never have a stale "test-results" because
of this change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-27 16:35:40 -07:00
Ævar Arnfjörð Bjarmason
366bd129dc test-lib: add a SANITIZE=leak logging mode
Add the ability to run the test suite under a new
"GIT_TEST_SANITIZE_LEAK_LOG=true" mode, when true we'll log the leaks
we find an a new "test-results/<test-name>.leak" directory.

That new path is consistent with the existing
"test-results/<test-name>.<type>" results, except that those are all
files, not directories.

We also set "log_exe_name=1" to include the name of the executable in
the filename. This gives us files like "trace.git.<pid>" instead of
the default of "trace.<pid>". I.e. we'll be able to distinguish "git"
leaks from "test-tool", "git-daemon" etc.

We then set "dedup_token_length" to non-zero ("0" is the default) to
succinctly log a token we can de-duplicate these stacktraces on. The
string is simply a one-line stack-trace with only function names up to
N frames, which we limit at "9999" as a shorthand for
"infinite" (there appears to be no way to say "no limit").

With these combined we can now easily get e.g. the top 10 leaks in the
test suite grouped by full stacktrace:

    grep -o -P -h '(?<=DEDUP_TOKEN: ).*' test-results/*.leak/trace.git.* | sort | uniq -c | sort -nr | head -n 10

Or add "grep -E -o '[^-]+'" to that to group by functions instead of
stack traces:

    grep -o -P -h '(?<=DEDUP_TOKEN: ).*' test-results/*.leak/trace.git.* | grep -E -o '[^-]+' | sort | uniq -c | sort -nr | head -n 20

This new mode requires git to be compiled with SANITIZE=leak, rather
than explaining that in the documentation let's make it
self-documenting by bailing out if the user asks for this without git
having been compiled with SANITIZE=leak, as we do with
GIT_TEST_PASSING_SANITIZE_LEAK=true.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-27 16:35:40 -07:00
Ævar Arnfjörð Bjarmason
ac8e3e94e5 t/README: reword the "GIT_TEST_PASSING_SANITIZE_LEAK" description
Reword the documentation added in 956d2e4639 (tests: add a test mode
for SANITIZE=leak, run it in CI, 2021-09-23) for brevity.

The comment added in the same commit was also misleading: We skip
certain tests if SANITIZE=leak and GIT_TEST_PASSING_SANITIZE_LEAK=true,
not if we're compiled with SANITIZE=leak. Let's just remove the
comment, the control flow here is obvious enough that the code can
speak for itself.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-27 16:35:40 -07:00
Ævar Arnfjörð Bjarmason
46fb057aaa test-lib: add a --invert-exit-code switch
Add the ability to have those tests that fail return 0, and those
tests that succeed return 1. This is useful e.g. to run "--stress"
tests on tests that fail 99% of the time on some setup, i.e. to smoke
out the flaky run which yielded success.

In a subsequent commit a new SANITIZE=leak mode will make use of this.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-27 16:35:40 -07:00
Ævar Arnfjörð Bjarmason
25c2351d85 test-lib: fix GIT_EXIT_OK logic errors, use BAIL_OUT
Change various "exit 1" checks that happened after our "die" handler
had been set up to use BAIL_OUT instead. See 234383cd40 (test-lib.sh:
use "Bail out!" syntax on bad SANITIZE=leak use, 2021-10-14) for the
benefits of the BAIL_OUT function.

The previous use of "error" here was not a logic error, but the "exit"
without "GIT_EXIT_OK" would emit the "FATAL: Unexpected exit with code
$code" message on top of the error we wanted to emit.

Since we'd also like to stop "prove" in its tracks here, the right
thing to do is to emit a "Bail out!" message.

Let's also move the "GIT_EXIT_OK=t" assignments to just above the
"exit [01]" in "test_done". It's not OK if we exit in
e.g. finalize_test_output.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-27 16:35:39 -07:00
Ævar Arnfjörð Bjarmason
e0258f15cb test-lib: don't set GIT_EXIT_OK before calling test_atexit_handler
Change the control flow in test_done so that we'll set GIT_EXIT_OK=t
after we call test_atexit_handler(). This seems to have been a mistake
in 900721e15c (test-lib: introduce 'test_atexit', 2019-03-13). It
doesn't make sense to allow our "atexit" handling to call "exit"
without us emitting the errors we'll emit without GIT_EXIT_OK=t being
set.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-27 16:35:39 -07:00
Ævar Arnfjörð Bjarmason
6d00680de2 test-lib: use $1, not $@ in test_known_broken_{ok,failure}_
Clarify that these two functions never take N arguments, they'll only
ever receive one. They've needlessly used $@ over $1 since
41ac414ea2 (Sane use of test_expect_failure, 2008-02-01).

In the future we might want to pass the test source to these, but now
that's not the case. This preparatory change helps to clarify a
follow-up change.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-27 16:35:39 -07:00
Junio C Hamano
a31dbaebb1 Merge branch 'js/ci-github-workflow-markup'
A fix for a regression in test framework.

* js/ci-github-workflow-markup:
  tests: fix incorrect --write-junit-xml code
2022-07-22 15:04:03 -07:00
Junio C Hamano
18bbc795fc Merge branch 'gc/bare-repo-discovery'
Introduce a discovery.barerepository configuration variable that
allows users to forbid discovery of bare repositories.

* gc/bare-repo-discovery:
  setup.c: create `safe.bareRepository`
  safe.directory: use git_protected_config()
  config: learn `git_protected_config()`
  Documentation: define protected configuration
  Documentation/git-config.txt: add SCOPES section
2022-07-22 15:04:02 -07:00
Junio C Hamano
4b8cdff8ba Merge branch 'll/curl-accept-language'
Earlier, HTTP transport clients learned to tell the server side
what locale they are in by sending Accept-Language HTTP header, but
this was done only for some requests but not others.

* ll/curl-accept-language:
  remote-curl: send Accept-Language header to server
2022-07-19 16:40:19 -07:00
Junio C Hamano
cf92cb29e9 Merge branch 'jk/clone-unborn-confusion'
"git clone" from a repository with some ref whose HEAD is unborn
did not set the HEAD in the resulting repository correctly, which
has been corrected.

* jk/clone-unborn-confusion:
  clone: move unborn head creation to update_head()
  clone: use remote branch if it matches default HEAD
  clone: propagate empty remote HEAD even with other branches
  clone: drop extra newline from warning message
2022-07-19 16:40:17 -07:00
Junio C Hamano
99c0d94eaa Merge branch 'hx/lookup-commit-in-graph-fix'
A corner case bug where lazily fetching objects from a promisor
remote resulted in infinite recursion has been corrected.

* hx/lookup-commit-in-graph-fix:
  t5330: remove run_with_limited_processses()
  commit-graph.c: no lazy fetch in lookup_commit_in_graph()
2022-07-19 16:40:16 -07:00
Junio C Hamano
418aef9055 Merge branch 'jc/resolve-undo'
The resolve-undo information in the index was not protected against
GC, which has been corrected.

* jc/resolve-undo:
  fsck: do not dereference NULL while checking resolve-undo data
  revision: mark blobs needed for resolve-undo as reachable
2022-07-19 16:40:16 -07:00
Junio C Hamano
6d003858e5 Merge branch 'gc/submodule-use-super-prefix'
Another step to rewrite more parts of "git submodule" in C.

* gc/submodule-use-super-prefix:
  submodule--helper: remove display path helper
  submodule--helper update: use --super-prefix
  submodule--helper: remove unused SUPPORT_SUPER_PREFIX flags
  submodule--helper: use correct display path helper
  submodule--helper: don't recreate recursive prefix
  submodule--helper update: use display path helper
  submodule--helper tests: add missing "display path" coverage
2022-07-18 13:31:56 -07:00
Junio C Hamano
e3349f2888 Merge branch 'en/merge-dual-dir-renames-fix'
Fixes a long-standing corner case bug around directory renames in
the merge-ort strategy.

* en/merge-dual-dir-renames-fix:
  merge-ort: fix issue with dual rename and add/add conflict
  merge-ort: shuffle the computation and cleanup of potential collisions
  merge-ort: make a separate function for freeing struct collisions
  merge-ort: small cleanups of check_for_directory_rename
  t6423: add tests of dual directory rename plus add/add conflict
2022-07-18 13:31:56 -07:00
Junio C Hamano
3d3874d537 Merge branch 'ab/test-without-templates'
Tweak tests so that they still work when the "git init" template
did not create .git/info directory.

* ab/test-without-templates:
  tests: don't assume a .git/info for .git/info/sparse-checkout
  tests: don't assume a .git/info for .git/info/exclude
  tests: don't assume a .git/info for .git/info/refs
  tests: don't assume a .git/info for .git/info/attributes
  tests: don't assume a .git/info for .git/info/grafts
  tests: don't depend on template-created .git/branches
  t0008: don't rely on default ".git/info/exclude"
2022-07-18 13:31:55 -07:00
Junio C Hamano
48e88a4862 Merge branch 'ab/build-gitweb'
Teach "make all" to build gitweb as well.

* ab/build-gitweb:
  gitweb/Makefile: add a "NO_GITWEB" parameter
  Makefile: build 'gitweb' in the default target
  gitweb/Makefile: include in top-level Makefile
  gitweb: remove "test" and "test-installed" targets
  gitweb/Makefile: prepare to merge into top-level Makefile
  gitweb/Makefile: clear up and de-duplicate the gitweb.{css,js} vars
  gitweb/Makefile: add a $(GITWEB_ALL) variable
  gitweb/Makefile: define all .PHONY prerequisites inline
2022-07-18 13:31:55 -07:00
Junio C Hamano
f63ac61fbf Merge branch 'ab/test-tool-leakfix'
Plug various memory leaks in test-tool commands.

* ab/test-tool-leakfix:
  test-tool delta: fix a memory leak
  test-tool ref-store: fix a memory leak
  test-tool bloom: fix memory leaks
  test-tool json-writer: fix memory leaks
  test-tool regex: call regfree(), fix memory leaks
  test-tool urlmatch-normalization: fix a memory leak
  test-tool {dump,scrap}-cache-tree: fix memory leaks
  test-tool path-utils: fix a memory leak
  test-tool test-hash: fix a memory leak
2022-07-18 13:31:54 -07:00
Junio C Hamano
44357f64f6 Merge branch 'ab/leakfix'
Plug various memory leaks.

* ab/leakfix:
  pull: fix a "struct oid_array" memory leak
  cat-file: fix a common "struct object_context" memory leak
  gc: fix a memory leak
  checkout: avoid "struct unpack_trees_options" leak
  merge-file: fix memory leaks on error path
  merge-file: refactor for subsequent memory leak fix
  cat-file: fix a memory leak in --batch-command mode
  revert: free "struct replay_opts" members
  submodule.c: free() memory from xgetcwd()
  clone: fix memory leak in wanted_peer_refs()
  check-ref-format: fix trivial memory leak
2022-07-18 13:31:54 -07:00
Glen Choo
8d1a744820 setup.c: create safe.bareRepository
There is a known social engineering attack that takes advantage of the
fact that a working tree can include an entire bare repository,
including a config file. A user could run a Git command inside the bare
repository thinking that the config file of the 'outer' repository would
be used, but in reality, the bare repository's config file (which is
attacker-controlled) is used, which may result in arbitrary code
execution. See [1] for a fuller description and deeper discussion.

A simple mitigation is to forbid bare repositories unless specified via
`--git-dir` or `GIT_DIR`. In environments that don't use bare
repositories, this would be minimally disruptive.

Create a config variable, `safe.bareRepository`, that tells Git whether
or not to die() when working with a bare repository. This config is an
enum of:

- "all": allow all bare repositories (this is the default)
- "explicit": only allow bare repositories specified via --git-dir
  or GIT_DIR.

If we want to protect users from such attacks by default, neither value
will suffice - "all" provides no protection, but "explicit" is
impractical for bare repository users. A more usable default would be to
allow only non-embedded bare repositories ([2] contains one such
proposal), but detecting if a repository is embedded is potentially
non-trivial, so this work is not implemented in this series.

[1]: https://lore.kernel.org/git/kl6lsfqpygsj.fsf@chooglen-macbookpro.roam.corp.google.com
[2]: https://lore.kernel.org/git/5b969c5e-e802-c447-ad25-6acc0b784582@github.com

Signed-off-by: Glen Choo <chooglen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-14 15:08:29 -07:00
Glen Choo
6061601d9f safe.directory: use git_protected_config()
Use git_protected_config() to read `safe.directory` instead of
read_very_early_config(), making it 'protected configuration only'.

As a result, `safe.directory` now respects "-c", so update the tests and
docs accordingly. It used to ignore "-c" due to how it was implemented,
not because of security or correctness concerns [1].

[1] https://lore.kernel.org/git/xmqqlevabcsu.fsf@gitster.g/

Signed-off-by: Glen Choo <chooglen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-14 15:08:29 -07:00
Glen Choo
5b3c650777 config: learn git_protected_config()
`uploadpack.packObjectsHook` is the only 'protected configuration only'
variable today, but we've noted that `safe.directory` and the upcoming
`safe.bareRepository` should also be 'protected configuration only'. So,
for consistency, we'd like to have a single implementation for protected
configuration.

The primary constraints are:

1. Reading from protected configuration should be fast. Nearly all "git"
   commands inside a bare repository will read both `safe.directory` and
   `safe.bareRepository`, so we cannot afford to be slow.

2. Protected configuration must be readable when the gitdir is not
   known. `safe.directory` and `safe.bareRepository` both affect
   repository discovery and the gitdir is not known at that point [1].

The chosen implementation in this commit is to read protected
configuration and cache the values in a global configset. This is
similar to the caching behavior we get with the_repository->config.

Introduce git_protected_config(), which reads protected configuration
and caches them in the global configset protected_config. Then, refactor
`uploadpack.packObjectsHook` to use git_protected_config().

The protected configuration functions are named similarly to their
non-protected counterparts, e.g. git_protected_config_check_init() vs
git_config_check_init().

In light of constraint 1, this implementation can still be improved.
git_protected_config() iterates through every variable in
protected_config, which is wasteful, but it makes the conversion simple
because it matches existing patterns. We will likely implement constant
time lookup functions for protected configuration in a future series
(such functions already exist for non-protected configuration, i.e.
repo_config_get_*()).

An alternative that avoids introducing another configset is to continue
to read all config using git_config(), but only accept values that have
the correct config scope [2]. This technically fulfills constraint 2,
because git_config() simply ignores the local and worktree config when
the gitdir is not known. However, this would read incomplete config into
the_repository->config, which would need to be reset when the gitdir is
known and git_config() needs to read the local and worktree config.
Resetting the_repository->config might be reasonable while we only have
these 'protected configuration only' variables, but it's not clear
whether this extends well to future variables.

[1] In this case, we do have a candidate gitdir though, so with a little
refactoring, it might be possible to provide a gitdir.
[2] This is how `uploadpack.packObjectsHook` was implemented prior to
this commit.

Signed-off-by: Glen Choo <chooglen@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-14 15:08:29 -07:00
Junio C Hamano
361cbe6d6d Merge branch 'ab/submodule-cleanup'
Further preparation to turn git-submodule.sh into a builtin.

* ab/submodule-cleanup:
  git-sh-setup.sh: remove "say" function, change last users
  git-submodule.sh: use "$quiet", not "$GIT_QUIET"
  submodule--helper: eliminate internal "--update" option
  submodule--helper: understand --checkout, --merge and --rebase synonyms
  submodule--helper: report "submodule" as our name in some "-h" output
  submodule--helper: rename "absorb-git-dirs" to "absorbgitdirs"
  submodule update: remove "-v" option
  submodule--helper: have --require-init imply --init
  git-submodule.sh: remove unused top-level "--branch" argument
  git-submodule.sh: make the "$cached" variable a boolean
  git-submodule.sh: remove unused $prefix variable
  git-submodule.sh: remove unused sanitize_submodule_env()
2022-07-14 15:04:00 -07:00
Junio C Hamano
0455aad1e3 Merge branch 'sy/mv-out-of-cone'
"git mv A B" in a sparsely populated working tree can be asked to
move a path between directories that are "in cone" (i.e. expected
to be materialized in the working tree) and "out of cone"
(i.e. expected to be hidden).  The handling of such cases has been
improved.

* sy/mv-out-of-cone:
  mv: add check_dir_in_index() and solve general dir check issue
  mv: use flags mode for update_mode
  mv: check if <destination> exists in index to handle overwriting
  mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit
  mv: decouple if/else-if checks using goto
  mv: update sparsity after moving from out-of-cone to in-cone
  t1092: mv directory from out-of-cone to in-cone
  t7002: add tests for moving out-of-cone file/directory
2022-07-14 15:04:00 -07:00
Junio C Hamano
73b9ef6ab1 Merge branch 'hx/unpack-streaming'
Allow large objects read from a packstream to be streamed into a
loose object file straight, without having to keep it in-core as a
whole.

* hx/unpack-streaming:
  unpack-objects: use stream_loose_object() to unpack large objects
  core doc: modernize core.bigFileThreshold documentation
  object-file.c: add "stream_loose_object()" to handle large object
  object-file.c: factor out deflate part of write_loose_object()
  object-file.c: refactor write_loose_object() to several steps
  unpack-objects: low memory footprint for get_data() in dry_run mode
2022-07-14 15:03:59 -07:00
Junio C Hamano
be733e1200 Merge branch 'en/merge-tree'
"git merge-tree" learned a new mode where it takes two commits and
computes a tree that would result in the merge commit, if the
histories leading to these two commits were to be merged.

* en/merge-tree:
  git-merge-tree.txt: add a section on potentional usage mistakes
  merge-tree: add a --allow-unrelated-histories flag
  merge-tree: allow `ls-files -u` style info to be NUL terminated
  merge-ort: optionally produce machine-readable output
  merge-ort: store more specific conflict information
  merge-ort: make `path_messages` a strmap to a string_list
  merge-ort: store messages in a list, not in a single strbuf
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: provide a list of which files have conflicts
  merge-ort: remove command-line-centric submodule message from merge-ort
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: support including merge messages in output
  merge-ort: split out a separate display_update_messages() function
  merge-tree: implement real merges
  merge-tree: add option parsing and initial shell for real merge function
  merge-tree: move logic for existing merge into new function
  merge-tree: rename merge_trees() to trivial_merge_trees()
2022-07-14 15:03:59 -07:00
Junio C Hamano
dc6315e1fc Merge branch 'gg/worktree-from-the-above'
In a non-bare repository, the behavior of Git when the
core.worktree configuration variable points at a directory that has
a repository as its subdirectory, regressed in Git 2.27 days.

* gg/worktree-from-the-above:
  dir: minor refactoring / clean-up
  dir: traverse into repository
2022-07-14 15:03:58 -07:00
Johannes Schindelin
7253f7ca9f tests: fix incorrect --write-junit-xml code
In 78d5e4cfb4 (tests: refactor --write-junit-xml code, 2022-05-21),
this developer refactored the `--write-junit-xml` code a bit, including
the part where the current test case's title was used in a `set`
invocation, but failed to account for the fact that some test cases'
titles start with a long option, which the `set` misinterprets as being
intended for parsing.

Let's fix this by using the `set -- <...>` form.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-14 10:02:06 -07:00
Junio C Hamano
8c4f65e0bf Merge branch 'cl/grep-max-count'
"git grep -m<max-hits>" is a way to limit the hits shown per file.

* cl/grep-max-count:
  grep: add --max-count command line option
2022-07-13 14:54:55 -07:00
Junio C Hamano
81705c4ee6 Merge branch 'zk/push-use-bitmaps'
"git push" sometimes perform poorly when reachability bitmaps are
used, even in a repository where other operations are helped by
bitmaps.  The push.useBitmaps configuration variable is introduced
to allow disabling use of reachability bitmaps only for "git push".

* zk/push-use-bitmaps:
  send-pack.c: add config push.useBitmaps
2022-07-13 14:54:54 -07:00
Junio C Hamano
33f448b5fc Merge branch 'jk/remote-show-with-negative-refspecs'
"git remote show [-n] frotz" now pays attention to negative
pathspec.

* jk/remote-show-with-negative-refspecs:
  remote: handle negative refspecs in git remote show
2022-07-13 14:54:54 -07:00
Junio C Hamano
ee493108e5 Merge branch 'll/ls-files-tests-update'
Test update.

* ll/ls-files-tests-update:
  ls-files: update test style
2022-07-13 14:54:53 -07:00
Junio C Hamano
92a25a8897 Merge branch 'ab/test-quoting-fix'
Fixes for tests when the source directory has unusual characters in
its path, e.g. whitespaces, double-quotes, etc.

* ab/test-quoting-fix:
  config tests: fix harmless but broken "rm -r" cleanup
  test-lib.sh: fix prepend_var() quoting issue
  tests: add missing double quotes to included library paths
2022-07-13 14:54:52 -07:00
Junio C Hamano
db791e6e8f Merge branch 'ds/t5510-brokequote'
Test fix.

* ds/t5510-brokequote:
  t5510: replace 'origin' with URL more carefully
2022-07-13 14:54:52 -07:00
Junio C Hamano
8da79e7250 Merge branch 'en/t6429-test-must-be-empty-fix'
A test fix.

* en/t6429-test-must-be-empty-fix:
  t6429: fix use of non-existent function
2022-07-13 14:54:51 -07:00
Han Xin
cb88b37cb9 t5330: remove run_with_limited_processses()
run_with_limited_processses() is used to end the loop faster when an
infinite loop happen. But "ulimit" is tied to the entire development
station, and the test will fail due to too many other processes or using
"--stress".

Without run_with_limited_processses() the infinite loop can also be
stopped due to global configrations or quotas, and the verification
still works fine. So let's remove run_with_limited_processses().

Signed-off-by: Han Xin <hanxin.hx@bytedance.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-12 07:47:43 -07:00
Junio C Hamano
f2e5255fc2 Git 2.37.1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE4fA2sf7nIh/HeOzvsLXohpav5ssFAmLDUXkACgkQsLXohpav
 5sv4SA/6AyzOuaMiBVTLtiYETFj9UU1Z3C12XtUlnu4qmw4Ddd1rq8/E4BgNDYJ7
 cC4MDVfOp7tvyaVGBBSfzIvIieBnGa7PIQl4z1eqZqIm0xt4T1D65jx1CHeJ+UIK
 k63L879dodQnIgwd1ThoGng0BUvmTREFODbGzX6JYPlRuEYkOpswLdvMO14epjLu
 T+TV9etoD5UELTrwnXDOq2ydH1MguyFj21g6NhMvBDTMCbZlQFb9skuF8dx2mC7T
 TktGntaEnFbm1UoZAoHg7g9AzO0iD+Vl6MVkkkolxJyAqiNUf030Ct6RQq9kRj1W
 7kYJDrgN9Oh3g93tJGsnTHmeOSoNiEJsHIeddH4HU0gzRtcx32ygL+KYE6exl2F6
 S1aoWJMdiQ9lYgQRef6aTQEHl2A08rIr3a3wFhVZBNZZk4NpwGblXfI2oIgKIVAx
 cFt0ABGX6RlokUUFGP+F/pk2noPi4m2tMaYfpUsd3sca+uKhFgtf26tQtmLpXAbq
 LKOA9FE+kjGgcOKMSIBjZYFod1HYHgZ+0F87JAURiUJnK2zAYSj+Sq4EatyvcHlb
 QcWQX5+Zxf+/rA1ACEBY+y4iZoSA0f1VQv8aANRmQwoIcjYjcu+W8dQKGCpJB68I
 ftTTxoM/OeQONkdzCcVNtS6ZbgjgKaaaaurvzLMwkQowtWEHd08=
 =pXYr
 -----END PGP SIGNATURE-----

Sync with Git 2.37.1
2022-07-11 16:08:49 -07:00
Junio C Hamano
b5a2d6cc49 Merge branch 'rs/archive-with-internal-gzip'
Teach "git archive" to (optionally and then by default) avoid
spawning an external "gzip" process when creating ".tar.gz" (and
".tgz") archives.

* rs/archive-with-internal-gzip:
  archive-tar: use internal gzip by default
  archive-tar: use OS_CODE 3 (Unix) for internal gzip
  archive-tar: add internal gzip implementation
  archive-tar: factor out write_block()
  archive: rename archiver data field to filter_command
  archive: update format documentation
2022-07-11 15:38:51 -07:00
Junio C Hamano
c2d01098fb Merge branch 'ds/branch-checked-out'
Introduce a helper to see if a branch is already being worked on
(hence should not be newly checked out in a working tree), which
performs much better than the existing find_shared_symref() to
replace many uses of the latter.

* ds/branch-checked-out:
  branch: drop unused worktrees variable
  fetch: stop passing around unused worktrees variable
  branch: fix branch_checked_out() leaks
  branch: use branch_checked_out() when deleting refs
  fetch: use new branch_checked_out() and add tests
  branch: check for bisects and rebases
  branch: add branch_checked_out() helper
2022-07-11 15:38:51 -07:00
Li Linchao
b0c4adcdd7 remote-curl: send Accept-Language header to server
Git server end's ability to accept Accept-Language header was introduced
in f18604bbf2 (http: add Accept-Language header if possible, 2015-01-28),
but this is only used by very early phase of the transfer, which is HTTP
GET request to discover references. For other phases, like POST request
in the smart HTTP, the server does not know what language the client
speaks.

Teach git client to learn end-user's preferred language and throw
accept-language header to the server side. Once the server gets this header,
it has the ability to talk to end-user with language they understand.
This would be very helpful for many non-English speakers.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Li Linchao <lilinchao@oschina.cn>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-11 12:24:28 -07:00
Jeff King
cc8fcd1e1a clone: use remote branch if it matches default HEAD
Usually clone tries to use the same local HEAD as the remote (unless the
user has given --branch explicitly). Even if the remote HEAD is detached
or unborn, we can detect those situations with modern versions of Git.
If the remote is too old to support the "unborn" extension (or it has
been disabled via config), then we can't know the name of the remote's
unborn HEAD, and we fall back whatever the local default branch name is
configured to be.

But that leads to one weird corner case. It's rare because it needs a
number of factors:

  - the remote has an unborn HEAD

  - the remote is too old to support "unborn", or has disabled it

  - the remote has another branch "foo"

  - the local default branch name is "foo"

In that case you end up with a local clone on an unborn "foo" branch,
disconnected completely from the remote's "foo". This is rare in
practice, but the result is quite confusing.

When choosing "foo", we can double check whether the remote has such a
name, and if so, start our local "foo" at the same spot, rather than
making it unborn.

Note that this causes a test failure in t5605, which is cloning from a
bundle that doesn't contain HEAD (so it behaves like a remote that
doesn't support "unborn"), but has a single "main" branch. That test
expects that we end up in the weird "unborn main" case, where we don't
actually check out the remote branch of the same name. Even though we
have to update the test, this seems like an argument in favor of this
patch: checking out main is what I'd expect from such a bundle.

So this patch updates the test for the new behavior and adds an adjacent
one that checks what the original was going for: if there's no HEAD and
the bundle _doesn't_ have a branch that matches our local default name,
then we end up with nothing checked out.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-07 20:57:54 -07:00
Jeff King
3d8314f8d1 clone: propagate empty remote HEAD even with other branches
Unless "--branch" was given, clone generally tries to match the local
HEAD to the remote one. For most repositories, this is easy: the remote
tells us which branch HEAD was pointing to, and we call our local
checkout() function on that branch.

When cloning an empty repository, it's a little more tricky: we have
special code that checks the transport's "unborn" extension, or falls back
to our local idea of what the default branch should be. In either case,
we point the new HEAD to that, and set up the branch.* config.

But that leaves one case unhandled: when the remote repository _isn't_
empty, but its HEAD is unborn. The checkout() function is smart enough
to realize we didn't fetch the remote HEAD and it bails with a warning.
But we'll have ignored any information the remote gave us via the unborn
extension. This leads to nonsense outcomes:

  - If the remote has its HEAD pointing to an unborn "foo" and contains
    another branch "bar", cloning will get branch "bar" but leave the
    local HEAD pointing at "master" (or whatever our local default is),
    which is useless. The project does not use "master" as a branch.

  - Worse, if the other branch "bar" is instead called "master" (but
    again, the remote HEAD is not pointing to it), then we end up with a
    local unborn branch "master", which is not connected to the remote
    "master" (it shares no history, and there's no branch.* config).

Instead, we should try to use the remote's HEAD, even if its unborn, to
be consistent with the other cases.

The reason this case was missed is that cmd_clone() handles empty and
non-empty repositories on two different sides of a conditional:

  if (we have any refs) {
      fetch refs;
      check for --branch;
      otherwise, try to point our head at remote head;
      otherwise, our head is NULL;
  } else {
      check for --branch;
      otherwise, try to use "unborn" extension;
      otherwise, fall back to our default name name;
  }

So the smallest change would be to repeat the "unborn" logic at the end
of the first block. But we can note some other overlaps and
inconsistencies:

  - both sides have to handle --branch (though note that it's always an
    error for the empty repo case, since an empty repo by definition
    does not have a matching branch)

  - the fall back to the default name is much more explicit in the
    empty-repo case. The non-empty case eventually ends up bailing
    from checkout() with a warning, which produces a similar result, but
    fails to set up the branch config we do in the empty case.

So let's pull the HEAD setup out of this conditional entirely. This
de-duplicates some of the code and the result is easy to follow, because
helper functions like find_ref_by_name() do the right thing even in the
empty-repo case (i.e., by returning NULL).

There are two subtleties:

  - for a remote with a detached HEAD, it will advertise an oid for HEAD
    (which we store in our "remote_head" variable), but we won't find a
    matching refname (so our "remote_head_points_at" is NULL). In this
    case we make a local detached HEAD to match. Right now this happens
    implicitly by reaching update_head() with a non-NULL remote_head
    (since we skip all of the unborn-fallback). We'll now need to
    account for it explicitly before doing the fallback.

  - for an empty repo, we issue a warning to the user that they've
    cloned an empty repo. The text of that warning doesn't make sense
    for a non-empty repo with an unborn HEAD, so we'll have to
    differentiate the two cases there. We could just use different text,
    but instead let's allow the code to continue down to checkout(),
    which will issue an appropriate warning, like:

      remote HEAD refers to nonexistent ref, unable to checkout

    Continuing down to checkout() will make it easier to do more fixes
    on top (see below).

Note that this patch fixes the case where the other side reports an
unborn head to us using the protocol extension. It _doesn't_ fix the
case where the other side doesn't tell us, we locally guess "master",
and the other side happens to have a "master" which its HEAD doesn't
point. But it doesn't make anything worse there, and it should actually
make it easier to fix that problem on top.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-07 20:57:54 -07:00
Li Linchao
18337d406f ls-files: update test style
Update test style in t/t30[*].sh for uniformity, that's to
keep test title the same line with helper function itself,
and fix some indentions.

Add a new section "recommended style" in t/README to
encourage people to use more modern style in test.

Signed-off-by: Li Linchao <lilinchao@oschina.cn>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-06 10:01:04 -07:00
Elijah Newren
751e165424 merge-ort: fix issue with dual rename and add/add conflict
There is code in both merge-recursive and merge-ort for avoiding doubly
transitive renames (i.e. one side renames directory A/ -> B/, and the
other side renames directory B/ -> C/), because this combination would
otherwise make a mess for new files added to A/ on the first side and
wondering which directory they end up in -- especially if there were
even more renames such as the first side renaming C/ -> D/.  In such
cases, it just turns "off" directory rename detection for the higher
order transitive cases.

The testcases added in t6423 a couple commits ago are slightly different
but similar in principle.  They involve a similar case of paired
renaming but instead of A/ -> B/ and B/ -> C/, the second side renames
a leading directory of B/ to C/.  And both sides add a new file
somewhere under the directory that the other side will rename.  While
the new files added start within different directories and thus could
logically end up within different directories, it is weird for a file
on one side to end up where the other one started and not move along
with it.  So, let's just turn off directory rename detection in this
case as well.

Another way to look at this is that if the source name involved in a
directory rename on one side is the target name of a directory rename
operation for a file from the other side, then we avoid the doubly
transitive rename.  (More concretely, if a directory rename on side D
wants to rename a file on side E from OLD_NAME -> NEW_NAME, and side D
already had a file named NEW_NAME, and a directory rename on side E
wants to rename side D's NEW_NAME -> NEWER_NAME, then we turn off the
directory rename detection for NEW_NAME to prevent the
NEW_NAME -> NEWER_NAME rename, and instead end up with an add/add
conflict on NEW_NAME.)

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-06 09:39:46 -07:00
Elijah Newren
0565cee5e4 t6423: add tests of dual directory rename plus add/add conflict
This is an attempt at minimalizing a testcase reported by Glen Choo
with tensorflow where merge-ort would report an assertion failure:

    Assertion failed: (ci->filemask == 2 || ci->filemask == 4), function apply_directory_rename_modifications, file merge-ort.c, line 2410

reversing the direction of the merge provides a different error:

    error: cache entry has null sha1: ...
    fatal: unable to write .git/index

so we add testcases for both.  With these new testcases, the
recursive strategy differs in that it returns the latter error for
both merge directions.

These testcases are somehow a little different than Glen's original
tensorflow testcase in that these ones trigger a bug with the recursive
algorithm whereas his testcase didn't.  I figure that means these
testcases somehow manage to be more comprehensive.

Reported-by: Glen Choo <chooglen@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-06 09:39:46 -07:00
Junio C Hamano
a631e99807 Merge 'js/add-i-delete' into maint-2.37
Rewrite of "git add -i" in C that appeared in Git 2.25 didn't
correctly record a removed file to the index, which is an old
regression but has become widely known because the C version
has become the default in the latest release.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-04 13:40:59 -07:00
Junio C Hamano
0f0bc2124b Merge branch 'js/add-i-delete'
Rewrite of "git add -i" in C that appeared in Git 2.25 didn't
correctly record a removed file to the index, which was fixed.

* js/add-i-delete:
  add --interactive: allow `update` to stage deleted files
2022-07-02 21:56:08 -07:00
Shaoxuan Yuan
b91a2b6594 mv: add check_dir_in_index() and solve general dir check issue
Originally, moving a <source> directory which is not on-disk due
to its existence outside of sparse-checkout cone, "giv mv" command
errors out with "bad source".

Add a helper check_dir_in_index() function to see if a directory
name exists in the index. Also add a SKIP_WORKTREE_DIR bit to mark
such directories.

Change the checking logic, so that such <source> directory makes
"giv mv" command warns with "advise_on_updating_sparse_paths()"
instead of "bad source"; also user now can supply a "--sparse" flag so
this operation can be carried out successfully.

Helped-by: Victoria Dye <vdye@github.com>
Helped-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Shaoxuan Yuan <shaoxuan.yuan02@gmail.com>
Acked-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-01 14:50:16 -07:00
Shaoxuan Yuan
8a26a3915f mv: check if <destination> exists in index to handle overwriting
Originally, moving a sparse file into cone can result in unwarned
overwrite of existing entry. The expected behavior is that if the
<destination> exists in the entry, user should be prompted to supply
a [-f|--force] to carry out the operation, or the operation should
fail.

Add a check mechanism to do that.

Signed-off-by: Shaoxuan Yuan <shaoxuan.yuan02@gmail.com>
Acked-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-01 14:50:16 -07:00
Shaoxuan Yuan
6645b03ca5 mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit
Originally, moving a <source> file which is not on-disk but exists in
index as a SKIP_WORKTREE enabled cache entry, "giv mv" command errors
out with "bad source".

Change the checking logic, so that such <source>
file makes "giv mv" command warns with "advise_on_updating_sparse_paths()"
instead of "bad source"; also user now can supply a "--sparse" flag so
this operation can be carried out successfully.

Signed-off-by: Shaoxuan Yuan <shaoxuan.yuan02@gmail.com>
Acked-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-01 14:50:16 -07:00