2019-05-19 16:26:49 +02:00
|
|
|
#!/bin/sh
|
|
|
|
|
|
|
|
|
|
test_description='Test cloning repos with submodules using remote-tracking branches'
|
|
|
|
|
|
2020-11-19 00:44:35 +01:00
|
|
|
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
|
tests: mark tests relying on the current default for `init.defaultBranch`
In addition to the manual adjustment to let the `linux-gcc` CI job run
the test suite with `master` and then with `main`, this patch makes sure
that GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME is set in all test scripts
that currently rely on the initial branch name being `master by default.
To determine which test scripts to mark up, the first step was to
force-set the default branch name to `master` in
- all test scripts that contain the keyword `master`,
- t4211, which expects `t/t4211/history.export` with a hard-coded ref to
initialize the default branch,
- t5560 because it sources `t/t556x_common` which uses `master`,
- t8002 and t8012 because both source `t/annotate-tests.sh` which also
uses `master`)
This trick was performed by this command:
$ sed -i '/^ *\. \.\/\(test-lib\|lib-\(bash\|cvs\|git-svn\)\|gitweb-lib\)\.sh$/i\
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\
' $(git grep -l master t/t[0-9]*.sh) \
t/t4211*.sh t/t5560*.sh t/t8002*.sh t/t8012*.sh
After that, careful, manual inspection revealed that some of the test
scripts containing the needle `master` do not actually rely on a
specific default branch name: either they mention `master` only in a
comment, or they initialize that branch specificially, or they do not
actually refer to the current default branch. Therefore, the
aforementioned modification was undone in those test scripts thusly:
$ git checkout HEAD -- \
t/t0027-auto-crlf.sh t/t0060-path-utils.sh \
t/t1011-read-tree-sparse-checkout.sh \
t/t1305-config-include.sh t/t1309-early-config.sh \
t/t1402-check-ref-format.sh t/t1450-fsck.sh \
t/t2024-checkout-dwim.sh \
t/t2106-update-index-assume-unchanged.sh \
t/t3040-subprojects-basic.sh t/t3301-notes.sh \
t/t3308-notes-merge.sh t/t3423-rebase-reword.sh \
t/t3436-rebase-more-options.sh \
t/t4015-diff-whitespace.sh t/t4257-am-interactive.sh \
t/t5323-pack-redundant.sh t/t5401-update-hooks.sh \
t/t5511-refspec.sh t/t5526-fetch-submodules.sh \
t/t5529-push-errors.sh t/t5530-upload-pack-error.sh \
t/t5548-push-porcelain.sh \
t/t5552-skipping-fetch-negotiator.sh \
t/t5572-pull-submodule.sh t/t5608-clone-2gb.sh \
t/t5614-clone-submodules-shallow.sh \
t/t7508-status.sh t/t7606-merge-custom.sh \
t/t9302-fast-import-unpack-limit.sh
We excluded one set of test scripts in these commands, though: the range
of `git p4` tests. The reason? `git p4` stores the (foreign) remote
branch in the branch called `p4/master`, which is obviously not the
default branch. Manual analysis revealed that only five of these tests
actually require a specific default branch name to pass; They were
modified thusly:
$ sed -i '/^ *\. \.\/lib-git-p4\.sh$/i\
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\
' t/t980[0167]*.sh t/t9811*.sh
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-19 00:44:19 +01:00
|
|
|
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
|
|
|
|
|
|
2022-11-08 19:17:47 +01:00
|
|
|
TEST_PASSES_SANITIZE_LEAK=true
|
2019-05-19 16:26:49 +02:00
|
|
|
. ./test-lib.sh
|
|
|
|
|
|
|
|
|
|
pwd=$(pwd)
|
|
|
|
|
|
|
|
|
|
test_expect_success 'setup' '
|
2022-07-29 21:21:06 +02:00
|
|
|
git config --global protocol.file.allow always &&
|
2020-11-19 00:44:35 +01:00
|
|
|
git checkout -b main &&
|
2019-05-19 16:26:49 +02:00
|
|
|
test_commit commit1 &&
|
|
|
|
|
mkdir sub &&
|
|
|
|
|
(
|
|
|
|
|
cd sub &&
|
|
|
|
|
git init &&
|
|
|
|
|
test_commit subcommit1 &&
|
2020-02-21 04:10:27 +01:00
|
|
|
git tag sub_when_added_to_super &&
|
|
|
|
|
git branch other
|
2019-05-19 16:26:49 +02:00
|
|
|
) &&
|
|
|
|
|
git submodule add "file://$pwd/sub" sub &&
|
|
|
|
|
git commit -m "add submodule" &&
|
|
|
|
|
(
|
|
|
|
|
cd sub &&
|
|
|
|
|
test_commit subcommit2
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
# bare clone giving "srv.bare" for use as our server.
|
|
|
|
|
test_expect_success 'setup bare clone for server' '
|
|
|
|
|
git clone --bare "file://$(pwd)/." srv.bare &&
|
|
|
|
|
git -C srv.bare config --local uploadpack.allowfilter 1 &&
|
|
|
|
|
git -C srv.bare config --local uploadpack.allowanysha1inwant 1
|
|
|
|
|
'
|
|
|
|
|
|
2019-05-19 16:26:49 +02:00
|
|
|
test_expect_success 'clone with --no-remote-submodules' '
|
|
|
|
|
test_when_finished "rm -rf super_clone" &&
|
|
|
|
|
git clone --recurse-submodules --no-remote-submodules "file://$pwd/." super_clone &&
|
|
|
|
|
(
|
|
|
|
|
cd super_clone/sub &&
|
|
|
|
|
git diff --exit-code sub_when_added_to_super
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
|
|
|
|
test_expect_success 'clone with --remote-submodules' '
|
|
|
|
|
test_when_finished "rm -rf super_clone" &&
|
|
|
|
|
git clone --recurse-submodules --remote-submodules "file://$pwd/." super_clone &&
|
|
|
|
|
(
|
|
|
|
|
cd super_clone/sub &&
|
2020-11-19 00:44:35 +01:00
|
|
|
git diff --exit-code remotes/origin/main
|
2019-05-19 16:26:49 +02:00
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
|
|
|
|
test_expect_success 'check the default is --no-remote-submodules' '
|
|
|
|
|
test_when_finished "rm -rf super_clone" &&
|
|
|
|
|
git clone --recurse-submodules "file://$pwd/." super_clone &&
|
|
|
|
|
(
|
|
|
|
|
cd super_clone/sub &&
|
|
|
|
|
git diff --exit-code sub_when_added_to_super
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
2020-02-21 04:10:27 +01:00
|
|
|
test_expect_success 'clone with --single-branch' '
|
|
|
|
|
test_when_finished "rm -rf super_clone" &&
|
|
|
|
|
git clone --recurse-submodules --single-branch "file://$pwd/." super_clone &&
|
|
|
|
|
(
|
|
|
|
|
cd super_clone/sub &&
|
2020-11-19 00:44:35 +01:00
|
|
|
git rev-parse --verify origin/main &&
|
2020-02-21 04:10:27 +01:00
|
|
|
test_must_fail git rev-parse --verify origin/other
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
# do basic partial clone from "srv.bare"
|
|
|
|
|
# confirm partial clone was registered in the local config for super and sub.
|
|
|
|
|
test_expect_success 'clone with --filter' '
|
|
|
|
|
git clone --recurse-submodules \
|
|
|
|
|
--filter blob:none --also-filter-submodules \
|
|
|
|
|
"file://$pwd/srv.bare" super_clone &&
|
|
|
|
|
test_cmp_config -C super_clone true remote.origin.promisor &&
|
|
|
|
|
test_cmp_config -C super_clone blob:none remote.origin.partialclonefilter &&
|
|
|
|
|
test_cmp_config -C super_clone/sub true remote.origin.promisor &&
|
|
|
|
|
test_cmp_config -C super_clone/sub blob:none remote.origin.partialclonefilter
|
|
|
|
|
'
|
|
|
|
|
|
|
|
|
|
# check that clone.filterSubmodules works (--also-filter-submodules can be
|
|
|
|
|
# omitted)
|
|
|
|
|
test_expect_success 'filters applied with clone.filterSubmodules' '
|
|
|
|
|
test_config_global clone.filterSubmodules true &&
|
|
|
|
|
git clone --recurse-submodules --filter blob:none \
|
|
|
|
|
"file://$pwd/srv.bare" super_clone2 &&
|
|
|
|
|
test_cmp_config -C super_clone2 true remote.origin.promisor &&
|
|
|
|
|
test_cmp_config -C super_clone2 blob:none remote.origin.partialclonefilter &&
|
|
|
|
|
test_cmp_config -C super_clone2/sub true remote.origin.promisor &&
|
|
|
|
|
test_cmp_config -C super_clone2/sub blob:none remote.origin.partialclonefilter
|
|
|
|
|
'
|
|
|
|
|
|
|
|
|
|
test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodules=true' '
|
|
|
|
|
test_config_global clone.filterSubmodules true &&
|
|
|
|
|
git clone --recurse-submodules --filter blob:none \
|
|
|
|
|
--no-also-filter-submodules \
|
|
|
|
|
"file://$pwd/srv.bare" super_clone3 &&
|
|
|
|
|
test_cmp_config -C super_clone3 true remote.origin.promisor &&
|
|
|
|
|
test_cmp_config -C super_clone3 blob:none remote.origin.partialclonefilter &&
|
|
|
|
|
test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor
|
|
|
|
|
'
|
|
|
|
|
|
2019-05-19 16:26:49 +02:00
|
|
|
test_done
|