2007-05-26 15:56:40 +02:00
|
|
|
#!/bin/sh
|
|
|
|
#
|
2012-06-25 12:56:59 +02:00
|
|
|
# git-submodule.sh: add, init, update or list git submodules
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
|
|
|
# Copyright (c) 2007 Lars Hjemli
|
|
|
|
|
2009-08-19 03:45:19 +02:00
|
|
|
dashless=$(basename "$0" | sed -e 's/-/ /')
|
2019-02-15 10:26:41 +01:00
|
|
|
USAGE="[--quiet] [--cached]
|
|
|
|
or: $dashless [--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
|
2009-08-19 03:45:24 +02:00
|
|
|
or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
|
2009-08-19 03:45:19 +02:00
|
|
|
or: $dashless [--quiet] init [--] [<path>...]
|
2016-05-05 21:52:32 +02:00
|
|
|
or: $dashless [--quiet] deinit [-f|--force] (--all| [--] <path>...)
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
or: $dashless [--quiet] update [--init [--filter=<filter-spec>]] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--[no-]recommend-shallow] [--reference <repository>] [--recursive] [--[no-]single-branch] [--] [<path>...]
|
2019-02-08 12:21:34 +01:00
|
|
|
or: $dashless [--quiet] set-branch (--default|--branch <branch>) [--] <path>
|
2019-10-29 18:01:52 +01:00
|
|
|
or: $dashless [--quiet] set-url [--] <path> <newurl>
|
2009-08-28 01:59:25 +02:00
|
|
|
or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
|
2009-08-19 03:45:22 +02:00
|
|
|
or: $dashless [--quiet] foreach [--recursive] <command>
|
2017-01-11 21:59:17 +01:00
|
|
|
or: $dashless [--quiet] sync [--recursive] [--] [<path>...]
|
|
|
|
or: $dashless [--quiet] absorbgitdirs [--] [<path>...]"
|
2007-11-06 10:50:02 +01:00
|
|
|
OPTIONS_SPEC=
|
2013-06-16 16:18:18 +02:00
|
|
|
SUBDIRECTORY_OK=Yes
|
2007-05-26 15:56:40 +02:00
|
|
|
. git-sh-setup
|
|
|
|
require_work_tree
|
2013-06-16 16:18:18 +02:00
|
|
|
wt_prefix=$(git rev-parse --show-prefix)
|
|
|
|
cd_to_toplevel
|
2007-05-26 15:56:40 +02:00
|
|
|
|
2016-12-14 23:39:52 +01:00
|
|
|
# Tell the rest of git that any URLs we get don't come
|
|
|
|
# directly from the user, so it can apply policy as appropriate.
|
|
|
|
GIT_PROTOCOL_FROM_USER=0
|
|
|
|
export GIT_PROTOCOL_FROM_USER
|
2015-09-16 19:13:12 +02:00
|
|
|
|
2008-01-15 11:48:45 +01:00
|
|
|
command=
|
2007-06-24 23:06:07 +02:00
|
|
|
branch=
|
2010-07-17 17:11:43 +02:00
|
|
|
force=
|
2009-05-04 21:30:01 +02:00
|
|
|
reference=
|
2007-05-26 15:56:40 +02:00
|
|
|
cached=
|
2010-04-26 11:50:39 +02:00
|
|
|
recursive=
|
|
|
|
init=
|
clone --recurse-submodules: prevent name squatting on Windows
In addition to preventing `.git` from being tracked by Git, on Windows
we also have to prevent `git~1` from being tracked, as the default NTFS
short name (also known as the "8.3 filename") for the file name `.git`
is `git~1`, otherwise it would be possible for malicious repositories to
write directly into the `.git/` directory, e.g. a `post-checkout` hook
that would then be executed _during_ a recursive clone.
When we implemented appropriate protections in 2b4c6efc821 (read-cache:
optionally disallow NTFS .git variants, 2014-12-16), we had analyzed
carefully that the `.git` directory or file would be guaranteed to be
the first directory entry to be written. Otherwise it would be possible
e.g. for a file named `..git` to be assigned the short name `git~1` and
subsequently, the short name generated for `.git` would be `git~2`. Or
`git~3`. Or even `~9999999` (for a detailed explanation of the lengths
we have to go to protect `.gitmodules`, see the commit message of
e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)).
However, by exploiting two issues (that will be addressed in a related
patch series close by), it is currently possible to clone a submodule
into a non-empty directory:
- On Windows, file names cannot end in a space or a period (for
historical reasons: the period separating the base name from the file
extension was not actually written to disk, and the base name/file
extension was space-padded to the full 8/3 characters, respectively).
Helpfully, when creating a directory under the name, say, `sub.`, that
trailing period is trimmed automatically and the actual name on disk
is `sub`.
This means that while Git thinks that the submodule names `sub` and
`sub.` are different, they both access `.git/modules/sub/`.
- While the backslash character is a valid file name character on Linux,
it is not so on Windows. As Git tries to be cross-platform, it
therefore allows backslash characters in the file names stored in tree
objects.
Which means that it is totally possible that a submodule `c` sits next
to a file `c\..git`, and on Windows, during recursive clone a file
called `..git` will be written into `c/`, of course _before_ the
submodule is cloned.
Note that the actual exploit is not quite as simple as having a
submodule `c` next to a file `c\..git`, as we have to make sure that the
directory `.git/modules/b` already exists when the submodule is checked
out, otherwise a different code path is taken in `module_clone()` that
does _not_ allow a non-empty submodule directory to exist already.
Even if we will address both issues nearby (the next commit will
disallow backslash characters in tree entries' file names on Windows,
and another patch will disallow creating directories/files with trailing
spaces or periods), it is a wise idea to defend in depth against this
sort of attack vector: when submodules are cloned recursively, we now
_require_ the directory to be empty, addressing CVE-2019-1349.
Note: the code path we patch is shared with the code path of `git
submodule update --init`, which must not expect, in general, that the
directory is empty. Hence we have to introduce the new option
`--force-init` and hand it all the way down from `git submodule` to the
actual `git submodule--helper` process that performs the initial clone.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
|
|
|
require_init=
|
2009-08-13 21:32:50 +02:00
|
|
|
files=
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
remote=
|
2009-02-05 23:18:32 +01:00
|
|
|
nofetch=
|
Rename submodule.<name>.rebase to submodule.<name>.update
The addition of "submodule.<name>.rebase" demonstrates the usefulness of
alternatives to the default behaviour of "git submodule update". However,
by naming the config variable "submodule.<name>.rebase", and making it a
boolean choice, we are artificially constraining future git versions that
may want to add _more_ alternatives than just "rebase".
Therefore, while "submodule.<name>.rebase" is not yet in a stable git
release, future-proof it, by changing it from
submodule.<name>.rebase = true/false
to
submodule.<name>.update = rebase/checkout
where "checkout" specifies the default behaviour of "git submodule update"
(checking out the new commit to a detached HEAD), and "rebase" specifies
the --rebase behaviour (where the current local branch in the submodule is
rebase onto the new commit). Thus .update == checkout is equivalent to
.rebase == false, and .update == rebase is equivalent to .rebase == true.
Finally, leaving .update unset is equivalent to leaving .rebase unset.
In future git versions, other alternatives to "git submodule update"
behaviour can be included by adding them to the list of allowable values
for the submodule.<name>.update variable.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-03 08:27:06 +02:00
|
|
|
update=
|
2009-08-19 03:45:22 +02:00
|
|
|
prefix=
|
2012-09-30 01:05:58 +02:00
|
|
|
custom_name=
|
2013-07-02 23:42:56 +02:00
|
|
|
depth=
|
clone: pass --progress decision to recursive submodules
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.
In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.
That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.
This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.
This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:
git clone --recursive --progress ... 2>&1 | cat
work by triggering progress explicitly in the children.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
|
|
|
progress=
|
2018-05-03 12:53:46 +02:00
|
|
|
dissociate=
|
2020-02-21 04:10:27 +01:00
|
|
|
single_branch=
|
2020-04-02 10:42:51 +02:00
|
|
|
jobs=
|
|
|
|
recommend_shallow=
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
filter=
|
2007-05-26 15:56:40 +02:00
|
|
|
|
2013-04-01 15:06:27 +02:00
|
|
|
isnumber()
|
|
|
|
{
|
|
|
|
n=$(($1 + 0)) 2>/dev/null && test "$n" = "$1"
|
|
|
|
}
|
|
|
|
|
2016-02-29 23:58:35 +01:00
|
|
|
# Sanitize the local git environment for use within a submodule. We
|
|
|
|
# can't simply use clear_local_git_env since we want to preserve some
|
|
|
|
# of the settings from GIT_CONFIG_PARAMETERS.
|
|
|
|
sanitize_submodule_env()
|
|
|
|
{
|
submodule: stop sanitizing config options
The point of having a whitelist of command-line config
options to pass to submodules was two-fold:
1. It prevented obvious nonsense like using core.worktree
for multiple repos.
2. It could prevent surprise when the user did not mean
for the options to leak to the submodules (e.g.,
http.sslverify=false).
For case 1, the answer is mostly "if it hurts, don't do
that". For case 2, we can note that any such example has a
matching inverted surprise (e.g., a user who meant
http.sslverify=true to apply everywhere, but it didn't).
So this whitelist is probably not giving us any benefit, and
is already creating a hassle as people propose things to put
on it. Let's just drop it entirely.
Note that we still need to keep a special code path for
"prepare the submodule environment", because we still have
to take care to pass through $GIT_CONFIG_PARAMETERS (and
block the rest of the repo-specific environment variables).
We can do this easily from within the submodule shell
script, which lets us drop the submodule--helper option
entirely (and it's OK to do so because as a "--" program, it
is entirely a private implementation detail).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-05 03:22:19 +02:00
|
|
|
save_config=$GIT_CONFIG_PARAMETERS
|
2016-02-29 23:58:35 +01:00
|
|
|
clear_local_git_env
|
submodule: stop sanitizing config options
The point of having a whitelist of command-line config
options to pass to submodules was two-fold:
1. It prevented obvious nonsense like using core.worktree
for multiple repos.
2. It could prevent surprise when the user did not mean
for the options to leak to the submodules (e.g.,
http.sslverify=false).
For case 1, the answer is mostly "if it hurts, don't do
that". For case 2, we can note that any such example has a
matching inverted surprise (e.g., a user who meant
http.sslverify=true to apply everywhere, but it didn't).
So this whitelist is probably not giving us any benefit, and
is already creating a hassle as people propose things to put
on it. Let's just drop it entirely.
Note that we still need to keep a special code path for
"prepare the submodule environment", because we still have
to take care to pass through $GIT_CONFIG_PARAMETERS (and
block the rest of the repo-specific environment variables).
We can do this easily from within the submodule shell
script, which lets us drop the submodule--helper option
entirely (and it's OK to do so because as a "--" program, it
is entirely a private implementation detail).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-05 03:22:19 +02:00
|
|
|
GIT_CONFIG_PARAMETERS=$save_config
|
2016-04-28 15:37:44 +02:00
|
|
|
export GIT_CONFIG_PARAMETERS
|
2016-02-29 23:58:35 +01:00
|
|
|
}
|
|
|
|
|
2007-06-24 23:06:07 +02:00
|
|
|
#
|
|
|
|
# Add a new submodule to the working tree, .gitmodules and the index
|
|
|
|
#
|
git-submodule - make "submodule add" more strict, and document it
This change makes "submodule add" much more strict in the arguments it
takes, and is intended to address confusion as recently noted on the
git-list. With this change, the required syntax is:
$ git submodule add URL path
Specifically, this eliminates the form
$ git submodule add URL
which was confused by more than one person as
$ git submodule add path
With this patch, the URL locating the submodule's origin repository can be
either an absolute URL, or (if it begins with ./ or ../) can express the
submodule's repository location relative to the superproject's origin.
This patch also eliminates a third form of URL, which was relative to the
superproject's top-level directory (not its repository). Any URL that was
neither absolute nor matched ./*|../* was assumed to point to a
subdirectory of the superproject as the location of the submodule's origin
repository. This URL form was confusing and does not seem to correspond
to an important use-case. Specifically, no-one has identified the need to
clone from a repository already in the superproject's tree, but if this is
needed it is easily done using an absolute URL: $(pwd)/relative-path. So,
no functionality is lost with this patch. (t6008-rev-list-submodule.sh did
rely upon this relative URL, fixed by using $(pwd).)
Following this change, there are exactly four variants of
submodule-add, as both arguments have two flavors:
URL can be absolute, or can begin with ./|../ and thus names the
submodule's origin relative to the superproject's origin.
Note: With this patch, "submodule add" discerns an absolute URL as
matching /*|*:*: e.g., URL begins with /, or it contains a :. This works
for all valid URLs, an absolute path in POSIX, as well as an absolute path
on Windows).
path can either already exist as a valid git repo, or will be cloned from
the given URL. The first form here eases creation of a new submodule in
an existing superproject as the submodule can be added and tested in-tree
before pushing to the public repository. However, the more usual form is
the second, where the repo is cloned from the given URL.
This specifically addresses the issue of
$ git submodule add a/b/c
attempting to clone from a repository at "a/b/c" to create a new module
in "c". This also simplifies description of "relative URL" as there is now
exactly *one* form: a URL relative to the parent's origin repo.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-10 03:05:40 +02:00
|
|
|
# $@ = repo path
|
2007-06-24 23:06:07 +02:00
|
|
|
#
|
|
|
|
# optional branch is stored in global branch variable
|
|
|
|
#
|
2008-01-15 11:35:49 +01:00
|
|
|
cmd_add()
|
2007-06-24 23:06:07 +02:00
|
|
|
{
|
2008-01-15 11:48:45 +01:00
|
|
|
# parse $args after "submodule ... add".
|
2013-06-16 16:18:18 +02:00
|
|
|
reference_path=
|
2008-01-15 11:48:45 +01:00
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-b | --branch)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
branch=$2
|
|
|
|
shift
|
|
|
|
;;
|
2010-07-17 17:11:43 +02:00
|
|
|
-f | --force)
|
|
|
|
force=$1
|
|
|
|
;;
|
2008-01-15 11:48:45 +01:00
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-01-15 11:48:45 +01:00
|
|
|
;;
|
2018-05-03 12:53:45 +02:00
|
|
|
--progress)
|
|
|
|
progress=1
|
|
|
|
;;
|
2009-05-04 21:30:01 +02:00
|
|
|
--reference)
|
|
|
|
case "$2" in '') usage ;; esac
|
2013-06-16 16:18:18 +02:00
|
|
|
reference_path=$2
|
2009-05-04 21:30:01 +02:00
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--reference=*)
|
2013-06-16 16:18:18 +02:00
|
|
|
reference_path="${1#--reference=}"
|
2009-05-04 21:30:01 +02:00
|
|
|
;;
|
2018-05-03 12:53:46 +02:00
|
|
|
--dissociate)
|
|
|
|
dissociate=1
|
|
|
|
;;
|
2012-09-30 01:05:58 +02:00
|
|
|
--name)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
custom_name=$2
|
|
|
|
shift
|
|
|
|
;;
|
2013-07-02 23:42:56 +02:00
|
|
|
--depth)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
depth="--depth=$2"
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--depth=*)
|
|
|
|
depth=$1
|
|
|
|
;;
|
2008-01-15 11:48:45 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-08-10 13:46:37 +02:00
|
|
|
if test -z "$1"
|
2018-10-25 18:18:12 +02:00
|
|
|
then
|
2007-06-24 23:06:07 +02:00
|
|
|
usage
|
|
|
|
fi
|
|
|
|
|
2021-08-10 13:46:37 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper add ${GIT_QUIET:+--quiet} ${force:+--force} ${progress:+"--progress"} ${branch:+--branch "$branch"} ${reference_path:+--reference "$reference_path"} ${dissociate:+--dissociate} ${custom_name:+--name "$custom_name"} ${depth:+"$depth"} -- "$@"
|
2007-06-24 23:06:07 +02:00
|
|
|
}
|
|
|
|
|
2008-08-11 01:10:04 +02:00
|
|
|
#
|
|
|
|
# Execute an arbitrary command sequence in each checked out
|
|
|
|
# submodule
|
|
|
|
#
|
|
|
|
# $@ = command to execute
|
|
|
|
#
|
|
|
|
cmd_foreach()
|
|
|
|
{
|
2009-08-19 03:45:19 +02:00
|
|
|
# parse $args after "submodule ... foreach".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
|
|
|
GIT_QUIET=1
|
|
|
|
;;
|
2009-08-19 03:45:22 +02:00
|
|
|
--recursive)
|
|
|
|
recursive=1
|
|
|
|
;;
|
2009-08-19 03:45:19 +02:00
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} -- "$@"
|
2008-08-11 01:10:04 +02:00
|
|
|
}
|
|
|
|
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
2007-06-06 11:13:02 +02:00
|
|
|
# Register submodules in .git/config
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
|
|
|
# $@ = requested paths (default to all)
|
|
|
|
#
|
2008-01-15 11:35:49 +01:00
|
|
|
cmd_init()
|
2007-05-26 15:56:40 +02:00
|
|
|
{
|
2008-01-15 11:48:45 +01:00
|
|
|
# parse $args after "submodule ... init".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-01-15 11:48:45 +01:00
|
|
|
;;
|
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2019-04-12 12:08:19 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper init ${GIT_QUIET:+--quiet} -- "$@"
|
2007-05-26 15:56:40 +02:00
|
|
|
}
|
|
|
|
|
2013-03-04 22:20:24 +01:00
|
|
|
#
|
|
|
|
# Unregister submodules from .git/config and remove their work tree
|
|
|
|
#
|
|
|
|
cmd_deinit()
|
|
|
|
{
|
|
|
|
# parse $args after "submodule ... deinit".
|
2016-05-05 21:52:32 +02:00
|
|
|
deinit_all=
|
2013-03-04 22:20:24 +01:00
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-f|--force)
|
|
|
|
force=$1
|
|
|
|
;;
|
|
|
|
-q|--quiet)
|
|
|
|
GIT_QUIET=1
|
|
|
|
;;
|
2016-05-05 21:52:32 +02:00
|
|
|
--all)
|
|
|
|
deinit_all=t
|
|
|
|
;;
|
2013-03-04 22:20:24 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${force:+--force} ${deinit_all:+--all} -- "$@"
|
2013-03-04 22:20:24 +01:00
|
|
|
}
|
|
|
|
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
2007-06-06 11:13:02 +02:00
|
|
|
# Update each submodule path to correct revision, using clone and checkout as needed
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
|
|
|
# $@ = requested paths (default to all)
|
|
|
|
#
|
2008-01-15 11:35:49 +01:00
|
|
|
cmd_update()
|
2007-05-26 15:56:40 +02:00
|
|
|
{
|
2008-01-15 11:48:45 +01:00
|
|
|
# parse $args after "submodule ... update".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-01-15 11:48:45 +01:00
|
|
|
;;
|
2018-08-14 20:22:02 +02:00
|
|
|
-v)
|
2020-09-30 21:50:53 +02:00
|
|
|
unset GIT_QUIET
|
2018-08-14 20:22:02 +02:00
|
|
|
;;
|
clone: pass --progress decision to recursive submodules
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.
In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.
That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.
This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.
This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:
git clone --recursive --progress ... 2>&1 | cat
work by triggering progress explicitly in the children.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
|
|
|
--progress)
|
2018-05-03 12:53:44 +02:00
|
|
|
progress=1
|
clone: pass --progress decision to recursive submodules
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.
In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.
That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.
This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.
This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:
git clone --recursive --progress ... 2>&1 | cat
work by triggering progress explicitly in the children.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
|
|
|
;;
|
2008-05-16 12:23:03 +02:00
|
|
|
-i|--init)
|
2009-05-04 21:30:01 +02:00
|
|
|
init=1
|
2008-05-16 12:23:03 +02:00
|
|
|
;;
|
clone --recurse-submodules: prevent name squatting on Windows
In addition to preventing `.git` from being tracked by Git, on Windows
we also have to prevent `git~1` from being tracked, as the default NTFS
short name (also known as the "8.3 filename") for the file name `.git`
is `git~1`, otherwise it would be possible for malicious repositories to
write directly into the `.git/` directory, e.g. a `post-checkout` hook
that would then be executed _during_ a recursive clone.
When we implemented appropriate protections in 2b4c6efc821 (read-cache:
optionally disallow NTFS .git variants, 2014-12-16), we had analyzed
carefully that the `.git` directory or file would be guaranteed to be
the first directory entry to be written. Otherwise it would be possible
e.g. for a file named `..git` to be assigned the short name `git~1` and
subsequently, the short name generated for `.git` would be `git~2`. Or
`git~3`. Or even `~9999999` (for a detailed explanation of the lengths
we have to go to protect `.gitmodules`, see the commit message of
e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)).
However, by exploiting two issues (that will be addressed in a related
patch series close by), it is currently possible to clone a submodule
into a non-empty directory:
- On Windows, file names cannot end in a space or a period (for
historical reasons: the period separating the base name from the file
extension was not actually written to disk, and the base name/file
extension was space-padded to the full 8/3 characters, respectively).
Helpfully, when creating a directory under the name, say, `sub.`, that
trailing period is trimmed automatically and the actual name on disk
is `sub`.
This means that while Git thinks that the submodule names `sub` and
`sub.` are different, they both access `.git/modules/sub/`.
- While the backslash character is a valid file name character on Linux,
it is not so on Windows. As Git tries to be cross-platform, it
therefore allows backslash characters in the file names stored in tree
objects.
Which means that it is totally possible that a submodule `c` sits next
to a file `c\..git`, and on Windows, during recursive clone a file
called `..git` will be written into `c/`, of course _before_ the
submodule is cloned.
Note that the actual exploit is not quite as simple as having a
submodule `c` next to a file `c\..git`, as we have to make sure that the
directory `.git/modules/b` already exists when the submodule is checked
out, otherwise a different code path is taken in `module_clone()` that
does _not_ allow a non-empty submodule directory to exist already.
Even if we will address both issues nearby (the next commit will
disallow backslash characters in tree entries' file names on Windows,
and another patch will disallow creating directories/files with trailing
spaces or periods), it is a wise idea to defend in depth against this
sort of attack vector: when submodules are cloned recursively, we now
_require_ the directory to be empty, addressing CVE-2019-1349.
Note: the code path we patch is shared with the code path of `git
submodule update --init`, which must not expect, in general, that the
directory is empty. Hence we have to introduce the new option
`--force-init` and hand it all the way down from `git submodule` to the
actual `git submodule--helper` process that performs the initial clone.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
|
|
|
--require-init)
|
|
|
|
init=1
|
|
|
|
require_init=1
|
|
|
|
;;
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
--remote)
|
|
|
|
remote=1
|
|
|
|
;;
|
2009-02-05 23:18:32 +01:00
|
|
|
-N|--no-fetch)
|
|
|
|
nofetch=1
|
|
|
|
;;
|
2011-04-01 11:42:03 +02:00
|
|
|
-f|--force)
|
|
|
|
force=$1
|
|
|
|
;;
|
2009-04-24 01:06:38 +02:00
|
|
|
-r|--rebase)
|
Rename submodule.<name>.rebase to submodule.<name>.update
The addition of "submodule.<name>.rebase" demonstrates the usefulness of
alternatives to the default behaviour of "git submodule update". However,
by naming the config variable "submodule.<name>.rebase", and making it a
boolean choice, we are artificially constraining future git versions that
may want to add _more_ alternatives than just "rebase".
Therefore, while "submodule.<name>.rebase" is not yet in a stable git
release, future-proof it, by changing it from
submodule.<name>.rebase = true/false
to
submodule.<name>.update = rebase/checkout
where "checkout" specifies the default behaviour of "git submodule update"
(checking out the new commit to a detached HEAD), and "rebase" specifies
the --rebase behaviour (where the current local branch in the submodule is
rebase onto the new commit). Thus .update == checkout is equivalent to
.rebase == false, and .update == rebase is equivalent to .rebase == true.
Finally, leaving .update unset is equivalent to leaving .rebase unset.
In future git versions, other alternatives to "git submodule update"
behaviour can be included by adding them to the list of allowable values
for the submodule.<name>.update variable.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-03 08:27:06 +02:00
|
|
|
update="rebase"
|
2009-04-24 01:06:38 +02:00
|
|
|
;;
|
2009-05-04 21:30:01 +02:00
|
|
|
--reference)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
reference="--reference=$2"
|
2010-11-03 07:26:25 +01:00
|
|
|
shift
|
2009-05-04 21:30:01 +02:00
|
|
|
;;
|
|
|
|
--reference=*)
|
|
|
|
reference="$1"
|
|
|
|
;;
|
2018-05-03 12:53:46 +02:00
|
|
|
--dissociate)
|
|
|
|
dissociate=1
|
|
|
|
;;
|
2009-06-03 00:59:12 +02:00
|
|
|
-m|--merge)
|
|
|
|
update="merge"
|
|
|
|
;;
|
2009-08-19 03:45:23 +02:00
|
|
|
--recursive)
|
|
|
|
recursive=1
|
|
|
|
;;
|
2011-10-11 00:56:16 +02:00
|
|
|
--checkout)
|
|
|
|
update="checkout"
|
|
|
|
;;
|
2016-05-26 23:59:43 +02:00
|
|
|
--recommend-shallow)
|
|
|
|
recommend_shallow="--recommend-shallow"
|
|
|
|
;;
|
|
|
|
--no-recommend-shallow)
|
|
|
|
recommend_shallow="--no-recommend-shallow"
|
|
|
|
;;
|
2013-07-02 23:42:56 +02:00
|
|
|
--depth)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
depth="--depth=$2"
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--depth=*)
|
|
|
|
depth=$1
|
|
|
|
;;
|
2016-03-01 03:07:19 +01:00
|
|
|
-j|--jobs)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
jobs="--jobs=$2"
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--jobs=*)
|
|
|
|
jobs=$1
|
|
|
|
;;
|
2020-02-21 04:10:27 +01:00
|
|
|
--single-branch)
|
|
|
|
single_branch="--single-branch"
|
|
|
|
;;
|
|
|
|
--no-single-branch)
|
|
|
|
single_branch="--no-single-branch"
|
|
|
|
;;
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
--filter)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
filter="--filter=$2"
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--filter=*)
|
|
|
|
filter="$1"
|
|
|
|
;;
|
2008-01-15 11:48:45 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
2010-11-03 07:26:25 +01:00
|
|
|
shift
|
2008-01-15 11:48:45 +01:00
|
|
|
done
|
|
|
|
|
2022-03-15 22:09:24 +01:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update \
|
2022-03-05 01:13:58 +01:00
|
|
|
${GIT_QUIET:+--quiet} \
|
2022-03-15 22:09:24 +01:00
|
|
|
${force:+--force} \
|
2018-05-03 12:53:44 +02:00
|
|
|
${progress:+"--progress"} \
|
2022-03-15 22:09:24 +01:00
|
|
|
${remote:+--remote} \
|
|
|
|
${recursive:+--recursive} \
|
2022-03-05 01:13:58 +01:00
|
|
|
${init:+--init} \
|
2022-03-15 22:09:24 +01:00
|
|
|
${nofetch:+--no-fetch} \
|
2016-03-01 03:07:17 +01:00
|
|
|
${wt_prefix:+--prefix "$wt_prefix"} \
|
|
|
|
${prefix:+--recursive-prefix "$prefix"} \
|
|
|
|
${update:+--update "$update"} \
|
2016-08-12 01:14:01 +02:00
|
|
|
${reference:+"$reference"} \
|
2018-05-03 12:53:46 +02:00
|
|
|
${dissociate:+"--dissociate"} \
|
2022-03-15 22:09:22 +01:00
|
|
|
${depth:+"$depth"} \
|
clone --recurse-submodules: prevent name squatting on Windows
In addition to preventing `.git` from being tracked by Git, on Windows
we also have to prevent `git~1` from being tracked, as the default NTFS
short name (also known as the "8.3 filename") for the file name `.git`
is `git~1`, otherwise it would be possible for malicious repositories to
write directly into the `.git/` directory, e.g. a `post-checkout` hook
that would then be executed _during_ a recursive clone.
When we implemented appropriate protections in 2b4c6efc821 (read-cache:
optionally disallow NTFS .git variants, 2014-12-16), we had analyzed
carefully that the `.git` directory or file would be guaranteed to be
the first directory entry to be written. Otherwise it would be possible
e.g. for a file named `..git` to be assigned the short name `git~1` and
subsequently, the short name generated for `.git` would be `git~2`. Or
`git~3`. Or even `~9999999` (for a detailed explanation of the lengths
we have to go to protect `.gitmodules`, see the commit message of
e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)).
However, by exploiting two issues (that will be addressed in a related
patch series close by), it is currently possible to clone a submodule
into a non-empty directory:
- On Windows, file names cannot end in a space or a period (for
historical reasons: the period separating the base name from the file
extension was not actually written to disk, and the base name/file
extension was space-padded to the full 8/3 characters, respectively).
Helpfully, when creating a directory under the name, say, `sub.`, that
trailing period is trimmed automatically and the actual name on disk
is `sub`.
This means that while Git thinks that the submodule names `sub` and
`sub.` are different, they both access `.git/modules/sub/`.
- While the backslash character is a valid file name character on Linux,
it is not so on Windows. As Git tries to be cross-platform, it
therefore allows backslash characters in the file names stored in tree
objects.
Which means that it is totally possible that a submodule `c` sits next
to a file `c\..git`, and on Windows, during recursive clone a file
called `..git` will be written into `c/`, of course _before_ the
submodule is cloned.
Note that the actual exploit is not quite as simple as having a
submodule `c` next to a file `c\..git`, as we have to make sure that the
directory `.git/modules/b` already exists when the submodule is checked
out, otherwise a different code path is taken in `module_clone()` that
does _not_ allow a non-empty submodule directory to exist already.
Even if we will address both issues nearby (the next commit will
disallow backslash characters in tree entries' file names on Windows,
and another patch will disallow creating directories/files with trailing
spaces or periods), it is a wise idea to defend in depth against this
sort of attack vector: when submodules are cloned recursively, we now
_require_ the directory to be empty, addressing CVE-2019-1349.
Note: the code path we patch is shared with the code path of `git
submodule update --init`, which must not expect, in general, that the
directory is empty. Hence we have to introduce the new option
`--force-init` and hand it all the way down from `git submodule` to the
actual `git submodule--helper` process that performs the initial clone.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
|
|
|
${require_init:+--require-init} \
|
2022-03-15 22:09:24 +01:00
|
|
|
${dissociate:+"--dissociate"} \
|
2020-02-21 04:10:27 +01:00
|
|
|
$single_branch \
|
2018-05-03 12:53:44 +02:00
|
|
|
$recommend_shallow \
|
|
|
|
$jobs \
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
$filter \
|
2019-04-12 12:08:19 +02:00
|
|
|
-- \
|
2022-03-15 22:09:24 +01:00
|
|
|
"$@"
|
2007-05-26 15:56:40 +02:00
|
|
|
}
|
|
|
|
|
2019-02-08 12:21:34 +01:00
|
|
|
#
|
|
|
|
# Configures a submodule's default branch
|
|
|
|
#
|
|
|
|
# $@ = requested path
|
|
|
|
#
|
|
|
|
cmd_set_branch() {
|
2020-06-02 18:35:23 +02:00
|
|
|
default=
|
2019-02-08 12:21:34 +01:00
|
|
|
branch=
|
|
|
|
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
|
|
|
# we don't do anything with this but we need to accept it
|
|
|
|
;;
|
|
|
|
-d|--default)
|
2020-06-02 18:35:23 +02:00
|
|
|
default=1
|
2019-02-08 12:21:34 +01:00
|
|
|
;;
|
|
|
|
-b|--branch)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
branch=$2
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper set-branch ${GIT_QUIET:+--quiet} ${branch:+--branch "$branch"} ${default:+--default} -- "$@"
|
2019-02-08 12:21:34 +01:00
|
|
|
}
|
|
|
|
|
2019-10-29 18:01:52 +01:00
|
|
|
#
|
|
|
|
# Configures a submodule's remote url
|
|
|
|
#
|
|
|
|
# $@ = requested path, requested url
|
|
|
|
#
|
|
|
|
cmd_set_url() {
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
|
|
|
GIT_QUIET=1
|
|
|
|
;;
|
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper set-url ${GIT_QUIET:+--quiet} -- "$@"
|
2019-10-29 18:01:52 +01:00
|
|
|
}
|
|
|
|
|
2008-03-11 14:52:15 +01:00
|
|
|
#
|
|
|
|
# Show commit summary for submodules in index or working tree
|
|
|
|
#
|
|
|
|
# If '--cached' is given, show summary between index and given commit,
|
|
|
|
# or between working tree and given commit
|
|
|
|
#
|
|
|
|
# $@ = [commit (default 'HEAD'),] requested paths (default all)
|
|
|
|
#
|
|
|
|
cmd_summary() {
|
2008-03-11 14:52:17 +01:00
|
|
|
summary_limit=-1
|
2008-04-12 17:05:31 +02:00
|
|
|
for_status=
|
2009-08-13 21:32:50 +02:00
|
|
|
diff_cmd=diff-index
|
2008-03-11 14:52:17 +01:00
|
|
|
|
2008-03-11 14:52:15 +01:00
|
|
|
# parse $args after "submodule ... summary".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
--cached)
|
|
|
|
cached="$1"
|
|
|
|
;;
|
2009-08-13 21:32:50 +02:00
|
|
|
--files)
|
|
|
|
files="$1"
|
|
|
|
;;
|
2008-04-12 17:05:31 +02:00
|
|
|
--for-status)
|
|
|
|
for_status="$1"
|
|
|
|
;;
|
2008-03-11 14:52:17 +01:00
|
|
|
-n|--summary-limit)
|
2013-04-01 15:06:27 +02:00
|
|
|
summary_limit="$2"
|
|
|
|
isnumber "$summary_limit" || usage
|
2008-03-11 14:52:17 +01:00
|
|
|
shift
|
|
|
|
;;
|
2013-04-01 15:06:27 +02:00
|
|
|
--summary-limit=*)
|
|
|
|
summary_limit="${1#--summary-limit=}"
|
|
|
|
isnumber "$summary_limit" || usage
|
|
|
|
;;
|
2008-03-11 14:52:15 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
2007-06-27 01:40:58 +02:00
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${files:+--files} ${cached:+--cached} ${for_status:+--for-status} ${summary_limit:+-n $summary_limit} -- "$@"
|
2008-03-11 14:52:15 +01:00
|
|
|
}
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
2007-06-11 21:12:24 +02:00
|
|
|
# List all submodules, prefixed with:
|
2007-05-26 15:56:40 +02:00
|
|
|
# - submodule not initialized
|
|
|
|
# + different revision checked out
|
|
|
|
#
|
|
|
|
# If --cached was specified the revision in the index will be printed
|
|
|
|
# instead of the currently checked out revision.
|
|
|
|
#
|
|
|
|
# $@ = requested paths (default to all)
|
|
|
|
#
|
2008-01-15 11:35:49 +01:00
|
|
|
cmd_status()
|
2007-05-26 15:56:40 +02:00
|
|
|
{
|
2008-01-15 11:48:45 +01:00
|
|
|
# parse $args after "submodule ... status".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-01-15 11:48:45 +01:00
|
|
|
;;
|
|
|
|
--cached)
|
|
|
|
cached=1
|
|
|
|
;;
|
2009-08-19 03:45:24 +02:00
|
|
|
--recursive)
|
|
|
|
recursive=1
|
|
|
|
;;
|
2008-01-15 11:48:45 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} -- "$@"
|
2007-05-26 15:56:40 +02:00
|
|
|
}
|
2008-08-24 21:43:37 +02:00
|
|
|
#
|
|
|
|
# Sync remote urls for submodules
|
|
|
|
# This makes the value for remote.$remote.url match the value
|
|
|
|
# specified in .gitmodules.
|
|
|
|
#
|
|
|
|
cmd_sync()
|
|
|
|
{
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-08-24 21:43:37 +02:00
|
|
|
shift
|
|
|
|
;;
|
2012-10-26 21:44:42 +02:00
|
|
|
--recursive)
|
|
|
|
recursive=1
|
|
|
|
shift
|
|
|
|
;;
|
2008-08-24 21:43:37 +02:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
done
|
2017-03-16 23:29:45 +01:00
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} -- "$@"
|
2008-08-24 21:43:37 +02:00
|
|
|
}
|
2007-05-26 15:56:40 +02:00
|
|
|
|
2016-12-12 20:04:35 +01:00
|
|
|
cmd_absorbgitdirs()
|
|
|
|
{
|
|
|
|
git submodule--helper absorb-git-dirs --prefix "$wt_prefix" "$@"
|
|
|
|
}
|
|
|
|
|
2008-01-15 11:48:45 +01:00
|
|
|
# This loop parses the command line arguments to find the
|
|
|
|
# subcommand name to dispatch. Parsing of the subcommand specific
|
|
|
|
# options are primarily done by the subcommand implementations.
|
|
|
|
# Subcommand specific options such as --branch and --cached are
|
|
|
|
# parsed here as well, for backward compatibility.
|
|
|
|
|
|
|
|
while test $# != 0 && test -z "$command"
|
2007-05-26 15:56:40 +02:00
|
|
|
do
|
|
|
|
case "$1" in
|
2019-10-29 18:01:52 +01:00
|
|
|
add | foreach | init | deinit | update | set-branch | set-url | status | summary | sync | absorbgitdirs)
|
2008-01-15 11:48:45 +01:00
|
|
|
command=$1
|
2007-05-26 15:56:40 +02:00
|
|
|
;;
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2007-05-26 15:56:40 +02:00
|
|
|
;;
|
2007-06-24 23:06:07 +02:00
|
|
|
-b|--branch)
|
|
|
|
case "$2" in
|
|
|
|
'')
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
branch="$2"; shift
|
|
|
|
;;
|
2007-05-26 15:56:40 +02:00
|
|
|
--cached)
|
2008-03-11 14:52:15 +01:00
|
|
|
cached="$1"
|
2007-05-26 15:56:40 +02:00
|
|
|
;;
|
|
|
|
--)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2008-01-15 11:48:45 +01:00
|
|
|
# No command word defaults to "status"
|
2012-09-22 13:27:59 +02:00
|
|
|
if test -z "$command"
|
|
|
|
then
|
|
|
|
if test $# = 0
|
|
|
|
then
|
|
|
|
command=status
|
|
|
|
else
|
|
|
|
usage
|
|
|
|
fi
|
|
|
|
fi
|
2008-01-15 11:48:45 +01:00
|
|
|
|
2019-02-08 12:21:34 +01:00
|
|
|
# "-b branch" is accepted only by "add" and "set-branch"
|
|
|
|
if test -n "$branch" && (test "$command" != add || test "$command" != set-branch)
|
2008-01-15 11:48:45 +01:00
|
|
|
then
|
2007-06-24 23:06:07 +02:00
|
|
|
usage
|
2008-01-15 11:48:45 +01:00
|
|
|
fi
|
|
|
|
|
2008-03-11 14:52:15 +01:00
|
|
|
# "--cached" is accepted only by "status" and "summary"
|
2014-06-10 14:28:33 +02:00
|
|
|
if test -n "$cached" && test "$command" != status && test "$command" != summary
|
2008-01-15 11:48:45 +01:00
|
|
|
then
|
2007-05-26 15:56:40 +02:00
|
|
|
usage
|
2008-01-15 11:48:45 +01:00
|
|
|
fi
|
|
|
|
|
2019-02-08 12:21:34 +01:00
|
|
|
"cmd_$(echo $command | sed -e s/-/_/g)" "$@"
|