2007-05-26 15:56:40 +02:00
|
|
|
#!/bin/sh
|
|
|
|
#
|
2012-06-25 12:56:59 +02:00
|
|
|
# git-submodule.sh: add, init, update or list git submodules
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
|
|
|
# Copyright (c) 2007 Lars Hjemli
|
|
|
|
|
2009-08-19 03:45:19 +02:00
|
|
|
dashless=$(basename "$0" | sed -e 's/-/ /')
|
2019-02-15 10:26:41 +01:00
|
|
|
USAGE="[--quiet] [--cached]
|
|
|
|
or: $dashless [--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
|
2009-08-19 03:45:24 +02:00
|
|
|
or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
|
2009-08-19 03:45:19 +02:00
|
|
|
or: $dashless [--quiet] init [--] [<path>...]
|
2016-05-05 21:52:32 +02:00
|
|
|
or: $dashless [--quiet] deinit [-f|--force] (--all| [--] <path>...)
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
or: $dashless [--quiet] update [--init [--filter=<filter-spec>]] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--[no-]recommend-shallow] [--reference <repository>] [--recursive] [--[no-]single-branch] [--] [<path>...]
|
2019-02-08 12:21:34 +01:00
|
|
|
or: $dashless [--quiet] set-branch (--default|--branch <branch>) [--] <path>
|
2019-10-29 18:01:52 +01:00
|
|
|
or: $dashless [--quiet] set-url [--] <path> <newurl>
|
2009-08-28 01:59:25 +02:00
|
|
|
or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
|
2009-08-19 03:45:22 +02:00
|
|
|
or: $dashless [--quiet] foreach [--recursive] <command>
|
2017-01-11 21:59:17 +01:00
|
|
|
or: $dashless [--quiet] sync [--recursive] [--] [<path>...]
|
|
|
|
or: $dashless [--quiet] absorbgitdirs [--] [<path>...]"
|
2007-11-06 10:50:02 +01:00
|
|
|
OPTIONS_SPEC=
|
2013-06-16 16:18:18 +02:00
|
|
|
SUBDIRECTORY_OK=Yes
|
2007-05-26 15:56:40 +02:00
|
|
|
. git-sh-setup
|
|
|
|
require_work_tree
|
2013-06-16 16:18:18 +02:00
|
|
|
wt_prefix=$(git rev-parse --show-prefix)
|
|
|
|
cd_to_toplevel
|
2007-05-26 15:56:40 +02:00
|
|
|
|
2016-12-14 23:39:52 +01:00
|
|
|
# Tell the rest of git that any URLs we get don't come
|
|
|
|
# directly from the user, so it can apply policy as appropriate.
|
|
|
|
GIT_PROTOCOL_FROM_USER=0
|
|
|
|
export GIT_PROTOCOL_FROM_USER
|
2015-09-16 19:13:12 +02:00
|
|
|
|
2008-01-15 11:48:45 +01:00
|
|
|
command=
|
2007-06-24 23:06:07 +02:00
|
|
|
branch=
|
2010-07-17 17:11:43 +02:00
|
|
|
force=
|
2009-05-04 21:30:01 +02:00
|
|
|
reference=
|
2007-05-26 15:56:40 +02:00
|
|
|
cached=
|
2010-04-26 11:50:39 +02:00
|
|
|
recursive=
|
|
|
|
init=
|
clone --recurse-submodules: prevent name squatting on Windows
In addition to preventing `.git` from being tracked by Git, on Windows
we also have to prevent `git~1` from being tracked, as the default NTFS
short name (also known as the "8.3 filename") for the file name `.git`
is `git~1`, otherwise it would be possible for malicious repositories to
write directly into the `.git/` directory, e.g. a `post-checkout` hook
that would then be executed _during_ a recursive clone.
When we implemented appropriate protections in 2b4c6efc821 (read-cache:
optionally disallow NTFS .git variants, 2014-12-16), we had analyzed
carefully that the `.git` directory or file would be guaranteed to be
the first directory entry to be written. Otherwise it would be possible
e.g. for a file named `..git` to be assigned the short name `git~1` and
subsequently, the short name generated for `.git` would be `git~2`. Or
`git~3`. Or even `~9999999` (for a detailed explanation of the lengths
we have to go to protect `.gitmodules`, see the commit message of
e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)).
However, by exploiting two issues (that will be addressed in a related
patch series close by), it is currently possible to clone a submodule
into a non-empty directory:
- On Windows, file names cannot end in a space or a period (for
historical reasons: the period separating the base name from the file
extension was not actually written to disk, and the base name/file
extension was space-padded to the full 8/3 characters, respectively).
Helpfully, when creating a directory under the name, say, `sub.`, that
trailing period is trimmed automatically and the actual name on disk
is `sub`.
This means that while Git thinks that the submodule names `sub` and
`sub.` are different, they both access `.git/modules/sub/`.
- While the backslash character is a valid file name character on Linux,
it is not so on Windows. As Git tries to be cross-platform, it
therefore allows backslash characters in the file names stored in tree
objects.
Which means that it is totally possible that a submodule `c` sits next
to a file `c\..git`, and on Windows, during recursive clone a file
called `..git` will be written into `c/`, of course _before_ the
submodule is cloned.
Note that the actual exploit is not quite as simple as having a
submodule `c` next to a file `c\..git`, as we have to make sure that the
directory `.git/modules/b` already exists when the submodule is checked
out, otherwise a different code path is taken in `module_clone()` that
does _not_ allow a non-empty submodule directory to exist already.
Even if we will address both issues nearby (the next commit will
disallow backslash characters in tree entries' file names on Windows,
and another patch will disallow creating directories/files with trailing
spaces or periods), it is a wise idea to defend in depth against this
sort of attack vector: when submodules are cloned recursively, we now
_require_ the directory to be empty, addressing CVE-2019-1349.
Note: the code path we patch is shared with the code path of `git
submodule update --init`, which must not expect, in general, that the
directory is empty. Hence we have to introduce the new option
`--force-init` and hand it all the way down from `git submodule` to the
actual `git submodule--helper` process that performs the initial clone.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
|
|
|
require_init=
|
2009-08-13 21:32:50 +02:00
|
|
|
files=
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
remote=
|
2009-02-05 23:18:32 +01:00
|
|
|
nofetch=
|
Rename submodule.<name>.rebase to submodule.<name>.update
The addition of "submodule.<name>.rebase" demonstrates the usefulness of
alternatives to the default behaviour of "git submodule update". However,
by naming the config variable "submodule.<name>.rebase", and making it a
boolean choice, we are artificially constraining future git versions that
may want to add _more_ alternatives than just "rebase".
Therefore, while "submodule.<name>.rebase" is not yet in a stable git
release, future-proof it, by changing it from
submodule.<name>.rebase = true/false
to
submodule.<name>.update = rebase/checkout
where "checkout" specifies the default behaviour of "git submodule update"
(checking out the new commit to a detached HEAD), and "rebase" specifies
the --rebase behaviour (where the current local branch in the submodule is
rebase onto the new commit). Thus .update == checkout is equivalent to
.rebase == false, and .update == rebase is equivalent to .rebase == true.
Finally, leaving .update unset is equivalent to leaving .rebase unset.
In future git versions, other alternatives to "git submodule update"
behaviour can be included by adding them to the list of allowable values
for the submodule.<name>.update variable.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-03 08:27:06 +02:00
|
|
|
update=
|
2009-08-19 03:45:22 +02:00
|
|
|
prefix=
|
2012-09-30 01:05:58 +02:00
|
|
|
custom_name=
|
2013-07-02 23:42:56 +02:00
|
|
|
depth=
|
clone: pass --progress decision to recursive submodules
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.
In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.
That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.
This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.
This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:
git clone --recursive --progress ... 2>&1 | cat
work by triggering progress explicitly in the children.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
|
|
|
progress=
|
2018-05-03 12:53:46 +02:00
|
|
|
dissociate=
|
2020-02-21 04:10:27 +01:00
|
|
|
single_branch=
|
2020-04-02 10:42:51 +02:00
|
|
|
jobs=
|
|
|
|
recommend_shallow=
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
filter=
|
2007-05-26 15:56:40 +02:00
|
|
|
|
2012-08-14 22:35:27 +02:00
|
|
|
die_if_unmatched ()
|
|
|
|
{
|
|
|
|
if test "$1" = "#unmatched"
|
|
|
|
then
|
2016-07-22 21:14:38 +02:00
|
|
|
exit ${2:-1}
|
2012-08-14 22:35:27 +02:00
|
|
|
fi
|
|
|
|
}
|
|
|
|
|
2013-04-01 15:06:27 +02:00
|
|
|
isnumber()
|
|
|
|
{
|
|
|
|
n=$(($1 + 0)) 2>/dev/null && test "$n" = "$1"
|
|
|
|
}
|
|
|
|
|
2016-02-29 23:58:35 +01:00
|
|
|
# Sanitize the local git environment for use within a submodule. We
|
|
|
|
# can't simply use clear_local_git_env since we want to preserve some
|
|
|
|
# of the settings from GIT_CONFIG_PARAMETERS.
|
|
|
|
sanitize_submodule_env()
|
|
|
|
{
|
submodule: stop sanitizing config options
The point of having a whitelist of command-line config
options to pass to submodules was two-fold:
1. It prevented obvious nonsense like using core.worktree
for multiple repos.
2. It could prevent surprise when the user did not mean
for the options to leak to the submodules (e.g.,
http.sslverify=false).
For case 1, the answer is mostly "if it hurts, don't do
that". For case 2, we can note that any such example has a
matching inverted surprise (e.g., a user who meant
http.sslverify=true to apply everywhere, but it didn't).
So this whitelist is probably not giving us any benefit, and
is already creating a hassle as people propose things to put
on it. Let's just drop it entirely.
Note that we still need to keep a special code path for
"prepare the submodule environment", because we still have
to take care to pass through $GIT_CONFIG_PARAMETERS (and
block the rest of the repo-specific environment variables).
We can do this easily from within the submodule shell
script, which lets us drop the submodule--helper option
entirely (and it's OK to do so because as a "--" program, it
is entirely a private implementation detail).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-05 03:22:19 +02:00
|
|
|
save_config=$GIT_CONFIG_PARAMETERS
|
2016-02-29 23:58:35 +01:00
|
|
|
clear_local_git_env
|
submodule: stop sanitizing config options
The point of having a whitelist of command-line config
options to pass to submodules was two-fold:
1. It prevented obvious nonsense like using core.worktree
for multiple repos.
2. It could prevent surprise when the user did not mean
for the options to leak to the submodules (e.g.,
http.sslverify=false).
For case 1, the answer is mostly "if it hurts, don't do
that". For case 2, we can note that any such example has a
matching inverted surprise (e.g., a user who meant
http.sslverify=true to apply everywhere, but it didn't).
So this whitelist is probably not giving us any benefit, and
is already creating a hassle as people propose things to put
on it. Let's just drop it entirely.
Note that we still need to keep a special code path for
"prepare the submodule environment", because we still have
to take care to pass through $GIT_CONFIG_PARAMETERS (and
block the rest of the repo-specific environment variables).
We can do this easily from within the submodule shell
script, which lets us drop the submodule--helper option
entirely (and it's OK to do so because as a "--" program, it
is entirely a private implementation detail).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-05 03:22:19 +02:00
|
|
|
GIT_CONFIG_PARAMETERS=$save_config
|
2016-04-28 15:37:44 +02:00
|
|
|
export GIT_CONFIG_PARAMETERS
|
2016-02-29 23:58:35 +01:00
|
|
|
}
|
|
|
|
|
2007-06-24 23:06:07 +02:00
|
|
|
#
|
|
|
|
# Add a new submodule to the working tree, .gitmodules and the index
|
|
|
|
#
|
git-submodule - make "submodule add" more strict, and document it
This change makes "submodule add" much more strict in the arguments it
takes, and is intended to address confusion as recently noted on the
git-list. With this change, the required syntax is:
$ git submodule add URL path
Specifically, this eliminates the form
$ git submodule add URL
which was confused by more than one person as
$ git submodule add path
With this patch, the URL locating the submodule's origin repository can be
either an absolute URL, or (if it begins with ./ or ../) can express the
submodule's repository location relative to the superproject's origin.
This patch also eliminates a third form of URL, which was relative to the
superproject's top-level directory (not its repository). Any URL that was
neither absolute nor matched ./*|../* was assumed to point to a
subdirectory of the superproject as the location of the submodule's origin
repository. This URL form was confusing and does not seem to correspond
to an important use-case. Specifically, no-one has identified the need to
clone from a repository already in the superproject's tree, but if this is
needed it is easily done using an absolute URL: $(pwd)/relative-path. So,
no functionality is lost with this patch. (t6008-rev-list-submodule.sh did
rely upon this relative URL, fixed by using $(pwd).)
Following this change, there are exactly four variants of
submodule-add, as both arguments have two flavors:
URL can be absolute, or can begin with ./|../ and thus names the
submodule's origin relative to the superproject's origin.
Note: With this patch, "submodule add" discerns an absolute URL as
matching /*|*:*: e.g., URL begins with /, or it contains a :. This works
for all valid URLs, an absolute path in POSIX, as well as an absolute path
on Windows).
path can either already exist as a valid git repo, or will be cloned from
the given URL. The first form here eases creation of a new submodule in
an existing superproject as the submodule can be added and tested in-tree
before pushing to the public repository. However, the more usual form is
the second, where the repo is cloned from the given URL.
This specifically addresses the issue of
$ git submodule add a/b/c
attempting to clone from a repository at "a/b/c" to create a new module
in "c". This also simplifies description of "relative URL" as there is now
exactly *one* form: a URL relative to the parent's origin repo.
Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-10 03:05:40 +02:00
|
|
|
# $@ = repo path
|
2007-06-24 23:06:07 +02:00
|
|
|
#
|
|
|
|
# optional branch is stored in global branch variable
|
|
|
|
#
|
2008-01-15 11:35:49 +01:00
|
|
|
cmd_add()
|
2007-06-24 23:06:07 +02:00
|
|
|
{
|
2008-01-15 11:48:45 +01:00
|
|
|
# parse $args after "submodule ... add".
|
2013-06-16 16:18:18 +02:00
|
|
|
reference_path=
|
2008-01-15 11:48:45 +01:00
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-b | --branch)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
branch=$2
|
|
|
|
shift
|
|
|
|
;;
|
2010-07-17 17:11:43 +02:00
|
|
|
-f | --force)
|
|
|
|
force=$1
|
|
|
|
;;
|
2008-01-15 11:48:45 +01:00
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-01-15 11:48:45 +01:00
|
|
|
;;
|
2018-05-03 12:53:45 +02:00
|
|
|
--progress)
|
|
|
|
progress=1
|
|
|
|
;;
|
2009-05-04 21:30:01 +02:00
|
|
|
--reference)
|
|
|
|
case "$2" in '') usage ;; esac
|
2013-06-16 16:18:18 +02:00
|
|
|
reference_path=$2
|
2009-05-04 21:30:01 +02:00
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--reference=*)
|
2013-06-16 16:18:18 +02:00
|
|
|
reference_path="${1#--reference=}"
|
2009-05-04 21:30:01 +02:00
|
|
|
;;
|
2018-05-03 12:53:46 +02:00
|
|
|
--dissociate)
|
|
|
|
dissociate=1
|
|
|
|
;;
|
2012-09-30 01:05:58 +02:00
|
|
|
--name)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
custom_name=$2
|
|
|
|
shift
|
|
|
|
;;
|
2013-07-02 23:42:56 +02:00
|
|
|
--depth)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
depth="--depth=$2"
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--depth=*)
|
|
|
|
depth=$1
|
|
|
|
;;
|
2008-01-15 11:48:45 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-08-10 13:46:37 +02:00
|
|
|
if test -z "$1"
|
2018-10-25 18:18:12 +02:00
|
|
|
then
|
2007-06-24 23:06:07 +02:00
|
|
|
usage
|
|
|
|
fi
|
|
|
|
|
2021-08-10 13:46:37 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper add ${GIT_QUIET:+--quiet} ${force:+--force} ${progress:+"--progress"} ${branch:+--branch "$branch"} ${reference_path:+--reference "$reference_path"} ${dissociate:+--dissociate} ${custom_name:+--name "$custom_name"} ${depth:+"$depth"} -- "$@"
|
2007-06-24 23:06:07 +02:00
|
|
|
}
|
|
|
|
|
2008-08-11 01:10:04 +02:00
|
|
|
#
|
|
|
|
# Execute an arbitrary command sequence in each checked out
|
|
|
|
# submodule
|
|
|
|
#
|
|
|
|
# $@ = command to execute
|
|
|
|
#
|
|
|
|
cmd_foreach()
|
|
|
|
{
|
2009-08-19 03:45:19 +02:00
|
|
|
# parse $args after "submodule ... foreach".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
|
|
|
GIT_QUIET=1
|
|
|
|
;;
|
2009-08-19 03:45:22 +02:00
|
|
|
--recursive)
|
|
|
|
recursive=1
|
|
|
|
;;
|
2009-08-19 03:45:19 +02:00
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} -- "$@"
|
2008-08-11 01:10:04 +02:00
|
|
|
}
|
|
|
|
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
2007-06-06 11:13:02 +02:00
|
|
|
# Register submodules in .git/config
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
|
|
|
# $@ = requested paths (default to all)
|
|
|
|
#
|
2008-01-15 11:35:49 +01:00
|
|
|
cmd_init()
|
2007-05-26 15:56:40 +02:00
|
|
|
{
|
2008-01-15 11:48:45 +01:00
|
|
|
# parse $args after "submodule ... init".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-01-15 11:48:45 +01:00
|
|
|
;;
|
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2019-04-12 12:08:19 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper init ${GIT_QUIET:+--quiet} -- "$@"
|
2007-05-26 15:56:40 +02:00
|
|
|
}
|
|
|
|
|
2013-03-04 22:20:24 +01:00
|
|
|
#
|
|
|
|
# Unregister submodules from .git/config and remove their work tree
|
|
|
|
#
|
|
|
|
cmd_deinit()
|
|
|
|
{
|
|
|
|
# parse $args after "submodule ... deinit".
|
2016-05-05 21:52:32 +02:00
|
|
|
deinit_all=
|
2013-03-04 22:20:24 +01:00
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-f|--force)
|
|
|
|
force=$1
|
|
|
|
;;
|
|
|
|
-q|--quiet)
|
|
|
|
GIT_QUIET=1
|
|
|
|
;;
|
2016-05-05 21:52:32 +02:00
|
|
|
--all)
|
|
|
|
deinit_all=t
|
|
|
|
;;
|
2013-03-04 22:20:24 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${force:+--force} ${deinit_all:+--all} -- "$@"
|
2013-03-04 22:20:24 +01:00
|
|
|
}
|
|
|
|
|
submodule: fix fetch_in_submodule logic
Commit 1c1518071c (submodule: use "fetch" logic instead of custom remote
discovery, 2020-11-14) rewrote the logic in fetch_in_submodule to do:
elif test "$2" -ne ""
But this is nonsense in shell: -ne is for numeric comparisons. This
should be "=" or more idiomatically:
elif test -n "$2"
But once we fix that, many tests start failing. Because that commit
introduced another problem. The caller that passes 3 arguments looks
like this:
fetch_in_submodule "$sm_path" $depth "$sha1"
Note the unquoted $depth parameter. When it isn't set, the function will
see only 2 arguments, and the function has no idea if what it sees in $2
is an option to go on the command line, or a refspec to pass on stdin.
In the old code before that commit:
fetch_in_submodule () (
sanitize_submodule_env &&
cd "$1" &&
- case "$2" in
- '')
- git fetch ;;
- *)
- shift
- git fetch $(get_default_remote) "$@" ;;
- esac
we treated those the same, so it didn't matter. But in the new logic
(with my fix above):
+ if test $# -eq 3
+ then
+ echo "$3" | git fetch --stdin "$2"
+ elif test -n "$n"
+ then
+ git fetch "$2"
+ else
+ git fetch
+ fi
we use the number of parameters to distinguish the two. Let's insist
that the caller pass an empty string for positional parameter two if
they want to have a third parameter after it.
But that still leaves one problem. In the --stdin block, we
unconditionally pass "$2" to git-fetch, even if it's the empty string.
Rather than add another conditional, we can use :+ parameter expansion
to include it only if it's non-empty. In fact, we can do the same for
the elif, too, simplifying it further. Technically this is overkill,
since we know the --depth parameter will not have whitespace (and
indeed, most callers do not bother quoting it), but it doesn't hurt for
the function to be careful.
It's somewhat amazing that no tests were failing. I think what happened
is that:
- the 3-arg form rarely triggered; any call with a non-empty $depth
and a $sha1 would work, but one with an empty $depth would only have
2 arguments
- because of the wrong arguments to "test", the shell would complain
and exit non-zero. So we never ran the middle conditional at all
- that left every call running "git fetch" with no arguments. A
well-written test could have detected the distinction here, but in
practice omitting --depth just means fetching more commits, and
fetching everything (rather than a single sha1) works as long as the
commit in question is reachable
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-24 10:06:05 +01:00
|
|
|
# usage: fetch_in_submodule <module_path> [<depth>] [<sha1>]
|
|
|
|
# Because arguments are positional, use an empty string to omit <depth>
|
|
|
|
# but include <sha1>.
|
2016-02-24 04:32:13 +01:00
|
|
|
fetch_in_submodule () (
|
2016-04-06 20:39:12 +02:00
|
|
|
sanitize_submodule_env &&
|
2016-02-24 04:32:13 +01:00
|
|
|
cd "$1" &&
|
2020-11-14 13:21:30 +01:00
|
|
|
if test $# -eq 3
|
|
|
|
then
|
2021-04-30 11:59:06 +02:00
|
|
|
echo "$3" | git fetch ${GIT_QUIET:+--quiet} --stdin ${2:+"$2"}
|
2020-11-14 13:21:30 +01:00
|
|
|
else
|
2021-04-30 11:59:06 +02:00
|
|
|
git fetch ${GIT_QUIET:+--quiet} ${2:+"$2"}
|
2020-11-14 13:21:30 +01:00
|
|
|
fi
|
2016-02-24 04:32:13 +01:00
|
|
|
)
|
|
|
|
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
2007-06-06 11:13:02 +02:00
|
|
|
# Update each submodule path to correct revision, using clone and checkout as needed
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
|
|
|
# $@ = requested paths (default to all)
|
|
|
|
#
|
2008-01-15 11:35:49 +01:00
|
|
|
cmd_update()
|
2007-05-26 15:56:40 +02:00
|
|
|
{
|
2008-01-15 11:48:45 +01:00
|
|
|
# parse $args after "submodule ... update".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-01-15 11:48:45 +01:00
|
|
|
;;
|
2018-08-14 20:22:02 +02:00
|
|
|
-v)
|
2020-09-30 21:50:53 +02:00
|
|
|
unset GIT_QUIET
|
2018-08-14 20:22:02 +02:00
|
|
|
;;
|
clone: pass --progress decision to recursive submodules
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.
In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.
That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.
This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.
This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:
git clone --recursive --progress ... 2>&1 | cat
work by triggering progress explicitly in the children.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
|
|
|
--progress)
|
2018-05-03 12:53:44 +02:00
|
|
|
progress=1
|
clone: pass --progress decision to recursive submodules
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.
In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.
That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.
This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.
This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:
git clone --recursive --progress ... 2>&1 | cat
work by triggering progress explicitly in the children.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
|
|
|
;;
|
2008-05-16 12:23:03 +02:00
|
|
|
-i|--init)
|
2009-05-04 21:30:01 +02:00
|
|
|
init=1
|
2008-05-16 12:23:03 +02:00
|
|
|
;;
|
clone --recurse-submodules: prevent name squatting on Windows
In addition to preventing `.git` from being tracked by Git, on Windows
we also have to prevent `git~1` from being tracked, as the default NTFS
short name (also known as the "8.3 filename") for the file name `.git`
is `git~1`, otherwise it would be possible for malicious repositories to
write directly into the `.git/` directory, e.g. a `post-checkout` hook
that would then be executed _during_ a recursive clone.
When we implemented appropriate protections in 2b4c6efc821 (read-cache:
optionally disallow NTFS .git variants, 2014-12-16), we had analyzed
carefully that the `.git` directory or file would be guaranteed to be
the first directory entry to be written. Otherwise it would be possible
e.g. for a file named `..git` to be assigned the short name `git~1` and
subsequently, the short name generated for `.git` would be `git~2`. Or
`git~3`. Or even `~9999999` (for a detailed explanation of the lengths
we have to go to protect `.gitmodules`, see the commit message of
e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)).
However, by exploiting two issues (that will be addressed in a related
patch series close by), it is currently possible to clone a submodule
into a non-empty directory:
- On Windows, file names cannot end in a space or a period (for
historical reasons: the period separating the base name from the file
extension was not actually written to disk, and the base name/file
extension was space-padded to the full 8/3 characters, respectively).
Helpfully, when creating a directory under the name, say, `sub.`, that
trailing period is trimmed automatically and the actual name on disk
is `sub`.
This means that while Git thinks that the submodule names `sub` and
`sub.` are different, they both access `.git/modules/sub/`.
- While the backslash character is a valid file name character on Linux,
it is not so on Windows. As Git tries to be cross-platform, it
therefore allows backslash characters in the file names stored in tree
objects.
Which means that it is totally possible that a submodule `c` sits next
to a file `c\..git`, and on Windows, during recursive clone a file
called `..git` will be written into `c/`, of course _before_ the
submodule is cloned.
Note that the actual exploit is not quite as simple as having a
submodule `c` next to a file `c\..git`, as we have to make sure that the
directory `.git/modules/b` already exists when the submodule is checked
out, otherwise a different code path is taken in `module_clone()` that
does _not_ allow a non-empty submodule directory to exist already.
Even if we will address both issues nearby (the next commit will
disallow backslash characters in tree entries' file names on Windows,
and another patch will disallow creating directories/files with trailing
spaces or periods), it is a wise idea to defend in depth against this
sort of attack vector: when submodules are cloned recursively, we now
_require_ the directory to be empty, addressing CVE-2019-1349.
Note: the code path we patch is shared with the code path of `git
submodule update --init`, which must not expect, in general, that the
directory is empty. Hence we have to introduce the new option
`--force-init` and hand it all the way down from `git submodule` to the
actual `git submodule--helper` process that performs the initial clone.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
|
|
|
--require-init)
|
|
|
|
init=1
|
|
|
|
require_init=1
|
|
|
|
;;
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
--remote)
|
|
|
|
remote=1
|
|
|
|
;;
|
2009-02-05 23:18:32 +01:00
|
|
|
-N|--no-fetch)
|
|
|
|
nofetch=1
|
|
|
|
;;
|
2011-04-01 11:42:03 +02:00
|
|
|
-f|--force)
|
|
|
|
force=$1
|
|
|
|
;;
|
2009-04-24 01:06:38 +02:00
|
|
|
-r|--rebase)
|
Rename submodule.<name>.rebase to submodule.<name>.update
The addition of "submodule.<name>.rebase" demonstrates the usefulness of
alternatives to the default behaviour of "git submodule update". However,
by naming the config variable "submodule.<name>.rebase", and making it a
boolean choice, we are artificially constraining future git versions that
may want to add _more_ alternatives than just "rebase".
Therefore, while "submodule.<name>.rebase" is not yet in a stable git
release, future-proof it, by changing it from
submodule.<name>.rebase = true/false
to
submodule.<name>.update = rebase/checkout
where "checkout" specifies the default behaviour of "git submodule update"
(checking out the new commit to a detached HEAD), and "rebase" specifies
the --rebase behaviour (where the current local branch in the submodule is
rebase onto the new commit). Thus .update == checkout is equivalent to
.rebase == false, and .update == rebase is equivalent to .rebase == true.
Finally, leaving .update unset is equivalent to leaving .rebase unset.
In future git versions, other alternatives to "git submodule update"
behaviour can be included by adding them to the list of allowable values
for the submodule.<name>.update variable.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-03 08:27:06 +02:00
|
|
|
update="rebase"
|
2009-04-24 01:06:38 +02:00
|
|
|
;;
|
2009-05-04 21:30:01 +02:00
|
|
|
--reference)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
reference="--reference=$2"
|
2010-11-03 07:26:25 +01:00
|
|
|
shift
|
2009-05-04 21:30:01 +02:00
|
|
|
;;
|
|
|
|
--reference=*)
|
|
|
|
reference="$1"
|
|
|
|
;;
|
2018-05-03 12:53:46 +02:00
|
|
|
--dissociate)
|
|
|
|
dissociate=1
|
|
|
|
;;
|
2009-06-03 00:59:12 +02:00
|
|
|
-m|--merge)
|
|
|
|
update="merge"
|
|
|
|
;;
|
2009-08-19 03:45:23 +02:00
|
|
|
--recursive)
|
|
|
|
recursive=1
|
|
|
|
;;
|
2011-10-11 00:56:16 +02:00
|
|
|
--checkout)
|
|
|
|
update="checkout"
|
|
|
|
;;
|
2016-05-26 23:59:43 +02:00
|
|
|
--recommend-shallow)
|
|
|
|
recommend_shallow="--recommend-shallow"
|
|
|
|
;;
|
|
|
|
--no-recommend-shallow)
|
|
|
|
recommend_shallow="--no-recommend-shallow"
|
|
|
|
;;
|
2013-07-02 23:42:56 +02:00
|
|
|
--depth)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
depth="--depth=$2"
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--depth=*)
|
|
|
|
depth=$1
|
|
|
|
;;
|
2016-03-01 03:07:19 +01:00
|
|
|
-j|--jobs)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
jobs="--jobs=$2"
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--jobs=*)
|
|
|
|
jobs=$1
|
|
|
|
;;
|
2020-02-21 04:10:27 +01:00
|
|
|
--single-branch)
|
|
|
|
single_branch="--single-branch"
|
|
|
|
;;
|
|
|
|
--no-single-branch)
|
|
|
|
single_branch="--no-single-branch"
|
|
|
|
;;
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
--filter)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
filter="--filter=$2"
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--filter=*)
|
|
|
|
filter="$1"
|
|
|
|
;;
|
2008-01-15 11:48:45 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
2010-11-03 07:26:25 +01:00
|
|
|
shift
|
2008-01-15 11:48:45 +01:00
|
|
|
done
|
|
|
|
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
if test -n "$filter" && test "$init" != "1"
|
|
|
|
then
|
|
|
|
usage
|
|
|
|
fi
|
|
|
|
|
2009-05-04 21:30:01 +02:00
|
|
|
if test -n "$init"
|
|
|
|
then
|
|
|
|
cmd_init "--" "$@" || return
|
|
|
|
fi
|
|
|
|
|
2016-03-01 03:07:17 +01:00
|
|
|
{
|
|
|
|
git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
|
2018-05-03 12:53:44 +02:00
|
|
|
${progress:+"--progress"} \
|
2016-03-01 03:07:17 +01:00
|
|
|
${wt_prefix:+--prefix "$wt_prefix"} \
|
|
|
|
${prefix:+--recursive-prefix "$prefix"} \
|
|
|
|
${update:+--update "$update"} \
|
2016-08-12 01:14:01 +02:00
|
|
|
${reference:+"$reference"} \
|
2018-05-03 12:53:46 +02:00
|
|
|
${dissociate:+"--dissociate"} \
|
2016-03-01 03:07:17 +01:00
|
|
|
${depth:+--depth "$depth"} \
|
clone --recurse-submodules: prevent name squatting on Windows
In addition to preventing `.git` from being tracked by Git, on Windows
we also have to prevent `git~1` from being tracked, as the default NTFS
short name (also known as the "8.3 filename") for the file name `.git`
is `git~1`, otherwise it would be possible for malicious repositories to
write directly into the `.git/` directory, e.g. a `post-checkout` hook
that would then be executed _during_ a recursive clone.
When we implemented appropriate protections in 2b4c6efc821 (read-cache:
optionally disallow NTFS .git variants, 2014-12-16), we had analyzed
carefully that the `.git` directory or file would be guaranteed to be
the first directory entry to be written. Otherwise it would be possible
e.g. for a file named `..git` to be assigned the short name `git~1` and
subsequently, the short name generated for `.git` would be `git~2`. Or
`git~3`. Or even `~9999999` (for a detailed explanation of the lengths
we have to go to protect `.gitmodules`, see the commit message of
e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)).
However, by exploiting two issues (that will be addressed in a related
patch series close by), it is currently possible to clone a submodule
into a non-empty directory:
- On Windows, file names cannot end in a space or a period (for
historical reasons: the period separating the base name from the file
extension was not actually written to disk, and the base name/file
extension was space-padded to the full 8/3 characters, respectively).
Helpfully, when creating a directory under the name, say, `sub.`, that
trailing period is trimmed automatically and the actual name on disk
is `sub`.
This means that while Git thinks that the submodule names `sub` and
`sub.` are different, they both access `.git/modules/sub/`.
- While the backslash character is a valid file name character on Linux,
it is not so on Windows. As Git tries to be cross-platform, it
therefore allows backslash characters in the file names stored in tree
objects.
Which means that it is totally possible that a submodule `c` sits next
to a file `c\..git`, and on Windows, during recursive clone a file
called `..git` will be written into `c/`, of course _before_ the
submodule is cloned.
Note that the actual exploit is not quite as simple as having a
submodule `c` next to a file `c\..git`, as we have to make sure that the
directory `.git/modules/b` already exists when the submodule is checked
out, otherwise a different code path is taken in `module_clone()` that
does _not_ allow a non-empty submodule directory to exist already.
Even if we will address both issues nearby (the next commit will
disallow backslash characters in tree entries' file names on Windows,
and another patch will disallow creating directories/files with trailing
spaces or periods), it is a wise idea to defend in depth against this
sort of attack vector: when submodules are cloned recursively, we now
_require_ the directory to be empty, addressing CVE-2019-1349.
Note: the code path we patch is shared with the code path of `git
submodule update --init`, which must not expect, in general, that the
directory is empty. Hence we have to introduce the new option
`--force-init` and hand it all the way down from `git submodule` to the
actual `git submodule--helper` process that performs the initial clone.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
|
|
|
${require_init:+--require-init} \
|
2020-02-21 04:10:27 +01:00
|
|
|
$single_branch \
|
2018-05-03 12:53:44 +02:00
|
|
|
$recommend_shallow \
|
|
|
|
$jobs \
|
clone, submodule: pass partial clone filters to submodules
When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-05 06:00:49 +01:00
|
|
|
$filter \
|
2019-04-12 12:08:19 +02:00
|
|
|
-- \
|
2016-07-22 21:14:38 +02:00
|
|
|
"$@" || echo "#unmatched" $?
|
2016-03-01 03:07:17 +01:00
|
|
|
} | {
|
2011-06-13 19:15:26 +02:00
|
|
|
err=
|
2018-08-04 00:23:17 +02:00
|
|
|
while read -r quickabort sha1 just_cloned sm_path
|
2007-05-26 15:56:40 +02:00
|
|
|
do
|
2018-08-04 00:23:17 +02:00
|
|
|
die_if_unmatched "$quickabort" "$sha1"
|
2016-03-01 03:07:17 +01:00
|
|
|
|
2019-01-18 22:55:19 +01:00
|
|
|
git submodule--helper ensure-core-worktree "$sm_path" || exit 1
|
2018-08-14 00:42:34 +02:00
|
|
|
|
2016-06-01 02:27:59 +02:00
|
|
|
displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
|
2013-06-16 16:18:18 +02:00
|
|
|
|
2016-03-01 03:07:17 +01:00
|
|
|
if test $just_cloned -eq 1
|
2014-04-02 23:15:36 +02:00
|
|
|
then
|
2007-06-11 21:12:22 +02:00
|
|
|
subsha1=
|
|
|
|
else
|
2021-08-24 16:06:09 +02:00
|
|
|
just_cloned=
|
2016-02-29 23:58:35 +01:00
|
|
|
subsha1=$(sanitize_submodule_env; cd "$sm_path" &&
|
2007-07-03 07:52:14 +02:00
|
|
|
git rev-parse --verify HEAD) ||
|
2021-07-10 09:47:59 +02:00
|
|
|
die "fatal: $(eval_gettext "Unable to find current revision in submodule path '\$displaypath'")"
|
2007-05-26 15:56:40 +02:00
|
|
|
fi
|
2007-06-06 11:13:02 +02:00
|
|
|
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
if test -n "$remote"
|
|
|
|
then
|
2016-08-03 22:44:03 +02:00
|
|
|
branch=$(git submodule--helper remote-branch "$sm_path")
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
if test -z "$nofetch"
|
|
|
|
then
|
|
|
|
# Fetch remote before determining tracking $sha1
|
2016-07-29 02:44:04 +02:00
|
|
|
fetch_in_submodule "$sm_path" $depth ||
|
2021-07-10 09:47:59 +02:00
|
|
|
die "fatal: $(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
fi
|
2020-11-14 13:21:31 +01:00
|
|
|
remote_name=$(sanitize_submodule_env; cd "$sm_path" && git submodule--helper print-default-remote)
|
2016-02-29 23:58:35 +01:00
|
|
|
sha1=$(sanitize_submodule_env; cd "$sm_path" &&
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
git rev-parse --verify "${remote_name}/${branch}") ||
|
2021-07-10 09:47:59 +02:00
|
|
|
die "fatal: $(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
|
submodule update: add --remote for submodule's upstream changes
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
|
|
|
fi
|
|
|
|
|
2021-08-24 16:06:09 +02:00
|
|
|
out=$(git submodule--helper run-update-procedure \
|
|
|
|
${wt_prefix:+--prefix "$wt_prefix"} \
|
|
|
|
${GIT_QUIET:+--quiet} \
|
|
|
|
${force:+--force} \
|
|
|
|
${just_cloned:+--just-cloned} \
|
|
|
|
${nofetch:+--no-fetch} \
|
|
|
|
${depth:+"$depth"} \
|
|
|
|
${update:+--update "$update"} \
|
|
|
|
${prefix:+--recursive-prefix "$prefix"} \
|
|
|
|
${sha1:+--oid "$sha1"} \
|
|
|
|
${subsha1:+--suboid "$subsha1"} \
|
|
|
|
"--" \
|
|
|
|
"$sm_path")
|
|
|
|
|
|
|
|
# exit codes for run-update-procedure:
|
|
|
|
# 0: update was successful, say command output
|
|
|
|
# 1: update procedure failed, but should not die
|
|
|
|
# 2 or 128: subcommand died during execution
|
|
|
|
# 3: no update procedure was run
|
|
|
|
res="$?"
|
|
|
|
case $res in
|
|
|
|
0)
|
|
|
|
say "$out"
|
|
|
|
;;
|
|
|
|
1)
|
|
|
|
err="${err};fatal: $out"
|
|
|
|
continue
|
|
|
|
;;
|
|
|
|
2|128)
|
|
|
|
die_with_status $res "fatal: $out"
|
|
|
|
;;
|
|
|
|
esac
|
2009-08-19 03:45:23 +02:00
|
|
|
|
|
|
|
if test -n "$recursive"
|
|
|
|
then
|
2013-03-02 20:44:59 +01:00
|
|
|
(
|
2016-06-01 02:27:59 +02:00
|
|
|
prefix=$(git submodule--helper relative-path "$prefix$sm_path/" "$wt_prefix")
|
submodule: port init from shell to C
By having the `submodule init` functionality in C, we can reference it
easier from other parts in the code in later patches. The code is split
up to have one function to initialize one submodule and a calling function
that takes care of the rest, such as argument handling and translating the
arguments to the paths of the submodules.
This is the first submodule subcommand that is fully converted to C
except for the usage string, so this is actually removing a call to
the `submodule--helper list` function, which is supposed to be used in
this transition. Instead we'll make a direct call to `module_list_compute`.
An explanation why we need to edit the prefixes in cmd_update in
git-submodule.sh in this patch:
By having no processing in the shell part, we need to convey the notion
of wt_prefix and prefix to the C parts, which former patches punted on
and did the processing of displaying path in the shell.
`wt_prefix` used to hold the path from the repository root to the current
directory, e.g. wt_prefix would be t/ if the user invoked the
`git submodule` command in ~/repo/t and ~repo is the GIT_DIR.
`prefix` used to hold the relative path from the repository root to the
operation, e.g. if you have recursive submodules, the shell script would
modify the `prefix` in each recursive step by adding the submodule path.
We will pass `wt_prefix` into the C helper via `git -C <dir>` as that
will setup git in the directory the user actually called git-submodule.sh
from. The `prefix` will be passed in via the `--prefix` option.
Having `prefix` and `wt_prefix` relative to the GIT_DIR of the
calling superproject is unfortunate with this patch as the C code doesn't
know about a possible recursion from a superproject via `submodule update
--init --recursive`.
To fix this, we change the meaning of `wt_prefix` to point to the current
project instead of the superproject and `prefix` to include any relative
paths issues in the superproject. That way `prefix` will become the leading
part for displaying paths and `wt_prefix` will be empty in recursive
calls for now.
The new notion of `wt_prefix` and `prefix` still allows us to reconstruct
the calling directory in the superproject by just traveling reverse of
`prefix`.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-16 02:50:13 +02:00
|
|
|
wt_prefix=
|
2016-02-29 23:58:35 +01:00
|
|
|
sanitize_submodule_env
|
2013-03-02 20:44:59 +01:00
|
|
|
cd "$sm_path" &&
|
2013-11-11 21:55:52 +01:00
|
|
|
eval cmd_update
|
2013-03-02 20:44:59 +01:00
|
|
|
)
|
2011-06-13 19:15:26 +02:00
|
|
|
res=$?
|
|
|
|
if test $res -gt 0
|
|
|
|
then
|
2021-07-10 09:47:59 +02:00
|
|
|
die_msg="fatal: $(eval_gettext "Failed to recurse into submodule path '\$displaypath'")"
|
2016-06-09 21:06:37 +02:00
|
|
|
if test $res -ne 2
|
2011-06-13 19:15:26 +02:00
|
|
|
then
|
2011-07-13 23:31:35 +02:00
|
|
|
err="${err};$die_msg"
|
2011-06-13 19:15:26 +02:00
|
|
|
continue
|
|
|
|
else
|
2011-07-13 23:31:35 +02:00
|
|
|
die_with_status $res "$die_msg"
|
2011-06-13 19:15:26 +02:00
|
|
|
fi
|
|
|
|
fi
|
2009-08-19 03:45:23 +02:00
|
|
|
fi
|
2007-05-26 15:56:40 +02:00
|
|
|
done
|
2011-06-13 19:15:26 +02:00
|
|
|
|
|
|
|
if test -n "$err"
|
|
|
|
then
|
|
|
|
OIFS=$IFS
|
|
|
|
IFS=';'
|
|
|
|
for e in $err
|
|
|
|
do
|
|
|
|
if test -n "$e"
|
|
|
|
then
|
|
|
|
echo >&2 "$e"
|
|
|
|
fi
|
|
|
|
done
|
|
|
|
IFS=$OIFS
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
}
|
2007-05-26 15:56:40 +02:00
|
|
|
}
|
|
|
|
|
2019-02-08 12:21:34 +01:00
|
|
|
#
|
|
|
|
# Configures a submodule's default branch
|
|
|
|
#
|
|
|
|
# $@ = requested path
|
|
|
|
#
|
|
|
|
cmd_set_branch() {
|
2020-06-02 18:35:23 +02:00
|
|
|
default=
|
2019-02-08 12:21:34 +01:00
|
|
|
branch=
|
|
|
|
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
|
|
|
# we don't do anything with this but we need to accept it
|
|
|
|
;;
|
|
|
|
-d|--default)
|
2020-06-02 18:35:23 +02:00
|
|
|
default=1
|
2019-02-08 12:21:34 +01:00
|
|
|
;;
|
|
|
|
-b|--branch)
|
|
|
|
case "$2" in '') usage ;; esac
|
|
|
|
branch=$2
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper set-branch ${GIT_QUIET:+--quiet} ${branch:+--branch "$branch"} ${default:+--default} -- "$@"
|
2019-02-08 12:21:34 +01:00
|
|
|
}
|
|
|
|
|
2019-10-29 18:01:52 +01:00
|
|
|
#
|
|
|
|
# Configures a submodule's remote url
|
|
|
|
#
|
|
|
|
# $@ = requested path, requested url
|
|
|
|
#
|
|
|
|
cmd_set_url() {
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
|
|
|
GIT_QUIET=1
|
|
|
|
;;
|
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper set-url ${GIT_QUIET:+--quiet} -- "$@"
|
2019-10-29 18:01:52 +01:00
|
|
|
}
|
|
|
|
|
2008-03-11 14:52:15 +01:00
|
|
|
#
|
|
|
|
# Show commit summary for submodules in index or working tree
|
|
|
|
#
|
|
|
|
# If '--cached' is given, show summary between index and given commit,
|
|
|
|
# or between working tree and given commit
|
|
|
|
#
|
|
|
|
# $@ = [commit (default 'HEAD'),] requested paths (default all)
|
|
|
|
#
|
|
|
|
cmd_summary() {
|
2008-03-11 14:52:17 +01:00
|
|
|
summary_limit=-1
|
2008-04-12 17:05:31 +02:00
|
|
|
for_status=
|
2009-08-13 21:32:50 +02:00
|
|
|
diff_cmd=diff-index
|
2008-03-11 14:52:17 +01:00
|
|
|
|
2008-03-11 14:52:15 +01:00
|
|
|
# parse $args after "submodule ... summary".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
--cached)
|
|
|
|
cached="$1"
|
|
|
|
;;
|
2009-08-13 21:32:50 +02:00
|
|
|
--files)
|
|
|
|
files="$1"
|
|
|
|
;;
|
2008-04-12 17:05:31 +02:00
|
|
|
--for-status)
|
|
|
|
for_status="$1"
|
|
|
|
;;
|
2008-03-11 14:52:17 +01:00
|
|
|
-n|--summary-limit)
|
2013-04-01 15:06:27 +02:00
|
|
|
summary_limit="$2"
|
|
|
|
isnumber "$summary_limit" || usage
|
2008-03-11 14:52:17 +01:00
|
|
|
shift
|
|
|
|
;;
|
2013-04-01 15:06:27 +02:00
|
|
|
--summary-limit=*)
|
|
|
|
summary_limit="${1#--summary-limit=}"
|
|
|
|
isnumber "$summary_limit" || usage
|
|
|
|
;;
|
2008-03-11 14:52:15 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
2007-06-27 01:40:58 +02:00
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${files:+--files} ${cached:+--cached} ${for_status:+--for-status} ${summary_limit:+-n $summary_limit} -- "$@"
|
2008-03-11 14:52:15 +01:00
|
|
|
}
|
2007-05-26 15:56:40 +02:00
|
|
|
#
|
2007-06-11 21:12:24 +02:00
|
|
|
# List all submodules, prefixed with:
|
2007-05-26 15:56:40 +02:00
|
|
|
# - submodule not initialized
|
|
|
|
# + different revision checked out
|
|
|
|
#
|
|
|
|
# If --cached was specified the revision in the index will be printed
|
|
|
|
# instead of the currently checked out revision.
|
|
|
|
#
|
|
|
|
# $@ = requested paths (default to all)
|
|
|
|
#
|
2008-01-15 11:35:49 +01:00
|
|
|
cmd_status()
|
2007-05-26 15:56:40 +02:00
|
|
|
{
|
2008-01-15 11:48:45 +01:00
|
|
|
# parse $args after "submodule ... status".
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-01-15 11:48:45 +01:00
|
|
|
;;
|
|
|
|
--cached)
|
|
|
|
cached=1
|
|
|
|
;;
|
2009-08-19 03:45:24 +02:00
|
|
|
--recursive)
|
|
|
|
recursive=1
|
|
|
|
;;
|
2008-01-15 11:48:45 +01:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} -- "$@"
|
2007-05-26 15:56:40 +02:00
|
|
|
}
|
2008-08-24 21:43:37 +02:00
|
|
|
#
|
|
|
|
# Sync remote urls for submodules
|
|
|
|
# This makes the value for remote.$remote.url match the value
|
|
|
|
# specified in .gitmodules.
|
|
|
|
#
|
|
|
|
cmd_sync()
|
|
|
|
{
|
|
|
|
while test $# -ne 0
|
|
|
|
do
|
|
|
|
case "$1" in
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2008-08-24 21:43:37 +02:00
|
|
|
shift
|
|
|
|
;;
|
2012-10-26 21:44:42 +02:00
|
|
|
--recursive)
|
|
|
|
recursive=1
|
|
|
|
shift
|
|
|
|
;;
|
2008-08-24 21:43:37 +02:00
|
|
|
--)
|
|
|
|
shift
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
done
|
2017-03-16 23:29:45 +01:00
|
|
|
|
2021-06-22 20:14:52 +02:00
|
|
|
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} -- "$@"
|
2008-08-24 21:43:37 +02:00
|
|
|
}
|
2007-05-26 15:56:40 +02:00
|
|
|
|
2016-12-12 20:04:35 +01:00
|
|
|
cmd_absorbgitdirs()
|
|
|
|
{
|
|
|
|
git submodule--helper absorb-git-dirs --prefix "$wt_prefix" "$@"
|
|
|
|
}
|
|
|
|
|
2008-01-15 11:48:45 +01:00
|
|
|
# This loop parses the command line arguments to find the
|
|
|
|
# subcommand name to dispatch. Parsing of the subcommand specific
|
|
|
|
# options are primarily done by the subcommand implementations.
|
|
|
|
# Subcommand specific options such as --branch and --cached are
|
|
|
|
# parsed here as well, for backward compatibility.
|
|
|
|
|
|
|
|
while test $# != 0 && test -z "$command"
|
2007-05-26 15:56:40 +02:00
|
|
|
do
|
|
|
|
case "$1" in
|
2019-10-29 18:01:52 +01:00
|
|
|
add | foreach | init | deinit | update | set-branch | set-url | status | summary | sync | absorbgitdirs)
|
2008-01-15 11:48:45 +01:00
|
|
|
command=$1
|
2007-05-26 15:56:40 +02:00
|
|
|
;;
|
|
|
|
-q|--quiet)
|
2009-06-17 00:33:00 +02:00
|
|
|
GIT_QUIET=1
|
2007-05-26 15:56:40 +02:00
|
|
|
;;
|
2007-06-24 23:06:07 +02:00
|
|
|
-b|--branch)
|
|
|
|
case "$2" in
|
|
|
|
'')
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
branch="$2"; shift
|
|
|
|
;;
|
2007-05-26 15:56:40 +02:00
|
|
|
--cached)
|
2008-03-11 14:52:15 +01:00
|
|
|
cached="$1"
|
2007-05-26 15:56:40 +02:00
|
|
|
;;
|
|
|
|
--)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
-*)
|
|
|
|
usage
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
|
|
|
esac
|
|
|
|
shift
|
|
|
|
done
|
|
|
|
|
2008-01-15 11:48:45 +01:00
|
|
|
# No command word defaults to "status"
|
2012-09-22 13:27:59 +02:00
|
|
|
if test -z "$command"
|
|
|
|
then
|
|
|
|
if test $# = 0
|
|
|
|
then
|
|
|
|
command=status
|
|
|
|
else
|
|
|
|
usage
|
|
|
|
fi
|
|
|
|
fi
|
2008-01-15 11:48:45 +01:00
|
|
|
|
2019-02-08 12:21:34 +01:00
|
|
|
# "-b branch" is accepted only by "add" and "set-branch"
|
|
|
|
if test -n "$branch" && (test "$command" != add || test "$command" != set-branch)
|
2008-01-15 11:48:45 +01:00
|
|
|
then
|
2007-06-24 23:06:07 +02:00
|
|
|
usage
|
2008-01-15 11:48:45 +01:00
|
|
|
fi
|
|
|
|
|
2008-03-11 14:52:15 +01:00
|
|
|
# "--cached" is accepted only by "status" and "summary"
|
2014-06-10 14:28:33 +02:00
|
|
|
if test -n "$cached" && test "$command" != status && test "$command" != summary
|
2008-01-15 11:48:45 +01:00
|
|
|
then
|
2007-05-26 15:56:40 +02:00
|
|
|
usage
|
2008-01-15 11:48:45 +01:00
|
|
|
fi
|
|
|
|
|
2019-02-08 12:21:34 +01:00
|
|
|
"cmd_$(echo $command | sed -e s/-/_/g)" "$@"
|