git-commit-vandalism/git-submodule.sh

955 lines
20 KiB
Bash
Raw Normal View History

#!/bin/sh
#
# git-submodule.sh: add, init, update or list git submodules
#
# Copyright (c) 2007 Lars Hjemli
dashless=$(basename "$0" | sed -e 's/-/ /')
USAGE="[--quiet] [--cached]
or: $dashless [--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
or: $dashless [--quiet] init [--] [<path>...]
or: $dashless [--quiet] deinit [-f|--force] (--all| [--] <path>...)
or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--[no-]recommend-shallow] [--reference <repository>] [--recursive] [--[no-]single-branch] [--] [<path>...]
or: $dashless [--quiet] set-branch (--default|--branch <branch>) [--] <path>
or: $dashless [--quiet] set-url [--] <path> <newurl>
or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
or: $dashless [--quiet] foreach [--recursive] <command>
or: $dashless [--quiet] sync [--recursive] [--] [<path>...]
or: $dashless [--quiet] absorbgitdirs [--] [<path>...]"
OPTIONS_SPEC=
SUBDIRECTORY_OK=Yes
. git-sh-setup
require_work_tree
wt_prefix=$(git rev-parse --show-prefix)
cd_to_toplevel
transport: add protocol policy config option Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to specify a whitelist of protocols to be used in clone/fetch/push commands. This patch introduces new configuration options for more fine-grained control for allowing/disallowing protocols. This also has the added benefit of allowing easier construction of a protocol whitelist on systems where setting an environment variable is non-trivial. Now users can specify a policy to be used for each type of protocol via the 'protocol.<name>.allow' config option. A default policy for all unconfigured protocols can be set with the 'protocol.allow' config option. If no user configured default is made git will allow known-safe protocols (http, https, git, ssh, file), disallow known-dangerous protocols (ext), and have a default policy of `user` for all other protocols. The supported policies are `always`, `never`, and `user`. The `user` policy can be used to configure a protocol to be usable when explicitly used by a user, while disallowing it for commands which run clone/fetch/push commands without direct user intervention (e.g. recursive initialization of submodules). Commands which can potentially clone/fetch/push from untrusted repositories without user intervention can export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent protocols configured to the `user` policy from being used. Fix remote-ext tests to use the new config to allow the ext protocol to be tested. Based on a patch by Jeff King <peff@peff.net> Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-14 23:39:52 +01:00
# Tell the rest of git that any URLs we get don't come
# directly from the user, so it can apply policy as appropriate.
GIT_PROTOCOL_FROM_USER=0
export GIT_PROTOCOL_FROM_USER
command=
branch=
force=
reference=
cached=
recursive=
init=
clone --recurse-submodules: prevent name squatting on Windows In addition to preventing `.git` from being tracked by Git, on Windows we also have to prevent `git~1` from being tracked, as the default NTFS short name (also known as the "8.3 filename") for the file name `.git` is `git~1`, otherwise it would be possible for malicious repositories to write directly into the `.git/` directory, e.g. a `post-checkout` hook that would then be executed _during_ a recursive clone. When we implemented appropriate protections in 2b4c6efc821 (read-cache: optionally disallow NTFS .git variants, 2014-12-16), we had analyzed carefully that the `.git` directory or file would be guaranteed to be the first directory entry to be written. Otherwise it would be possible e.g. for a file named `..git` to be assigned the short name `git~1` and subsequently, the short name generated for `.git` would be `git~2`. Or `git~3`. Or even `~9999999` (for a detailed explanation of the lengths we have to go to protect `.gitmodules`, see the commit message of e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)). However, by exploiting two issues (that will be addressed in a related patch series close by), it is currently possible to clone a submodule into a non-empty directory: - On Windows, file names cannot end in a space or a period (for historical reasons: the period separating the base name from the file extension was not actually written to disk, and the base name/file extension was space-padded to the full 8/3 characters, respectively). Helpfully, when creating a directory under the name, say, `sub.`, that trailing period is trimmed automatically and the actual name on disk is `sub`. This means that while Git thinks that the submodule names `sub` and `sub.` are different, they both access `.git/modules/sub/`. - While the backslash character is a valid file name character on Linux, it is not so on Windows. As Git tries to be cross-platform, it therefore allows backslash characters in the file names stored in tree objects. Which means that it is totally possible that a submodule `c` sits next to a file `c\..git`, and on Windows, during recursive clone a file called `..git` will be written into `c/`, of course _before_ the submodule is cloned. Note that the actual exploit is not quite as simple as having a submodule `c` next to a file `c\..git`, as we have to make sure that the directory `.git/modules/b` already exists when the submodule is checked out, otherwise a different code path is taken in `module_clone()` that does _not_ allow a non-empty submodule directory to exist already. Even if we will address both issues nearby (the next commit will disallow backslash characters in tree entries' file names on Windows, and another patch will disallow creating directories/files with trailing spaces or periods), it is a wise idea to defend in depth against this sort of attack vector: when submodules are cloned recursively, we now _require_ the directory to be empty, addressing CVE-2019-1349. Note: the code path we patch is shared with the code path of `git submodule update --init`, which must not expect, in general, that the directory is empty. Hence we have to introduce the new option `--force-init` and hand it all the way down from `git submodule` to the actual `git submodule--helper` process that performs the initial clone. Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
require_init=
files=
submodule update: add --remote for submodule's upstream changes The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule.<name>.branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule.<name>.branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule.<name>.branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules/<name>/config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule.<name>.branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King <wking@tremily.us> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
remote=
nofetch=
update=
prefix=
custom_name=
depth=
clone: pass --progress decision to recursive submodules When cloning with "--recursive", we'd generally expect submodules to show progress reports if the main clone did, too. In older versions of git, this mostly worked out of the box. Since we show progress by default when stderr is a tty, and since the child clones inherit the parent stderr, then both processes would come to the same decision by default. If the parent clone was asked for "--quiet", we passed down "--quiet" to the child. However, if stderr was not a tty and the user specified "--progress", we did not propagate this to the child. That's a minor bug, but things got much worse when we switched recently to submodule--helper's update_clone command. With that change, the stderr of the child clones are always connected to a pipe, and we never output progress at all. This patch teaches git-submodule and git-submodule--helper how to pass down an explicit "--progress" flag when cloning. The clone command then decides to propagate that flag based on the cloning decision made earlier (which takes into account isatty(2) of the parent process, existing --progress or --quiet flags, etc). Since the child processes always run without a tty on stderr, we don't have to worry about passing an explicit "--no-progress"; it's the default for them. This fixes the recent loss of progress during recursive clones. And as a bonus, it makes: git clone --recursive --progress ... 2>&1 | cat work by triggering progress explicitly in the children. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
progress=
dissociate=
single_branch=
jobs=
recommend_shallow=
die_if_unmatched ()
{
if test "$1" = "#unmatched"
then
exit ${2:-1}
fi
}
isnumber()
{
n=$(($1 + 0)) 2>/dev/null && test "$n" = "$1"
}
# Given a full hex object ID, is this the zero OID?
is_zero_oid () {
echo "$1" | sane_egrep '^0+$' >/dev/null 2>&1
}
# Sanitize the local git environment for use within a submodule. We
# can't simply use clear_local_git_env since we want to preserve some
# of the settings from GIT_CONFIG_PARAMETERS.
sanitize_submodule_env()
{
save_config=$GIT_CONFIG_PARAMETERS
clear_local_git_env
GIT_CONFIG_PARAMETERS=$save_config
export GIT_CONFIG_PARAMETERS
}
#
# Add a new submodule to the working tree, .gitmodules and the index
#
git-submodule - make "submodule add" more strict, and document it This change makes "submodule add" much more strict in the arguments it takes, and is intended to address confusion as recently noted on the git-list. With this change, the required syntax is: $ git submodule add URL path Specifically, this eliminates the form $ git submodule add URL which was confused by more than one person as $ git submodule add path With this patch, the URL locating the submodule's origin repository can be either an absolute URL, or (if it begins with ./ or ../) can express the submodule's repository location relative to the superproject's origin. This patch also eliminates a third form of URL, which was relative to the superproject's top-level directory (not its repository). Any URL that was neither absolute nor matched ./*|../* was assumed to point to a subdirectory of the superproject as the location of the submodule's origin repository. This URL form was confusing and does not seem to correspond to an important use-case. Specifically, no-one has identified the need to clone from a repository already in the superproject's tree, but if this is needed it is easily done using an absolute URL: $(pwd)/relative-path. So, no functionality is lost with this patch. (t6008-rev-list-submodule.sh did rely upon this relative URL, fixed by using $(pwd).) Following this change, there are exactly four variants of submodule-add, as both arguments have two flavors: URL can be absolute, or can begin with ./|../ and thus names the submodule's origin relative to the superproject's origin. Note: With this patch, "submodule add" discerns an absolute URL as matching /*|*:*: e.g., URL begins with /, or it contains a :. This works for all valid URLs, an absolute path in POSIX, as well as an absolute path on Windows). path can either already exist as a valid git repo, or will be cloned from the given URL. The first form here eases creation of a new submodule in an existing superproject as the submodule can be added and tested in-tree before pushing to the public repository. However, the more usual form is the second, where the repo is cloned from the given URL. This specifically addresses the issue of $ git submodule add a/b/c attempting to clone from a repository at "a/b/c" to create a new module in "c". This also simplifies description of "relative URL" as there is now exactly *one* form: a URL relative to the parent's origin repo. Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-10 03:05:40 +02:00
# $@ = repo path
#
# optional branch is stored in global branch variable
#
cmd_add()
{
# parse $args after "submodule ... add".
reference_path=
while test $# -ne 0
do
case "$1" in
-b | --branch)
case "$2" in '') usage ;; esac
branch=$2
shift
;;
-f | --force)
force=$1
;;
-q|--quiet)
GIT_QUIET=1
;;
--progress)
progress=1
;;
--reference)
case "$2" in '') usage ;; esac
reference_path=$2
shift
;;
--reference=*)
reference_path="${1#--reference=}"
;;
--dissociate)
dissociate=1
;;
--name)
case "$2" in '') usage ;; esac
custom_name=$2
shift
;;
--depth)
case "$2" in '') usage ;; esac
depth="--depth=$2"
shift
;;
--depth=*)
depth=$1
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
submodule: support reading .gitmodules when it's not in the working tree When the .gitmodules file is not available in the working tree, try using the content from the index and from the current branch. This covers the case when the file is part of the repository but for some reason it is not checked out, for example because of a sparse checkout. This makes it possible to use at least the 'git submodule' commands which *read* the gitmodules configuration file without fully populating the working tree. Writing to .gitmodules will still require that the file is checked out, so check for that before calling config_set_in_gitmodules_file_gently. Add a similar check also in git-submodule.sh::cmd_add() to anticipate the eventual failure of the "git submodule add" command when .gitmodules is not safely writeable; this prevents the command from leaving the repository in a spurious state (e.g. the submodule repository was cloned but .gitmodules was not updated because config_set_in_gitmodules_file_gently failed). Moreover, since config_from_gitmodules() now accesses the global object store, it is necessary to protect all code paths which call the function against concurrent access to the global object store. Currently this only happens in builtin/grep.c::grep_submodules(), so call grep_read_lock() before invoking code involving config_from_gitmodules(). Finally, add t7418-submodule-sparse-gitmodules.sh to verify that reading from .gitmodules succeeds and that writing to it fails when the file is not checked out. NOTE: there is one rare case where this new feature does not work properly yet: nested submodules without .gitmodules in their working tree. This has been documented with a warning and a test_expect_failure item in t7814, and in this case the current behavior is not altered: no config is read. Signed-off-by: Antonio Ospite <ao2@ao2.it> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-25 18:18:12 +02:00
if ! git submodule--helper config --check-writeable >/dev/null 2>&1
then
die "$(eval_gettext "please make sure that the .gitmodules file is in the working tree")"
fi
if test -n "$reference_path"
then
is_absolute_path "$reference_path" ||
reference_path="$wt_prefix$reference_path"
reference="--reference=$reference_path"
fi
repo=$1
git-submodule.sh: Don't use $path variable in eval_gettext string The eval_gettext (and eval_gettextln) i18n shell functions call git-sh-i18n--envsubst to process the variable references in the string parameter. Unfortunately, environment variables are case insensitive on windows, which leads to failure on cygwin when eval_gettext exports $path. Commit df599e9 (Windows: teach getenv to do a case-sensitive search, 06-06-2011) attempts to solve this problem on MinGW by overriding the system getenv() function to allow git-sh-i18n--envsubst to read $path rather than $PATH from the environment. However, this commit does not address cygwin at all and, furthermore, does not fix all problems on MinGW. In particular, when executing test #38 in t7400-submodule-basic.sh, an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog pops up saying that the application "failed to start because libiconv2.dll was not found." After studying the voluminous trace output from the process monitor, it is clear that the system is attempting to use $path, rather than $PATH, to search for the DLL file. (Note that, after dismissing the dialog, the test passes anyway!) As an alternative, we finesse the problem by renaming the $path variable to $sm_path (submodule path). This fixes the problem on MinGW along with all test failures on cygwin (t7400.{7,32,34}, t7406.3 and t7407.{2,6}). We note that the foreach subcommand provides $path to user scripts (ie it is part of the API), so we can't simply rename it to $sm_path. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jens Lehmann <Jens.Lehmann@web.de> Tested-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-17 20:00:58 +02:00
sm_path=$2
git-submodule.sh: Don't use $path variable in eval_gettext string The eval_gettext (and eval_gettextln) i18n shell functions call git-sh-i18n--envsubst to process the variable references in the string parameter. Unfortunately, environment variables are case insensitive on windows, which leads to failure on cygwin when eval_gettext exports $path. Commit df599e9 (Windows: teach getenv to do a case-sensitive search, 06-06-2011) attempts to solve this problem on MinGW by overriding the system getenv() function to allow git-sh-i18n--envsubst to read $path rather than $PATH from the environment. However, this commit does not address cygwin at all and, furthermore, does not fix all problems on MinGW. In particular, when executing test #38 in t7400-submodule-basic.sh, an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog pops up saying that the application "failed to start because libiconv2.dll was not found." After studying the voluminous trace output from the process monitor, it is clear that the system is attempting to use $path, rather than $PATH, to search for the DLL file. (Note that, after dismissing the dialog, the test passes anyway!) As an alternative, we finesse the problem by renaming the $path variable to $sm_path (submodule path). This fixes the problem on MinGW along with all test failures on cygwin (t7400.{7,32,34}, t7406.3 and t7407.{2,6}). We note that the foreach subcommand provides $path to user scripts (ie it is part of the API), so we can't simply rename it to $sm_path. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jens Lehmann <Jens.Lehmann@web.de> Tested-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-17 20:00:58 +02:00
if test -z "$sm_path"; then
sm_path=$(printf '%s\n' "$repo" |
sed -e 's|/$||' -e 's|:*/*\.git$||' -e 's|.*[/:]||g')
fi
if test -z "$repo" || test -z "$sm_path"; then
usage
fi
is_absolute_path "$sm_path" || sm_path="$wt_prefix$sm_path"
git-submodule - make "submodule add" more strict, and document it This change makes "submodule add" much more strict in the arguments it takes, and is intended to address confusion as recently noted on the git-list. With this change, the required syntax is: $ git submodule add URL path Specifically, this eliminates the form $ git submodule add URL which was confused by more than one person as $ git submodule add path With this patch, the URL locating the submodule's origin repository can be either an absolute URL, or (if it begins with ./ or ../) can express the submodule's repository location relative to the superproject's origin. This patch also eliminates a third form of URL, which was relative to the superproject's top-level directory (not its repository). Any URL that was neither absolute nor matched ./*|../* was assumed to point to a subdirectory of the superproject as the location of the submodule's origin repository. This URL form was confusing and does not seem to correspond to an important use-case. Specifically, no-one has identified the need to clone from a repository already in the superproject's tree, but if this is needed it is easily done using an absolute URL: $(pwd)/relative-path. So, no functionality is lost with this patch. (t6008-rev-list-submodule.sh did rely upon this relative URL, fixed by using $(pwd).) Following this change, there are exactly four variants of submodule-add, as both arguments have two flavors: URL can be absolute, or can begin with ./|../ and thus names the submodule's origin relative to the superproject's origin. Note: With this patch, "submodule add" discerns an absolute URL as matching /*|*:*: e.g., URL begins with /, or it contains a :. This works for all valid URLs, an absolute path in POSIX, as well as an absolute path on Windows). path can either already exist as a valid git repo, or will be cloned from the given URL. The first form here eases creation of a new submodule in an existing superproject as the submodule can be added and tested in-tree before pushing to the public repository. However, the more usual form is the second, where the repo is cloned from the given URL. This specifically addresses the issue of $ git submodule add a/b/c attempting to clone from a repository at "a/b/c" to create a new module in "c". This also simplifies description of "relative URL" as there is now exactly *one* form: a URL relative to the parent's origin repo. Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-10 03:05:40 +02:00
# assure repo is absolute or relative to parent
case "$repo" in
./*|../*)
test -z "$wt_prefix" ||
die "$(gettext "Relative path can only be used from the toplevel of the working tree")"
git-submodule - make "submodule add" more strict, and document it This change makes "submodule add" much more strict in the arguments it takes, and is intended to address confusion as recently noted on the git-list. With this change, the required syntax is: $ git submodule add URL path Specifically, this eliminates the form $ git submodule add URL which was confused by more than one person as $ git submodule add path With this patch, the URL locating the submodule's origin repository can be either an absolute URL, or (if it begins with ./ or ../) can express the submodule's repository location relative to the superproject's origin. This patch also eliminates a third form of URL, which was relative to the superproject's top-level directory (not its repository). Any URL that was neither absolute nor matched ./*|../* was assumed to point to a subdirectory of the superproject as the location of the submodule's origin repository. This URL form was confusing and does not seem to correspond to an important use-case. Specifically, no-one has identified the need to clone from a repository already in the superproject's tree, but if this is needed it is easily done using an absolute URL: $(pwd)/relative-path. So, no functionality is lost with this patch. (t6008-rev-list-submodule.sh did rely upon this relative URL, fixed by using $(pwd).) Following this change, there are exactly four variants of submodule-add, as both arguments have two flavors: URL can be absolute, or can begin with ./|../ and thus names the submodule's origin relative to the superproject's origin. Note: With this patch, "submodule add" discerns an absolute URL as matching /*|*:*: e.g., URL begins with /, or it contains a :. This works for all valid URLs, an absolute path in POSIX, as well as an absolute path on Windows). path can either already exist as a valid git repo, or will be cloned from the given URL. The first form here eases creation of a new submodule in an existing superproject as the submodule can be added and tested in-tree before pushing to the public repository. However, the more usual form is the second, where the repo is cloned from the given URL. This specifically addresses the issue of $ git submodule add a/b/c attempting to clone from a repository at "a/b/c" to create a new module in "c". This also simplifies description of "relative URL" as there is now exactly *one* form: a URL relative to the parent's origin repo. Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-10 03:05:40 +02:00
# dereference source url relative to parent's url
realrepo=$(git submodule--helper resolve-relative-url "$repo") || exit
git-submodule - make "submodule add" more strict, and document it This change makes "submodule add" much more strict in the arguments it takes, and is intended to address confusion as recently noted on the git-list. With this change, the required syntax is: $ git submodule add URL path Specifically, this eliminates the form $ git submodule add URL which was confused by more than one person as $ git submodule add path With this patch, the URL locating the submodule's origin repository can be either an absolute URL, or (if it begins with ./ or ../) can express the submodule's repository location relative to the superproject's origin. This patch also eliminates a third form of URL, which was relative to the superproject's top-level directory (not its repository). Any URL that was neither absolute nor matched ./*|../* was assumed to point to a subdirectory of the superproject as the location of the submodule's origin repository. This URL form was confusing and does not seem to correspond to an important use-case. Specifically, no-one has identified the need to clone from a repository already in the superproject's tree, but if this is needed it is easily done using an absolute URL: $(pwd)/relative-path. So, no functionality is lost with this patch. (t6008-rev-list-submodule.sh did rely upon this relative URL, fixed by using $(pwd).) Following this change, there are exactly four variants of submodule-add, as both arguments have two flavors: URL can be absolute, or can begin with ./|../ and thus names the submodule's origin relative to the superproject's origin. Note: With this patch, "submodule add" discerns an absolute URL as matching /*|*:*: e.g., URL begins with /, or it contains a :. This works for all valid URLs, an absolute path in POSIX, as well as an absolute path on Windows). path can either already exist as a valid git repo, or will be cloned from the given URL. The first form here eases creation of a new submodule in an existing superproject as the submodule can be added and tested in-tree before pushing to the public repository. However, the more usual form is the second, where the repo is cloned from the given URL. This specifically addresses the issue of $ git submodule add a/b/c attempting to clone from a repository at "a/b/c" to create a new module in "c". This also simplifies description of "relative URL" as there is now exactly *one* form: a URL relative to the parent's origin repo. Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-10 03:05:40 +02:00
;;
*:*|/*)
# absolute url
realrepo=$repo
;;
*)
die "$(eval_gettext "repo URL: '\$repo' must be absolute or begin with ./|../")"
git-submodule - make "submodule add" more strict, and document it This change makes "submodule add" much more strict in the arguments it takes, and is intended to address confusion as recently noted on the git-list. With this change, the required syntax is: $ git submodule add URL path Specifically, this eliminates the form $ git submodule add URL which was confused by more than one person as $ git submodule add path With this patch, the URL locating the submodule's origin repository can be either an absolute URL, or (if it begins with ./ or ../) can express the submodule's repository location relative to the superproject's origin. This patch also eliminates a third form of URL, which was relative to the superproject's top-level directory (not its repository). Any URL that was neither absolute nor matched ./*|../* was assumed to point to a subdirectory of the superproject as the location of the submodule's origin repository. This URL form was confusing and does not seem to correspond to an important use-case. Specifically, no-one has identified the need to clone from a repository already in the superproject's tree, but if this is needed it is easily done using an absolute URL: $(pwd)/relative-path. So, no functionality is lost with this patch. (t6008-rev-list-submodule.sh did rely upon this relative URL, fixed by using $(pwd).) Following this change, there are exactly four variants of submodule-add, as both arguments have two flavors: URL can be absolute, or can begin with ./|../ and thus names the submodule's origin relative to the superproject's origin. Note: With this patch, "submodule add" discerns an absolute URL as matching /*|*:*: e.g., URL begins with /, or it contains a :. This works for all valid URLs, an absolute path in POSIX, as well as an absolute path on Windows). path can either already exist as a valid git repo, or will be cloned from the given URL. The first form here eases creation of a new submodule in an existing superproject as the submodule can be added and tested in-tree before pushing to the public repository. However, the more usual form is the second, where the repo is cloned from the given URL. This specifically addresses the issue of $ git submodule add a/b/c attempting to clone from a repository at "a/b/c" to create a new module in "c". This also simplifies description of "relative URL" as there is now exactly *one* form: a URL relative to the parent's origin repo. Signed-off-by: Mark Levedahl <mlevedahl@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-10 03:05:40 +02:00
;;
esac
# normalize path:
# multiple //; leading ./; /./; /../; trailing /
git-submodule.sh: Don't use $path variable in eval_gettext string The eval_gettext (and eval_gettextln) i18n shell functions call git-sh-i18n--envsubst to process the variable references in the string parameter. Unfortunately, environment variables are case insensitive on windows, which leads to failure on cygwin when eval_gettext exports $path. Commit df599e9 (Windows: teach getenv to do a case-sensitive search, 06-06-2011) attempts to solve this problem on MinGW by overriding the system getenv() function to allow git-sh-i18n--envsubst to read $path rather than $PATH from the environment. However, this commit does not address cygwin at all and, furthermore, does not fix all problems on MinGW. In particular, when executing test #38 in t7400-submodule-basic.sh, an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog pops up saying that the application "failed to start because libiconv2.dll was not found." After studying the voluminous trace output from the process monitor, it is clear that the system is attempting to use $path, rather than $PATH, to search for the DLL file. (Note that, after dismissing the dialog, the test passes anyway!) As an alternative, we finesse the problem by renaming the $path variable to $sm_path (submodule path). This fixes the problem on MinGW along with all test failures on cygwin (t7400.{7,32,34}, t7406.3 and t7407.{2,6}). We note that the foreach subcommand provides $path to user scripts (ie it is part of the API), so we can't simply rename it to $sm_path. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jens Lehmann <Jens.Lehmann@web.de> Tested-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-17 20:00:58 +02:00
sm_path=$(printf '%s/\n' "$sm_path" |
sed -e '
s|//*|/|g
s|^\(\./\)*||
s|/\(\./\)*|/|g
:start
s|\([^/]*\)/\.\./||
tstart
s|/*$||
')
if test -z "$force"
then
git ls-files --error-unmatch "$sm_path" > /dev/null 2>&1 &&
die "$(eval_gettext "'\$sm_path' already exists in the index")"
else
git ls-files -s "$sm_path" | sane_grep -v "^160000" > /dev/null 2>&1 &&
die "$(eval_gettext "'\$sm_path' already exists in the index and is not a submodule")"
fi
if test -d "$sm_path" &&
test -z $(git -C "$sm_path" rev-parse --show-cdup 2>/dev/null)
then
git -C "$sm_path" rev-parse --verify -q HEAD >/dev/null ||
die "$(eval_gettext "'\$sm_path' does not have a commit checked out")"
fi
submodule add: show 'add --dry-run' stderr when aborting Unless --force is specified, 'submodule add' checks if the destination path is ignored by calling 'git add --dry-run --ignore-missing', and, if that call fails, aborts with a custom "path is ignored" message (a slight variant of what 'git add' shows). Aborting early rather than letting the downstream 'git add' call fail is done so that the command exits before cloning into the destination path. However, in rare cases where the dry-run call fails for a reason other than the path being ignored---for example, due to a preexisting index.lock file---displaying the "ignored path" error message hides the real source of the failure. Instead of displaying the tailored "ignored path" message, let's report the standard error from the dry run to give the caller more accurate information about failures that are not due to an ignored path. For the ignored path case, this leads to the following change in the error message: The following [-path is-]{+paths are+} ignored by one of your .gitignore files: <destination path> Use -f if you really want to add [-it.-]{+them.+} The new phrasing is a bit awkward, because 'submodule add' is only dealing with one destination path. Alternatively, we could continue to use the tailored message when the exit code is 1 (the expected status for a failure due to an ignored path) and relay the standard error for all other non-zero exits. That, however, risks hiding the message of unrelated failures that share an exit code of 1, so it doesn't seem worth doing just to avoid a clunkier, but still clear, error message. Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-08 01:31:21 +01:00
if test -z "$force"
then
submodule add: show 'add --dry-run' stderr when aborting Unless --force is specified, 'submodule add' checks if the destination path is ignored by calling 'git add --dry-run --ignore-missing', and, if that call fails, aborts with a custom "path is ignored" message (a slight variant of what 'git add' shows). Aborting early rather than letting the downstream 'git add' call fail is done so that the command exits before cloning into the destination path. However, in rare cases where the dry-run call fails for a reason other than the path being ignored---for example, due to a preexisting index.lock file---displaying the "ignored path" error message hides the real source of the failure. Instead of displaying the tailored "ignored path" message, let's report the standard error from the dry run to give the caller more accurate information about failures that are not due to an ignored path. For the ignored path case, this leads to the following change in the error message: The following [-path is-]{+paths are+} ignored by one of your .gitignore files: <destination path> Use -f if you really want to add [-it.-]{+them.+} The new phrasing is a bit awkward, because 'submodule add' is only dealing with one destination path. Alternatively, we could continue to use the tailored message when the exit code is 1 (the expected status for a failure due to an ignored path) and relay the standard error for all other non-zero exits. That, however, risks hiding the message of unrelated failures that share an exit code of 1, so it doesn't seem worth doing just to avoid a clunkier, but still clear, error message. Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-08 01:31:21 +01:00
dryerr=$(git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" 2>&1 >/dev/null)
res=$?
if test $res -ne 0
then
echo >&2 "$dryerr"
exit $res
fi
fi
if test -n "$custom_name"
then
sm_name="$custom_name"
else
sm_name="$sm_path"
fi
submodule-config: verify submodule names as paths Submodule "names" come from the untrusted .gitmodules file, but we blindly append them to $GIT_DIR/modules to create our on-disk repo paths. This means you can do bad things by putting "../" into the name (among other things). Let's sanity-check these names to avoid building a path that can be exploited. There are two main decisions: 1. What should the allowed syntax be? It's tempting to reuse verify_path(), since submodule names typically come from in-repo paths. But there are two reasons not to: a. It's technically more strict than what we need, as we really care only about breaking out of the $GIT_DIR/modules/ hierarchy. E.g., having a submodule named "foo/.git" isn't actually dangerous, and it's possible that somebody has manually given such a funny name. b. Since we'll eventually use this checking logic in fsck to prevent downstream repositories, it should be consistent across platforms. Because verify_path() relies on is_dir_sep(), it wouldn't block "foo\..\bar" on a non-Windows machine. 2. Where should we enforce it? These days most of the .gitmodules reads go through submodule-config.c, so I've put it there in the reading step. That should cover all of the C code. We also construct the name for "git submodule add" inside the git-submodule.sh script. This is probably not a big deal for security since the name is coming from the user anyway, but it would be polite to remind them if the name they pick is invalid (and we need to expose the name-checker to the shell anyway for our test scripts). This patch issues a warning when reading .gitmodules and just ignores the related config entry completely. This will generally end up producing a sensible error, as it works the same as a .gitmodules file which is missing a submodule entry (so "submodule update" will barf, but "git clone --recurse-submodules" will print an error but not abort the clone. There is one minor oddity, which is that we print the warning once per malformed config key (since that's how the config subsystem gives us the entries). So in the new test, for example, the user would see three warnings. That's OK, since the intent is that this case should never come up outside of malicious repositories (and then it might even benefit the user to see the message multiple times). Credit for finding this vulnerability and the proof of concept from which the test script was adapted goes to Etienne Stalmans. Signed-off-by: Jeff King <peff@peff.net>
2018-04-30 09:25:25 +02:00
if ! git submodule--helper check-name "$sm_name"
then
die "$(eval_gettext "'$sm_name' is not a valid submodule name")"
fi
# perhaps the path exists and is already a git repo, else clone it
git-submodule.sh: Don't use $path variable in eval_gettext string The eval_gettext (and eval_gettextln) i18n shell functions call git-sh-i18n--envsubst to process the variable references in the string parameter. Unfortunately, environment variables are case insensitive on windows, which leads to failure on cygwin when eval_gettext exports $path. Commit df599e9 (Windows: teach getenv to do a case-sensitive search, 06-06-2011) attempts to solve this problem on MinGW by overriding the system getenv() function to allow git-sh-i18n--envsubst to read $path rather than $PATH from the environment. However, this commit does not address cygwin at all and, furthermore, does not fix all problems on MinGW. In particular, when executing test #38 in t7400-submodule-basic.sh, an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog pops up saying that the application "failed to start because libiconv2.dll was not found." After studying the voluminous trace output from the process monitor, it is clear that the system is attempting to use $path, rather than $PATH, to search for the DLL file. (Note that, after dismissing the dialog, the test passes anyway!) As an alternative, we finesse the problem by renaming the $path variable to $sm_path (submodule path). This fixes the problem on MinGW along with all test failures on cygwin (t7400.{7,32,34}, t7406.3 and t7407.{2,6}). We note that the foreach subcommand provides $path to user scripts (ie it is part of the API), so we can't simply rename it to $sm_path. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jens Lehmann <Jens.Lehmann@web.de> Tested-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-17 20:00:58 +02:00
if test -e "$sm_path"
then
if test -d "$sm_path"/.git || test -f "$sm_path"/.git
then
git-submodule.sh: Don't use $path variable in eval_gettext string The eval_gettext (and eval_gettextln) i18n shell functions call git-sh-i18n--envsubst to process the variable references in the string parameter. Unfortunately, environment variables are case insensitive on windows, which leads to failure on cygwin when eval_gettext exports $path. Commit df599e9 (Windows: teach getenv to do a case-sensitive search, 06-06-2011) attempts to solve this problem on MinGW by overriding the system getenv() function to allow git-sh-i18n--envsubst to read $path rather than $PATH from the environment. However, this commit does not address cygwin at all and, furthermore, does not fix all problems on MinGW. In particular, when executing test #38 in t7400-submodule-basic.sh, an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog pops up saying that the application "failed to start because libiconv2.dll was not found." After studying the voluminous trace output from the process monitor, it is clear that the system is attempting to use $path, rather than $PATH, to search for the DLL file. (Note that, after dismissing the dialog, the test passes anyway!) As an alternative, we finesse the problem by renaming the $path variable to $sm_path (submodule path). This fixes the problem on MinGW along with all test failures on cygwin (t7400.{7,32,34}, t7406.3 and t7407.{2,6}). We note that the foreach subcommand provides $path to user scripts (ie it is part of the API), so we can't simply rename it to $sm_path. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jens Lehmann <Jens.Lehmann@web.de> Tested-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-17 20:00:58 +02:00
eval_gettextln "Adding existing repo at '\$sm_path' to the index"
else
git-submodule.sh: Don't use $path variable in eval_gettext string The eval_gettext (and eval_gettextln) i18n shell functions call git-sh-i18n--envsubst to process the variable references in the string parameter. Unfortunately, environment variables are case insensitive on windows, which leads to failure on cygwin when eval_gettext exports $path. Commit df599e9 (Windows: teach getenv to do a case-sensitive search, 06-06-2011) attempts to solve this problem on MinGW by overriding the system getenv() function to allow git-sh-i18n--envsubst to read $path rather than $PATH from the environment. However, this commit does not address cygwin at all and, furthermore, does not fix all problems on MinGW. In particular, when executing test #38 in t7400-submodule-basic.sh, an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog pops up saying that the application "failed to start because libiconv2.dll was not found." After studying the voluminous trace output from the process monitor, it is clear that the system is attempting to use $path, rather than $PATH, to search for the DLL file. (Note that, after dismissing the dialog, the test passes anyway!) As an alternative, we finesse the problem by renaming the $path variable to $sm_path (submodule path). This fixes the problem on MinGW along with all test failures on cygwin (t7400.{7,32,34}, t7406.3 and t7407.{2,6}). We note that the foreach subcommand provides $path to user scripts (ie it is part of the API), so we can't simply rename it to $sm_path. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jens Lehmann <Jens.Lehmann@web.de> Tested-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-17 20:00:58 +02:00
die "$(eval_gettext "'\$sm_path' already exists and is not a valid git repo")"
fi
else
if test -d ".git/modules/$sm_name"
then
if test -z "$force"
then
eval_gettextln >&2 "A git directory for '\$sm_name' is found locally with remote(s):"
GIT_DIR=".git/modules/$sm_name" GIT_WORK_TREE=. git remote -v | grep '(fetch)' | sed -e s,^," ", -e s,' (fetch)',, >&2
die "$(eval_gettextln "\
If you want to reuse this local git directory instead of cloning again from
\$realrepo
use the '--force' option. If the local git directory is not the correct repo
or you are unsure what this means choose another name with the '--name' option.")"
else
eval_gettextln "Reactivating local git directory for submodule '\$sm_name'."
fi
fi
git submodule--helper clone ${GIT_QUIET:+--quiet} ${progress:+"--progress"} --prefix "$wt_prefix" --path "$sm_path" --name "$sm_name" --url "$realrepo" ${reference:+"$reference"} ${dissociate:+"--dissociate"} ${depth:+"$depth"} || exit
(
sanitize_submodule_env
cd "$sm_path" &&
# ash fails to wordsplit ${branch:+-b "$branch"...}
case "$branch" in
'') git checkout -f -q ;;
?*) git checkout -f -q -B "$branch" "origin/$branch" ;;
esac
) || die "$(eval_gettext "Unable to checkout submodule '\$sm_path'")"
fi
git config submodule."$sm_name".url "$realrepo"
add: warn when adding an embedded repository It's an easy mistake to add a repository inside another repository, like: git clone $url git add . The resulting entry is a gitlink, but there's no matching .gitmodules entry. Trying to use "submodule init" (or clone with --recursive) doesn't do anything useful. Prior to v2.13, such an entry caused git-submodule to barf entirely. In v2.13, the entry is considered "inactive" and quietly ignored. Either way, no clone of your repository can do anything useful with the gitlink without the user manually adding the submodule config. In most cases, the user probably meant to either add a real submodule, or they forgot to put the embedded repository in their .gitignore file. Let's issue a warning when we see this case. There are a few things to note: - the warning will go in the git-add porcelain; anybody wanting to do low-level manipulation of the index is welcome to create whatever funny states they want. - we detect the case by looking for a newly added gitlink; updates via "git add submodule" are perfectly reasonable, and this avoids us having to investigate .gitmodules entirely - there's a command-line option to suppress the warning. This is needed for git-submodule itself (which adds the entry before adding any submodule config), but also provides a mechanism for other scripts doing submodule-like things. We could make this a hard error instead of a warning. However, we do add lots of sub-repos in our test suite. It's not _wrong_ to do so. It just creates a state where users may be surprised. Pointing them in the right direction with a gentle hint is probably the best option. There is a config knob that can disable the (long) hint. But I intentionally omitted a config knob to disable the warning entirely. Whether the warning is sensible or not is generally about context, not about the user's preferences. If there's a tool or workflow that adds gitlinks without matching .gitmodules, it should probably be taught about the new command-line option, rather than blanket-disabling the warning. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-14 12:58:22 +02:00
git add --no-warn-embedded-repo $force "$sm_path" ||
git-submodule.sh: Don't use $path variable in eval_gettext string The eval_gettext (and eval_gettextln) i18n shell functions call git-sh-i18n--envsubst to process the variable references in the string parameter. Unfortunately, environment variables are case insensitive on windows, which leads to failure on cygwin when eval_gettext exports $path. Commit df599e9 (Windows: teach getenv to do a case-sensitive search, 06-06-2011) attempts to solve this problem on MinGW by overriding the system getenv() function to allow git-sh-i18n--envsubst to read $path rather than $PATH from the environment. However, this commit does not address cygwin at all and, furthermore, does not fix all problems on MinGW. In particular, when executing test #38 in t7400-submodule-basic.sh, an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog pops up saying that the application "failed to start because libiconv2.dll was not found." After studying the voluminous trace output from the process monitor, it is clear that the system is attempting to use $path, rather than $PATH, to search for the DLL file. (Note that, after dismissing the dialog, the test passes anyway!) As an alternative, we finesse the problem by renaming the $path variable to $sm_path (submodule path). This fixes the problem on MinGW along with all test failures on cygwin (t7400.{7,32,34}, t7406.3 and t7407.{2,6}). We note that the foreach subcommand provides $path to user scripts (ie it is part of the API), so we can't simply rename it to $sm_path. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jens Lehmann <Jens.Lehmann@web.de> Tested-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-17 20:00:58 +02:00
die "$(eval_gettext "Failed to add submodule '\$sm_path'")"
git submodule--helper config submodule."$sm_name".path "$sm_path" &&
git submodule--helper config submodule."$sm_name".url "$repo" &&
if test -n "$branch"
then
git submodule--helper config submodule."$sm_name".branch "$branch"
fi &&
git submodule: add submodules with git add -f <path> Change `git submodule add' to add the new submodule <path> with `git add --force'. I keep my /etc in .git with a .gitignore that contains just "*". I.e. `git status' will ignore everything that isn't in the tree already. When I do: git submodule add <url> hlagh git-submodule will get as far as checking out the remote repository into hlagh, but it'll die right afterwards when it fails to add the new path: The following paths are ignored by one of your .gitignore files: hlagh Use -f if you really want to add them. fatal: no files added Failed to add submodule 'hlagh' Currently there's no way to add a submodule in this situation other than to remove the ignored path from the .gitignore while I'm at it. That's silly, when you run `git submodule add' you're explicitly saying that you want to add something *new* to the repository. Instead it should just add the path with `git add --force'. Initially I implemented this by adding new -f and --force options to `git submodule add'. But if the --force option isn't supplied it'll get as far as cloning `hlagh', but won't add it. So the first thing the user has to do is to remove `hlagh' and then try again with the --force option. That sucks, it should just add the path to begin with. I can't think of any usecase where you've gone through the trouble of typing out `git submodule add ..', but wish to be overriden by a `gitignore'. The submodule semantics should be more like `git init', not `git add'. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-05 19:33:03 +02:00
git add --force .gitmodules ||
git-submodule.sh: Don't use $path variable in eval_gettext string The eval_gettext (and eval_gettextln) i18n shell functions call git-sh-i18n--envsubst to process the variable references in the string parameter. Unfortunately, environment variables are case insensitive on windows, which leads to failure on cygwin when eval_gettext exports $path. Commit df599e9 (Windows: teach getenv to do a case-sensitive search, 06-06-2011) attempts to solve this problem on MinGW by overriding the system getenv() function to allow git-sh-i18n--envsubst to read $path rather than $PATH from the environment. However, this commit does not address cygwin at all and, furthermore, does not fix all problems on MinGW. In particular, when executing test #38 in t7400-submodule-basic.sh, an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog pops up saying that the application "failed to start because libiconv2.dll was not found." After studying the voluminous trace output from the process monitor, it is clear that the system is attempting to use $path, rather than $PATH, to search for the DLL file. (Note that, after dismissing the dialog, the test passes anyway!) As an alternative, we finesse the problem by renaming the $path variable to $sm_path (submodule path). This fixes the problem on MinGW along with all test failures on cygwin (t7400.{7,32,34}, t7406.3 and t7407.{2,6}). We note that the foreach subcommand provides $path to user scripts (ie it is part of the API), so we can't simply rename it to $sm_path. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Jens Lehmann <Jens.Lehmann@web.de> Tested-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-17 20:00:58 +02:00
die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
# NEEDSWORK: In a multi-working-tree world, this needs to be
# set in the per-worktree config.
if git config --get submodule.active >/dev/null
then
# If the submodule being adding isn't already covered by the
# current configured pathspec, set the submodule's active flag
if ! git submodule--helper is-active "$sm_path"
then
git config submodule."$sm_name".active "true"
fi
else
git config submodule."$sm_name".active "true"
fi
}
#
# Execute an arbitrary command sequence in each checked out
# submodule
#
# $@ = command to execute
#
cmd_foreach()
{
# parse $args after "submodule ... foreach".
while test $# -ne 0
do
case "$1" in
-q|--quiet)
GIT_QUIET=1
;;
--recursive)
recursive=1
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
submodule foreach: fix "<command> --quiet" not being respected Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12 12:08:19 +02:00
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} -- "$@"
}
#
# Register submodules in .git/config
#
# $@ = requested paths (default to all)
#
cmd_init()
{
# parse $args after "submodule ... init".
while test $# -ne 0
do
case "$1" in
-q|--quiet)
GIT_QUIET=1
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
submodule foreach: fix "<command> --quiet" not being respected Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12 12:08:19 +02:00
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper init ${GIT_QUIET:+--quiet} -- "$@"
}
#
# Unregister submodules from .git/config and remove their work tree
#
cmd_deinit()
{
# parse $args after "submodule ... deinit".
deinit_all=
while test $# -ne 0
do
case "$1" in
-f|--force)
force=$1
;;
-q|--quiet)
GIT_QUIET=1
;;
--all)
deinit_all=t
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
submodule foreach: fix "<command> --quiet" not being respected Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12 12:08:19 +02:00
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${prefix:+--prefix "$prefix"} ${force:+--force} ${deinit_all:+--all} -- "$@"
}
is_tip_reachable () (
sanitize_submodule_env &&
cd "$1" &&
rev=$(git rev-list -n 1 "$2" --not --all 2>/dev/null) &&
test -z "$rev"
)
submodule: fix fetch_in_submodule logic Commit 1c1518071c (submodule: use "fetch" logic instead of custom remote discovery, 2020-11-14) rewrote the logic in fetch_in_submodule to do: elif test "$2" -ne "" But this is nonsense in shell: -ne is for numeric comparisons. This should be "=" or more idiomatically: elif test -n "$2" But once we fix that, many tests start failing. Because that commit introduced another problem. The caller that passes 3 arguments looks like this: fetch_in_submodule "$sm_path" $depth "$sha1" Note the unquoted $depth parameter. When it isn't set, the function will see only 2 arguments, and the function has no idea if what it sees in $2 is an option to go on the command line, or a refspec to pass on stdin. In the old code before that commit: fetch_in_submodule () ( sanitize_submodule_env && cd "$1" && - case "$2" in - '') - git fetch ;; - *) - shift - git fetch $(get_default_remote) "$@" ;; - esac we treated those the same, so it didn't matter. But in the new logic (with my fix above): + if test $# -eq 3 + then + echo "$3" | git fetch --stdin "$2" + elif test -n "$n" + then + git fetch "$2" + else + git fetch + fi we use the number of parameters to distinguish the two. Let's insist that the caller pass an empty string for positional parameter two if they want to have a third parameter after it. But that still leaves one problem. In the --stdin block, we unconditionally pass "$2" to git-fetch, even if it's the empty string. Rather than add another conditional, we can use :+ parameter expansion to include it only if it's non-empty. In fact, we can do the same for the elif, too, simplifying it further. Technically this is overkill, since we know the --depth parameter will not have whitespace (and indeed, most callers do not bother quoting it), but it doesn't hurt for the function to be careful. It's somewhat amazing that no tests were failing. I think what happened is that: - the 3-arg form rarely triggered; any call with a non-empty $depth and a $sha1 would work, but one with an empty $depth would only have 2 arguments - because of the wrong arguments to "test", the shell would complain and exit non-zero. So we never ran the middle conditional at all - that left every call running "git fetch" with no arguments. A well-written test could have detected the distinction here, but in practice omitting --depth just means fetching more commits, and fetching everything (rather than a single sha1) works as long as the commit in question is reachable Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-24 10:06:05 +01:00
# usage: fetch_in_submodule <module_path> [<depth>] [<sha1>]
# Because arguments are positional, use an empty string to omit <depth>
# but include <sha1>.
fetch_in_submodule () (
sanitize_submodule_env &&
cd "$1" &&
if test $# -eq 3
then
submodule: fix fetch_in_submodule logic Commit 1c1518071c (submodule: use "fetch" logic instead of custom remote discovery, 2020-11-14) rewrote the logic in fetch_in_submodule to do: elif test "$2" -ne "" But this is nonsense in shell: -ne is for numeric comparisons. This should be "=" or more idiomatically: elif test -n "$2" But once we fix that, many tests start failing. Because that commit introduced another problem. The caller that passes 3 arguments looks like this: fetch_in_submodule "$sm_path" $depth "$sha1" Note the unquoted $depth parameter. When it isn't set, the function will see only 2 arguments, and the function has no idea if what it sees in $2 is an option to go on the command line, or a refspec to pass on stdin. In the old code before that commit: fetch_in_submodule () ( sanitize_submodule_env && cd "$1" && - case "$2" in - '') - git fetch ;; - *) - shift - git fetch $(get_default_remote) "$@" ;; - esac we treated those the same, so it didn't matter. But in the new logic (with my fix above): + if test $# -eq 3 + then + echo "$3" | git fetch --stdin "$2" + elif test -n "$n" + then + git fetch "$2" + else + git fetch + fi we use the number of parameters to distinguish the two. Let's insist that the caller pass an empty string for positional parameter two if they want to have a third parameter after it. But that still leaves one problem. In the --stdin block, we unconditionally pass "$2" to git-fetch, even if it's the empty string. Rather than add another conditional, we can use :+ parameter expansion to include it only if it's non-empty. In fact, we can do the same for the elif, too, simplifying it further. Technically this is overkill, since we know the --depth parameter will not have whitespace (and indeed, most callers do not bother quoting it), but it doesn't hurt for the function to be careful. It's somewhat amazing that no tests were failing. I think what happened is that: - the 3-arg form rarely triggered; any call with a non-empty $depth and a $sha1 would work, but one with an empty $depth would only have 2 arguments - because of the wrong arguments to "test", the shell would complain and exit non-zero. So we never ran the middle conditional at all - that left every call running "git fetch" with no arguments. A well-written test could have detected the distinction here, but in practice omitting --depth just means fetching more commits, and fetching everything (rather than a single sha1) works as long as the commit in question is reachable Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-24 10:06:05 +01:00
echo "$3" | git fetch --stdin ${2:+"$2"}
else
submodule: fix fetch_in_submodule logic Commit 1c1518071c (submodule: use "fetch" logic instead of custom remote discovery, 2020-11-14) rewrote the logic in fetch_in_submodule to do: elif test "$2" -ne "" But this is nonsense in shell: -ne is for numeric comparisons. This should be "=" or more idiomatically: elif test -n "$2" But once we fix that, many tests start failing. Because that commit introduced another problem. The caller that passes 3 arguments looks like this: fetch_in_submodule "$sm_path" $depth "$sha1" Note the unquoted $depth parameter. When it isn't set, the function will see only 2 arguments, and the function has no idea if what it sees in $2 is an option to go on the command line, or a refspec to pass on stdin. In the old code before that commit: fetch_in_submodule () ( sanitize_submodule_env && cd "$1" && - case "$2" in - '') - git fetch ;; - *) - shift - git fetch $(get_default_remote) "$@" ;; - esac we treated those the same, so it didn't matter. But in the new logic (with my fix above): + if test $# -eq 3 + then + echo "$3" | git fetch --stdin "$2" + elif test -n "$n" + then + git fetch "$2" + else + git fetch + fi we use the number of parameters to distinguish the two. Let's insist that the caller pass an empty string for positional parameter two if they want to have a third parameter after it. But that still leaves one problem. In the --stdin block, we unconditionally pass "$2" to git-fetch, even if it's the empty string. Rather than add another conditional, we can use :+ parameter expansion to include it only if it's non-empty. In fact, we can do the same for the elif, too, simplifying it further. Technically this is overkill, since we know the --depth parameter will not have whitespace (and indeed, most callers do not bother quoting it), but it doesn't hurt for the function to be careful. It's somewhat amazing that no tests were failing. I think what happened is that: - the 3-arg form rarely triggered; any call with a non-empty $depth and a $sha1 would work, but one with an empty $depth would only have 2 arguments - because of the wrong arguments to "test", the shell would complain and exit non-zero. So we never ran the middle conditional at all - that left every call running "git fetch" with no arguments. A well-written test could have detected the distinction here, but in practice omitting --depth just means fetching more commits, and fetching everything (rather than a single sha1) works as long as the commit in question is reachable Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-24 10:06:05 +01:00
git fetch ${2:+"$2"}
fi
)
#
# Update each submodule path to correct revision, using clone and checkout as needed
#
# $@ = requested paths (default to all)
#
cmd_update()
{
# parse $args after "submodule ... update".
while test $# -ne 0
do
case "$1" in
-q|--quiet)
GIT_QUIET=1
;;
-v)
unset GIT_QUIET
;;
clone: pass --progress decision to recursive submodules When cloning with "--recursive", we'd generally expect submodules to show progress reports if the main clone did, too. In older versions of git, this mostly worked out of the box. Since we show progress by default when stderr is a tty, and since the child clones inherit the parent stderr, then both processes would come to the same decision by default. If the parent clone was asked for "--quiet", we passed down "--quiet" to the child. However, if stderr was not a tty and the user specified "--progress", we did not propagate this to the child. That's a minor bug, but things got much worse when we switched recently to submodule--helper's update_clone command. With that change, the stderr of the child clones are always connected to a pipe, and we never output progress at all. This patch teaches git-submodule and git-submodule--helper how to pass down an explicit "--progress" flag when cloning. The clone command then decides to propagate that flag based on the cloning decision made earlier (which takes into account isatty(2) of the parent process, existing --progress or --quiet flags, etc). Since the child processes always run without a tty on stderr, we don't have to worry about passing an explicit "--no-progress"; it's the default for them. This fixes the recent loss of progress during recursive clones. And as a bonus, it makes: git clone --recursive --progress ... 2>&1 | cat work by triggering progress explicitly in the children. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
--progress)
progress=1
clone: pass --progress decision to recursive submodules When cloning with "--recursive", we'd generally expect submodules to show progress reports if the main clone did, too. In older versions of git, this mostly worked out of the box. Since we show progress by default when stderr is a tty, and since the child clones inherit the parent stderr, then both processes would come to the same decision by default. If the parent clone was asked for "--quiet", we passed down "--quiet" to the child. However, if stderr was not a tty and the user specified "--progress", we did not propagate this to the child. That's a minor bug, but things got much worse when we switched recently to submodule--helper's update_clone command. With that change, the stderr of the child clones are always connected to a pipe, and we never output progress at all. This patch teaches git-submodule and git-submodule--helper how to pass down an explicit "--progress" flag when cloning. The clone command then decides to propagate that flag based on the cloning decision made earlier (which takes into account isatty(2) of the parent process, existing --progress or --quiet flags, etc). Since the child processes always run without a tty on stderr, we don't have to worry about passing an explicit "--no-progress"; it's the default for them. This fixes the recent loss of progress during recursive clones. And as a bonus, it makes: git clone --recursive --progress ... 2>&1 | cat work by triggering progress explicitly in the children. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 07:24:46 +02:00
;;
-i|--init)
init=1
;;
clone --recurse-submodules: prevent name squatting on Windows In addition to preventing `.git` from being tracked by Git, on Windows we also have to prevent `git~1` from being tracked, as the default NTFS short name (also known as the "8.3 filename") for the file name `.git` is `git~1`, otherwise it would be possible for malicious repositories to write directly into the `.git/` directory, e.g. a `post-checkout` hook that would then be executed _during_ a recursive clone. When we implemented appropriate protections in 2b4c6efc821 (read-cache: optionally disallow NTFS .git variants, 2014-12-16), we had analyzed carefully that the `.git` directory or file would be guaranteed to be the first directory entry to be written. Otherwise it would be possible e.g. for a file named `..git` to be assigned the short name `git~1` and subsequently, the short name generated for `.git` would be `git~2`. Or `git~3`. Or even `~9999999` (for a detailed explanation of the lengths we have to go to protect `.gitmodules`, see the commit message of e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)). However, by exploiting two issues (that will be addressed in a related patch series close by), it is currently possible to clone a submodule into a non-empty directory: - On Windows, file names cannot end in a space or a period (for historical reasons: the period separating the base name from the file extension was not actually written to disk, and the base name/file extension was space-padded to the full 8/3 characters, respectively). Helpfully, when creating a directory under the name, say, `sub.`, that trailing period is trimmed automatically and the actual name on disk is `sub`. This means that while Git thinks that the submodule names `sub` and `sub.` are different, they both access `.git/modules/sub/`. - While the backslash character is a valid file name character on Linux, it is not so on Windows. As Git tries to be cross-platform, it therefore allows backslash characters in the file names stored in tree objects. Which means that it is totally possible that a submodule `c` sits next to a file `c\..git`, and on Windows, during recursive clone a file called `..git` will be written into `c/`, of course _before_ the submodule is cloned. Note that the actual exploit is not quite as simple as having a submodule `c` next to a file `c\..git`, as we have to make sure that the directory `.git/modules/b` already exists when the submodule is checked out, otherwise a different code path is taken in `module_clone()` that does _not_ allow a non-empty submodule directory to exist already. Even if we will address both issues nearby (the next commit will disallow backslash characters in tree entries' file names on Windows, and another patch will disallow creating directories/files with trailing spaces or periods), it is a wise idea to defend in depth against this sort of attack vector: when submodules are cloned recursively, we now _require_ the directory to be empty, addressing CVE-2019-1349. Note: the code path we patch is shared with the code path of `git submodule update --init`, which must not expect, in general, that the directory is empty. Hence we have to introduce the new option `--force-init` and hand it all the way down from `git submodule` to the actual `git submodule--helper` process that performs the initial clone. Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
--require-init)
init=1
require_init=1
;;
submodule update: add --remote for submodule's upstream changes The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule.<name>.branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule.<name>.branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule.<name>.branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules/<name>/config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule.<name>.branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King <wking@tremily.us> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
--remote)
remote=1
;;
-N|--no-fetch)
nofetch=1
;;
-f|--force)
force=$1
;;
-r|--rebase)
update="rebase"
;;
--reference)
case "$2" in '') usage ;; esac
reference="--reference=$2"
shift
;;
--reference=*)
reference="$1"
;;
--dissociate)
dissociate=1
;;
-m|--merge)
update="merge"
;;
--recursive)
recursive=1
;;
--checkout)
update="checkout"
;;
--recommend-shallow)
recommend_shallow="--recommend-shallow"
;;
--no-recommend-shallow)
recommend_shallow="--no-recommend-shallow"
;;
--depth)
case "$2" in '') usage ;; esac
depth="--depth=$2"
shift
;;
--depth=*)
depth=$1
;;
-j|--jobs)
case "$2" in '') usage ;; esac
jobs="--jobs=$2"
shift
;;
--jobs=*)
jobs=$1
;;
--single-branch)
single_branch="--single-branch"
;;
--no-single-branch)
single_branch="--no-single-branch"
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
if test -n "$init"
then
cmd_init "--" "$@" || return
fi
{
git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
${progress:+"--progress"} \
${wt_prefix:+--prefix "$wt_prefix"} \
${prefix:+--recursive-prefix "$prefix"} \
${update:+--update "$update"} \
${reference:+"$reference"} \
${dissociate:+"--dissociate"} \
${depth:+--depth "$depth"} \
clone --recurse-submodules: prevent name squatting on Windows In addition to preventing `.git` from being tracked by Git, on Windows we also have to prevent `git~1` from being tracked, as the default NTFS short name (also known as the "8.3 filename") for the file name `.git` is `git~1`, otherwise it would be possible for malicious repositories to write directly into the `.git/` directory, e.g. a `post-checkout` hook that would then be executed _during_ a recursive clone. When we implemented appropriate protections in 2b4c6efc821 (read-cache: optionally disallow NTFS .git variants, 2014-12-16), we had analyzed carefully that the `.git` directory or file would be guaranteed to be the first directory entry to be written. Otherwise it would be possible e.g. for a file named `..git` to be assigned the short name `git~1` and subsequently, the short name generated for `.git` would be `git~2`. Or `git~3`. Or even `~9999999` (for a detailed explanation of the lengths we have to go to protect `.gitmodules`, see the commit message of e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)). However, by exploiting two issues (that will be addressed in a related patch series close by), it is currently possible to clone a submodule into a non-empty directory: - On Windows, file names cannot end in a space or a period (for historical reasons: the period separating the base name from the file extension was not actually written to disk, and the base name/file extension was space-padded to the full 8/3 characters, respectively). Helpfully, when creating a directory under the name, say, `sub.`, that trailing period is trimmed automatically and the actual name on disk is `sub`. This means that while Git thinks that the submodule names `sub` and `sub.` are different, they both access `.git/modules/sub/`. - While the backslash character is a valid file name character on Linux, it is not so on Windows. As Git tries to be cross-platform, it therefore allows backslash characters in the file names stored in tree objects. Which means that it is totally possible that a submodule `c` sits next to a file `c\..git`, and on Windows, during recursive clone a file called `..git` will be written into `c/`, of course _before_ the submodule is cloned. Note that the actual exploit is not quite as simple as having a submodule `c` next to a file `c\..git`, as we have to make sure that the directory `.git/modules/b` already exists when the submodule is checked out, otherwise a different code path is taken in `module_clone()` that does _not_ allow a non-empty submodule directory to exist already. Even if we will address both issues nearby (the next commit will disallow backslash characters in tree entries' file names on Windows, and another patch will disallow creating directories/files with trailing spaces or periods), it is a wise idea to defend in depth against this sort of attack vector: when submodules are cloned recursively, we now _require_ the directory to be empty, addressing CVE-2019-1349. Note: the code path we patch is shared with the code path of `git submodule update --init`, which must not expect, in general, that the directory is empty. Hence we have to introduce the new option `--force-init` and hand it all the way down from `git submodule` to the actual `git submodule--helper` process that performs the initial clone. Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-09-12 14:20:39 +02:00
${require_init:+--require-init} \
$single_branch \
$recommend_shallow \
$jobs \
submodule foreach: fix "<command> --quiet" not being respected Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12 12:08:19 +02:00
-- \
"$@" || echo "#unmatched" $?
} | {
err=
while read -r quickabort sha1 just_cloned sm_path
do
die_if_unmatched "$quickabort" "$sha1"
git submodule--helper ensure-core-worktree "$sm_path" || exit 1
update_module=$(git submodule--helper update-module-mode $just_cloned "$sm_path" $update)
displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
if test $just_cloned -eq 1
then
subsha1=
else
subsha1=$(sanitize_submodule_env; cd "$sm_path" &&
git rev-parse --verify HEAD) ||
die "$(eval_gettext "Unable to find current revision in submodule path '\$displaypath'")"
fi
submodule update: add --remote for submodule's upstream changes The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule.<name>.branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule.<name>.branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule.<name>.branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules/<name>/config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule.<name>.branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King <wking@tremily.us> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
if test -n "$remote"
then
branch=$(git submodule--helper remote-branch "$sm_path")
submodule update: add --remote for submodule's upstream changes The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule.<name>.branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule.<name>.branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule.<name>.branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules/<name>/config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule.<name>.branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King <wking@tremily.us> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
if test -z "$nofetch"
then
# Fetch remote before determining tracking $sha1
fetch_in_submodule "$sm_path" $depth ||
submodule update: add --remote for submodule's upstream changes The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule.<name>.branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule.<name>.branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule.<name>.branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules/<name>/config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule.<name>.branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King <wking@tremily.us> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
fi
remote_name=$(sanitize_submodule_env; cd "$sm_path" && git submodule--helper print-default-remote)
sha1=$(sanitize_submodule_env; cd "$sm_path" &&
submodule update: add --remote for submodule's upstream changes The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule.<name>.branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule.<name>.branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule.<name>.branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules/<name>/config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule.<name>.branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King <wking@tremily.us> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
git rev-parse --verify "${remote_name}/${branch}") ||
die "$(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
submodule update: add --remote for submodule's upstream changes The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule.<name>.branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule.<name>.branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule.<name>.branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules/<name>/config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule.<name>.branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King <wking@tremily.us> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-19 17:03:32 +01:00
fi
if test "$subsha1" != "$sha1" || test -n "$force"
then
subforce=$force
# If we don't already have a -f flag and the submodule has never been checked out
if test -z "$subsha1" && test -z "$force"
then
subforce="-f"
fi
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
is_tip_reachable "$sm_path" "$sha1" ||
fetch_in_submodule "$sm_path" $depth ||
submodule: explain first attempt failure clearly When cloning with --recurse-submodules a superproject with at least one submodule with HEAD pointing to an unborn branch, the clone goes something like this: Cloning into 'test'... <messages about cloning of superproject> Submodule '<name>' (<uri>) registered for path '<submodule path>' Cloning into '<submodule path>'... fatal: Couldn't find remote ref HEAD Unable to fetch in submodule path '<submodule path>' <messages about fetching with SHA-1> From <uri> * branch <hash> -> FETCH_HEAD Submodule path '<submodule path>': checked out '<hash>' In other words, first, a fetch is done with no hash arguments (that is, a fetch of HEAD) resulting in a "Couldn't find remote ref HEAD" error; then, a fetch is done given a hash, which succeeds. The fetch given a hash was added in fb43e31f2b ("submodule: try harder to fetch needed sha1 by direct fetching sha1", 2016-02-24), and the "Unable to fetch..." message was downgraded from a fatal error to a notice in e30d833671 ("git-submodule.sh: try harder to fetch a submodule", 2018-05-16). This commit improves the notice to be clearer that we are retrying the fetch, and that the previous messages (in particular, the fatal errors from fetch) do not necessarily indicate that the whole command fails. In other words: - If the HEAD-fetch succeeds and we then have the commit we want, git-submodule prints no explanation. - If the HEAD-fetch succeeds and we do not have the commit we want, but the hash-fetch succeeds, git-submodule prints no explanation. - If the HEAD-fetch succeeds and we do not have the commit we want, but the hash-fetch fails, git-submodule prints a fatal error. - If the HEAD-fetch fails, fetch prints a fatal error, and git-submodule informs the user that it will retry by fetching specific commits by hash. - If the hash-fetch then succeeds, git-submodule prints no explanation (besides the ones already printed). - If the HEAD-fetch then fails, git-submodule prints a fatal error. It could be said that we should just eliminate the HEAD-fetch altogether, but that changes some behavior (in particular, some refs that were opportunistically updated would no longer be), so I have left that alone for now. There is an analogous situation with the fetching code in fetch_finish() and surrounding functions. For now, I have added a NEEDSWORK. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-13 18:57:38 +01:00
say "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'; trying to directly fetch \$sha1:")"
# Now we tried the usual fetch, but $sha1 may
# not be reachable from any of the refs
is_tip_reachable "$sm_path" "$sha1" ||
submodule: fix fetch_in_submodule logic Commit 1c1518071c (submodule: use "fetch" logic instead of custom remote discovery, 2020-11-14) rewrote the logic in fetch_in_submodule to do: elif test "$2" -ne "" But this is nonsense in shell: -ne is for numeric comparisons. This should be "=" or more idiomatically: elif test -n "$2" But once we fix that, many tests start failing. Because that commit introduced another problem. The caller that passes 3 arguments looks like this: fetch_in_submodule "$sm_path" $depth "$sha1" Note the unquoted $depth parameter. When it isn't set, the function will see only 2 arguments, and the function has no idea if what it sees in $2 is an option to go on the command line, or a refspec to pass on stdin. In the old code before that commit: fetch_in_submodule () ( sanitize_submodule_env && cd "$1" && - case "$2" in - '') - git fetch ;; - *) - shift - git fetch $(get_default_remote) "$@" ;; - esac we treated those the same, so it didn't matter. But in the new logic (with my fix above): + if test $# -eq 3 + then + echo "$3" | git fetch --stdin "$2" + elif test -n "$n" + then + git fetch "$2" + else + git fetch + fi we use the number of parameters to distinguish the two. Let's insist that the caller pass an empty string for positional parameter two if they want to have a third parameter after it. But that still leaves one problem. In the --stdin block, we unconditionally pass "$2" to git-fetch, even if it's the empty string. Rather than add another conditional, we can use :+ parameter expansion to include it only if it's non-empty. In fact, we can do the same for the elif, too, simplifying it further. Technically this is overkill, since we know the --depth parameter will not have whitespace (and indeed, most callers do not bother quoting it), but it doesn't hurt for the function to be careful. It's somewhat amazing that no tests were failing. I think what happened is that: - the 3-arg form rarely triggered; any call with a non-empty $depth and a $sha1 would work, but one with an empty $depth would only have 2 arguments - because of the wrong arguments to "test", the shell would complain and exit non-zero. So we never ran the middle conditional at all - that left every call running "git fetch" with no arguments. A well-written test could have detected the distinction here, but in practice omitting --depth just means fetching more commits, and fetching everything (rather than a single sha1) works as long as the commit in question is reachable Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-24 10:06:05 +01:00
fetch_in_submodule "$sm_path" "$depth" "$sha1" ||
die "$(eval_gettext "Fetched in submodule path '\$displaypath', but it did not contain \$sha1. Direct fetching of that commit failed.")"
fi
must_die_on_failure=
case "$update_module" in
checkout)
command="git checkout $subforce -q"
die_msg="$(eval_gettext "Unable to checkout '\$sha1' in submodule path '\$displaypath'")"
say_msg="$(eval_gettext "Submodule path '\$displaypath': checked out '\$sha1'")"
;;
rebase)
command="git rebase ${GIT_QUIET:+--quiet}"
die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$displaypath'")"
say_msg="$(eval_gettext "Submodule path '\$displaypath': rebased into '\$sha1'")"
must_die_on_failure=yes
;;
merge)
command="git merge ${GIT_QUIET:+--quiet}"
die_msg="$(eval_gettext "Unable to merge '\$sha1' in submodule path '\$displaypath'")"
say_msg="$(eval_gettext "Submodule path '\$displaypath': merged in '\$sha1'")"
must_die_on_failure=yes
;;
!*)
command="${update_module#!}"
die_msg="$(eval_gettext "Execution of '\$command \$sha1' failed in submodule path '\$displaypath'")"
say_msg="$(eval_gettext "Submodule path '\$displaypath': '\$command \$sha1'")"
must_die_on_failure=yes
;;
*)
die "$(eval_gettext "Invalid update mode '$update_module' for submodule path '$path'")"
esac
if (sanitize_submodule_env; cd "$sm_path" && $command "$sha1")
then
say "$say_msg"
elif test -n "$must_die_on_failure"
then
die_with_status 2 "$die_msg"
else
err="${err};$die_msg"
continue
fi
fi
if test -n "$recursive"
then
(
prefix=$(git submodule--helper relative-path "$prefix$sm_path/" "$wt_prefix")
submodule: port init from shell to C By having the `submodule init` functionality in C, we can reference it easier from other parts in the code in later patches. The code is split up to have one function to initialize one submodule and a calling function that takes care of the rest, such as argument handling and translating the arguments to the paths of the submodules. This is the first submodule subcommand that is fully converted to C except for the usage string, so this is actually removing a call to the `submodule--helper list` function, which is supposed to be used in this transition. Instead we'll make a direct call to `module_list_compute`. An explanation why we need to edit the prefixes in cmd_update in git-submodule.sh in this patch: By having no processing in the shell part, we need to convey the notion of wt_prefix and prefix to the C parts, which former patches punted on and did the processing of displaying path in the shell. `wt_prefix` used to hold the path from the repository root to the current directory, e.g. wt_prefix would be t/ if the user invoked the `git submodule` command in ~/repo/t and ~repo is the GIT_DIR. `prefix` used to hold the relative path from the repository root to the operation, e.g. if you have recursive submodules, the shell script would modify the `prefix` in each recursive step by adding the submodule path. We will pass `wt_prefix` into the C helper via `git -C <dir>` as that will setup git in the directory the user actually called git-submodule.sh from. The `prefix` will be passed in via the `--prefix` option. Having `prefix` and `wt_prefix` relative to the GIT_DIR of the calling superproject is unfortunate with this patch as the C code doesn't know about a possible recursion from a superproject via `submodule update --init --recursive`. To fix this, we change the meaning of `wt_prefix` to point to the current project instead of the superproject and `prefix` to include any relative paths issues in the superproject. That way `prefix` will become the leading part for displaying paths and `wt_prefix` will be empty in recursive calls for now. The new notion of `wt_prefix` and `prefix` still allows us to reconstruct the calling directory in the superproject by just traveling reverse of `prefix`. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-16 02:50:13 +02:00
wt_prefix=
sanitize_submodule_env
cd "$sm_path" &&
eval cmd_update
)
res=$?
if test $res -gt 0
then
die_msg="$(eval_gettext "Failed to recurse into submodule path '\$displaypath'")"
if test $res -ne 2
then
err="${err};$die_msg"
continue
else
die_with_status $res "$die_msg"
fi
fi
fi
done
if test -n "$err"
then
OIFS=$IFS
IFS=';'
for e in $err
do
if test -n "$e"
then
echo >&2 "$e"
fi
done
IFS=$OIFS
exit 1
fi
}
}
#
# Configures a submodule's default branch
#
# $@ = requested path
#
cmd_set_branch() {
default=
branch=
while test $# -ne 0
do
case "$1" in
-q|--quiet)
# we don't do anything with this but we need to accept it
;;
-d|--default)
default=1
;;
-b|--branch)
case "$2" in '') usage ;; esac
branch=$2
shift
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper set-branch ${GIT_QUIET:+--quiet} ${branch:+--branch "$branch"} ${default:+--default} -- "$@"
}
#
# Configures a submodule's remote url
#
# $@ = requested path, requested url
#
cmd_set_url() {
while test $# -ne 0
do
case "$1" in
-q|--quiet)
GIT_QUIET=1
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper set-url ${GIT_QUIET:+--quiet} -- "$@"
}
#
# Show commit summary for submodules in index or working tree
#
# If '--cached' is given, show summary between index and given commit,
# or between working tree and given commit
#
# $@ = [commit (default 'HEAD'),] requested paths (default all)
#
cmd_summary() {
summary_limit=-1
for_status=
diff_cmd=diff-index
# parse $args after "submodule ... summary".
while test $# -ne 0
do
case "$1" in
--cached)
cached="$1"
;;
--files)
files="$1"
;;
--for-status)
for_status="$1"
;;
-n|--summary-limit)
summary_limit="$2"
isnumber "$summary_limit" || usage
shift
;;
--summary-limit=*)
summary_limit="${1#--summary-limit=}"
isnumber "$summary_limit" || usage
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
submodule: port submodule subcommand 'summary' from shell to C Convert submodule subcommand 'summary' to a builtin and call it via 'git-submodule.sh'. The shell version had to call $diff_cmd twice, once to find the modified modules cared by the user and then again, with that list of modules to do various operations for computing the summary of those modules. On the other hand, the C version does not need a second call to $diff_cmd since it reuses the module list from the first call to do the aforementioned tasks. In the C version, we use the combination of setting a child process' working directory to the submodule path and then calling 'prepare_submodule_repo_env()' which also sets the 'GIT_DIR' to '.git', so that we can be certain that those spawned processes will not access the superproject's ODB by mistake. A behavioural difference between the C and the shell version is that the shell version outputs two line feeds after the 'git log' output when run outside of the tests while the C version outputs one line feed in any case. The reason for this is that the shell version calls log with '--pretty=format:<fmt>' whose output is followed by two echo calls; 'format' does not have "terminator" semantics like its 'tformat' counterpart. So, the log output is terminated by a newline only when invoked by the user and not when invoked from the scripts. This results in the one & two line feed differences in the shell version. On the other hand, the C version calls log with '--pretty=<fmt>' which is equivalent to '--pretty:tformat:<fmt>' which is then followed by a 'printf("\n")'. Due to its "terminator" semantics the log output is always terminated by newline and hence one line feed in any case. Also, when we try to pass an option-like argument after a non-option argument, for instance: git submodule summary HEAD --foo-bar (or) git submodule summary HEAD --cached That argument would be treated like a path to the submodule for which the user is requesting a summary. So, the option ends up having no effect. Though, passing '--quiet' is an exception to this: git submodule summary HEAD --quiet While 'summary' doesn't support '--quiet', we don't get an output for the above command as '--quiet' is treated as a path which means we get an output only if a submodule whose path is '--quiet' exists. The error message in case of computing a summary for non-existent submodules in the C version is different from that of the shell version. Since the new error message is not marked for translation, change the 'test_i18ngrep' in t7421.4 to 'grep'. Mentored-by: Christian Couder <chriscool@tuxfamily.org> Mentored-by: Stefan Beller <stefanbeller@gmail.com> Mentored-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com> Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-12 21:44:04 +02:00
git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${prefix:+--prefix "$prefix"} ${files:+--files} ${cached:+--cached} ${for_status:+--for-status} ${summary_limit:+-n $summary_limit} -- "$@"
}
#
# List all submodules, prefixed with:
# - submodule not initialized
# + different revision checked out
#
# If --cached was specified the revision in the index will be printed
# instead of the currently checked out revision.
#
# $@ = requested paths (default to all)
#
cmd_status()
{
# parse $args after "submodule ... status".
while test $# -ne 0
do
case "$1" in
-q|--quiet)
GIT_QUIET=1
;;
--cached)
cached=1
;;
--recursive)
recursive=1
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
submodule foreach: fix "<command> --quiet" not being respected Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12 12:08:19 +02:00
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper status ${GIT_QUIET:+--quiet} ${cached:+--cached} ${recursive:+--recursive} -- "$@"
}
#
# Sync remote urls for submodules
# This makes the value for remote.$remote.url match the value
# specified in .gitmodules.
#
cmd_sync()
{
while test $# -ne 0
do
case "$1" in
-q|--quiet)
GIT_QUIET=1
shift
;;
--recursive)
recursive=1
shift
;;
--)
shift
break
;;
-*)
usage
;;
*)
break
;;
esac
done
submodule foreach: fix "<command> --quiet" not being respected Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12 12:08:19 +02:00
git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper sync ${GIT_QUIET:+--quiet} ${recursive:+--recursive} -- "$@"
}
cmd_absorbgitdirs()
{
git submodule--helper absorb-git-dirs --prefix "$wt_prefix" "$@"
}
# This loop parses the command line arguments to find the
# subcommand name to dispatch. Parsing of the subcommand specific
# options are primarily done by the subcommand implementations.
# Subcommand specific options such as --branch and --cached are
# parsed here as well, for backward compatibility.
while test $# != 0 && test -z "$command"
do
case "$1" in
add | foreach | init | deinit | update | set-branch | set-url | status | summary | sync | absorbgitdirs)
command=$1
;;
-q|--quiet)
GIT_QUIET=1
;;
-b|--branch)
case "$2" in
'')
usage
;;
esac
branch="$2"; shift
;;
--cached)
cached="$1"
;;
--)
break
;;
-*)
usage
;;
*)
break
;;
esac
shift
done
# No command word defaults to "status"
if test -z "$command"
then
if test $# = 0
then
command=status
else
usage
fi
fi
# "-b branch" is accepted only by "add" and "set-branch"
if test -n "$branch" && (test "$command" != add || test "$command" != set-branch)
then
usage
fi
# "--cached" is accepted only by "status" and "summary"
if test -n "$cached" && test "$command" != status && test "$command" != summary
then
usage
fi
"cmd_$(echo $command | sed -e s/-/_/g)" "$@"