In the submodule helper we did not correctly handled the display path
for initializing submodules when both the submodule is inside a
subdirectory as well as the command being invoked from a subdirectory
(as viewed from the superproject).
This was broken in 3604242f08, which was written at a time where
there was no super-prefix available, so we abused the --prefix option
for the same purpose and could get only one case right (the call from
within a subdirectory, not the submodule being in a subdirectory).
Test-provided-by: David Turner <novalis@novalis.org>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This part was missing in f6f85861 (submodule: add absorb-git-dir function,
2016-12-12).
Noticed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A new submodule helper "git submodule embedgitdirs" to make it
easier to move embedded .git/ directory for submodules in a
superproject to .git/modules/ (and point the latter with the former
that is turned into a "gitdir:" file) has been added.
* sb/submodule-embed-gitdir:
worktree: initialize return value for submodule_uses_worktrees
submodule: add absorb-git-dir function
move connect_work_tree_and_git_dir to dir.h
worktree: check if a submodule uses worktrees
test-lib-functions.sh: teach test_commit -C <dir>
submodule helper: support super prefix
submodule: use absolute path for computing relative path connecting
Finer-grained control of what protocols are allowed for transports
during clone/fetch/push have been enabled via a new configuration
mechanism.
* bw/transport-protocol-policy:
http: respect protocol.*.allow=user for http-alternates
transport: add from_user parameter to is_transport_allowed
http: create function to get curl allowed protocols
transport: add protocol policy config option
http: always warn if libcurl version is too old
lib-proto-disable: variable name fix
Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to
specify a whitelist of protocols to be used in clone/fetch/push
commands. This patch introduces new configuration options for more
fine-grained control for allowing/disallowing protocols. This also has
the added benefit of allowing easier construction of a protocol
whitelist on systems where setting an environment variable is
non-trivial.
Now users can specify a policy to be used for each type of protocol via
the 'protocol.<name>.allow' config option. A default policy for all
unconfigured protocols can be set with the 'protocol.allow' config
option. If no user configured default is made git will allow known-safe
protocols (http, https, git, ssh, file), disallow known-dangerous
protocols (ext), and have a default policy of `user` for all other
protocols.
The supported policies are `always`, `never`, and `user`. The `user`
policy can be used to configure a protocol to be usable when explicitly
used by a user, while disallowing it for commands which run
clone/fetch/push commands without direct user intervention (e.g.
recursive initialization of submodules). Commands which can potentially
clone/fetch/push from untrusted repositories without user intervention
can export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent
protocols configured to the `user` policy from being used.
Fix remote-ext tests to use the new config to allow the ext
protocol to be tested.
Based on a patch by Jeff King <peff@peff.net>
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a submodule has its git dir inside the working dir, the submodule
support for checkout that we plan to add in a later patch will fail.
Add functionality to migrate the git directory to be absorbed
into the superprojects git directory.
The newly added code in this patch is structured such that other areas of
Git can also make use of it. The code in the submodule--helper is a mere
wrapper and option parser for the function
`absorb_git_dir_into_superproject`, that takes care of embedding the
submodules git directory into the superprojects git dir. That function
makes use of the more abstract function for this use case
`relocate_gitdir`, which can be used by e.g. the worktree code eventually
to move around a git directory.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently the force flag in `git submodule add` takes care of possibly
ignored files or when a name collision occurs.
However there is another situation where submodule add comes in handy:
When you already have a gitlink recorded, but no configuration was
done (i.e. no .gitmodules file nor any entry in .git/config) and you
want to generate these config entries. For this situation allow
`git submodule add` to proceed if there is already a submodule at the
given path in the index.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.
In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.
That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.
This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.
This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:
git clone --recursive --progress ... 2>&1 | cat
work by triggering progress explicitly in the children.
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git clone --resurse-submodules --reference $path $URL" is a way to
reduce network transfer cost by borrowing objects in an existing
$path repository when cloning the superproject from $URL; it
learned to also peek into $path for presense of corresponding
repositories of submodules and borrow objects from there when able.
* sb/submodule-clone-rr:
clone: recursive and reference option triggers submodule alternates
clone: implement optional references
clone: clarify option_reference as required
clone: factor out checking for an alternate path
submodule--helper update-clone: allow multiple references
submodule--helper module-clone: allow multiple references
t7408: merge short tests, factor out testing method
t7408: modernize style
Allow the user to pass in multiple references to update_clone.
Currently this is only internal API, but once the shell script is
replaced by a C version, this is needed.
This fixes an API bug between the shell script and the helper.
Currently the helper accepts "--reference" "--reference=foo"
as a OPT_STRING whose value happens to be "--reference=foo", and
then uses
if (suc->reference)
argv_array_push(&child->args, suc->reference)
where suc->reference _is_ "--reference=foo" when invoking the
underlying "git clone", it cancels out.
With this change we omit one of the "--reference" arguments when
passing references from the shell script to the helper.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A few updates to "git submodule update".
Use of "| wc -l" break with BSD variant of 'wc'.
* sb/submodule-update-dot-branch:
t7406: fix breakage on OSX
submodule update: allow '.' for branch value
submodule--helper: add remote-branch helper
submodule-config: keep configured branch around
submodule--helper: fix usage string for relative-path
submodule update: narrow scope of local variable
submodule update: respect depth in subsequent fetches
t7406: future proof tests with hard coded depth
An earlier tweak to make "submodule update" retry a failing clone
of submodules was buggy and caused segfault, which has been fixed.
* sb/submodule-clone-retry:
submodule-helper: fix indexing in clone retry error reporting path
git-submodule: forward exit code of git-submodule--helper more faithfully
In a later patch we want to enhance the logic for the branch selection.
Rewrite the current logic to be in C, so we can directly use C when
we enhance the logic.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When depth is given the user may have a reasonable expectation that
any remote operation is using the given depth. Add a test to demonstrate
we still get the desired sha1 even if the depth is too short to
include the actual commit.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-submodule--helper is invoked as the upstream of a pipe in several
places. Usually, the failure of a program in this position is not
detected by the shell. For this reason, the code inserts a token in the
output stream when git-submodule--helper fails that is detected
downstream, where the shell script is quit with exit code 1.
There happens to be a bug in git-submodule--helper that leads to a
segmentation fault. The test suite triggers the crash in several places,
all of which are protected by 'test_must_fail'. But due to the inspecific
exit code 1, the crash remains undiagnosed.
Extend the failure protocol such that git-submodule--helper's exit code
is passed downstream (only in the case of failure). This enables the
downstream to use it as its own exit code, and 'test_must_fail' to
identify the segmentation fault as an unexpected failure.
The bug itself is fixed in the next commit.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
More markings of messages for i18n, with updates to various tests
to pass GETTEXT_POISON tests.
One patch from the original submission dropped due to conflicts
with jk/upload-pack-hook, which is still in flux.
* va/i18n-even-more: (38 commits)
t5541: become resilient to GETTEXT_POISON
i18n: branch: mark comment when editing branch description for translation
i18n: unmark die messages for translation
i18n: submodule: escape shell variables inside eval_gettext
i18n: submodule: join strings marked for translation
i18n: init-db: join message pieces
i18n: remote: allow translations to reorder message
i18n: remote: mark URL fallback text for translation
i18n: standardise messages
i18n: sequencer: add period to error message
i18n: merge: change command option help to lowercase
i18n: merge: mark messages for translation
i18n: notes: mark options for translation
i18n: notes: mark strings for translation
i18n: transport-helper.c: change N_() call to _()
i18n: bisect: mark strings for translation
t5523: use test_i18ngrep for negation
t4153: fix negated test_i18ngrep call
t9003: become resilient to GETTEXT_POISON
tests: unpack-trees: update to use test_i18n* functions
...
"git submodule update" that drives many "git clone" could
eventually hit flaky servers/network conditions on one of the
submodules; the command learned to retry the attempt.
* sb/submodule-clone-retry:
submodule update: continue when a clone fails
submodule--helper: initial clone learns retry logic
An upstream project can make a recommendation to shallowly clone
some submodules in the .gitmodules file it ships.
* sb/submodule-recommend-shallowness:
submodule update: learn `--[no-]recommend-shallow` option
submodule-config: keep shallow recommendation around
According to the gettext manual [1], references to shell variables inside
eval_gettext call must be escaped so that eval_gettext receives the
translatable string before the variable values are substituted into it.
[1] http://www.gnu.org/software/gettext/manual/html_node/Preparing-Shell-Scripts.html
Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Join strings marked for translation since that would facilitate and
improve translations result.
Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Positional arguments, such as $0, $1, etc, need to be stored on shell
variables for use in translatable strings, according to gettext manual
[1].
Add git-sh-setup.sh to LOCALIZED_SH variable in Makefile to enable
extraction of string marked for translation by xgettext.
Source git-sh-i18n in git-sh-setup.sh for gettext support.
git-sh-setup.sh is a shell library to be sourced by other shell scripts.
In order to avoid other scripts from sourcing git-sh-i18n twice, remove
line that sources it from them. Not sourcing git-sh-i18n in any script
that uses gettext would lead to failure due to, for instance, gettextln
not being found.
[1] http://www.gnu.org/software/gettext/manual/html_node/Preparing-Shell-Scripts.html
Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 15ffb7cde4 (2011-06-13, submodule update: continue when a checkout
fails), we reasoned it is ok to continue, when there is not much of
a mental burden by the failure. If a recursive submodule fails to clone
because a .gitmodules file is broken (e.g. :
fatal: No url found for submodule path 'foo/bar' in .gitmodules
Failed to recurse into submodule path 'foo', signaled by exit code 128),
this is one of the cases where the user is not expected to have much of
a burden afterwards, so we can also continue in that case.
This means we only want to stop for updating submodules in case of rebase,
merge or custom update command failures, which are all signaled with
exit code 2.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio pointed out `relative_path` was using bashisms via the
local variables. As the longer term goal is to rewrite most of the
submodule code in C, do it now.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 48308681 (2016-02-29, git submodule update: have a dedicated helper
for cloning), the helper communicated errors back only via exit code,
and dance with printing '#unmatched' in case of error was left to
git-submodule.sh as it uses the output of the helper and pipes it into
shell commands. This change makes the helper consistent by never
printing '#unmatched' in the helper but always handling these piping
issues in the actual shell script.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sometimes the history of a submodule is not considered important by
the projects upstream. To make it easier for downstream users, allow
a boolean field 'submodule.<name>.shallow' in .gitmodules, which can
be used to recommend whether upstream considers the history important.
This field is honored in the initial clone by default, it can be
ignored by giving the `--no-recommend-shallow` option.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
An earlier addition of "sanitize_submodule_env" with 14111fc4 (git:
submodule honor -c credential.* from command line, 2016-02-29)
turned out to be a convoluted no-op; implement what it wanted to do
correctly, and stop filtering settings given via "git -c var=val".
* jk/submodule-c-credential:
submodule: stop sanitizing config options
submodule: use prepare_submodule_repo_env consistently
submodule--helper: move config-sanitizing to submodule.c
submodule: export sanitized GIT_CONFIG_PARAMETERS
t5550: break submodule config test into multiple sub-tests
t5550: fix typo in $HTTPD_URL
Correct faulty recommendation to use "git submodule deinit ." when
de-initialising all submodules, which would result in a strange
error message in a pathological corner case.
* sb/submodule-deinit-all:
submodule deinit: require '--all' instead of '.' for all submodules
Update of "git submodule" to move pieces of logic to C continues.
* sb/submodule-init:
submodule init: redirect stdout to stderr
submodule--helper update-clone: abort gracefully on missing .gitmodules
submodule init: fail gracefully with a missing .gitmodules file
submodule: port init from shell to C
submodule: port resolve_relative_url from shell to C
The point of having a whitelist of command-line config
options to pass to submodules was two-fold:
1. It prevented obvious nonsense like using core.worktree
for multiple repos.
2. It could prevent surprise when the user did not mean
for the options to leak to the submodules (e.g.,
http.sslverify=false).
For case 1, the answer is mostly "if it hurts, don't do
that". For case 2, we can note that any such example has a
matching inverted surprise (e.g., a user who meant
http.sslverify=true to apply everywhere, but it didn't).
So this whitelist is probably not giving us any benefit, and
is already creating a hassle as people propose things to put
on it. Let's just drop it entirely.
Note that we still need to keep a special code path for
"prepare the submodule environment", because we still have
to take care to pass through $GIT_CONFIG_PARAMETERS (and
block the rest of the repo-specific environment variables).
We can do this easily from within the submodule shell
script, which lets us drop the submodule--helper option
entirely (and it's OK to do so because as a "--" program, it
is entirely a private implementation detail).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The discussion in [1] pointed out that '.' is a faulty suggestion as
there is a corner case where it fails:
> "submodule deinit ." may have "worked" in the sense that you would
> have at least one path in your tree and avoided this "nothing
> matches" most of the time. It would have still failed with the
> exactly same error if run in an empty repository, i.e.
>
> $ E=/var/tmp/x/empty && rm -fr "$E" && mkdir -p "$E" && cd "$E"
> $ git init
> $ rungit v2.6.6 submodule deinit .
> error: pathspec '.' did not match any file(s) known to git.
> Did you forget to 'git add'?
> $ >file && git add file
> $ rungit v2.6.6 submodule deinit .
> $ echo $?
> 0
So instead of a pathspec add the '--all' option to deinit all submodules
and add a test to check for the corner case of an empty repository.
The code only needs to learn about the '--all' option and doesn't
require further changes as `git submodule--helper list "$@"` will list
all submodules when "$@" is empty.
[1] http://news.gmane.org/gmane.comp.version-control.git/289535
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 14111fc (git: submodule honor -c credential.* from
command line, 2016-02-29) taught git-submodule.sh to save
the sanitized value of $GIT_CONFIG_PARAMETERS when clearing
the environment for a submodule. However, it failed to
export the result, meaning that it had no effect for any
sub-programs.
We didn't catch this in our initial tests because we checked
only the "clone" case, which does not go through the shell
script at all. Provoking "git submodule update" to do a
fetch demonstrates the bug.
Noticed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule" reports the paths of submodules the command
recurses into, but this was incorrect when the command was not run
from the root level of the superproject.
* sb/submodule-path-misc-bugs:
t7407: make expectation as clear as possible
submodule update: test recursive path reporting from subdirectory
submodule update: align reporting path for custom command execution
submodule status: correct path handling in recursive submodules
submodule update --init: correct path handling in recursive submodules
submodule foreach: correct path display in recursive submodules
By having the `submodule init` functionality in C, we can reference it
easier from other parts in the code in later patches. The code is split
up to have one function to initialize one submodule and a calling function
that takes care of the rest, such as argument handling and translating the
arguments to the paths of the submodules.
This is the first submodule subcommand that is fully converted to C
except for the usage string, so this is actually removing a call to
the `submodule--helper list` function, which is supposed to be used in
this transition. Instead we'll make a direct call to `module_list_compute`.
An explanation why we need to edit the prefixes in cmd_update in
git-submodule.sh in this patch:
By having no processing in the shell part, we need to convey the notion
of wt_prefix and prefix to the C parts, which former patches punted on
and did the processing of displaying path in the shell.
`wt_prefix` used to hold the path from the repository root to the current
directory, e.g. wt_prefix would be t/ if the user invoked the
`git submodule` command in ~/repo/t and ~repo is the GIT_DIR.
`prefix` used to hold the relative path from the repository root to the
operation, e.g. if you have recursive submodules, the shell script would
modify the `prefix` in each recursive step by adding the submodule path.
We will pass `wt_prefix` into the C helper via `git -C <dir>` as that
will setup git in the directory the user actually called git-submodule.sh
from. The `prefix` will be passed in via the `--prefix` option.
Having `prefix` and `wt_prefix` relative to the GIT_DIR of the
calling superproject is unfortunate with this patch as the C code doesn't
know about a possible recursion from a superproject via `submodule update
--init --recursive`.
To fix this, we change the meaning of `wt_prefix` to point to the current
project instead of the superproject and `prefix` to include any relative
paths issues in the superproject. That way `prefix` will become the leading
part for displaying paths and `wt_prefix` will be empty in recursive
calls for now.
The new notion of `wt_prefix` and `prefix` still allows us to reconstruct
the calling directory in the superproject by just traveling reverse of
`prefix`.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Later on we want to automatically call `git submodule init` from
other commands, such that the users don't have to initialize the
submodule themselves. As these other commands are written in C
already, we'd need the init functionality in C, too. The
`resolve_relative_url` function is a large part of that init
functionality, so start by porting this function to C.
To create the tests in t0060, the function `resolve_relative_url`
was temporarily enhanced to write all inputs and output to disk
when running the test suite. The added tests in this patch are
a small selection thereof.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule" reports the paths of submodules the command
recurses into, but this was incorrect when the command was not run
from the root level of the superproject.
Any further comments? Otherwise will merge to 'next'.
* sb/submodule-path-misc-bugs: (600 commits)
t7407: make expectation as clear as possible
submodule update: test recursive path reporting from subdirectory
submodule update: align reporting path for custom command execution
submodule status: correct path handling in recursive submodules
submodule update --init: correct path handling in recursive submodules
submodule foreach: correct path display in recursive submodules
Git 2.8
Documentation: fix git-p4 AsciiDoc formatting
mingw: skip some tests in t9115 due to file name issues
t1300: fix the new --show-origin tests on Windows
t1300-repo-config: make it resilient to being run via 'sh -x'
config --show-origin: report paths with forward slashes
submodule: fix regression for deinit without submodules
l10n: pt_PT: Update and add new translations
l10n: ca.po: update translation
Git 2.8-rc4
Documentation: fix broken linkgit to git-config
Documentation: use ASCII quotation marks in git-p4
Revert "config.mak.uname: use clang for Mac OS X 10.6"
git-compat-util: st_add4: work around gcc 4.2.x compiler crash
...
"git -c credential.<var>=<value> submodule" can now be used to
propagate configuration variables related to credential helper
down to the submodules.
* jk/submodule-c-credential:
git_config_push_parameter: handle empty GIT_CONFIG_PARAMETERS
git: submodule honor -c credential.* from command line
quote: implement sq_quotef()
submodule: fix segmentation fault in submodule--helper clone
submodule: fix submodule--helper clone usage
submodule: check argc count for git submodule--helper clone
submodule: don't pass empty string arguments to submodule--helper clone
A major part of "git submodule update" has been ported to C to take
advantage of the recently added framework to run download tasks in
parallel.
* sb/submodule-parallel-update:
clone: allow an explicit argument for parallel submodule clones
submodule update: expose parallelism to the user
submodule helper: remove double 'fatal: ' prefix
git submodule update: have a dedicated helper for cloning
run_processes_parallel: rename parameters for the callbacks
run_processes_parallel: treat output of children as byte array
submodule update: direct error message to stderr
fetching submodules: respect `submodule.fetchJobs` config option
submodule-config: drop check against NULL
submodule-config: keep update strategy around
In the predefined actions (merge, rebase, none, checkout), we use
the display path, which is relative to the current working directory.
Also use the display path when running a custom command.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The new test which is a replica of the previous test except
that it executes from a sub directory. Prior to this patch
the test failed by having too many '../' prefixed:
--- expect 2016-03-29 19:02:33.087336115 +0000
+++ actual 2016-03-29 19:02:33.359343311 +0000
@@ -1,7 +1,7 @@
b23f134787d96fae589a6b76da41f4db112fc8db ../nested1 (heads/master)
-+25d56d1ddfb35c3e91ff7d8f12331c2e53147dcc ../nested1/nested2 (file2)
- 5ec83512b76a0b8170b899f8e643913c3e9b72d9 ../nested1/nested2/nested3 (heads/master)
- 509f622a4f36a3e472affcf28fa959174f3dd5b5 ../nested1/nested2/nested3/submodule (heads/master)
++25d56d1ddfb35c3e91ff7d8f12331c2e53147dcc ../../nested1/nested2 (file2)
+ 5ec83512b76a0b8170b899f8e643913c3e9b72d9 ../../../nested1/nested2/nested3 (heads/master)
+ 509f622a4f36a3e472affcf28fa959174f3dd5b5 ../../../../nested1/nested2/nested3/submodule (heads/master)
0c90624ab7f1aaa301d3bb79f60dcfed1ec4897f ../sub1 (0c90624)
0c90624ab7f1aaa301d3bb79f60dcfed1ec4897f ../sub2 (0c90624)
509f622a4f36a3e472affcf28fa959174f3dd5b5 ../sub3 (heads/master)
The path code in question:
displaypath=$(relative_path "$prefix$sm_path")
prefix=$displaypath
if recursive:
eval cmd_status
That way we change `prefix` each iteration to contain another
'../', because of the the relative_path computation is done
on an already computed relative path.
We must call relative_path exactly once with `wt_prefix` non empty.
Further calls in recursive instances to to calculate the displaypath
already incorporate the correct prefix from before. Fix the issue by
clearing `wt_prefix` in recursive calls.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When calling `git submodule init` from a recursive instance of
`git submodule update --recursive`, the reported path is wrong as it
skips the nested submodules.
The new test demonstrates a failure in the code prior to this patch.
Instead of getting the expected
Submodule 'submodule' (${pwd}/submodule) registered for path '../super/submodule'
the `super` directory is omitted and you get
Submodule 'submodule' (${pwd}/submodule) registered for path '../submodule'
instead.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `prefix` was put in front of the display path unconditionally.
This is wrong as any relative path computation would need to be at
the front, so include the prefix into the display path.
The new test replicates the previous test with the difference of executing
from a sub directory. By executing from a sub directory all we would
expect all displayed paths to be prefixed by '../'.
Prior to this patch the test would report
Entering 'nested1/nested2/../nested3'
instead of the expected
Entering '../nested1/nested2/nested3'
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Due to the way that the git-submodule code works, it clears all local
git environment variables before entering submodules. This is normally
a good thing since we want to clear settings such as GIT_WORKTREE and
other variables which would affect the operation of submodule commands.
However, GIT_CONFIG_PARAMETERS is special, and we actually do want to
preserve these settings. However, we do not want to preserve all
configuration as many things should be left specific to the parent
project.
Add a git submodule--helper function, sanitize-config, which shall be
used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs
except a small subset that are known to be safe and necessary.
Replace all the calls to clear_local_git_env with a wrapped function
that filters GIT_CONFIG_PARAMETERS using the new helper and then
restores it to the filtered subset after clearing the rest of the
environment.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When --reference or --depth are unused, the current git-submodule.sh
results in empty "" arguments appended to the end of the argv array
inside git submodule--helper clone. This is not caught because the argc
count is not checked today.
Fix git-submodule.sh to only pass an argument when --reference or
--depth are used, preventing the addition of two empty string arguments
on the tail of the argv array.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Expose possible parallelism either via the "--jobs" CLI parameter or
the "submodule.fetchJobs" setting.
By having the variable initialized to -1, we make sure 0 can be passed
into the parallel processing machine, which will then pick as many parallel
workers as there are CPUs.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This introduces a new helper function in git submodule--helper
which takes care of cloning all submodules, which we want to
parallelize eventually.
Some tests (such as empty URL, update_mode=none) are required in the
helper to make the decision for cloning. These checks have been
moved into the C function as well (no need to repeat them in the
shell script).
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reroute the error message for specified but initialized submodules
to stderr instead of stdout.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When reviewing a change that also updates a submodule in Gerrit, a
common review practice is to download and cherry-pick the patch
locally to test it. However when testing it locally, the 'git
submodule update' may fail fetching the correct submodule sha1 as
the corresponding commit in the submodule is not yet part of the
project history, but also just a proposed change.
If $sha1 was not part of the default fetch, we try to fetch the $sha1
directly. Some servers however do not support direct fetch by sha1,
which leads git-fetch to fail quickly. We can fail ourselves here as
the still missing sha1 would lead to a failure later in the checkout
stage anyway, so failing here is as good as we can get.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The infrastructure to rewrite "git submodule" in C is being built
incrementally. Let's polish these early parts well enough and make
them graduate to 'next' and 'master', so that the more involved
follow-up can start cooking on a solid ground.
* sb/submodule-helper:
submodule: rewrite `module_clone` shell function in C
submodule: rewrite `module_name` shell function in C
submodule: rewrite `module_list` shell function in C
Some protocols (like git-remote-ext) can execute arbitrary
code found in the URL. The URLs that submodules use may come
from arbitrary sources (e.g., .gitmodules files in a remote
repository). Let's restrict submodules to fetching from a
known-good subset of protocols.
Note that we apply this restriction to all submodule
commands, whether the URL comes from .gitmodules or not.
This is more restrictive than we need to be; for example, in
the tests we run:
git submodule add ext::...
which should be trusted, as the URL comes directly from the
command line provided by the user. But doing it this way is
simpler, and makes it much less likely that we would miss a
case. And since such protocols should be an exception
(especially because nobody who clones from them will be able
to update the submodules!), it's not likely to inconvenience
anyone in practice.
Reported-by: Blake Burkhart <bburky@bburky.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reimplements the helper function `module_clone` in shell
in C as `clone`. This functionality is needed for converting
`git submodule update` later on, which we want to add threading
to.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This implements the helper `name` in C instead of shell,
yielding a nice performance boost.
Before this patch, I measured a time (best out of three):
$ time ./t7400-submodule-basic.sh >/dev/null
real 0m11.066s
user 0m3.348s
sys 0m8.534s
With this patch applied I measured (also best out of three)
$ time ./t7400-submodule-basic.sh >/dev/null
real 0m10.063s
user 0m3.044s
sys 0m7.487s
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of the submodule operations work on a set of submodules.
Calculating and using this set is usually done via:
module_list "$@" | {
while read mode sha1 stage sm_path
do
# the actual operation
done
}
Currently the function `module_list` is implemented in the
git-submodule.sh as a shell script wrapping a perl script.
The rewrite is in C, such that it is faster and can later be
easily adapted when other functions are rewritten in C.
git-submodule.sh, similar to the builtin commands, will navigate
to the top-most directory of the repository and keep the
subdirectory as a variable. As the helper is called from
within the git-submodule.sh script, we are already navigated
to the root level, but the path arguments are still relative
to the subdirectory we were in when calling git-submodule.sh.
That's why there is a `--prefix` option pointing to an alternative
path which to anchor relative path arguments.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Alex Henrie <alexhenrie24@gmail.com>
Acked-by: Chris Packham <judge.packham@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we add a new submodule the path of the submodule is being
normalized. We fail to normalize multiple adjacent '/./', though.
Thus 'path/to/././submodule' will become 'path/to/./submodule' where
it should be 'path/to/submodule' instead.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
SysV-derived implementation of "echo" interprets some backslash
sequences as special instruction, e.g. "echo 'ab\c'" shows an
incomplete line with 'a' and 'b' on it. Avoid using it when showing
a path-like values in the script.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The construct is error-prone; "test" being built-in in most modern
shells, the reason to avoid "test <cond> && test <cond>" spawning
one extra process by using a single "test <cond> -a <cond>" no
longer exists.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 4dce7d9b40,
which was originally done to help Windows but was almost
immediately reverted in msysGit, and the codebase kept this
unnecessary divergence for almost two years.
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 23d25e48f5, as it is
broken for users who haven't opted into the new feature of checking
out submodule.*.branch with update mode set to checkout.
Commit 322bb6e12f (add update 'none' flag to disable update of submodule
by default) added the '--checkout' option to "git submodule update" but
forgot to explicitly document it in synopsis, usage string and man page
(It is only mentioned implicitly in the man page). In 23d25e48 (submodule:
explicit local branch creation in module_clone) the synopsis of the man
page was updated, but the "OPTIONS" section of the man page and the usage
string of the git-submodule script still do not mention the '--checkout'
option.
Fix that by documenting this option in usage string and the "OPTIONS"
section of man page too. While at it group the update-mode options into
a single set in the usage string.
Reported-by: Matthijs Kooijman <matthijs@stdin.nl>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make sure 'submodule update' modes that do not detach HEADs can
be used more pleasantly by checking out a concrete branch when
cloning them to prime the well.
* wk/submodule-on-branch:
Documentation: describe 'submodule update --remote' use case
submodule: explicit local branch creation in module_clone
submodule: document module_clone arguments in comments
submodule: make 'checkout' update_module mode more explicit
The previous code only checked out branches in cmd_add. This commit
moves the branch-checkout logic into module_clone, where it can be
shared by cmd_add and cmd_update. I also update the initial checkout
command to use 'reset' to preserve branches setup during module_clone.
With this change, folks cloning submodules for the first time via:
$ git submodule update ...
will get a local branch instead of a detached HEAD, unless they are
using the default checkout-mode updates. This is a change from the
previous situation where cmd_update always used checkout-mode logic
(regardless of the requested update mode) for updates that triggered
an initial clone, which always resulted in a detached HEAD.
This commit does not change the logic for updates after the initial
clone, which will continue to create detached HEADs for checkout-mode
updates, and integrate remote work with the local HEAD (detached or
not) in other modes.
The motivation for the change is that developers doing local work
inside the submodule are likely to select a non-checkout-mode for
updates so their local work is integrated with upstream work.
Developers who are not doing local submodule work stick with
checkout-mode updates so any apparently local work is blown away
during updates. For example, if upstream rolls back the remote branch
or gitlinked commit to an earlier version, the checkout-mode developer
wants their old submodule checkout to be rolled back as well, instead
of getting a no-op merge/rebase with the rolled-back reference.
By using the update mode to distinguish submodule developers from
black-box submodule consumers, we can setup local branches for the
developers who will want local branches, and stick with detached HEADs
for the developers that don't care.
Testing
=======
In t7406, just-cloned checkouts now update to the gitlinked hash with
'reset', to preserve the local branch for situations where we're not
on a detached HEAD.
I also added explicit tests to t7406 for HEAD attachement after
cloning updates, showing that it depends on their update mode:
* Checkout-mode updates get detached HEADs
* Everyone else gets a local branch, matching the configured
submodule.<name>.branch and defaulting to master.
The 'initial-setup' tag makes it easy to reset the superproject to a
known state, as several earlier tests commit to submodules and commit
the changed gitlinks to the superproject, but don't push the new
submodule commits to the upstream subprojects. This makes it
impossible to checkout the current super master, because it references
submodule commits that don't exist in the upstream subprojects. For a
specific example, see the tests that currently generate the
'two_new_submodule_commits' commits.
Documentation
=============
I updated the docs to describe the 'submodule update' modes in detail.
The old documentation did not distinguish between cloning and
non-cloning updates and lacked clarity on which operations would lead
to detached HEADs, and which would not. The new documentation
addresses these issues while updating the docs to reflect the changes
introduced by this commit's explicit local branch creation in
module_clone.
I also add '--checkout' to the usage summary and group the update-mode
options into a single set.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This avoids the current awkwardness of having either '' or 'checkout'
for checkout-mode updates, which makes testing for checkout-mode
updates (or non-checkout-mode updates) easier.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"submodule.*.update=checkout", when propagated from .gitmodules to
.git/config, turned into a "submodule.*.update=none", which did not
make much sense.
* fp/submodule-checkout-mode:
git-submodule.sh: 'checkout' is a valid update mode
'checkout' is documented as one of the valid values for the
'submodule.<name>.update' variable, and in a repository with the
variable set to 'checkout', "git submodule update" command does
update using the 'checkout' mode.
However, it has been an accident that the implementation works this
way; any unknown value would trigger the same codepath and update
using the 'checkout' mode.
Explicitly list 'checkout' as one of the known update modes, and
error out when an unknown update mode is used.
Teach the codepath that initializes the configuration variable from
an in-tree .gitmodules that 'checkout' is one of the valid values.
The code since ac1fbbda (submodule: do not copy unknown update mode
from .gitmodules, 2013-12-02) used to treat the value 'checkout' as
unknown and mapped it to 'none', which made little sense. With this
change, 'checkout' specified in .gitmodules will stay to be 'checkout'.
Signed-off-by: Francesco Pretto <ceztko@gmail.com>
Signed-off-by: Signed-off-by: Junio C Hamano <gitster@pobox.com>
A behavior change, but a worthwhile one: "git submodule foreach"
was treating its arguments as part of a single command to be
concatenated and passed to a shell, making writing buggy
scripts too easy.
This patch preserves the old "just pass it to the shell" behavior
when a single argument is passed to 'git submodule foreach' and
moves to a new "skip the shell and use the arguments passed
unmolested" behavior when more than one argument is passed.
The old behavior (always concatenating and passing to the shell)
was similar to the 'ssh' command, while the new behavior (switching
on the number of arguments) is what 'xterm -e' does.
May need more thought to make sure this change is advertised well
so that scripts that used multiple arguments but added their own
extra layer of quoting are not broken.
* ak/submodule-foreach-quoting:
submodule foreach: skip eval for more than one argument
When submodule.$name.update is given as hint from the upstream in
the .gitmodules file, we used to blindly copy it to .git/config,
unless there already is a value defined for the submodule.
However, there is no reason to expect that the update mode hinted by
the upstream is available in the version of Git the user is using,
and a really custom "!cmd" prepared by an upstream person running on
Linux may not even be available to a user on Windows. It is simply
irresponsible to copy the setting blindly and to attempt to use it
during a later "submodule update" without validating it first.
Just show the suggested value to the diagnostic output, and set the
value to 'none' in the configuration, if it is not one of the ones
that are known to be supported by this version of Git.
Helped-by: Jens Lehmann <Jens.Lehmann@web.de>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cmd_update() in the submodule script tries to preserve the options given
on the command line in the "orig_flags" variable to pass them on into the
recursion when the '--recursive' option is given. But this isn't necessary
because all the variables set by the options will be seen in the recursion
too as that is achieved by executing "eval cmd_update".
The same has already been done for cmd_status() in e15bec0ec, so let's
clean up cmd_update() likewise. Also add a test to make sure that a
submodule name given on the command line is not passed into the recursion
(which was the goal of adding the orig_flags variable in 98dbe63db).
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Several of the built shell commands invoke a bare "perl" to
perform some one-liners. This will use the first perl in the
PATH rather than the one specified by the user's SHELL_PATH.
We are not asking these perl invocations to do anything
exotic, so typically any old system perl will do; however,
in some cases the system perl may have unexpected behavior
(e.g., by handling line endings differently). We should err
on the side of using the perl the user pointed us to.
The downside of this is that on systems with a sane perl
setup, we no longer find the perl at runtime, but instead
point to a static perl (like /usr/bin/perl). That means we
will not handle somebody moving perl without rebuilding git,
whereas before we tracked it just fine. This is probably not
a big deal, though, as the built perl scripts already
suffered from this.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'eval "$@"' creates an extra layer of shell interpretation, which is
probably not expected by a user who passes multiple arguments to git
submodule foreach:
$ git grep "'"
[searches for single quotes]
$ git submodule foreach git grep "'"
Entering '[submodule]'
/usr/lib/git-core/git-submodule: 1: eval: Syntax error: Unterminated quoted string
Stopping at '[submodule]'; script returned non-zero status.
To fix this, if the user passes more than one argument, execute "$@"
directly instead of passing it to eval.
Examples:
* Typical usage when adding an extra level of quoting is to pass a
single argument representing the entire command to be passed to the
shell. This doesn't change that.
* One can imagine someone feeding untrusted input as an argument:
git submodule foreach git grep "$variable"
That currently results in a nonobvious shell code injection
vulnerability. Executing the command named by the arguments
directly, as in this patch, fixes it.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Acked-by: Johan Herland <johan@herland.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
* bc/submodule-status-ignored:
Improve documentation concerning the status.submodulesummary setting
submodule: don't print status output with ignore=all
submodule: fix confusing variable name
The --for-status option was an undocumented option used only by
wt-status.c, which inserted a header and commented out the output. We can
achieve the same result within wt-status.c, without polluting the
submodule command-line options.
This will make it easier to disable the comments from wt-status.c later.
The --for-status is kept so that another topic in flight
(bc/submodule-status-ignored) can continue relying on it, although it is
currently a no-op.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git status prints information for submodules, but it should ignore the status of
those which have submodule.<name>.ignore set to all. Fix it so that it does
properly ignore those which have that setting either in .git/config or in
.gitmodules.
Not ignored are submodules that are added, deleted, or moved (which is
essentially a combination of the first two) because it is not easily possible to
determine the old path once a move has occurred, nor is it easily possible to
detect which adds and deletions are moves and which are not. This also
preserves the previous behavior of always listing modules which are to be
deleted.
Tests are included which verify that this change has no effect on git submodule
summary without the --for-status option.
Signed-off-by: Brian M. Carlson <sandals@crustytoothpaste.net>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cmd_summary reads the output of git diff, but reads in the submodule path into a
variable called name. Since this variable does not contain the name of the
submodule, but the path, rename it to be clearer what data it actually holds.
Signed-off-by: Brian M. Carlson <sandals@crustytoothpaste.net>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the --depth option to the add and update commands of "git submodule",
which is then passed on to the clone command. This is useful when the
submodule(s) are huge and you're not really interested in anything but
the latest commit.
Tests are added and some indention adjustments were made to conform to the
rest of the testfile on "submodule update can handle symbolic links in pwd".
Signed-off-by: Fredrik Gustafsson <iveqy@iveqy.com>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can set submodule.$name.update to '!command' which will cause
'command' to be run instead of checkout/merge/rebase. This allows
the user finer-grained control over how the update is done.
The primary motivation for this was interoperability with stgit;
however being able to intercept the submodule update process may
prove useful for integrating with or extending other tools.
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allow various subcommands of "git submodule" to be run not from the
top of the working tree of the superproject.
* jk/submodule-subdirectory-ok:
submodule: drop the top-level requirement
rev-parse: add --prefix option
submodule: show full path in error message
t7403: add missing && chaining
t7403: modernize style
t7401: make indentation consistent
Many "git submodule" operations do not work on a submodule at a
path whose name is not in ASCII.
* fg/submodule-non-ascii-path:
t7400: test of UTF-8 submodule names pass under Mac OS
handle multibyte characters in name
Use the new rev-parse --prefix option to process all paths given to the
submodule command, dropping the requirement that it be run from the
top-level of the repository.
Since the interpretation of a relative submodule URL depends on whether
or not "remote.origin.url" is configured, explicitly block relative URLs
in "git submodule add" when not at the top level of the working tree.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When --recursive was added to "submodule foreach" in commit 15fc56a (git
submodule foreach: Add --recursive to recurse into nested submodules,
2009-08-19), the error message when the script returns a non-zero status
was not updated to contain $prefix to show the full path. Fix this.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
set_rev_name is a possiblly expensive operation. If a submodule has
changes in it, set_rev_name was called twice.
Move call to set_rev_name so it's only called once, no matter which
codepath is taken.
Signed-off-by: Fredrik Gustafsson <iveqy@iveqy.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Many "git submodule" operations do not work on a submodule at a path whose
name is not in ASCII.
This is because "git ls-files" is used to find which paths are bound to
submodules to the current working tree, and the output is C-quoted by default
for non ASCII pathnames.
Tell "git ls-files" to not C-quote its output, which is easier than unwrapping
C-quote ourselves.
Signed-off-by: Fredrik Gustafsson <iveqy@iveqy.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"submodule summary --summary-limit" option did not support
"--option=value" form.
* rs/submodule-summary-limit:
submodule summary: support --summary-limit=<n>
The output of "git submodule deinit sub" of a populated submodule prints
rm 'sub'
as the first line unless used with the -f option.
The "rm 'sub'" line is exactly the same output the user gets when using
"git rm sub" (because that command is used with the --dry-run option under
the hood to determine if the submodule is clean), which can easily lead to
the false impression that the submodule would be permanently removed. Also
users might be confused that the "rm 'submodule'" line won't show up when
the -f option is used, as the test is skipped in this case.
Silence the "rm 'submodule'" output by using the --quiet option for "git
rm" and always print
Cleared directory 'submodule'
instead as the first output line. This line is printed as long as the
directory exists, no matter if empty or not.
Also extend the tests in t7400 to make sure the "Cleared directory" line
is printed correctly.
Reported-by: Phil Hord <phil.hord@gmail.com>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In addition to "--summary-limit <n>" support the form "--summary-limit=<n>",
for consistency with other parameters and commands.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule update", when recursed into sub-submodules, did not
acccumulate the prefix paths.
* we/submodule-update-prefix-output:
submodule update: when using recursion, show full path
There was no Porcelain way to say "I no longer am interested in
this submodule", once you express your interest in a submodule with
"submodule init". "submodule deinit" is the way to do so.
* jl/submodule-deinit:
submodule: add 'deinit' command
"git submodule update", when recursed into sub-submodules, did not
acccumulate the prefix paths.
* we/submodule-update-prefix-output:
submodule update: when using recursion, show full path
With "git submodule init" the user is able to tell git he cares about one
or more submodules and wants to have it populated on the next call to "git
submodule update". But currently there is no easy way he could tell git he
does not care about a submodule anymore and wants to get rid of his local
work tree (except he knows a lot about submodule internals and removes the
"submodule.$name.url" setting from .git/config together with the work tree
himself).
Help those users by providing a 'deinit' command. This removes the
whole submodule.<name> section from .git/config (either for the given
submodule(s) or for all those which have been initialized if '.' is used)
together with their work tree. Fail if the current work tree contains
modifications (unless forced), but don't complain when either the work
tree is already removed or no settings are found in .git/config.
Add tests and link the man pages of "git submodule deinit" and "git rm"
to assist the user in deciding whether removing or unregistering the
submodule is the right thing to do for him. Also add the deinit subcommand
to the completion list.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously when using update with recursion, only the path for the
inner-most module was printed. Now the path is printed relative to
the directory the command was started from. This now matches the
behavior of submodule foreach.
Signed-off-by: William Entriken <github.com@phor.net>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some users do want to write a line that begin with a pound sign, #,
in their commit log message. Many tracking system recognise
a token of #<bugid> form, for example.
The support we offer these use cases is not very friendly to the end
users. They have a choice between
- Don't do it. Avoid such a line by rewrapping or indenting; and
- Use --cleanup=whitespace but remove all the hint lines we add.
Give them a way to set a custom comment char, e.g.
$ git -c core.commentchar="%" commit
so that they do not have to do either of the two workarounds.
[jc: although I started the topic, all the tests and documentation
updates, many of the call sites of the new strbuf_add_commented_*()
functions, and the change to git-submodule.sh scripted Porcelain are
from Ralf.]
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ralf Thielow <ralf.thielow@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This allows you to easily record a submodule.<name>.branch option in
.gitmodules when you add a new submodule. With this patch,
$ git submodule add -b <branch> <repository> [<path>]
$ git config -f .gitmodules submodule.<path>.branch <branch>
reduces to
$ git submodule add -b <branch> <repository> [<path>]
This means that future calls to
$ git submodule update --remote ...
will get updates from the same branch that you used to initialize the
submodule, which is usually what you want.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current `update` command incorporates the superproject's gitlinked
SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the
options you use, it may checkout $sha1, rebase the $subsha1 onto
$sha1, or merge $sha1 into $subsha1. This helps you keep up with
changes in the upstream superproject.
However, it's also useful to stay up to date with changes in the
upstream subproject. Previous workflows for incorporating such
changes include the ungainly:
$ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
With this patch, all of the useful functionality for incorporating
superproject changes can be reused to incorporate upstream subproject
updates. When you specify --remote, the target $sha1 is replaced with
a $sha1 of the submodule's origin/master tracking branch. If you want
to merge a different tracking branch, you can configure the
`submodule.<name>.branch` option in `.gitmodules`. You can override
the `.gitmodules` configuration setting for a particular superproject
by configuring the option in that superproject's default configuration
(using the usual configuration hierarchy, e.g. `.git/config`,
`~/.gitconfig`, etc.).
Previous use of submodule.<name>.branch
=======================================
Because we're adding a new configuration option, it's a good idea to
check if anyone else is already using the option. The foreach-pull
example above was described by Ævar in
commit f030c96d86
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Fri May 21 16:10:10 2010 +0000
git-submodule foreach: Add $toplevel variable
Gerrit uses the same interpretation for the setting, but because
Gerrit has direct access to the subproject repositories, it updates
the superproject repositories automatically when a subproject changes.
Gerrit also accepts the special value '.', which it expands into the
superproject's branch name.
Although the --remote functionality is using `submodule.<name>.branch`
slightly differently, the effect is the same. The foreach-pull
example uses the option to record the name of the local branch to
checkout before pulls. The tracking branch to be pulled is recorded
in `.git/modules/<name>/config`, which was initialized by the module
clone during `submodule add` or `submodule init`. Because the branch
name stored in `submodule.<name>.branch` was likely the same as the
branch name used during the initial `submodule add`, the same branch
will be pulled in each workflow.
Implementation details
======================
In order to ensure a current tracking branch state, `update --remote`
fetches the submodule's remote repository before calculating the
SHA-1. However, I didn't change the logic guarding the existing fetch:
if test -z "$nofetch"
then
# Run fetch only if $sha1 isn't present or it
# is not reachable from a ref.
(clear_local_git_env; cd "$path" &&
( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
test -z "$rev") || git-fetch)) ||
die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
fi
There will not be a double-fetch, because the new $sha1 determined
after the `--remote` triggered fetch should always exist in the
repository. If it doesn't, it's because some racy process removed it
from the submodule's repository and we *should* be re-fetching.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Several submodule configuration variables
(e.g. fetchRecurseSubmodules) are read from .gitmodules with local
overrides from the usual git config files. This shell function mimics
that logic to help initialize configuration variables in
git-submodule.sh.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Cleans up some leftover bits from an earlier submodule change.
* ph/maint-submodule-status-fix:
submodule status: remove unused orig_* variables
t7407: Fix recursive submodule test
If you remove a submodule, in order to keep the repository so that
"git checkout" to an older commit in the superproject history can
resurrect the submodule, the real repository will stay in $GIT_DIR
of the superproject. A later "git submodule add $path" to add a
different submodule at the same path will fail. Diagnose this case
a bit better, and if the user really wants to add an unrelated
submodule at the same path, give the "--name" option to give it a
place in $GIT_DIR of the superproject that does not conflict with
the original submodule.
* jl/submodule-add-by-name:
submodule add: Fail when .git/modules/<name> already exists unless forced
Teach "git submodule add" the --name option
The submodule sync command was somehow left out when
--recursive was added to the other submodule commands.
Teach sync to handle the --recursive switch by recursing
when we're in a submodule we are sync'ing.
Change the report during sync to show submodule-path
instead of submodule-name to be consistent with the other
submodule commands and to help recursed paths make sense.
Signed-off-by: Phil Hord <hordp@cisco.com>
Acked-By: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Jeff King <peff@peff.net>
When renaming orig_args to orig_flags in 98dbe63d (submodule: only
preserve flags across recursive status/update invocations) the call site
of the recursive cmd_status was forgotten. At that place orig_args is
still passed into the recursion, which is always empty since then. This
did not break anything because the orig_flags logic is not needed at all
when a function from the submodule script is called with eval, as that
inherits all the variables set by the option parsing done in the first
level of the recursion.
Now that we know that orig_flags and orig_args aren't needed at all,
let's just remove them from cmd_status().
Thanks-to: Phil Hord <hordp@cisco.com>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Jeff King <peff@peff.net>
Doing a shift here is wrong because there is no extra
argument to consume when "--reference=<repo>" is used (note
the '=' instead of a space).
Signed-off-by: Stefan Zager <szager@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Use "-b <branch>" instead of "-b branch". This brings the usage
strings in line with other options, e.g. "--reference <repository>".
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Jeff King <peff@peff.net>
"git submodule frotz" was not diagnosed as "frotz" being an unknown
subcommand to "git submodule"; the user instead got a complaint that
"git submodule status" was run with an unknown path "frotz".
* rr/maint-submodule-unknown-cmd:
submodule: if $command was not matched, don't parse other args
When adding a new submodule it can happen that .git/modules/<name> already
contains a submodule repo, e.g. when a submodule is removed from the work
tree and another submodule is added at the same path. But then the work
tree of the submodule will be populated using the existing repository and
not the one the user provided, which results in an incorrect work tree. On
the other hand the user might reactivate a submodule removed earlier, then
reusing that .git directory is the Right Thing to do.
As git can't decide what is the case, error out and tell the user she
should use either use a different name for the submodule with the "--name"
option or can reuse the .git directory for the newly added submodule by
providing the --force option (which only makes sense when the upstream
matches, so the error message lists all remotes of .git/modules/<name>).
In one test in t7406 the --force option had to be added to "git submodule
add", as that test re-adds a formerly removed submodule.
Reported-by: Jonathan Johnson <me@jondavidjohn.com>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule add" initializes the name of a submodule to its path. This
was ok as long as the .git directory lived inside the submodule's work
tree, but since 1.7.8 it is stored in the .git/modules/<name> directory of
the superproject, making the submodule name survive the removal of the
submodule's work tree. This leads to problems when the user tries to add a
different submodule at the same path - and thus the same name - later, as
that will happily try to restore the submodule from the old repository
instead of the one the user specified and will lead to a checkout of the
wrong repository.
Add the new "--name" option to let the user provide a name for the
submodule. This enables the user to solve this conflict without having to
remove .git/modules/<name> by hand (which is no viable solution as it
makes it impossible to checkout a commit that records the old submodule
and populate it, as that will still check out the new submodule for the
same reason).
To achieve that the submodule's name is added to the parameter list of
the module_clone() helper function. This makes it possible to remove the
call of module_name() there because both callers of module_clone() already
know the name and can provide it as argument number two.
Reported-by: Jonathan Johnson <me@jondavidjohn.com>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule" command DWIMs the command line and assumes a
unspecified action word for 'status' action. This is a UI mistake
that leads to a confusing behaviour. A mistyped command name is
instead treated as a request for 'status' of the submodule with that
name, e.g.
$ git submodule show
error: pathspec 'show' did not match any file(s) known to git.
Did you forget to 'git add'?
Stop DWIMming an unknown or mistyped subcommand name as pathspec
given to unspelled "status" subcommand. "git submodule" without any
argument is still interpreted as "git submodule status", but its
value is questionable.
Adjust t7400 to match, and stop advertising the default subcommand
being 'status' which does not help much in practice, other than
promoting laziness and confusion.
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule update --force" used to leave the working tree of the
submodule intact when there were local changes. It is more intiutive
to make "--force" a sign to run "checkout -f" to overwrite them.
* sz/submodule-force-update:
Make 'git submodule update --force' always check out submodules.
Currently, it will only do a checkout if the sha1 registered in the containing
repository doesn't match the HEAD of the submodule, regardless of whether the
submodule is dirty. As discussed on the mailing list, the '--force' flag is a
strong indicator that the state of the submodule is suspect, and should be reset
to HEAD.
Signed-off-by: Stefan Zager <szager@google.com>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Various subcommands of the "git submodule" command exited with 0
status even though the path given by the user did not exist.
The reason behind that was that they all pipe the output of
module_list into the while loop which then does the action on the
paths specified by the commandline. Since the exit code of the
command on the upstream side of the pipe is ignored by the shell,
the status code of "ls-files --error-unmatch" nor "module_list" was
not propagated.
In case ls-files returns with an error code, we write a special
string that is not possible in non error situations, and no other
output, so that the downstream can detect the error and die with an
error code.
The error message that there is an unmatched pathspec comes through
stderr directly from ls-files. So the user still gets a hint whats going
on.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add i18n support for scripted Porcelains, and mark strings in
merge(-recursive), am, and rebase for i18n.
* jx/i18n-1.7.11:
i18n: merge-recursive: mark strings for translation
Remove dead code which contains bad gettext block
i18n: am: mark more strings for translation
rebase: remove obsolete and unused LONG_USAGE which breaks xgettext
i18n: Rewrite gettext messages start with dash
i18n: rebase: mark messages for translation
i18n: New keywords for xgettext extraction from sh
When "git submodule add" clones a submodule repository, it can get
confused where to store the resulting submodule repository in the
superproject's .git/ directory when there is a symbolic link in the
path to the current directory.
* jl/maint-1.7.10-recurse-submodules-with-symlink:
submodules: don't stumble over symbolic links when cloning recursively
Gettext message in a shell script should not start with '-', one
workaround is adding '--' between gettext and the message, like:
gettext -- "--exec option ..."
But due to a bug in the xgettext extraction, xgettext can not
extract the actual message for this case. Rewriting the message
is a simpler and better solution.
Reported-by: Vincent van Ravesteijn <vfr@lyx.org>
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The combination of GIT_DIR and GIT_WORK_TREE can be used to manage
files in one directory hierarchy while keeping the repository that
keeps track of them outside the directory hierarchy. For example:
git init --bare /path/to/there
alias dotfiles="GIT_DIR=/path/to/there GIT_WORK_TREE=/path/to/here git"
cd /path/to/here
dotfiles add file
dotfiles commit -a -m "add /path/to/here/file"
...
lets you manage files under /path/to/here/ in the repository located
at /path/to/there.
git-submodule however fails to add submodules, as it is confused by
GIT_DIR and GIT_WORK_TREE environment variables when it tries to
work in the submodule, like so:
dotfiles submodule add http://path.to/submodule
fatal: working tree '/path/to/here' already exists.
Simply unsetting the environment where the command works on the
submodule is sufficient to fix this, as it has set things up so
that GIT_DIR and GIT_WORK_TREE do not even have to point at the
repository and the working tree of the submodule.
Signed-off-by: Daniel Graña <dangra@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When "git submodule add" clones a submodule repository, it can get
confused where to store the resulting submodule repository in the
superproject's .git/ directory when there is a symbolic link in the
path to the current directory.
* jl/maint-1.7.10-recurse-submodules-with-symlink:
submodules: don't stumble over symbolic links when cloning recursively
Since 69c3051 (submodules: refactor computation of relative gitdir path)
cloning a submodule recursively fails for nested submodules when a
symbolic link is part of the path to the work tree of the superproject.
This happens when module_clone() tries to find the relative paths between
the work tree and the git dir. When a symbolic link in current $PWD points
to a directory that is at a different level, then determining the number
of "../" needed to traverse to the superproject's work tree leads to a
wrong result.
As there is no portable way to say "pwd -P", use cd_to_toplevel to remove
the link from $PWD, which fixes this problem.
A test for this issue has been added to t7406.
Reported-by: Bob Halley <halley@play-bow.org>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach "git submodule" deal with nested submodule structure where a
module is contained within a module whose origin is specified as a
relative URL to its superproject's origin.
Currently git submodule init and git submodule sync fail with an error
if the superproject origin URL is of the form foo but succeed if the
superproject origin URL is of the form ./foo or ./foo/bar or foo/bar.
This change makes handling of the foo case behave like the handling
of the ./foo case and also ensures that superfluous leading and
embedded ./'s are removed from the resulting derived URLs.
Signed-off-by: Jon Seymour <jon.seymour@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the origin URL of the superproject is itself relative, git submodule sync
configures the remote.origin.url configuration property of the submodule
with a path that is relative to the work tree of the superproject
rather than the work tree of the submodule.
To fix this an 'up_path' that navigates from the work tree of the submodule
to the work tree of the superproject needs to be prepended to the URL
otherwise calculated.
Correct handling of superproject origin URLs like foo, ./foo and ./foo/bar is
left to a subsequent patch since an additional change is required to handle
these cases.
The documentation of resolve_relative_url() is expanded to give a more thorough
description of the function's objective.
Signed-off-by: Jon Seymour <jon.seymour@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 2cd9de3e (submodule add: always initialize .git/config entry) the
message "Submodule '\$name' (\$url) registered for path '\$sm_path'" is
printed every time cmd_init() is called, e.g. each time "git submodule
update" is used with the --init option.
This was not intended and leads to bogus output which can confuse users
and build systems. Apart from that the $url variable was not set after the
first run which did the actual initialization and only "()" was printed
in subsequent runs where "($url)" was meant to inform the user about the
upstream repo.
Fix that by moving the say command in question into the if block where the
url is initialized, restoring the behavior that was in place before the
2cd9de3e commit. While at it also remove the comment which still describes
the logic used before 2cd9de3e and add a comment about how things work now.
Reported-by: Nicolas Viennot and Sid Nair <nicolas@viennot.com>
Reported-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The eval_gettext (and eval_gettextln) i18n shell functions call
git-sh-i18n--envsubst to process the variable references in the
string parameter. Unfortunately, environment variables are case
insensitive on windows, which leads to failure on cygwin when
eval_gettext exports $path.
Commit df599e9 (Windows: teach getenv to do a case-sensitive search,
06-06-2011) attempts to solve this problem on MinGW by overriding
the system getenv() function to allow git-sh-i18n--envsubst to read
$path rather than $PATH from the environment. However, this commit
does not address cygwin at all and, furthermore, does not fix all
problems on MinGW.
In particular, when executing test #38 in t7400-submodule-basic.sh,
an 'git-sh-i18n-envsubst.exe - Unable To Locate Component' dialog
pops up saying that the application "failed to start because
libiconv2.dll was not found." After studying the voluminous trace
output from the process monitor, it is clear that the system is
attempting to use $path, rather than $PATH, to search for the DLL
file. (Note that, after dismissing the dialog, the test passes
anyway!)
As an alternative, we finesse the problem by renaming the $path
variable to $sm_path (submodule path). This fixes the problem on
MinGW along with all test failures on cygwin (t7400.{7,32,34},
t7406.3 and t7407.{2,6}). We note that the foreach subcommand
provides $path to user scripts (ie it is part of the API), so we
can't simply rename it to $sm_path.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Tested-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The sed provided by Solaris in /usr/xpg4/bin has a bug whereby an
unanchored regex using * for zero or more repetitions sees two
separate matches fed to the substitution engine in some cases.
This is evidenced by:
$ for sed in /usr/xpg4/bin/sed /usr/bin/sed /opt/csw/gnu/sed; do \
echo 'ab' | $sed -e 's|[a]*|X|g'; \
done
XXbX
XbX
XbX
This bug was triggered during a git submodule clone operation as
exercised in the setup stage of t5526-fetch-submodules when using the
default SANE_TOOL_PATH for Solaris. It led to paths such as
..../.. being used in the submodule .git gitdir reference.
Using the expression 's|\([^/]*\(/*\)\)|..\2|g' provides the desired
result with all three three tested sed implementations but is harder
to read. As we do not need to handle fully qualified paths though,
the expression could actually be [^/]+ which isn't properly handled
either. Instead, use [^/][^/]*, as suggested by Andreas Schwab, which
works on all three tested sed implementations.
The new expression is semantically different than the original one.
It will not place a leading '..' on a fully qualified path as the
original expression did. All of the paths being passed through this
regex are relative and did not rely on this behaviour so it's a safe
change.
Signed-off-by: Ben Walton <bwalton@artsci.utoronto.ca>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Under Windows the "git rev-parse --git-dir" and "pwd" commands may return
either drive-letter-colon or POSIX style paths. This makes module_clone()
behave badly because it expects absolute paths to always start with a '/'.
Fix that by always converting the "c:/" notation into "/c/" when computing
the relative paths from gitdir to the submodule work tree and back.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
