Commit Graph

309 Commits

Author SHA1 Message Date
Junio C Hamano
e801be066c Merge branch 'sb/submodule-init'
Error message fix.

* sb/submodule-init:
  submodule update --init: display correct path from submodule
2017-01-23 15:59:21 -08:00
Junio C Hamano
9dc7f78149 Merge branch 'sb/submodule-embed-gitdir'
Help-text fix.

* sb/submodule-embed-gitdir:
  submodule absorbgitdirs: mention in docstring help
2017-01-23 15:59:20 -08:00
Stefan Beller
6e7c14e65c submodule update --init: display correct path from submodule
In the submodule helper we did not correctly handled the display path
for initializing submodules when both the submodule is inside a
subdirectory as well as the command being invoked from a subdirectory
(as viewed from the superproject).

This was broken in 3604242f08, which was written at a time where
there was no super-prefix available, so we abused the --prefix option
for the same purpose and could get only one case right (the call from
within a subdirectory, not the submodule being in a subdirectory).

Test-provided-by: David Turner <novalis@novalis.org>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-12 14:36:20 -08:00
Stefan Beller
c32eaa8af1 submodule absorbgitdirs: mention in docstring help
This part was missing in f6f85861 (submodule: add absorb-git-dir function,
2016-12-12).

Noticed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-12 14:09:28 -08:00
Junio C Hamano
da2b74eeec Merge branch 'sb/submodule-embed-gitdir'
A new submodule helper "git submodule embedgitdirs" to make it
easier to move embedded .git/ directory for submodules in a
superproject to .git/modules/ (and point the latter with the former
that is turned into a "gitdir:" file) has been added.

* sb/submodule-embed-gitdir:
  worktree: initialize return value for submodule_uses_worktrees
  submodule: add absorb-git-dir function
  move connect_work_tree_and_git_dir to dir.h
  worktree: check if a submodule uses worktrees
  test-lib-functions.sh: teach test_commit -C <dir>
  submodule helper: support super prefix
  submodule: use absolute path for computing relative path connecting
2017-01-10 15:24:28 -08:00
Junio C Hamano
9d540e9726 Merge branch 'bw/transport-protocol-policy'
Finer-grained control of what protocols are allowed for transports
during clone/fetch/push have been enabled via a new configuration
mechanism.

* bw/transport-protocol-policy:
  http: respect protocol.*.allow=user for http-alternates
  transport: add from_user parameter to is_transport_allowed
  http: create function to get curl allowed protocols
  transport: add protocol policy config option
  http: always warn if libcurl version is too old
  lib-proto-disable: variable name fix
2016-12-27 00:11:41 -08:00
Brandon Williams
f1762d772e transport: add protocol policy config option
Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to
specify a whitelist of protocols to be used in clone/fetch/push
commands.  This patch introduces new configuration options for more
fine-grained control for allowing/disallowing protocols.  This also has
the added benefit of allowing easier construction of a protocol
whitelist on systems where setting an environment variable is
non-trivial.

Now users can specify a policy to be used for each type of protocol via
the 'protocol.<name>.allow' config option.  A default policy for all
unconfigured protocols can be set with the 'protocol.allow' config
option.  If no user configured default is made git will allow known-safe
protocols (http, https, git, ssh, file), disallow known-dangerous
protocols (ext), and have a default policy of `user` for all other
protocols.

The supported policies are `always`, `never`, and `user`.  The `user`
policy can be used to configure a protocol to be usable when explicitly
used by a user, while disallowing it for commands which run
clone/fetch/push commands without direct user intervention (e.g.
recursive initialization of submodules).  Commands which can potentially
clone/fetch/push from untrusted repositories without user intervention
can export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent
protocols configured to the `user` policy from being used.

Fix remote-ext tests to use the new config to allow the ext
protocol to be tested.

Based on a patch by Jeff King <peff@peff.net>

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-15 09:29:13 -08:00
Stefan Beller
f6f8586140 submodule: add absorb-git-dir function
When a submodule has its git dir inside the working dir, the submodule
support for checkout that we plan to add in a later patch will fail.

Add functionality to migrate the git directory to be absorbed
into the superprojects git directory.

The newly added code in this patch is structured such that other areas of
Git can also make use of it. The code in the submodule--helper is a mere
wrapper and option parser for the function
`absorb_git_dir_into_superproject`, that takes care of embedding the
submodules git directory into the superprojects git dir. That function
makes use of the more abstract function for this use case
`relocate_gitdir`, which can be used by e.g. the worktree code eventually
to move around a git directory.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12 15:15:07 -08:00
Jeff King
72c5f88311 clone: pass --progress decision to recursive submodules
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.

In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.

That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.

This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.

This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:

  git clone --recursive --progress ... 2>&1 | cat

work by triggering progress explicitly in the children.

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 11:22:47 -07:00
Junio C Hamano
02c6c14d6c Merge branch 'sb/submodule-clone-rr'
"git clone --resurse-submodules --reference $path $URL" is a way to
reduce network transfer cost by borrowing objects in an existing
$path repository when cloning the superproject from $URL; it
learned to also peek into $path for presense of corresponding
repositories of submodules and borrow objects from there when able.

* sb/submodule-clone-rr:
  clone: recursive and reference option triggers submodule alternates
  clone: implement optional references
  clone: clarify option_reference as required
  clone: factor out checking for an alternate path
  submodule--helper update-clone: allow multiple references
  submodule--helper module-clone: allow multiple references
  t7408: merge short tests, factor out testing method
  t7408: modernize style
2016-09-08 21:49:50 -07:00
Stefan Beller
5f50f33e87 submodule--helper update-clone: allow multiple references
Allow the user to pass in multiple references to update_clone.
Currently this is only internal API, but once the shell script is
replaced by a C version, this is needed.

This fixes an API bug between the shell script and the helper.
Currently the helper accepts "--reference" "--reference=foo"
as a OPT_STRING whose value happens to be "--reference=foo", and
then uses

        if (suc->reference)
                argv_array_push(&child->args, suc->reference)

where suc->reference _is_ "--reference=foo" when invoking the
underlying "git clone", it cancels out.

With this change we omit one of the "--reference" arguments when
passing references from the shell script to the helper.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 15:00:17 -07:00
Junio C Hamano
11b53957ac Merge branch 'sb/submodule-update-dot-branch'
A few updates to "git submodule update".

Use of "| wc -l" break with BSD variant of 'wc'.

* sb/submodule-update-dot-branch:
  t7406: fix breakage on OSX
  submodule update: allow '.' for branch value
  submodule--helper: add remote-branch helper
  submodule-config: keep configured branch around
  submodule--helper: fix usage string for relative-path
  submodule update: narrow scope of local variable
  submodule update: respect depth in subsequent fetches
  t7406: future proof tests with hard coded depth
2016-08-10 12:33:20 -07:00
Junio C Hamano
6a024a249f Merge branch 'sb/submodule-deinit-all' into maint
A comment update for a topic that was merged to Git v2.8.

* sb/submodule-deinit-all:
  submodule deinit: remove outdated comment
2016-08-08 14:21:46 -07:00
Junio C Hamano
1e9a4856fb Merge branch 'sb/submodule-clone-retry'
An earlier tweak to make "submodule update" retry a failing clone
of submodules was buggy and caused segfault, which has been fixed.

* sb/submodule-clone-retry:
  submodule-helper: fix indexing in clone retry error reporting path
  git-submodule: forward exit code of git-submodule--helper more faithfully
2016-08-04 14:39:17 -07:00
Stefan Beller
92bbe7ccf1 submodule--helper: add remote-branch helper
In a later patch we want to enhance the logic for the branch selection.
Rewrite the current logic to be in C, so we can directly use C when
we enhance the logic.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-03 16:11:35 -07:00
Stefan Beller
341238ebc4 submodule update: narrow scope of local variable
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-01 14:41:51 -07:00
Stefan Beller
6cbf454a2e submodule update: respect depth in subsequent fetches
When depth is given the user may have a reasonable expectation that
any remote operation is using the given depth. Add a test to demonstrate
we still get the desired sha1 even if the depth is too short to
include the actual commit.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-01 14:41:02 -07:00
Junio C Hamano
b8307836d2 Merge branch 'sb/submodule-deinit-all'
A comment update for a topic that was merged to Git v2.8.

* sb/submodule-deinit-all:
  submodule deinit: remove outdated comment
2016-07-28 10:34:45 -07:00
Stefan Beller
406621f43d submodule deinit: remove outdated comment
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-26 13:52:14 -07:00
Johannes Sixt
c4c02bf16c git-submodule: forward exit code of git-submodule--helper more faithfully
git-submodule--helper is invoked as the upstream of a pipe in several
places. Usually, the failure of a program in this position is not
detected by the shell. For this reason, the code inserts a token in the
output stream when git-submodule--helper fails that is detected
downstream, where the shell script is quit with exit code 1.

There happens to be a bug in git-submodule--helper that leads to a
segmentation fault. The test suite triggers the crash in several places,
all of which are protected by 'test_must_fail'. But due to the inspecific
exit code 1, the crash remains undiagnosed.

Extend the failure protocol such that git-submodule--helper's exit code
is passed downstream (only in the case of failure). This enables the
downstream to use it as its own exit code, and 'test_must_fail' to
identify the segmentation fault as an unexpected failure.

The bug itself is fixed in the next commit.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-22 13:43:50 -07:00
Junio C Hamano
2703572b3a Merge branch 'va/i18n-even-more'
More markings of messages for i18n, with updates to various tests
to pass GETTEXT_POISON tests.

One patch from the original submission dropped due to conflicts
with jk/upload-pack-hook, which is still in flux.

* va/i18n-even-more: (38 commits)
  t5541: become resilient to GETTEXT_POISON
  i18n: branch: mark comment when editing branch description for translation
  i18n: unmark die messages for translation
  i18n: submodule: escape shell variables inside eval_gettext
  i18n: submodule: join strings marked for translation
  i18n: init-db: join message pieces
  i18n: remote: allow translations to reorder message
  i18n: remote: mark URL fallback text for translation
  i18n: standardise messages
  i18n: sequencer: add period to error message
  i18n: merge: change command option help to lowercase
  i18n: merge: mark messages for translation
  i18n: notes: mark options for translation
  i18n: notes: mark strings for translation
  i18n: transport-helper.c: change N_() call to _()
  i18n: bisect: mark strings for translation
  t5523: use test_i18ngrep for negation
  t4153: fix negated test_i18ngrep call
  t9003: become resilient to GETTEXT_POISON
  tests: unpack-trees: update to use test_i18n* functions
  ...
2016-07-13 11:24:10 -07:00
Junio C Hamano
bb2d8a817d Merge branch 'sb/submodule-clone-retry'
"git submodule update" that drives many "git clone" could
eventually hit flaky servers/network conditions on one of the
submodules; the command learned to retry the attempt.

* sb/submodule-clone-retry:
  submodule update: continue when a clone fails
  submodule--helper: initial clone learns retry logic
2016-07-11 10:31:04 -07:00
Junio C Hamano
3807098cd6 Merge branch 'sb/submodule-recommend-shallowness'
An upstream project can make a recommendation to shallowly clone
some submodules in the .gitmodules file it ships.

* sb/submodule-recommend-shallowness:
  submodule update: learn `--[no-]recommend-shallow` option
  submodule-config: keep shallow recommendation around
2016-06-20 11:01:01 -07:00
Junio C Hamano
de76eb69d2 Merge branch 'sb/submodule-misc-cleanups'
Minor simplification.

* sb/submodule-misc-cleanups:
  submodule update: make use of the existing fetch_in_submodule function
2016-06-20 11:01:01 -07:00
Vasco Almeida
c87302bfe4 i18n: submodule: escape shell variables inside eval_gettext
According to the gettext manual [1], references to shell variables inside
eval_gettext call must be escaped so that eval_gettext receives the
translatable string before the variable values are substituted into it.

[1] http://www.gnu.org/software/gettext/manual/html_node/Preparing-Shell-Scripts.html

Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-17 15:46:10 -07:00
Vasco Almeida
0d71dbfd50 i18n: submodule: join strings marked for translation
Join strings marked for translation since that would facilitate and
improve translations result.

Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-17 15:46:10 -07:00
Vasco Almeida
d323c6b641 i18n: git-sh-setup.sh: mark strings for translation
Positional arguments, such as $0, $1, etc, need to be stored on shell
variables for use in translatable strings, according to gettext manual
[1].

Add git-sh-setup.sh to LOCALIZED_SH variable in Makefile to enable
extraction of string marked for translation by xgettext.

Source git-sh-i18n in git-sh-setup.sh for gettext support.
git-sh-setup.sh is a shell library to be sourced by other shell scripts.
In order to avoid other scripts from sourcing git-sh-i18n twice, remove
line that sources it from them.  Not sourcing git-sh-i18n in any script
that uses gettext would lead to failure due to, for instance, gettextln
not being found.

[1] http://www.gnu.org/software/gettext/manual/html_node/Preparing-Shell-Scripts.html

Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-17 15:45:48 -07:00
Stefan Beller
bb9d91b4ed submodule update: continue when a clone fails
In 15ffb7cde4 (2011-06-13, submodule update: continue when a checkout
fails), we reasoned it is ok to continue, when there is not much of
a mental burden by the failure. If a recursive submodule fails to clone
because a .gitmodules file is broken (e.g. :
fatal: No url found for submodule path 'foo/bar' in .gitmodules
Failed to recurse into submodule path 'foo', signaled by exit code 128),
this is one of the cases where the user is not expected to have much of
a burden afterwards, so we can also continue in that case.

This means we only want to stop for updating submodules in case of rebase,
merge or custom update command failures, which are all signaled with
exit code 2.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 10:29:06 -07:00
Stefan Beller
44431df024 submodule: remove bashism from shell script
Junio pointed out `relative_path` was using bashisms via the
local variables. As the longer term goal is to rewrite most of the
submodule code in C, do it now.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 11:32:53 -07:00
Stefan Beller
b0f4b40846 submodule--helper: offer a consistent API
In 48308681 (2016-02-29, git submodule update: have a dedicated helper
for cloning), the helper communicated errors back only via exit code,
and dance with printing '#unmatched' in case of error was left to
git-submodule.sh as it uses the output of the helper and pipes it into
shell commands. This change makes the helper consistent by never
printing '#unmatched' in the helper but always handling these piping
issues in the actual shell script.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 11:31:49 -07:00
Stefan Beller
abed000aca submodule update: learn --[no-]recommend-shallow option
Sometimes the history of a submodule is not considered important by
the projects upstream. To make it easier for downstream users, allow
a boolean field 'submodule.<name>.shallow' in .gitmodules, which can
be used to recommend whether upstream considers the history important.

This field is honored in the initial clone by default, it can be
ignored by giving the `--no-recommend-shallow` option.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-27 10:40:46 -07:00
Stefan Beller
a87bcd6d47 submodule update: make use of the existing fetch_in_submodule function
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-25 15:39:50 -07:00
Junio C Hamano
e059388fb2 Merge branch 'jk/submodule-c-credential'
An earlier addition of "sanitize_submodule_env" with 14111fc4 (git:
submodule honor -c credential.* from command line, 2016-02-29)
turned out to be a convoluted no-op; implement what it wanted to do
correctly, and stop filtering settings given via "git -c var=val".

* jk/submodule-c-credential:
  submodule: stop sanitizing config options
  submodule: use prepare_submodule_repo_env consistently
  submodule--helper: move config-sanitizing to submodule.c
  submodule: export sanitized GIT_CONFIG_PARAMETERS
  t5550: break submodule config test into multiple sub-tests
  t5550: fix typo in $HTTPD_URL
2016-05-17 14:38:25 -07:00
Junio C Hamano
21b2e60400 Merge branch 'sb/submodule-deinit-all'
Correct faulty recommendation to use "git submodule deinit ." when
de-initialising all submodules, which would result in a strange
error message in a pathological corner case.

* sb/submodule-deinit-all:
  submodule deinit: require '--all' instead of '.' for all submodules
2016-05-17 14:38:20 -07:00
Junio C Hamano
f2c96ceb57 Merge branch 'sb/submodule-init'
Update of "git submodule" to move pieces of logic to C continues.

* sb/submodule-init:
  submodule init: redirect stdout to stderr
  submodule--helper update-clone: abort gracefully on missing .gitmodules
  submodule init: fail gracefully with a missing .gitmodules file
  submodule: port init from shell to C
  submodule: port resolve_relative_url from shell to C
2016-05-17 14:38:17 -07:00
Jeff King
89044baa8b submodule: stop sanitizing config options
The point of having a whitelist of command-line config
options to pass to submodules was two-fold:

  1. It prevented obvious nonsense like using core.worktree
     for multiple repos.

  2. It could prevent surprise when the user did not mean
     for the options to leak to the submodules (e.g.,
     http.sslverify=false).

For case 1, the answer is mostly "if it hurts, don't do
that". For case 2, we can note that any such example has a
matching inverted surprise (e.g., a user who meant
http.sslverify=true to apply everywhere, but it didn't).

So this whitelist is probably not giving us any benefit, and
is already creating a hassle as people propose things to put
on it. Let's just drop it entirely.

Note that we still need to keep a special code path for
"prepare the submodule environment", because we still have
to take care to pass through $GIT_CONFIG_PARAMETERS (and
block the rest of the repo-specific environment variables).

We can do this easily from within the submodule shell
script, which lets us drop the submodule--helper option
entirely (and it's OK to do so because as a "--" program, it
is entirely a private implementation detail).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06 12:54:27 -07:00
Stefan Beller
f6a5279977 submodule deinit: require '--all' instead of '.' for all submodules
The discussion in [1] pointed out that '.' is a faulty suggestion as
there is a corner case where it fails:

> "submodule deinit ." may have "worked" in the sense that you would
> have at least one path in your tree and avoided this "nothing
> matches" most of the time.  It would have still failed with the
> exactly same error if run in an empty repository, i.e.
>
>        $ E=/var/tmp/x/empty && rm -fr "$E" && mkdir -p "$E" && cd "$E"
>        $ git init
>        $ rungit v2.6.6 submodule deinit .
>        error: pathspec '.' did not match any file(s) known to git.
>        Did you forget to 'git add'?
>        $ >file && git add file
>        $ rungit v2.6.6 submodule deinit .
>        $ echo $?
>        0

So instead of a pathspec add the '--all' option to deinit all submodules
and add a test to check for the corner case of an empty repository.

The code only needs to learn about the '--all' option and doesn't
require further changes as `git submodule--helper list "$@"` will list
all submodules when "$@" is empty.

[1] http://news.gmane.org/gmane.comp.version-control.git/289535

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-05 14:51:26 -07:00
Jeff King
860cba61a3 submodule: export sanitized GIT_CONFIG_PARAMETERS
Commit 14111fc (git: submodule honor -c credential.* from
command line, 2016-02-29) taught git-submodule.sh to save
the sanitized value of $GIT_CONFIG_PARAMETERS when clearing
the environment for a submodule. However, it failed to
export the result, meaning that it had no effect for any
sub-programs.

We didn't catch this in our initial tests because we checked
only the "clone" case, which does not go through the shell
script at all. Provoking "git submodule update" to do a
fetch demonstrates the bug.

Noticed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-28 10:47:34 -07:00
Junio C Hamano
6a0f105a21 Merge branch 'sb/submodule-path-misc-bugs'
"git submodule" reports the paths of submodules the command
recurses into, but this was incorrect when the command was not run
from the root level of the superproject.

* sb/submodule-path-misc-bugs:
  t7407: make expectation as clear as possible
  submodule update: test recursive path reporting from subdirectory
  submodule update: align reporting path for custom command execution
  submodule status: correct path handling in recursive submodules
  submodule update --init: correct path handling in recursive submodules
  submodule foreach: correct path display in recursive submodules
2016-04-25 15:17:16 -07:00
Stefan Beller
3604242f08 submodule: port init from shell to C
By having the `submodule init` functionality in C, we can reference it
easier from other parts in the code in later patches. The code is split
up to have one function to initialize one submodule and a calling function
that takes care of the rest, such as argument handling and translating the
arguments to the paths of the submodules.

This is the first submodule subcommand that is fully converted to C
except for the usage string, so this is actually removing a call to
the `submodule--helper list` function, which is supposed to be used in
this transition. Instead we'll make a direct call to `module_list_compute`.

An explanation why we need to edit the prefixes in cmd_update in
git-submodule.sh in this patch:

By having no processing in the shell part, we need to convey the notion
of wt_prefix and prefix to the C parts, which former patches punted on
and did the processing of displaying path in the shell.

`wt_prefix` used to hold the path from the repository root to the current
directory, e.g. wt_prefix would be t/ if the user invoked the
`git submodule` command in ~/repo/t and ~repo is the GIT_DIR.

`prefix` used to hold the relative path from the repository root to the
operation, e.g. if you have recursive submodules, the shell script would
modify the `prefix` in each recursive step by adding the submodule path.

We will pass `wt_prefix` into the C helper via `git -C <dir>` as that
will setup git in the directory the user actually called git-submodule.sh
from. The `prefix` will be passed in via the `--prefix` option.

Having `prefix` and `wt_prefix` relative to the GIT_DIR of the
calling superproject is unfortunate with this patch as the C code doesn't
know about a possible recursion from a superproject via `submodule update
--init --recursive`.

To fix this, we change the meaning of `wt_prefix` to point to the current
project instead of the superproject and `prefix` to include any relative
paths issues in the superproject. That way `prefix` will become the leading
part for displaying paths and `wt_prefix` will be empty in recursive
calls for now.

The new notion of `wt_prefix` and `prefix` still allows us to reconstruct
the calling directory in the superproject by just traveling reverse of
`prefix`.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-16 23:45:18 -07:00
Stefan Beller
63e95beb08 submodule: port resolve_relative_url from shell to C
Later on we want to automatically call `git submodule init` from
other commands, such that the users don't have to initialize the
submodule themselves.  As these other commands are written in C
already, we'd need the init functionality in C, too.  The
`resolve_relative_url` function is a large part of that init
functionality, so start by porting this function to C.

To create the tests in t0060, the function `resolve_relative_url`
was temporarily enhanced to write all inputs and output to disk
when running the test suite. The added tests in this patch are
a small selection thereof.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-16 23:44:01 -07:00
Junio C Hamano
ee30f17805 Merge branch 'sb/submodule-path-misc-bugs' into sb/submodule-init
"git submodule" reports the paths of submodules the command
recurses into, but this was incorrect when the command was not run
from the root level of the superproject.

Any further comments?  Otherwise will merge to 'next'.

* sb/submodule-path-misc-bugs: (600 commits)
  t7407: make expectation as clear as possible
  submodule update: test recursive path reporting from subdirectory
  submodule update: align reporting path for custom command execution
  submodule status: correct path handling in recursive submodules
  submodule update --init: correct path handling in recursive submodules
  submodule foreach: correct path display in recursive submodules
  Git 2.8
  Documentation: fix git-p4 AsciiDoc formatting
  mingw: skip some tests in t9115 due to file name issues
  t1300: fix the new --show-origin tests on Windows
  t1300-repo-config: make it resilient to being run via 'sh -x'
  config --show-origin: report paths with forward slashes
  submodule: fix regression for deinit without submodules
  l10n: pt_PT: Update and add new translations
  l10n: ca.po: update translation
  Git 2.8-rc4
  Documentation: fix broken linkgit to git-config
  Documentation: use ASCII quotation marks in git-p4
  Revert "config.mak.uname: use clang for Mac OS X 10.6"
  git-compat-util: st_add4: work around gcc 4.2.x compiler crash
  ...
2016-04-14 12:47:45 -07:00
Junio C Hamano
01e1d54418 Merge branch 'jk/submodule-c-credential'
"git -c credential.<var>=<value> submodule" can now be used to
propagate configuration variables related to credential helper
down to the submodules.

* jk/submodule-c-credential:
  git_config_push_parameter: handle empty GIT_CONFIG_PARAMETERS
  git: submodule honor -c credential.* from command line
  quote: implement sq_quotef()
  submodule: fix segmentation fault in submodule--helper clone
  submodule: fix submodule--helper clone usage
  submodule: check argc count for git submodule--helper clone
  submodule: don't pass empty string arguments to submodule--helper clone
2016-04-06 11:39:12 -07:00
Junio C Hamano
bdebbeb334 Merge branch 'sb/submodule-parallel-update'
A major part of "git submodule update" has been ported to C to take
advantage of the recently added framework to run download tasks in
parallel.

* sb/submodule-parallel-update:
  clone: allow an explicit argument for parallel submodule clones
  submodule update: expose parallelism to the user
  submodule helper: remove double 'fatal: ' prefix
  git submodule update: have a dedicated helper for cloning
  run_processes_parallel: rename parameters for the callbacks
  run_processes_parallel: treat output of children as byte array
  submodule update: direct error message to stderr
  fetching submodules: respect `submodule.fetchJobs` config option
  submodule-config: drop check against NULL
  submodule-config: keep update strategy around
2016-04-06 11:39:01 -07:00
Stefan Beller
b08238ac3f submodule update: align reporting path for custom command execution
In the predefined actions (merge, rebase, none, checkout), we use
the display path, which is relative to the current working directory.
Also use the display path when running a custom command.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-30 13:09:36 -07:00
Stefan Beller
10450cf72b submodule status: correct path handling in recursive submodules
The new test which is a replica of the previous test except
that it executes from a sub directory. Prior to this patch
the test failed by having too many '../' prefixed:

  --- expect	2016-03-29 19:02:33.087336115 +0000
  +++ actual	2016-03-29 19:02:33.359343311 +0000
  @@ -1,7 +1,7 @@
    b23f134787d96fae589a6b76da41f4db112fc8db ../nested1 (heads/master)
  -+25d56d1ddfb35c3e91ff7d8f12331c2e53147dcc ../nested1/nested2 (file2)
  - 5ec83512b76a0b8170b899f8e643913c3e9b72d9 ../nested1/nested2/nested3 (heads/master)
  - 509f622a4f36a3e472affcf28fa959174f3dd5b5 ../nested1/nested2/nested3/submodule (heads/master)
  ++25d56d1ddfb35c3e91ff7d8f12331c2e53147dcc ../../nested1/nested2 (file2)
  + 5ec83512b76a0b8170b899f8e643913c3e9b72d9 ../../../nested1/nested2/nested3 (heads/master)
  + 509f622a4f36a3e472affcf28fa959174f3dd5b5 ../../../../nested1/nested2/nested3/submodule (heads/master)
    0c90624ab7f1aaa301d3bb79f60dcfed1ec4897f ../sub1 (0c90624)
    0c90624ab7f1aaa301d3bb79f60dcfed1ec4897f ../sub2 (0c90624)
    509f622a4f36a3e472affcf28fa959174f3dd5b5 ../sub3 (heads/master)

The path code in question:
  displaypath=$(relative_path "$prefix$sm_path")
  prefix=$displaypath
  if recursive:
    eval cmd_status

That way we change `prefix` each iteration to contain another
'../', because of the the relative_path computation is done
on an already computed relative path.

We must call relative_path exactly once with `wt_prefix` non empty.
Further calls in recursive instances to to calculate the displaypath
already incorporate the correct prefix from before. Fix the issue by
clearing `wt_prefix` in recursive calls.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-30 13:07:23 -07:00
Stefan Beller
c1ab00fb26 submodule update --init: correct path handling in recursive submodules
When calling `git submodule init` from a recursive instance of
`git submodule update --recursive`, the reported path is wrong as it
skips the nested submodules.

The new test demonstrates a failure in the code prior to this patch.
Instead of getting the expected
    Submodule 'submodule' (${pwd}/submodule) registered for path '../super/submodule'
the `super` directory is omitted and you get
    Submodule 'submodule' (${pwd}/submodule) registered for path '../submodule'
instead.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-30 13:06:05 -07:00
Stefan Beller
ea2fa1040d submodule foreach: correct path display in recursive submodules
The `prefix` was put in front of the display path unconditionally.
This is wrong as any relative path computation would need to be at
the front, so include the prefix into the display path.

The new test replicates the previous test with the difference of executing
from a sub directory. By executing from a sub directory all we would
expect all displayed paths to be prefixed by '../'.

Prior to this patch the test would report
    Entering 'nested1/nested2/../nested3'
instead of the expected
    Entering '../nested1/nested2/nested3'

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-30 13:03:57 -07:00
Jacob Keller
14111fc492 git: submodule honor -c credential.* from command line
Due to the way that the git-submodule code works, it clears all local
git environment variables before entering submodules. This is normally
a good thing since we want to clear settings such as GIT_WORKTREE and
other variables which would affect the operation of submodule commands.
However, GIT_CONFIG_PARAMETERS is special, and we actually do want to
preserve these settings. However, we do not want to preserve all
configuration as many things should be left specific to the parent
project.

Add a git submodule--helper function, sanitize-config, which shall be
used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs
except a small subset that are known to be safe and necessary.

Replace all the calls to clear_local_git_env with a wrapped function
that filters GIT_CONFIG_PARAMETERS using the new helper and then
restores it to the filtered subset after clearing the rest of the
environment.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 12:24:22 -08:00
Jacob Keller
d10e3b4260 submodule: don't pass empty string arguments to submodule--helper clone
When --reference or --depth are unused, the current git-submodule.sh
results in empty "" arguments appended to the end of the argv array
inside git submodule--helper clone. This is not caught because the argc
count is not checked today.

Fix git-submodule.sh to only pass an argument when --reference or
--depth are used, preventing the addition of two empty string arguments
on the tail of the argv array.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 12:23:58 -08:00