When cloning a repo with a --filter and with --recurse-submodules
enabled, the partial clone filter only applies to the top-level repo.
This can lead to unexpected bandwidth and disk usage for projects which
include large submodules. For example, a user might wish to make a
partial clone of Gerrit and would run:
`git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`.
However, only the superproject would be a partial clone; all the
submodules would have all blobs downloaded regardless of their size.
With this change, the same filter can also be applied to submodules,
meaning the expected bandwidth and disk savings apply consistently.
To avoid changing default behavior, add a new clone flag,
`--also-filter-submodules`. When this is set along with `--filter` and
`--recurse-submodules`, the filter spec is passed along to git-submodule
and git-submodule--helper, such that submodule clones also have the
filter applied.
This applies the same filter to the superproject and all submodules.
Users who need to customize the filter per-submodule would need to clone
with `--no-recurse-submodules` and then manually initialize each
submodule with the proper filter.
Applying filters to submodules should be safe thanks to Jonathan Tan's
recent work [1, 2, 3] eliminating the use of alternates as a method of
accessing submodule objects, so any submodule object access now triggers
a lazy fetch from the submodule's promisor remote if the accessed object
is missing. This patch is a reworked version of [4], which was created
prior to Jonathan Tan's work.
[1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16)
[2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate',
2021-09-20)
[3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules',
2021-10-25)
[4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/
Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When `remote.<name>.branch` is not configured, `git submodule update`
currently falls back to using the branch name `master`. A much better
idea, however, is to use the remote `HEAD`: on all Git servers running
reasonably recent Git versions, the symref `HEAD` points to the main
branch.
Note: t7419 demonstrates that there _might_ be use cases out there that
_expect_ `git submodule update --remote` to update submodules to the
remote `master` branch even if the remote `HEAD` points to another
branch. Arguably, this patch makes the behavior more intuitive, but
there is a slight possibility that this might cause regressions in
obscure setups.
Even so, it should be okay to fix this behavior without anything like a
longer transition period:
- The `git submodule update --remote` command is not really common.
- Current Git's behavior when running this command is outright
confusing, unless the remote repository's current branch _is_ `master`
(in which case the proposed behavior matches the old behavior).
- If a user encounters a regression due to the changed behavior, the fix
is actually trivial: setting `submodule.<name>.branch` to `master`
will reinstate the old behavior.
Helped-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git clone --recurse-submodules --single-branch" now uses the same
single-branch option when cloning the submodules.
* es/recursive-single-branch-clone:
clone: pass --single-branch during --recurse-submodules
submodule--helper: use C99 named initializer
Previously, performing "git clone --recurse-submodules --single-branch"
resulted in submodules cloning all branches even though the superproject
cloned only one branch. Pipe --single-branch through the submodule
helper framework to make it to 'clone' later on.
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
f0fd0dc5c5 (submodule foreach: document '$sm_path' instead of '$path',
2018-05-08) updated the documentation to advise callers to favor
$sm_path over the deprecated synonym $path. However, the example in
that section still uses $path. Update it to use $sm_path.
Signed-off-by: Kyle Meyer <kyle@kyleam.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The sentence wants to talk about the superproject's possesive, not plural form.
Signed-off-by: Thomas Menzel <dev@tomsit.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule status" and "git submodule status --cached" show
different things, but the documentation did not cover them
correctly, which has been corrected.
* mg/doc-submodule-status-cached:
doc: document 'git submodule status --cached'
'git submodule update' will fetch new commits from the submodule remote
if the SHA-1 recorded in the superproject is not found. This was not
mentioned in the documentation.
Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git submodule status --cached' reports the SHAs recorded in the
index of the superproject, instead of the SHAs that are checked out
in the submodule.
Signed-off-by: Manish Goregaokar <manishsmail@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, in the event that a submodule's upstream URL changes, users
have to manually alter the URL in the .gitmodules file then run
`git submodule sync`. Let's make that process easier.
Teach submodule the set-url subcommand which will automatically change
the `submodule.$name.url` property in the .gitmodules file and then run
`git submodule sync` to complete the process.
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In b57e8119e6 (submodule: teach set-branch subcommand, 2019-02-08), the
`set-branch` subcommand was added for submodules. When the documentation
was written, the syntax for a "index term" in AsciiDoc was
accidentally used. This caused the documentation to be rendered as
set-branch -d|--default)|(-b|--branch <branch> [--] <path>
instead of
set-branch ((-d|--default)|(-b|--branch <branch>)) [--] <path>
In addition to this, the original documentation was possibly confusing
as it made it seem as if the `-b` option didn't accept a `<branch>`
argument.
Break `--default` and `--branch` into their own separate invocations to
make it obvious that these options are mutually exclusive. Also, this
removes the surrounding parentheses so that the "index term" syntax is
not triggered.
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Clean-up markup in the documentation suite.
* cb/doco-mono:
doc: format pathnames and URLs as monospace.
doc/CodingGuidelines: URLs and paths as monospace
This teaches git-submodule the set-branch subcommand which allows the
branch of a submodule to be set through a porcelain command without
having to manually manipulate the .gitmodules file.
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Applying CodingGuidelines about monospace on pathnames and URLs.
See Documentation/CodingGuidelines.txt for more information.
Signed-off-by: Corentin BOMPARD <corentin.bompard@etu.univ-lyon1.fr>
Signed-off-by: Nathan BERBEZIER <nathan.berbezier@etu.univ-lyon1.fr>
Signed-off-by: Pablo CHABANNE <pablo.chabanne@etu.univ-lyon1.fr>
Signed-off-by: Matthieu MOY <matthieu.moy@univ-lyon1.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
submodule's default behavior wasn't documented in both git-submodule.txt
and in the usage text of git-submodule. Document the default behavior
similar to how git-remote does it.
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This behavior is mentioned in gitmodules.txt but not in
git-submodule.txt so we copy the information over so that it is not
missed.
Also, add the missed argument to the -b/--branch option.
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The bulk of "git submodule foreach" has been rewritten in C.
* pc/submodule-helper-foreach:
submodule: port submodule subcommand 'foreach' from shell to C
submodule foreach: document variable '$displaypath'
submodule foreach: document '$sm_path' instead of '$path'
submodule foreach: correct '$path' in nested submodules from a subdirectory
Use the obvious consensus of hyphenated "remote-tracking branch", and
fix an obvious typo, all in documentation and comments.
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule update" and "git submodule add" supported the
"--reference" option to borrow objects from a neighbouring local
repository like "git clone" does, but lacked the more recent
invention "--dissociate". Also "git submodule add" has been taught
to take the "--progress" option.
* cf/submodule-progress-dissociate:
submodule: add --dissociate option to add/update commands
submodule: add --progress option to add command
submodule: clean up substitutions in script
Add --dissociate option to add and update commands, both clone helper commands
that already have the --reference option --dissociate pairs with.
Signed-off-by: Casey Fitzpatrick <kcghost@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The '--progress' was introduced in 72c5f88311 (clone: pass --progress
decision to recursive submodules, 2016-09-22) to fix the progress reporting
of the clone command. Also add the progress option to the 'submodule add'
command. The update command already supports the progress flag, but it
is not documented.
Signed-off-by: Casey Fitzpatrick <kcghost@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It was observed that the variable '$displaypath' was accessible but
undocumented. Hence, document it.
Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As using a variable '$path' may be harmful to users due to
capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
use $path variable in eval_gettext string, 2012-04-17). Adjust
the documentation to advocate for using $sm_path, which contains
the same value. We still make the 'path' variable available and
document it as a deprecated synonym of 'sm_path'.
Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We tend to quote command line examples using `` to set them in a
monospace font. The immediate motivation for this patch is to get rid of
another instance of \--. As noted in the previous commits, \-- has a
tendency of rendering badly. Here, it renders ok (at least with
AsciiDoc 8.6.9 and Asciidoctor 1.5.4), but by getting rid of this
instance, we reduce the chances of \-- cropping up in places where it
matters more.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
While at it, correctly quote important words.
Signed-off-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch aims to detangle (a) the usage of `git-submodule`
from (b) the concept of submodules and (c) how the actual
implementation looks like, such as where they are configured
and (d) what the best practices are.
To do so, move the conceptual parts of the 'git-submodule'
man page to a new man page gitsubmodules(7). This new page
is just like gitmodules(5), gitattributes(5), gitcredentials(7),
gitnamespaces(7), gittutorial(7), which introduce a concept
rather than explaining a specific command.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "add" section for 'git-submodule' is redundant in its
description and the short synopsis line. Fix it.
Remove the redundant mentioning of the 'repository' argument
being mandatory.
The text is hard to read because of back-references, so remove
those.
Replace the word "humanish" by "canonical" as that conveys better
what we do to guess the path.
While at it, quote all occurrences of '.gitmodules' as that is an
important file in the submodule context, also link to it on its
first mention.
Helped-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach `submodule init` to initialize submodules which have been
configured to be active by setting 'submodule.active' with a pathspec.
Now if no path arguments are given and 'submodule.active' is configured,
`init` will initialize all submodules which have been configured to be
active. If no path arguments are given and 'submodule.active' is not
configured, then `init` will retain the old behavior of initializing all
submodules.
This allows users to record more complex patterns as it saves retyping
them whenever you invoke update.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a submodule is initialized, the config variable 'submodule.<name>.url'
is set depending on the value of the same variable in the .gitmodules
file. When the URL indicates to be relative, then the url is computed
relative to its default remote. The default remote cannot be determined
accurately in all cases, such that it falls back to 'origin'.
The 'origin' remote may not exist, though. In that case we give up looking
for a suitable remote and we'll just assume it to be a local relative path.
This can be confusing to users as there is a lot of guessing involved,
which is not obvious to the user.
So in the corner case of assuming a local autoritative truth, warn the
user to lessen the confusion.
This behavior was introduced in 4d6893200 (submodule add: allow relative
repository path even when no url is set, 2011-06-06), which shared the
code with submodule-init and then ported to C in 3604242f08 (submodule:
port init from shell to C, 2016-04-15).
In case of submodule-add, this behavior makes sense in some use cases[1],
however for submodule-init there does not seem to be an immediate obvious
use case to fall back to a local submodule. However there might be, so
warn instead of die here.
While adding the warning, also clarify the behavior of relative URLs in
the documentation.
[1] e.g. http://stackoverflow.com/questions/8721984/git-ignore-files-for-public-repository-but-not-for-private
"store a secret locally in a submodule, with no intention to publish it"
Reported-by: Shawn Pearce <spearce@spearce.org>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The documentation gives an example of the submodule foreach
command that uses both backticks and single-quotes. We stick
the whole thing inside "+" markers to make it monospace, but
the inside punctuation still needs escaping. We handle the
backticks with "{backtick}", and use backslash-escaping for
the single-quotes.
But we missed the escaping on the second quote. Fortunately,
asciidoc renders this unbalanced quote as we want (showing
the quote), but asciidoctor does not. We could fix it by
adding the missing backslash.
However, let's take a step back. Even when rendered
correctly, it's hard to read a long command stuck into the
middle of a paragraph, and the important punctuation is hard
to notice. Let's instead bump it into its own single-line
code block. That makes both the source and the rendered
result more readable, and as a bonus we don't have to worry
about quoting at all.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The documentation for the `git submodule update` command, repeats itself
for each update option, "This is done when <option> is given, or no
option is given and `submodule.<name>.update` is set to <string>.
Avoid these repetitive clauses by stating the command line options take
precedence over configured options.
Also add 'none' to the list of options instead of mentioning it in the
following running text and split the list into two parts, one that is
accessible via the command line and one that is only reachable via the
configuration variables.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When reading up on a subcommand of `git submodule <subcommand>`,
it is convenient to have its options nearby and not just at the
top of the man page. Add the options to each subcommand.
While at it, also document the `--checkout` option for `update`.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a submodule has its git dir inside the working dir, the submodule
support for checkout that we plan to add in a later patch will fail.
Add functionality to migrate the git directory to be absorbed
into the superprojects git directory.
The newly added code in this patch is structured such that other areas of
Git can also make use of it. The code in the submodule--helper is a mere
wrapper and option parser for the function
`absorb_git_dir_into_superproject`, that takes care of embedding the
submodules git directory into the superprojects git dir. That function
makes use of the more abstract function for this use case
`relocate_gitdir`, which can be used by e.g. the worktree code eventually
to move around a git directory.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
4d7bc52b17 ("submodule update: allow '.' for branch value",
2016-08-03) adopted from Gerrit a feature to set "." as a special
value of "submodule.<name>.branch" in .gitmodules file to indicate
that the tracking branch in the submodule should be the same as the
current branch in the superproject.
Update the documentation to describe this.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sometimes the history of a submodule is not considered important by
the projects upstream. To make it easier for downstream users, allow
a boolean field 'submodule.<name>.shallow' in .gitmodules, which can
be used to recommend whether upstream considers the history important.
This field is honored in the initial clone by default, it can be
ignored by giving the `--no-recommend-shallow` option.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Correct faulty recommendation to use "git submodule deinit ." when
de-initialising all submodules, which would result in a strange
error message in a pathological corner case.
* sb/submodule-deinit-all:
submodule deinit: require '--all' instead of '.' for all submodules
The discussion in [1] pointed out that '.' is a faulty suggestion as
there is a corner case where it fails:
> "submodule deinit ." may have "worked" in the sense that you would
> have at least one path in your tree and avoided this "nothing
> matches" most of the time. It would have still failed with the
> exactly same error if run in an empty repository, i.e.
>
> $ E=/var/tmp/x/empty && rm -fr "$E" && mkdir -p "$E" && cd "$E"
> $ git init
> $ rungit v2.6.6 submodule deinit .
> error: pathspec '.' did not match any file(s) known to git.
> Did you forget to 'git add'?
> $ >file && git add file
> $ rungit v2.6.6 submodule deinit .
> $ echo $?
> 0
So instead of a pathspec add the '--all' option to deinit all submodules
and add a test to check for the corner case of an empty repository.
The code only needs to learn about the '--all' option and doesn't
require further changes as `git submodule--helper list "$@"` will list
all submodules when "$@" is empty.
[1] http://news.gmane.org/gmane.comp.version-control.git/289535
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Expose possible parallelism either via the "--jobs" CLI parameter or
the "submodule.fetchJobs" setting.
By having the variable initialized to -1, we make sure 0 can be passed
into the parallel processing machine, which will then pick as many parallel
workers as there are CPUs.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git-submodule(1) is inconsistent. In the synopsis, it says:
git submodule [--quiet] sync [--recursive] [--] [<path>...]
The description of the sync does not mention --recursive, and the
description of --recursive says that it is only available for foreach,
update and status.
The option was introduced (82f49f294c, Teach --recursive to submodule
sync, 2012-10-26) a while ago, so let's document it, too.
Reported-by: Per Cederqvist <cederp@opera.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
