Commit Graph

79 Commits

Author SHA1 Message Date
Junio C Hamano
d96e31e390 Merge branch 'js/fetch-jobs'
"git fetch --jobs=<n>" allowed <n> parallel jobs when fetching
submodules, but this did not apply to "git fetch --multiple" that
fetches from multiple remote repositories.  It now does.

* js/fetch-jobs:
  fetch: let --jobs=<n> parallelize --multiple, too
2019-10-15 13:48:00 +09:00
Johannes Schindelin
d54dea77db fetch: let --jobs=<n> parallelize --multiple, too
So far, `--jobs=<n>` only parallelizes submodule fetches/clones, not
`--multiple` fetches, which is unintuitive, given that the option's name
does not say anything about submodules in particular.

Let's change that. With this patch, also fetches from multiple remotes
are parallelized.

For backwards-compatibility (and to prepare for a use case where
submodule and multiple-remote fetches may need different parallelization
limits), the config setting `submodule.fetchJobs` still only controls
the submodule part of `git fetch`, while the newly-introduced setting
`fetch.parallel` controls both (but can be overridden for submodules
with `submodule.fetchJobs`).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-06 07:35:58 +09:00
Corentin BOMPARD
24bc1a1292 pull, fetch: add --set-upstream option
Add the --set-upstream option to git pull/fetch
which lets the user set the upstream configuration
(branch.<current-branch-name>.merge and
branch.<current-branch-name>.remote) for the current branch.

A typical use-case is:

    git clone http://example.com/my-public-fork
    git remote add main http://example.com/project-main-repo
    git pull --set-upstream main master

or, instead of the last line:

    git fetch --set-upstream main master
    git merge # or git rebase

This is mostly equivalent to cloning project-main-repo (which sets
upsteam) and then "git remote add" my-public-fork, but may feel more
natural for people using a hosting system which allows forking from
the web UI.

This functionality is analog to "git push --set-upstream".

Signed-off-by: Corentin BOMPARD <corentin.bompard@etu.univ-lyon1.fr>
Signed-off-by: Nathan BERBEZIER <nathan.berbezier@etu.univ-lyon1.fr>
Signed-off-by: Pablo CHABANNE <pablo.chabanne@etu.univ-lyon1.fr>
Signed-off-by: Matthieu Moy <git@matthieu-moy.fr>
Patch-edited-by: Matthieu Moy <git@matthieu-moy.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-19 13:05:58 -07:00
Junio C Hamano
cde9a64ea3 Merge branch 'ds/fetch-disable-force-notice'
"git fetch" and "git pull" reports when a fetch results in
non-fast-forward updates to let the user notice unusual situation.
The commands learned "--no-shown-forced-updates" option to disable
this safety feature.

* ds/fetch-disable-force-notice:
  pull: add --[no-]show-forced-updates passthrough
  fetch: warn about forced updates in branch listing
  fetch: add --[no-]show-forced-updates argument
2019-07-09 15:25:46 -07:00
Derrick Stolee
cdbd70c437 fetch: add --[no-]show-forced-updates argument
After updating a set of remove refs during a 'git fetch', we walk the
commits in the new ref value and not in the old ref value to discover
if the update was a forced update. This results in two things happening
during the command:

 1. The line including the ref update has an additional "(forced-update)"
    marker at the end.

 2. The ref log for that remote branch includes a bit saying that update
    is a forced update.

For many situations, this forced-update message happens infrequently, or
is a small bit of information among many ref updates. Many users ignore
these messages, but the calculation required here slows down their fetches
significantly. Keep in mind that they do not have the opportunity to
calculate a commit-graph file containing the newly-fetched commits, so
these comparisons can be very slow.

Add a '--[no-]show-forced-updates' option that allows a user to skip this
calculation. The only permanent result is dropping the forced-update bit
in the reflog.

Include a new fetch.showForcedUpdates config setting that allows this
behavior without including the argument in every command. The config
setting is overridden by the command-line arguments.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-21 09:38:29 -07:00
Nguyễn Thái Ngọc Duy
c3d6b70338 fetch: only run 'gc' once when fetching multiple remotes
In multiple remotes mode, git-fetch is launched for n-1 remotes and the
last remote is handled by the current process. Each of these processes
will in turn run 'gc' at the end.

This is not really a problem because even if multiple 'gc --auto' is run
at the same time we still handle it correctly. It does show multiple
"auto packing in the background" messages though. And we may waste some
resources when gc actually runs because we still do some stuff before
checking the lock and moving it to background.

So let's try to avoid that. We should only need one 'gc' run after all
objects and references are added anyway. Add a new option --no-auto-gc
that will be used by those n-1 processes. 'gc --auto' will always run on
the main fetch process (*).

(*) even if we fetch remotes in parallel at some point in future, this
    should still be fine because we should "join" all those processes
    before this step.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19 20:56:44 -07:00
Jonathan Tan
6e98305985 clone: send server options when using protocol v2
Commit 5e3548ef16 ("fetch: send server options when using protocol v2",
2018-04-24) taught "fetch" the ability to send server options when using
protocol v2, but not "clone". This ability is triggered by "-o" or
"--server-option".

Teach "clone" the same ability, except that because "clone" already
has "-o" for another parameter, teach "clone" only to receive
"--server-option".

Explain in the documentation, both for clone and for fetch, that server
handling of server options are server-specific. This is similar to
receive-pack's handling of push options - currently, they are just sent
to hooks to interpret as they see fit.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 10:53:51 +09:00
Junio C Hamano
d39cab3989 Merge branch 'ab/fetch-tags-noclobber'
The rules used by "git push" and "git fetch" to determine if a ref
can or cannot be updated were inconsistent; specifically, fetching
to update existing tags were allowed even though tags are supposed
to be unmoving anchoring points.  "git fetch" was taught to forbid
updates to existing tags without the "--force" option.

* ab/fetch-tags-noclobber:
  fetch: stop clobbering existing tags without --force
  fetch: document local ref updates with/without --force
  push doc: correct lies about how push refspecs work
  push doc: move mention of "tag <tag>" later in the prose
  push doc: remove confusing mention of remote merger
  fetch tests: add a test for clobbering tag behavior
  push tests: use spaces in interpolated string
  push tests: make use of unused $1 in test description
  fetch: change "branch" to "reference" in --force -h output
2018-09-17 13:54:00 -07:00
Ævar Arnfjörð Bjarmason
ae6a470334 fetch: document local ref updates with/without --force
Refer to the new git-push(1) documentation about when ref updates are
and aren't allowed with and without --force, noting how "git-fetch"
differs from the behavior of "git-push".

Perhaps it would be better to split this all out into a new
gitrefspecs(7) man page, or present this information using tables.

In lieu of that, this is accurate, and fixes a big omission in the
existing refspec docs.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-31 14:04:06 -07:00
Ævar Arnfjörð Bjarmason
526608284a fetch doc: cross-link two new negotiation options
Users interested in the fetch.negotiationAlgorithm variable added in
42cc7485a2 ("negotiator/skipping: skip commits during fetch",
2018-07-16) are probably interested in the related --negotiation-tip
option added in 3390e42adb ("fetch-pack: support negotiation tip
whitelist", 2018-07-02).

Change the documentation for those two to reference one another to
point readers in the right direction.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-01 11:07:48 -07:00
Jonathan Tan
3390e42adb fetch-pack: support negotiation tip whitelist
During negotiation, fetch-pack eventually reports as "have" lines all
commits reachable from all refs. Allow the user to restrict the commits
sent in this way by providing a whitelist of tips; only the tips
themselves and their ancestors will be sent.

Both globs and single objects are supported.

This feature is only supported for protocols that support connect or
stateless-connect (such as HTTP with protocol v2).

This will speed up negotiation when the repository has multiple
relatively independent branches (for example, when a repository
interacts with multiple repositories, such as with linux-next [1] and
torvalds/linux [2]), and the user knows which local branch is likely to
have commits in common with the upstream branch they are fetching.

[1] https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next/
[2] https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux/

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-03 15:00:41 -07:00
Brandon Williams
5e3548ef16 fetch: send server options when using protocol v2
Teach fetch to optionally accept server options by specifying them on
the cmdline via '-o' or '--server-option'.  These server options are
sent to the remote end when performing a fetch communicating using
protocol version 2.

If communicating using a protocol other than v2 the provided options are
ignored and not sent to the remote end.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-24 11:24:40 +09:00
Ævar Arnfjörð Bjarmason
97716d217c fetch: add a --prune-tags option and fetch.pruneTags config
Add a --prune-tags option to git-fetch, along with fetch.pruneTags
config option and a -P shorthand (-p is --prune). This allows for
doing any of:

    git fetch -p -P
    git fetch --prune --prune-tags
    git fetch -p -P origin
    git fetch --prune --prune-tags origin

Or simply:

    git config fetch.prune true &&
    git config fetch.pruneTags true &&
    git fetch

Instead of the much more verbose:

    git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'

Before this feature it was painful to support the use-case of pulling
from a repo which is having both its branches *and* tags deleted
regularly, and have our local references to reflect upstream.

At work we create deployment tags in the repo for each rollout, and
there's *lots* of those, so they're archived within weeks for
performance reasons.

Without this change it's hard to centrally configure such repos in
/etc/gitconfig (on servers that are only used for working with
them). You need to set fetch.prune=true globally, and then for each
repo:

    git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"

Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well,
and users running "git pull" will automatically get the pruning
semantics I want.

Even though "git remote" has corresponding "prune" and "update
--prune" subcommands I'm intentionally not adding a corresponding
prune-tags or "update --prune --prune-tags" mode to that command.

It's advertised (as noted in my recent "git remote doc: correct
dangerous lies about what prune does") as only modifying remote
tracking references, whereas any --prune-tags option is always going
to modify what from the user's perspective is a local copy of the tag,
since there's no such thing as a remote tracking tag.

Ideally add_prune_tags_to_fetch_refspec() would be something that
would use ALLOC_GROW() to grow the 'fetch` member of the 'remote'
struct. Instead I'm realloc-ing remote->fetch and adding the
tag_refspec to the end.

The reason is that parse_{fetch,push}_refspec which allocate the
refspec (ultimately remote->fetch) struct are called many places that
don't have access to a 'remote' struct. It would be hard to change all
their callsites to be amenable to carry around the bookkeeping
variables required for dynamic allocation.

All the other callers of the API first incrementally construct the
string version of the refspec in remote->fetch_refspec via
add_fetch_refspec(), before finally calling parse_fetch_refspec() via
some variation of remote_get().

It's less of a pain to deal with the one special case that needs to
modify already constructed refspecs than to chase down and change all
the other callsites. The API I'm adding is intentionally not
generalized because if we add more of these we'd probably want to
re-visit how this is done.

See my "Re: [BUG] git remote prune removes local tags, depending on
fetch config" (87po6ahx87.fsf@evledraar.gmail.com;
https://public-inbox.org/git/87po6ahx87.fsf@evledraar.gmail.com/) for
more background info.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-09 13:10:13 -08:00
Ævar Arnfjörð Bjarmason
627a129b46 git-fetch & config doc: link to the new PRUNING section
Amend the documentation for fetch.prune, fetch.<name>.prune and
--prune to link to the recently added PRUNING section.

I'd have liked to link directly to it with "<<PRUNING>>" from
fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt
also includes it, but doesn't include that option). However making a
reference across files yields this error:

    [...]/Documentation/git-fetch.xml:226: element xref: validity
    error : IDREF attribute linkend references an unknown ID "PRUNING"

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-09 13:10:12 -08:00
Junio C Hamano
a460ea4a3c Merge branch 'nd/shallow-deepen'
The existing "git fetch --depth=<n>" option was hard to use
correctly when making the history of an existing shallow clone
deeper.  A new option, "--deepen=<n>", has been added to make this
easier to use.  "git clone" also learned "--shallow-since=<date>"
and "--shallow-exclude=<tag>" options to make it easier to specify
"I am interested only in the recent N months worth of history" and
"Give me only the history since that version".

* nd/shallow-deepen: (27 commits)
  fetch, upload-pack: --deepen=N extends shallow boundary by N commits
  upload-pack: add get_reachable_list()
  upload-pack: split check_unreachable() in two, prep for get_reachable_list()
  t5500, t5539: tests for shallow depth excluding a ref
  clone: define shallow clone boundary with --shallow-exclude
  fetch: define shallow boundary with --shallow-exclude
  upload-pack: support define shallow boundary by excluding revisions
  refs: add expand_ref()
  t5500, t5539: tests for shallow depth since a specific date
  clone: define shallow clone boundary based on time with --shallow-since
  fetch: define shallow boundary with --shallow-since
  upload-pack: add deepen-since to cut shallow repos based on time
  shallow.c: implement a generic shallow boundary finder based on rev-list
  fetch-pack: use a separate flag for fetch in deepening mode
  fetch-pack.c: mark strings for translating
  fetch-pack: use a common function for verbose printing
  fetch-pack: use skip_prefix() instead of starts_with()
  upload-pack: move rev-list code out of check_non_tip()
  upload-pack: make check_non_tip() clean things up on error
  upload-pack: tighten number parsing at "deepen" lines
  ...
2016-10-10 14:03:50 -07:00
Junio C Hamano
ce18123cec Merge branch 'mm/doc-tt'
More mark-up updates to typeset strings that are expected to
literally typed by the end user in fixed-width font.

* mm/doc-tt:
  doc: typeset HEAD and variants as literal
  CodingGuidelines: formatting HEAD in documentation
  doc: typeset long options with argument as literal
  doc: typeset '--' as literal
  doc: typeset long command-line options as literal
  doc: typeset short command-line options as literal
  Documentation/git-mv.txt: fix whitespace indentation
2016-07-13 11:24:14 -07:00
Matthieu Moy
bcf9626a71 doc: typeset long command-line options as literal
Similarly to the previous commit, use backquotes instead of
forward-quotes, for long options.

This was obtained with:

  perl -pi -e "s/'(--[a-z][a-z=<>-]*)'/\`\$1\`/g" *.txt

and manual tweak to remove false positive in ascii-art (o'--o'--o' to
describe rewritten history).

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-28 08:36:45 -07:00
Junio C Hamano
4764053815 Merge branch 'jk/fetch-prune-doc'
* jk/fetch-prune-doc:
  fetch: document that pruning happens before fetching
2016-06-27 09:56:44 -07:00
Jeff King
9e70233a17 fetch: document that pruning happens before fetching
This was changed in 10a6cc8 (fetch --prune: Run prune before
fetching, 2014-01-02), but it seems that nobody in that
discussion realized we were advertising the "after"
explicitly.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-14 10:56:27 -07:00
Nguyễn Thái Ngọc Duy
cccf74e2da fetch, upload-pack: --deepen=N extends shallow boundary by N commits
In git-fetch, --depth argument is always relative with the latest
remote refs. This makes it a bit difficult to cover this use case,
where the user wants to make the shallow history, say 3 levels
deeper. It would work if remote refs have not moved yet, but nobody
can guarantee that, especially when that use case is performed a
couple months after the last clone or "git fetch --depth". Also,
modifying shallow boundary using --depth does not work well with
clones created by --since or --not.

This patch fixes that. A new argument --deepen=<N> will add <N> more (*)
parent commits to the current history regardless of where remote refs
are.

Have/Want negotiation is still respected. So if remote refs move, the
server will send two chunks: one between "have" and "want" and another
to extend shallow history. In theory, the client could send no "want"s
in order to get the second chunk only. But the protocol does not allow
that. Either you send no want lines, which means ls-remote; or you
have to send at least one want line that carries deep-relative to the
server..

The main work was done by Dongcan Jiang. I fixed it up here and there.
And of course all the bugs belong to me.

(*) We could even support --deepen=<N> where <N> is negative. In that
case we can cut some history from the shallow clone. This operation
(and --depth=<shorter depth>) does not require interaction with remote
side (and more complicated to implement as a result).

Helped-by: Duy Nguyen <pclouds@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Dongcan Jiang <dongcan.jiang@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 14:38:16 -07:00
Nguyễn Thái Ngọc Duy
a45a260086 fetch: define shallow boundary with --shallow-exclude
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 14:38:16 -07:00
Nguyễn Thái Ngọc Duy
508ea88226 fetch: define shallow boundary with --shallow-since
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 14:38:16 -07:00
Junio C Hamano
e84d5e9fa1 Merge branch 'ew/force-ipv4'
"git fetch" and friends that make network connections can now be
told to only use ipv4 (or ipv6).

* ew/force-ipv4:
  connect & http: support -4 and -6 switches for remote operations
2016-02-24 13:25:54 -08:00
Eric Wong
c915f11eb4 connect & http: support -4 and -6 switches for remote operations
Sometimes it is necessary to force IPv4-only or IPv6-only operation
on networks where name lookups may return a non-routable address and
stall remote operations.

The ssh(1) command has an equivalent switches which we may pass when
we run them.  There may be old ssh(1) implementations out there
which do not support these switches; they should report the
appropriate error in that case.

rsync support is untouched for now since it is deprecated and
scheduled to be removed.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Reviewed-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-12 11:34:14 -08:00
Junio C Hamano
85705cfb57 Merge branch 'ss/clone-depth-single-doc'
Documentation for "git fetch --depth" has been updated for clarity.

* ss/clone-depth-single-doc:
  docs: clarify that --depth for git-fetch works with newly initialized repos
  docs: say "commits" in the --depth option wording for git-clone
  docs: clarify that passing --depth to git-clone implies --single-branch
2016-01-20 11:43:35 -08:00
Sebastian Schuberth
60253a605d docs: clarify that --depth for git-fetch works with newly initialized repos
The original wording sounded as if --depth could only be used to deepen or
shorten the history of existing repos. However, that is not the case. In a
workflow like

    $ git init
    $ git remote add origin https://github.com/git/git.git
    $ git fetch --depth=1

The newly initialized repo is properly created as a shallow repo.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-08 12:40:12 -08:00
Stefan Beller
62104ba14a submodules: allow parallel fetching, add tests and documentation
This enables the work of the previous patches.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-16 12:06:08 -08:00
Nguyễn Thái Ngọc Duy
da0005b885 *config.txt: stick to camelCase naming convention
This should improve readability. Compare "thislongname" and
"thisLongName". The following keys are left in unchanged. We can
decide what to do with them later.

 - am.keepcr
 - core.autocrlf .safecrlf .trustctime
 - diff.dirstat .noprefix
 - gitcvs.usecrlfattr
 - gui.blamehistoryctx .trustmtime
 - pull.twohead
 - receive.autogc
 - sendemail.signedoffbycc .smtpsslcertpath .suppresscc

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-13 22:13:46 -07:00
Junio C Hamano
c5558f80c3 fetch: allow explicit --refmap to override configuration
Since the introduction of opportunisitic updates of remote-tracking
branches, started at around f2690487 (fetch: opportunistically
update tracking refs, 2013-05-11) with a few updates in v1.8.4 era,
the remote.*.fetch configuration always kicks in even when a refspec
to specify what to fetch is given on the command line, and there is
no way to disable or override it per-invocation.

Teach the command to pay attention to the --refmap=<lhs>:<rhs>
command-line options that can be used to override the use of
configured remote.*.fetch as the refmap.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
2014-06-05 15:13:12 -07:00
Junio C Hamano
92251b1b5b Merge branch 'nd/shallow-clone'
Fetching from a shallow-cloned repository used to be forbidden,
primarily because the codepaths involved were not carefully vetted
and we did not bother supporting such usage. This attempts to allow
object transfer out of a shallow-cloned repository in a controlled
way (i.e. the receiver become a shallow repository with truncated
history).

* nd/shallow-clone: (31 commits)
  t5537: fix incorrect expectation in test case 10
  shallow: remove unused code
  send-pack.c: mark a file-local function static
  git-clone.txt: remove shallow clone limitations
  prune: clean .git/shallow after pruning objects
  clone: use git protocol for cloning shallow repo locally
  send-pack: support pushing from a shallow clone via http
  receive-pack: support pushing to a shallow clone via http
  smart-http: support shallow fetch/clone
  remote-curl: pass ref SHA-1 to fetch-pack as well
  send-pack: support pushing to a shallow clone
  receive-pack: allow pushes that update .git/shallow
  connected.c: add new variant that runs with --shallow-file
  add GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses
  receive/send-pack: support pushing from a shallow clone
  receive-pack: reorder some code in unpack()
  fetch: add --update-shallow to accept refs that update .git/shallow
  upload-pack: make sure deepening preserves shallow roots
  fetch: support fetching from a shallow repository
  clone: support remote shallow repository
  ...
2014-01-17 12:21:20 -08:00
Nguyễn Thái Ngọc Duy
48d25cae22 fetch: add --update-shallow to accept refs that update .git/shallow
The same steps are done as in when --update-shallow is not given. The
only difference is we now add all shallow commits in "ours" and
"theirs" to .git/shallow (aka "step 8").

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10 16:14:17 -08:00
Nguyễn Thái Ngọc Duy
79d3a236c5 upload-pack: make sure deepening preserves shallow roots
When "fetch --depth=N" where N exceeds the longest chain of history in
the source repo, usually we just send an "unshallow" line to the
client so full history is obtained.

When the source repo is shallow we need to make sure to "unshallow"
the current shallow point _and_ "shallow" again when the commit
reaches its shallow bottom in the source repo.

This should fix both cases: large <N> and --unshallow.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10 16:14:17 -08:00
Michael Haggerty
01ca90c2e5 fetch-options.txt: simplify ifdef/ifndef/endif usage
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-30 14:16:41 -07:00
Michael Haggerty
0838bf47b3 fetch --prune: prune only based on explicit refspecs
The old behavior of "fetch --prune" was to prune whatever was being
fetched.  In particular, "fetch --prune --tags" caused tags not only
to be fetched, but also to be pruned.  This is inappropriate because
there is only one tags namespace that is shared among the local
repository and all remotes.  Therefore, if the user defines a local
tag and then runs "git fetch --prune --tags", then the local tag is
deleted.  Moreover, "--prune" and "--tags" can also be configured via
fetch.prune / remote.<name>.prune and remote.<name>.tagopt, making it
even less obvious that an invocation of "git fetch" could result in
tag lossage.

Since the command "git remote update" invokes "git fetch", it had the
same problem.

The command "git remote prune", on the other hand, disregarded the
setting of remote.<name>.tagopt, and so its behavior was inconsistent
with that of the other commands.

So the old behavior made it too easy to lose tags.  To fix this
problem, change "fetch --prune" to prune references based only on
refspecs specified explicitly by the user, either on the command line
or via remote.<name>.fetch.  Thus, tags are no longer made subject to
pruning by the --tags option or the remote.<name>.tagopt setting.

However, tags *are* still subject to pruning if they are fetched as
part of a refspec, and that is good.  For example:

* On the command line,

      git fetch --prune 'refs/tags/*:refs/tags/*'

  causes tags, and only tags, to be fetched and pruned, and is
  therefore a simple way for the user to get the equivalent of the old
  behavior of "--prune --tag".

* For a remote that was configured with the "--mirror" option, the
  configuration is set to include

      [remote "name"]
              fetch = +refs/*:refs/*

  , which causes tags to be subject to pruning along with all other
  references.  This is the behavior that will typically be desired for
  a mirror.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-30 14:16:37 -07:00
Michael Haggerty
c5a84e92a2 fetch --tags: fetch tags *in addition to* other stuff
Previously, fetch's "--tags" option was considered equivalent to
specifying the refspec "refs/tags/*:refs/tags/*" on the command line;
in particular, it caused the remote.<name>.refspec configuration to be
ignored.

But it is not very useful to fetch tags without also fetching other
references, whereas it *is* quite useful to be able to fetch tags *in
addition to* other references.  So change the semantics of this option
to do the latter.

If a user wants to fetch *only* tags, then it is still possible to
specifying an explicit refspec:

    git fetch <remote> 'refs/tags/*:refs/tags/*'

Please note that the documentation prior to 1.8.0.3 was ambiguous
about this aspect of "fetch --tags" behavior.  Commit

    f0cb2f137c 2012-12-14 fetch --tags: clarify documentation

made the documentation match the old behavior.  This commit changes
the documentation to match the new behavior.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-30 14:16:36 -07:00
Junio C Hamano
0f93608bfe Merge branch 'rr/maint-fetch-tag-doc-asterisks'
* rr/maint-fetch-tag-doc-asterisks:
  fetch-options.txt: prevent a wildcard refspec from getting misformatted
2013-06-11 13:31:41 -07:00
Junio C Hamano
9eb4754d76 fetch-options.txt: prevent a wildcard refspec from getting misformatted
When explaining the "--tags" option as an equivalent to giving an
explicit "refs/tags/*:refs/tags/*" refspec, the two asterisks were
misinterpreted by AsciiDoc as a request to typeset the string
segment between them in bold.

We could fix it in two ways.  We can replace them with {asterisk}s
while keeping the string as body text, or we can mark it as a
literal string with backquotes around it.

Let's do the latter, as it is teaching the user an "exactly as
typed" alternative.

Noticed-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-07 08:22:37 -07:00
Nguyễn Thái Ngọc Duy
cfb70e1fa5 fetch: elaborate --depth action
--depth is explained as deepen, but the way it's applied, it can
shorten the history as well. Keen users may have noticed the
implication by the phrase "the specified number of commits from the
tip of each remote branch". Put "shorten" in the description to make
it clearer.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-11 09:11:10 -08:00
Nguyễn Thái Ngọc Duy
4dcb167fc3 fetch: add --unshallow for turning shallow repo into complete one
The user can do --depth=2147483647 (*) for restoring complete repo
now. But it's hard to remember. Any other numbers larger than the
longest commit chain in the repository would also do, but some
guessing may be involved. Make easy-to-remember --unshallow an alias
for --depth=2147483647.

Make upload-pack recognize this special number as infinite depth. The
effect is essentially the same as before, except that upload-pack is
more efficient because it does not have to traverse to the bottom
anymore.

The chance of a user actually wanting exactly 2147483647 commits
depth, not infinite, on a repository with a history that long, is
probably too small to consider. The client can learn to add or
subtract one commit to avoid the special treatment when that actually
happens.

(*) This is the largest positive number a 32-bit signed integer can
    contain. JGit and older C Git store depth as "int" so both are OK
    with this number. Dulwich does not support shallow clone.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-11 09:09:30 -08:00
Junio C Hamano
f0cb2f137c fetch --tags: clarify documentation
Explain that --tags is just like another explicit refspec on the
command line and as such overrides the default refspecs configured
via the remote.$name.fetch variable.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-13 16:19:45 -08:00
Philip Oakley
bc40deabbc Doc: shallow clone deepens _to_ new depth
Clarify that 'depth=' specifies the new depth from the remote's
branch tip. It does not add the depth to the existing shallow clone.
(details from pack-protocol.txt).
Clarify that tags are not fetched. (details from shallow.txt)

Signed-off-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-18 21:47:12 -07:00
Jens Lehmann
c16c3e40b5 fetch/pull: Don't recurse into a submodule when commits are already present
When looking for submodules where new commits have been recorded in the
superproject ignore those cases where the submodules commits are already
present locally. This can happen e.g. when the submodule has been rewound
to an earlier state. Then there is no need to fetch the submodule again
as the commit recorded in the newly fetched superproject commit has
already been fetched earlier into the submodule.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09 13:10:35 -08:00
Jens Lehmann
8f0700dd33 fetch/pull: Add the 'on-demand' value to the --recurse-submodules option
Until now the --recurse-submodules option could only be used to either
fetch all populated submodules recursively or to disable recursion
completely. As fetch and pull now by default just fetch those submodules
for which new commits have been fetched in the superproject, a command
line option to enforce that behavior is needed to be able to override
configuration settings.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09 13:10:35 -08:00
Jens Lehmann
88a21979c5 fetch/pull: recurse into submodules when necessary
To be able to access all commits of populated submodules referenced by the
superproject it is sufficient to only then let "git fetch" recurse into a
submodule when the new commits fetched in the superproject record new
commits for it. Having these commits present is extremely useful when
using the "--submodule" option to "git diff" (which is what "git gui" and
"gitk" do since 1.6.6), as all submodule commits needed for creating a
descriptive output can be accessed. Also merging submodule commits (added
in 1.7.3) depends on the submodule commits in question being present to
work. Last but not least this enables disconnected operation when using
submodules, as all commits necessary for a successful "git submodule
update -N" will have been fetched automatically. So we choose this mode as
the default for fetch and pull.

Before a new or changed ref from upstream is updated in update_local_ref()
"git rev-list <new-sha1> --not --branches --remotes" is used to determine
all newly fetched commits. These are then walked and diffed against their
parent(s) to see if a submodule has been changed. If that is the case, its
path is stored to be fetched after the superproject fetch is completed.

Using the "--recurse-submodules" or the "--no-recurse-submodules" option
disables the examination of the fetched refs because the result will be
ignored anyway.

There is currently no infrastructure for storing deleted and new
submodules in the .git directory of the superproject. That's why fetch and
pull for now only fetch submodules that are already checked out and are
not renamed.

In t7403 the "--no-recurse-submodules" argument had to be added to "git
pull" to avoid failure because of the moved upstream submodule repo.

Thanks-to: Jonathan Nieder <jrnieder@gmail.com>
Thanks-to: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09 13:10:35 -08:00
Jens Lehmann
7811d9600f pull: Document the "--[no-]recurse-submodules" options
In commits be254a0ea9 and 7dce19d374 the handling of the new fetch options
"--[no-]recurse-submodules" had been added to git-pull.sh. But they were
not documented as the pull options they now are, so let's fix that.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-02-07 15:19:09 -08:00
Junio C Hamano
4bb4d30095 Merge branch 'jl/fetch-submodule-recursive'
* jl/fetch-submodule-recursive:
  fetch_populated_submodules(): document dynamic allocation
  Submodules: Add the "fetchRecurseSubmodules" config option
  Add the 'fetch.recurseSubmodules' config setting
  fetch/pull: Add the --recurse-submodules option

Conflicts:
	builtin/fetch.c
2010-12-16 12:57:15 -08:00
Junio C Hamano
786f174dab Merge branch 'mm/phrase-remote-tracking'
* mm/phrase-remote-tracking:
  git-branch.txt: mention --set-upstream as a way to change upstream configuration
  user-manual: remote-tracking can be checked out, with detached HEAD
  user-manual.txt: explain better the remote(-tracking) branch terms
  Change incorrect "remote branch" to "remote tracking branch" in C code
  Change incorrect uses of "remote branch" meaning "remote-tracking"
  Change "tracking branch" to "remote-tracking branch"
  everyday.txt: change "tracking branch" to "remote-tracking branch"
  Change remote tracking to remote-tracking in non-trivial places
  Replace "remote tracking" with "remote-tracking"
  Better "Changed but not updated" message in git-status
2010-11-24 15:55:05 -08:00
Jens Lehmann
c1a3c3640d Submodules: Add the "fetchRecurseSubmodules" config option
The new boolean "fetchRecurseSubmodules" config option controls the
behavior for "git fetch" and "git pull". It specifies if these commands
should recurse into submodules and fetch new commits there too and can be
set separately for each submodule.

In the .gitmodules file "submodule.<name>.fetchRecurseSubmodules" entries
are read before looking for them in .git/config. Thus settings found in
.git/config will override those from .gitmodules, thereby allowing the
user to ignore settings given by the remote side while also letting
upstream set reasonable defaults for those users who don't have special
needs.

This configuration can be overridden by the command line option
"--[no-]recurse-submodules" of "git fetch" and "git pull".

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-12 15:06:03 -08:00
Jens Lehmann
be254a0ea9 Add the 'fetch.recurseSubmodules' config setting
This new boolean option can be used to override the default for "git
fetch" and "git pull", which is to not recurse into populated submodules
and fetch all new commits there too.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-12 15:06:03 -08:00
Jens Lehmann
7dce19d374 fetch/pull: Add the --recurse-submodules option
Until now you had to call "git submodule update" (without -N|--no-fetch
option) or something like "git submodule foreach git fetch" to fetch
new commits in populated submodules from their remote.

This could lead to "(commits not present)" messages in the output of
"git diff --submodule" (which is used by "git gui" and "gitk") after
fetching or pulling new commits in the superproject and is an obstacle for
implementing recursive checkout of submodules. Also "git submodule
update" cannot fetch changes when disconnected, so it was very easy to
forget to fetch the submodule changes before disconnecting only to
discover later that they are needed.

This patch adds the "--recurse-submodules" option to recursively fetch
each populated submodule from the url configured in the .git/config of the
submodule at the end of each "git fetch" or during "git pull" in the
superproject. The submodule paths are taken from the index.

The hidden option "--submodule-prefix" is added to "git fetch" to be able
to print out the full paths of nested submodules.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-12 15:06:03 -08:00