The fetch-pack/upload-pack protocol relies on the underlying transport
(local pipe or TCP socket) to have enough slack to allow one window worth
of data in flight without blocking the writer. Traditionally we always
relied on being able to have two windows of 32 "have"s in flight (roughly
3k bytes) to stream.
The recent "progressive-stride" change allows "fetch-pack" to send up to
1024 "have"s without reading any response from "upload-pack". The
outgoing pipe of "upload-pack" can be clogged with many ACK and NAK that
are unread, while "fetch-pack" is still stuffing its outgoing pipe with
more "have"s, leading to a deadlock.
Revert the change unless we are in stateless rpc (aka smart-http) mode, as
using a large window full of "have"s is still a good way to help reduce
the number of back-and-forth, and there is no buffering issue there (it is
strictly "ping-pong" without an overlap).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Start the initial request small by halving the INITIAL_FLUSH (we will try
to stay one window ahead of the server, so we would end up giving twice as
many "have" in flight at the very beginning). We may want to tweak these
values even more, taking MTU into account.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Shawn Pearce <spearce@spearce.org>
The client has to dig the history deeper when more recent parts of its
history do not have any overlap with the server it is fetching from. Make
the handshake window exponentially larger as we dig deeper, with a
reasonable upper cap.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Shawn Pearce <spearce@spearce.org>
The "git fetch" client presents the most recent 32 commits it has to the
server and gives a chance to the server to say "ok, we heard enough", and
continues reporting what it has in chunks of 32 commits, digging its
history down to older commits.
Move the hardcoded size of the handshake window outside the code, so that
we can tweak it more easily.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Shawn Pearce <spearce@spearce.org>
If enabled on the connection "multi_ack_detailed no-done" as a
pair allows the remote upload-pack process to send a PACK down
to the client as soon as a "ACK %s ready" message was also sent.
Over git:// and ssh:// where a bi-directional stream is in place
this has very little difference over the classical version that
waits for the client to send a "done\n" line by itself. It does
slightly reduce the latency involved to start the pack stream as
there is one less round-trip from client->server required.
Over smart HTTP this avoids needing to send a final RPC that has
all of the prior common objects. Instead the server is able to
return a pack as soon as its ready to. For many common users the
smart HTTP fetch is now just 2 requests: GET .../info/refs, and
a POST .../git-upload-pack to not only negotiate but also receive
the pack stream. Only users who have more than 32 local unshared
commits with the remote will need additional requests to negotiate
a common merge base.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If multi_ack_detailed was selected in the protocol capabilities
(both client and server are >= Git 1.6.6) the upload-pack side will
send "ACK %s ready" when it knows how to safely cut the graph and
produce a reasonable pack for the want list that was already sent
on the connection.
Upon receiving "ACK %s ready" there is no point in looking at
the remaining commits inside of rev_list. Sending additional
"have %s" lines to the remote will not construct a smaller pack.
It is unlikely a commit older than the current cut point will have
a better delta base than the cut point itself has.
The original design of this code had fetch-pack empty rev_list by
marking a commit and its transitive ancestors COMMON whenever the
remote side said "ACK %s {continue,common}" and skipping over any
already COMMON commits during get_rev(). This approach does not
work when most of rev_list is actually COMMON_REF, commits that
are pointed to by a reference on the remote, which exist locally,
and which have not yet been sent to the remote as a "have %s" line.
Most of the common references are tags in the ref/tags namespace,
using points in the commit graph that are more than 1 commit apart.
In git.git itself, this is currently 340 tags, 339 of which point to
commits in the commit graph. fetch-pack pushes all of these into
rev_list, but is unable to mark them COMMON and discard during a
remote's "ACK %s {continue,common}" because it does not parse through
the entire parent chain. Not parsing the entire parent chain is
an optimization to avoid walking back to the roots of the repository.
Assuming the client is only following the remote (and does not make
its own local commits), the client needs 11 rounds to spin through
the entire list of tags (32 commits per round, ceil(339/32) == 11).
Unfortunately the server knows on the first "have %s" line that
it can produce a good pack, and does not need to see the remaining
320 tags in the other 10 rounds.
Over git:// and ssh:// this isn't as bad as it sounds, the client is
only transmitting an extra 16,000 bytes that it doesn't need to send.
Over smart HTTP, the client must do an additional 10 HTTP POST
requests, each of which incurs round-trip latency, and must upload
the entire state vector of all known common objects. On the final
POST request, this is 16 KiB worth of data.
Fix all of this by clearing rev_list as soon as the remote side
says it can construct a pack.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "--xhtml" option is supported only in highlight < 3.0. There is no option
to enforce (X)HTML output format compatible with both highlight < 3.0 and
highlight >= 3.0. However default output format is HTML so we don't need to
explicitly specify it.
Signed-off-by: Adam Tkac <atkac@redhat.com>
Helped-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
rebase -i: clarify in-editor documentation of "exec"
tests: sanitize more git environment variables
fast-import: treat filemodify with empty tree as delete
rebase: give a better error message for bogus branch
rebase: use explicit "--" with checkout
Conflicts:
t/t9300-fast-import.sh
The hints in the current "instruction sheet" template look like so:
# Rebase 3f14246..a1d7e01 onto 3f14246
#
# Commands:
# p, pick = use commit
# r, reword = use commit, but edit the commit message
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
# f, fixup = like "squash", but discard this commit's log message
# x <cmd>, exec <cmd> = Run a shell command <cmd>, and stop if it fails
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#
This does not make it clear that the format of each line is
<insn> <commit id> <explanatory text that will be printed>
but the reader will probably infer that from the automatically
generated pick examples above it.
What about the "exec" instruction? By analogy, I might imagine that
the format of that line is "exec <command> <explanatory text>", and
the "x <cmd>" hint does not address that question (at first I read it
as taking an argument <cmd> that is the name of a shell). Meanwhile,
the mention of <cmd> makes the hints harder to scan as a table.
So remove the <cmd> and add some words to remind the reader that
"exec" runs a command named by the rest of the line. To make room, it
is left to the manpage to explain that that command is run using
$SHELL and that nonzero status from that command will pause the
rebase.
Wording from Junio.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These variables should generally not be set in one's
environment, but they do get set by rebase, which means
doing an interactive rebase like:
pick abcd1234 foo
exec make test
will cause false negatives in the test suite.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Normal git processes do not allow one to build a tree with an empty
subtree entry without trying hard at it. This is in keeping with the
general UI philosophy: git tracks content, not empty directories.
v1.7.3-rc0~75^2 (2010-06-30) changed that by making it easy to include
an empty subtree in fast-import's active commit:
M 040000 4b825dc642 subdir
One can trigger this by reading an empty tree (for example, the tree
corresponding to an empty root commit) and trying to move it to a
subtree. It is better and more closely analogous to 'git read-tree
--prefix' to treat such commands as requests to remove the subtree.
Noticed-by: David Barr <david.barr@cordelta.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When you give a non-existent branch to git-rebase, it spits
out the usage. This can be confusing, since you may
understand the usage just fine, but simply have made a
mistake in the branch name.
Before:
$ git rebase origin bogus
Usage: git rebase ...
After:
$ git rebase origin bogus
fatal: no such branch: bogus
Usage: git rebase ...
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the case of a ref/pathname conflict, checkout will
already do the right thing and checkout the ref. However,
for a non-existant ref, this has two advantages:
1. If a file with that pathname exists, rebase will
refresh the file from the index and then rebase the
current branch instead of producing an error.
2. If no such file exists, the error message using an
explicit "--" is better:
# before
$ git rebase -i origin bogus
error: pathspec 'bogus' did not match any file(s) known to git.
Could not checkout bogus
# after
$ git rebase -i origin bogus
fatal: invalid reference: bogus
Could not checkout bogus
The problems seem to be trigger-able only through "git
rebase -i", as regular git-rebase checks the validity of the
branch parameter as a ref very early on. However, it doesn't
hurt to be defensive.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/setup-fixes:
t1510: fix typo in the comment of a test
Documentation updates for 'GIT_WORK_TREE without GIT_DIR' historical usecase
Subject: setup: officially support --work-tree without --git-dir
tests: compress the setup tests
tests: cosmetic improvements to the repo-setup test
t/README: hint about using $(pwd) rather than $PWD in tests
Fix expected values of setup tests on Windows
The original intention of --work-tree was to allow people to work in a
subdirectory of their working tree that does not have an embedded .git
directory. Because their working tree, which their $cwd was in, did not
have an embedded .git, they needed to use $GIT_DIR to specify where it is,
and because this meant there was no way to discover where the root level
of the working tree was, so we needed to add $GIT_WORK_TREE to tell git
where it was.
However, this facility has long been (mis)used by people's scripts to
start git from a working tree _with_ an embedded .git directory, let git
find .git directory, and then pretend as if an unrelated directory were
the associated working tree of the .git directory found by the discovery
process. It happens to work in simple cases, and is not worth causing
"regression" to these scripts.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current treatment of "git reset --keep" emphasizes how it
differs from --hard (treatment of local changes) and how it breaks
down into plumbing (git read-tree -m -u HEAD <commit> followed by git
update-ref HEAD <commit>). This can discourage people from using
it, since it might seem to be a complex or niche option.
Better to emphasize what the --keep flag is intended for --- moving
the index and worktree from one commit to another, like "git checkout"
would --- so the reader can make a more informed decision about the
appropriate situations in which to use it.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The errno check added in commit 3ba7a06 "A loose object is not corrupt
if it cannot be read due to EMFILE" only checked for whether errno is
not ENOENT and thus incorrectly treated "no error" as an error
condition.
Because of that, it never reached the code path that would report that
the object is corrupted and instead caused funny errors like:
fatal: failed to read object 333c4768ce595793fdab1ef3a036413e2a883853: Success
So we have to extend the check to cover the case in which the object
file was successfully read, but its contents are corrupted.
Reported-by: Will Palmer <wmpalmer@gmail.com>
Signed-off-by: Björn Steinbrink <B.Steinbrink@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
New test helpers:
- setup_repo, to initialize a repository or gitfile pointing to a
repository, with core.bare and core.worktree set as specified;
- try_case, to run setup from a given directory and validate the
result, with GIT_DIR and GIT_WORK_TREE set as specified;
- try_repo, to initialize a repository and call "try_case" from the
toplevel and a subdirectory;
- run_wt_tests, to run a battery of tests that check for sane
behavior when GIT_WORK_TREE is set to various positions relative to
the .git dir and cwd.
Use these helpers to make the test shorter, less repetitive, and (one
hopes) easier to understand and modify.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Give an overview in "sh t1510-repo-setup.sh --help" output.
Waste some vertical and horizontal space for clearer code.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This definition was added by commit 77cb17e9, but it's left unused since
commit 511707d. Remove the left-over definition.
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* rj/maint-test-fixes:
t9501-*.sh: Fix a test failure on Cygwin
lib-git-svn.sh: Add check for mis-configured web server variables
lib-git-svn.sh: Avoid setting web server variables unnecessarily
t9142: Move call to start_httpd into the setup test
t3600-rm.sh: Don't pass a non-existent prereq to test #15
* ak/describe-exact:
describe: Delay looking up commits until searching for an inexact match
describe: Store commit_names in a hash table by commit SHA1
describe: Do not use a flex array in struct commit_name
describe: Use for_each_rawref
Omit needless words ("Additionally ... <path> may also" is redundant).
While at it, place the explanation of this special case after the
general rules for paths to provide the reader with some context.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In particular, on systems that define uint32_t as an unsigned long,
gcc complains as follows:
CC vcs-svn/svndump.o
vcs-svn/svndump.c: In function `svndump_read':
vcs-svn/svndump.c:215: warning: int format, uint32_t arg (arg 2)
In order to suppress the warning we use the C99 format specifier
macro PRIu32 from <inttypes.h>.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of stripping space characters past the beginning of the
line and overflowing a buffer, stop at the beginning of the line
(mimicking the corresponding fix in remote-fd).
The argument to isspace does not need to be cast explicitly because
git isspace takes care of that already.
Noticed-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As long as sizeof(struct ll_merge_options) is small, there is not
much reason not to keep a copy of the default merge options in the BSS
section. In return, we get clearer code and one less stack frame in
the opts == NULL case.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* rj/test-fixes:
t4135-*.sh: Skip the "backslash" tests on cygwin
t3032-*.sh: Do not strip CR from line-endings while grepping on MinGW
t3032-*.sh: Pass the -b (--binary) option to sed on cygwin
t6038-*.sh: Pass the -b (--binary) option to sed on cygwin
Conflicts:
t/t3032-merge-recursive-options.sh
