This patch concludes the series, which makes
git-fetch-pack/git-upload-pack negotiate a potentially better set of
common revs. It should make a difference when fetching from a repository
with a few branches.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The code used to call git-rev-list to enumerate the local revisions. A
disadvantage of that method was that git-rev-list, lacking a control apart
from the command line, would happily enumerate ancestors of acknowledged
common commits, which was just taking unnecessary bandwidth.
Therefore, do not use git-rev-list on the fetching side, but rather
construct the list on the go. Send the revisions starting from the local
heads, ignoring the revisions known to be common.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The current fetch/upload protocol works like this:
- client sends revs it wants to have via "want" messages
- client sends a flush message (message with len 0)
- client sends revs it has via "have" messages
- after one window (32 revs), a flush is sent
- after each subsequent window, a flush is sent, and an ACK/NAK is received.
(NAK means that server does not have any of the transmitted revs;
ACK sends also the sha1 of the rev server has)
- when the first ACK is received, client sends "done", and does not expect
any further messages
One special case, though:
- if no ACK is received (only NAK's), and client runs out of revs to send,
"done" is sent, and server sends just one more "NAK"
A smarter scheme, which actually has a chance to detect more than one
common rev, would be to send more than just one ACK. This patch implements
the server side of the following extension to the protocol:
- client sends at least one "want" message with "multi_ack" appended, like
"want 1234567890123456789012345678901234567890 multi_ack"
- if the server understands that extension, it will send ACK messages for all
revs it has, not just the first one
- server appends "continue" to the ACK messages like
"ACK 1234567890123456789012345678901234567890 continue"
until it has MAX_HAS-1 revs. In this manner, client knows when to
stop sending revs by checking for the substring "continue" (and
further knows that server understands multi_ack)
In this manner, the protocol stays backwards compatible, since both client
must send "want ... multi_ack" and server must answer with "ACK ...
continue" to enable the extension.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This patch is based on Junio's proposal. It marks parents of common revs
so that they do not clutter up the has_sha1 array.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds a very git specific restricted shell, that can be
added to /etc/shells and set to the pw_shell in the /etc/passwd
file, to give users ability to push into repositories over ssh
without giving them full interactive shell acount.
[jc: I updated Linus' patch to match what the current sq_quote()
does.]
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
What we list as "Ignored files" are not "ignored". Rather, it
is the list of "not listed in the to-be-ignored files, but
exists -- you may be forgetting to add them".
Pointed out by Daniel.
Signed-off-by: Junio C Hamano <junkio@cox.net>
It supersedes git-rename by adding functionality to move multiple
files, directories or symlinks into another directory. It also
provides according documentation.
The implementation renames multiple files, using the arguments from
the command line to produce an array of sources and destinations. In
a first pass, all requested renames are checked for errors, and
overwriting of existing files is only allowed with '-f'. The actual
renaming is done in a second pass. This ensures that any error
condition is checked before anything is changed.
Signed-off-by: Josef Weidendorfer <Josef.Weidendorfer@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
git-http-fetch spits out curl 404 error message when unable to fetch an object,
but that's confusing since no error really happened and the object is usually
found in a pack it tries right after that. And if the object still cannot be
retrieved, it will say another error message anyway. OTOH other HTTP errors
(403 etc) are likely fatal and the user should be still informed about them.
Signed-off-by: Petr Baudis <pasky@suse.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is what the recent git-rev-list changes have all been gearing up for.
When we use a path filter to git-rev-list, the new "--dense" flag asks
git-rev-list to compress the history so that it _only_ contains commits
that change files in the path filter. It also rewrites the parent
information so that tools like "gitk" will see the result as a dense
history tree.
For example, on the current kernel archive:
[torvalds@g5 linux]$ git-rev-list HEAD | wc -l
9904
[torvalds@g5 linux]$ git-rev-list HEAD -- kernel | wc -l
5442
[torvalds@g5 linux]$ git-rev-list --dense HEAD -- kernel | wc -l
356
which shows that while we have almost ten thousand commits, we can prune
down the work to slightly more than half by only following the merges
that are interesting. But further, we can then compress the history to
just 356 entries that actually make changes to the kernel subdirectory.
To see this in action, try something like
gitk --dense -- gitk
to see just the history that affects gitk. Or, to show that true
parallel development still remains parallel, do
gitk --dense -- daemon.c
which shows some parallel commits in the current git tree.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is the first cut at a git-rev-list that knows to ignore commits that
don't change a certain file (or set of files).
NOTE! For now it only prunes _merge_ commits, and follows the parent where
there are no differences in the set of files specified. In the long run,
I'd like to make it re-write the straight-line history too, but for now
the merge simplification is much more fundamentally important (the
rewriting of straight-line history is largely a separate simplification
phase, but the merge simplification needs to happen early if we want to
optimize away unnecessary commit parsing).
If all parents of a merge change some of the files, the merge is left as
is, so the end result is in no way guaranteed to be a linear history, but
it will often be a lot /more/ linear than the full tree, since it prunes
out parents that didn't matter for that set of files.
As an example from the current kernel:
[torvalds@g5 linux]$ git-rev-list HEAD | wc -l
9885
[torvalds@g5 linux]$ git-rev-list HEAD -- Makefile | wc -l
4084
[torvalds@g5 linux]$ git-rev-list HEAD -- drivers/usb | wc -l
5206
and you can also use 'gitk' to more visually see the pruning of the
history tree, with something like
gitk -- drivers/usb
showing a simplified history that tries to follow the first parent in a
merge that is the parent that fully defines drivers/usb/.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This makes the tree diff functionality independent of the "git-diff-tree"
program, by splitting the core functionality up into a library file.
This will be needed for when we teach git-rev-list to only follow a
specified set of pathnames, rather than the global revision history.
Most of it is a fairly straightforward code move, but it also involves
some calling convention cleanup, and moving some of the static variables
from diff-tree.c into the options structure.
The actual tree change callback routines also become paramterized by the
diff_options structure, allowing the library functionality to do something
else than just show the diff on stdout.
Right now the only user of this functionality remains git-diff-tree
itself.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Martin Langhoff wants to use git-merge from outside git-pull and wants
to do further processing; for this, he wants git-merge no to commit
even when it cleanly merges. I think other script writers would want
something like that as well, so here it is.
Instead of the "merge commit message" parameter (which usually is made
for you by "git-pull" which calls this command), you pass an empty
string to it. Then it will not update your HEAD -- you can do whatever
you want with the resulting index file, which contains the merge results.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Later round would further improve fetch-pack not to send useless "have",
but in the meantime, increase it to help upload-pack to find more common
commits, as discussed on the list.
Signed-off-by: Junio C Hamano <junkio@cox.net>
New options --timeout, --init-timeout, --export-all and whitelist support
were added to git-daemon, but noone bothered to also add the proper
documentation. This patch aims to fix that.
Signed-off-by: Petr Baudis <pasky@suse.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The git-am script is nowhere called and nowhere (including itself)
explained, and the name isn't helpful either. For those like me who will
wonder what is it about, add some documentation stub for it to the
documentation.
I probably got something wrong and I don't feel like investigating all the
options - this is just kind of "emergency" docs.
Signed-off-by: Petr Baudis <pasky@suse.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If rev-parse output includes both flags and files, we should pass on any
"--" marker we see, so that the end result can also tell the difference
between a flag and a filename that begins with '-'.
[jc: merged a later one liner updates from Linus]
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Currently, the code would use getdomainname() call, which however returns
something usually unset and not necessarily related at all to the DNS
domain name (it seems to be mostly some scary NIS/YP thing).
This patch changes the code to actually use the DNS domain name, which is
also what tends to be used in emails, and we aim at emails with our ident
code.
Signed-off-by: Petr Baudis <pasky@suse.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Since git-cherry-pick is simply a copy of git-revert, it can be created
before installing (so that it can be used without installing, too).
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
With the '0' timeout given to poll, it returns instantly without any
events on my system, causing git-daemon to consume all the CPU time. Use
-1 as the timeout so poll() only returns in case of EINTR or actually
events being available.
Signed-off-by: Jens Axboe <axboe@suse.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Sorry for the mistake of rewinding something already pushed out.
This recovers the side branch lost by that mistake, specifically
ea5a65a599 commit.
Signed-off-by: Junio C Hamano <junio@hera.kernel.org>
Martin Langhoff noticed that ref^0 barfed correctly when we did not
have the commit in a broken repository, but ref^{commit} didn't.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Also Johannes noticed we use parse_object to look up if we know that
object already -- we should just ask the in-core object registry with
lookup_object() for that.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The documentation was lazily sharing the argument description across these
commands.
Lazy may be a way of life, but that does not justify confusing others ;-).
Signed-off-by: Junio C Hamano <junkio@cox.net>
The previous round to optimize fetch-pack has a small bug that
feeds SHA1^ ("parent commit") before making sure SHA1 is
actually a commit (or a tag that eventually dereferences to a
commit). Also it did not help culling the known-to-be-common
parents if the common one was a merge.
Signed-off-by: Junio C Hamano <junkio@cox.net>
It was all good and well to check if all remote refs are complete (local
refs or descendants thereof), but we can just as easily use the same
information to avoid sending "want" lines just for the complete objects in
the case that not all remote refs are complete (or their names differ).
Also, git-fetch-pack does not have to ask for descendants of remote refs
which are complete (for now, git-rev-list is told to ignore only the first
parent). That change also eliminates a code path where a popen()ed handle
was not pclose()ed.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This deprecates --keep and -q flags and always keeps the pack
sent from the remote site. Corresponding configuration
variables are also removed.
Signed-off-by: Junio C Hamano <junkio@cox.net>
On top of optimization by Linus not to ask refs that already match, we
can walk our refs and not issue "want" for things that are known to be
reachable from them.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Add configuration settings to abort HTTP requests if the transfer rate
drops below a threshold for a specified length of time. Environment
variables override config file settings.
Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
It turns out that not only did git-daemon do DWIM, but git-upload-pack
does as well. This is bad; security checks have to be performed *after*
canonicalization, not before.
Additionally, the current git-daemon can be trivially DoSed by spewing
SYNs at the target port.
This patch adds a --strict option to git-upload-pack to disable all
DWIM, a --timeout option to git-daemon and git-upload-pack, and an
--init-timeout option to git-daemon (which is typically set to a much
lower value, since the initial request should come immediately from the
client.)
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
On top of optimization by Linus not to ask refs that already match, we
can walk our refs and not issue "want" for things that are known to be
reachable from them.
Signed-off-by: Junio C Hamano <junkio@cox.net>
I took a look at webgit, and it looks like at least for the "projects"
page, the most common operation ends up being basically
git-rev-list --header --parents --max-count=1 HEAD
Now, the thing is, the way "git-rev-list" works, it always keeps on
popping the parents and parsing them in order to build the list of
parents, and it turns out that even though we just want a single commit,
git-rev-list will invariably look up _three_ generations of commits.
It will parse:
- the commit we want (it obviously needs this)
- it's parent(s) as part of the "pop_most_recent_commit()" logic
- it will then pop one of the parents before it notices that it doesn't
need any more
- and as part of popping the parent, it will parse the grandparent (again
due to "pop_most_recent_commit()".
Now, I've strace'd it, and it really is pretty efficient on the whole, but
if things aren't nicely cached, and with long-latency IO, doing those two
extra objects (at a minimum - if the parent is a merge it will be more) is
just wasted time, and potentially a lot of it.
So here's a quick special-case for the trivial case of "just one commit,
and no date-limits or other special rules".
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This patch adds some extra paranoia to the git-daemon filename test. In
particular, it now rejects pathnames containing //; it also adds a
redundant test for pathname absoluteness (belts and suspenders.)
A single / at the end of the path is still permitted, however, and the
.git and /.git append DWIM stuff is now handled in an integrated manner,
which means the resulting path will always be subjected to pathname checks.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
