We recently introduced a write_in_full() which would either write
the specified object or emit an error message and fail. In order
to fix the read side we now want to introduce a read_in_full()
but without an error emit. This patch cleans up the naming
of this family of calls:
1) convert the existing write_or_whine() to write_or_whine_pipe()
to better indicate its pipe specific nature,
2) convert the existing write_in_full() calls to write_or_whine()
to better indicate its nature,
3) introduce a write_in_full() providing a write or fail semantic,
and
4) convert write_or_whine() and write_or_whine_pipe() to use
write_in_full().
Signed-off-by: Andy Whitcroft <apw@shadowen.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
When passing the revisions list to pack-objects we do not check for
errors nor short writes. Introduce a new write_in_full which will
handle short writes and report errors to the caller. Use this to
short cut the send on failure, allowing us to wait for and report
the child in case the failure is its fault.
Signed-off-by: Andy Whitcroft <apw@shadowen.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
send-pack builds a pipeline that runs "rev-list | pack-objects"
and sends the output from pack-objects to the other side, while
feeding the input side of that pipe from itself. However, the
file descriptor that is given to this pipeline (so that it can
be dup2(2)'ed into file descriptor 1 of pack-objects) is closed
by the caller before the complex fork+exec dance! Worse yet,
the caller already dup2's it to 1, so the child process did not
even have to.
I do not understand how this code could possibly have been
working, but it somehow was working by accident.
Merging the sliding mmap() code reveals this problem, presumably
because it keeps one extra file descriptor open for a packfile
and changes the way file descriptors are allocated. I am too
tired to diagnose the problem now, but this seems to be a
sensible fix.
Signed-off-by: Junio C Hamano <junkio@cox.net>
"git push $URL HEAD~6" created a bogus ref HEAD~6 immediately
under $GIT_DIR of the remote repository. While we should keep
refspecs that have arbitrary extended SHA-1 expression on the
source side working (e.g. "HEAD~6:refs/tags/yesterday"), we
should not create bogus ref on the other end.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This allows you to say
git send-pack $URL :refs/heads/$branch
to delete the named remote branch. The refspec $src:$dst means
replace the destination ref with the object known as $src on the
local side, so this is a natural extension to make an empty $src
mean "No object" to delete the target.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Currently git-push displays progress status for the local packing of
objects to send, but nothing once it starts to push it over the
connection. Having progress status in that later case is especially
nice when pushing lots of objects over a slow network link.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds a "int *flag" parameter to resolve_ref() and makes
for_each_ref() family to call callback function with an extra
"int flag" parameter. They are used to give two bits of
information (REF_ISSYMREF and REF_ISPACKED) about the ref.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a long overdue fix to the API for for_each_ref() family
of functions. It allows the callers to specify a callback data
pointer, so that the caller does not have to use static
variables to communicate with the callback funciton.
The updated for_each_ref() family takes a function of type
int (*fn)(const char *, const unsigned char *, void *)
and a void pointer as parameters, and calls the function with
the name of the ref and its SHA-1 with the caller-supplied void
pointer as parameters.
The commit updates two callers, builtin-name-rev.c and
builtin-pack-refs.c as an example.
Signed-off-by: Junio C Hamano <junkio@cox.net>
When we are generating packs to update remote repositories we
want to supply as much information as possible about the revisions
that already exist to rev-list in order optimise the pack as much
as possible. We need to pass two revisions for each branch we are
updating in the remote repository and one for each additional branch.
Where the remote repository has numerous branches we can run out
of command line space to pass them.
Utilise the git-rev-list --stdin mode to allow unlimited numbers
of revision constraints. This allows us to move back to the much
simpler unordered revision selection code.
[jc: added some comments in the code to describe the pipe flow
a bit.]
Signed-off-by: Andy Whitcroft <apw@shadowen.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This abstracts away the size of the hash values when copying them
from memory location to memory location, much as the introduction
of hashcmp abstracted away hash value comparsion.
A few call sites were using char* rather than unsigned char* so
I added the cast rather than open hashcpy to be void*. This is a
reasonable tradeoff as most call sites already use unsigned char*
and the existing hashcmp is also declared to be unsigned char*.
[jc: Splitted the patch to "master" part, to be followed by a
patch for merge-recursive.c which is not in "master" yet.
Fixed the cast in the latter hunk to combine-diff.c which was
wrong in the original.
Also converted ones left-over in combine-diff.c, diff-lib.c and
upload-pack.c ]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Introduces global inline:
hashcmp(const unsigned char *sha1, const unsigned char *sha2)
Uses memcmp for comparison and returns the result based on the length of
the hash name (a future runtime decision).
Acked-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
[jc: I needed to hand merge the changes to the updated codebase,
so the result needs to be checked.]
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This updates the type-enumeration constants introduced to reduce
the memory footprint of "struct object" to match the type bits
already used in the packfile format, by removing the former
(i.e. TYPE_* constant macros) and using the latter (i.e. enum
object_type) throughout the code for consistency.
Eventually we can stop passing around the "type strings"
entirely, and this will help - no confusion about two different
integer enumeration.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This makes git-peek-remote able to basically do everything that
git-ls-remote does (but obviously just for the native protocol, so no
http[s]: or rsync: support).
The default behaviour is the same, but you can now give a mixture of
"--refs", "--tags" and "--heads" flags, where "--refs" forces
git-peek-remote to only show real refs (ie none of the fakey tag lookups,
but also not the special pseudo-refs like HEAD and MERGE_HEAD).
The "--tags" and "--heads" flags respectively limit the output to just
regular tags and heads, of course.
You can still also ask to limit them by name too.
You can combine the flags, so
git peek-remote --refs --tags .
will show all local _true_ tags, without the generated tag lookups
(compare the output without the "--refs" flag).
And "--tags --heads" will show both tags and heads, but will avoid (for
example) any special refs outside of the standard locations.
I'm also planning on adding a "--ignore-local" flag that allows us to ask
it to ignore any refs that we already have in the local tree, but that's
an independent thing.
All this is obviously gearing up to making "git fetch" cheaper.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This shrinks "struct object" by a small amount, by getting rid of the
"struct type *" pointer and replacing it with a 3-bit bitfield instead.
In addition, we merge the bitfields and the "flags" field, which
incidentally should also remove a useless 4-byte padding from the object
when in 64-bit mode.
Now, our "struct object" is still too damn large, but it's now less
obviously bloated, and of the remaining fields, only the "util" (which is
not used by most things) is clearly something that should be eventually
discarded.
This shrinks the "git-rev-list --all" memory use by about 2.5% on the
kernel archive (and, perhaps more importantly, on the larger mozilla
archive). That may not sound like much, but I suspect it's more on a
64-bit platform.
There are other remaining inefficiencies (the parent lists, for example,
probably have horrible malloc overhead), but this was pretty obvious.
Most of the patch is just changing the comparison of the "type" pointer
from one of the constant string pointers to the appropriate new TYPE_xxx
small integer constant.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This makes sure that many commands that take refs on the command
line to honor core.warnambiguousrefs configuration. Earlier,
the commands affected by this patch did not read the
configuration file.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Mark Wooding noticed there was a type mismatch warning in git.c; this
patch does things slightly differently (mostly tightening const) and
was what I was holding onto, waiting for the setup-revisions change
to be merged into the master branch.
Signed-off-by: Junio C Hamano <junkio@cox.net>
* jc/rev-list:
rev-list --objects: use full pathname to help hashing.
rev-list --objects-edge: remove duplicated edge commit output.
rev-list --objects-edge
* jc/pack-thin:
pack-objects: hash basename and direname a bit differently.
pack-objects: allow "thin" packs to exceed depth limits
pack-objects: use full pathname to help hashing with "thin" pack.
pack-objects: thin pack micro-optimization.
Use thin pack transfer in "git fetch".
Add git-push --thin.
send-pack --thin: use "thin pack" delta transfer.
Thin pack - create packfile with missing delta base.
Conflicts:
pack-objects.c (taking "next")
send-pack.c (taking "next")
Stephen C. Tweedie noticed that we give up running rev-list when
we see too many refs on the remote side. Limit the number of
negative references we give to rev-list and continue.
Not sending any negative references to rev-list is very bad --
we may be pushing a ref that is new to the other end.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The new flag loosens the usual "self containedness" requirment
of packfiles, and sends deltified representation of objects when
we know the other side has the base objects needed to unpack
them. This would help reducing the transfer size.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The git suite may not be in PATH (and thus programs such as
git-send-pack could not exec git-rev-list). Thus there is a need for
logic that will locate these programs. Modifying PATH is not
desirable as it result in behavior differing from the user's
intentions, as we may end up prepending "/usr/bin" to PATH.
- git C programs will use exec*_git_cmd() APIs to exec sub-commands.
- exec*_git_cmd() will execute a git program by searching for it in
the following directories:
1. --exec-path (as used by "git")
2. The GIT_EXEC_PATH environment variable.
3. $(gitexecdir) as set in Makefile (default value $(bindir)).
- git wrapper will modify PATH as before to enable shell scripts to
invoke "git-foo" commands.
Ideally, shell scripts should use the git wrapper to become independent
of PATH, and then modifying PATH will not be necessary.
[jc: with minor updates after a brief review.]
Signed-off-by: Michal Ostrowski <mostrows@watson.ibm.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This updates the protocol between git-send-pack/git-receive-pack
in a backward compatible way to allow failures at the receiving
end to be propagated back to the sender. Most notably, versions
of git-push before this could not notice if the update hook on
the receiving end refused to update the ref for its own policy
reasons.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Wnen refusing to push a head, we said cryptic "remote 'branch'
object X does not exist on local" or "remote ref 'branch' is not
a strict subset of local ref 'branch'". That was gittish.
Since the most likely reason this happens is because the pushed
head was not up-to-date, clarify the error message to say that
straight, and suggest pulling first.
First noticed by Johannes and seconded by Andreas.
Signed-off-by: Junio C Hamano <junkio@cox.net>
When a push fails (for example when the remote head does not fast forward
to the desired ref) it is not correct to print "Everything up-to-date".
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
It used to make sense to have git-send-pack talk about the things it sent
when (a) it was a new program and (b) nobody had a lot of tags and
branches.
These days, it's just distracting to see tons of
'refs/tags/xyz': up-to-date
...
when updating a remote repo.
So shut it up by default, and add a "--verbose" flag for those who really
want to see it.
Also, since this makes he case of everything being up-to-date just totally
silent, make it say "Everything up-to-date" if no refs needed updating.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
In case some refs couldn't be pushed out due to an error (mostly the
not-a-proper-subset error), make git-send-pack exit with non-zero status
after the push is over (that is, it still tries to push out the rest
of the refs).
[jc: I adjusted a test for this change.]
Signed-off-by: Petr Baudis <pasky@suse.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If you try to push into an empty repository with no ref arguments to
git push, it doesn't do anything and doesn't say anything. This adds a
warning when send-pack isn't going to push anything, so you don't
assume that it silently did what you wanted.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
These are whole-tree operations and there is not much point
making them operable from within a subdirectory, but it is easy
to do so, and using setup_git_directory() upfront helps git://
proxy specification picked up from the correct place.
Signed-off-by: Junio C Hamano <junkio@cox.net>
One caller of deref_tag() was not careful enough to make sure
what deref_tag() returned was not NULL (i.e. we found a tag
object that points at an object we do not have). Fix it, and
warn about refs that point at such an incomplete tag where
needed.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This allows the remote side (most notably, upload-pack) to show
additional information without affecting the downloader. Peek-remote
does not ignore them -- this is to make it useful for Pasky's
automatic tag following.
Signed-off-by: Junio C Hamano <junkio@cox.net>
When more than two references need to be checked with
ref_newer() function, the second and later calls did not work
correctly. This was because the later calls found commits
retained by the "struct object" layer that still had smudges
made by earlier calls.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Again I left the v2.6.11-tree tag behind. My bad.
This commit makes sure that we do not barf when pushing a ref
that is a non-commitish tag. You can update a remote ref under
the following conditions:
* You can always use --force.
* Creating a brand new ref is OK.
* If the remote ref is exactly the same as what you are
pushing, it is OK (nothing is pushed).
* You can replace a commitish with another commitish which is a
descendant of it, if you can verify the ancestry between them;
this and the above means you have to have what you are replacing.
* Otherwise you cannot update; you need to use --force.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This allows git-send-pack to push local refs to a destination
repository under different names.
Here is the name mapping rules for refs.
* If there is no ref mapping on the command line:
- if '--all' is specified, it is equivalent to specifying
<local> ":" <local> for all the existing local refs on the
command line
- otherwise, it is equivalent to specifying <ref> ":" <ref> for
all the refs that exist on both sides.
* <name> is just a shorthand for <name> ":" <name>
* <src> ":" <dst>
push ref that matches <src> to ref that matches <dst>.
- It is an error if <src> does not match exactly one of local
refs.
- It is an error if <dst> matches more than one remote refs.
- If <dst> does not match any remote refs, either
- it has to start with "refs/"; <dst> is used as the
destination literally in this case.
- <src> == <dst> and the ref that matched the <src> must not
exist in the set of remote refs; the ref matched <src>
locally is used as the name of the destination.
For example,
- "git-send-pack --all <remote>" works exactly as before;
- "git-send-pack <remote> master:upstream" pushes local master
to remote ref that matches "upstream". If there is no such
ref, it is an error.
- "git-send-pack <remote> master:refs/heads/upstream" pushes
local master to remote refs/heads/upstream, even when
refs/heads/upstream does not exist.
- "git-send-pack <remote> master" into an empty remote
repository pushes the local ref/heads/master to the remote
ref/heads/master.
Signed-off-by: Junio C Hamano <junkio@cox.net>
When pushing into multi-user repository, or when pushing to a
repository from a local repository that has rebased branches
that has been pruned, the destination repository can have head
commits that are missing from the local repository.
This should not matter as long as the local head of the branch
being pushed is a proper superset of the destination branch, but
we ended up trying to run rev-list telling it to exclude objects
reachable from those heads missing from the local repository,
causing it to barf. Prune those heads from the rev-list
parameter list, and make sure we do not try to push a branch
whose remote head is something we lack.
Signed-off-by: Junio C Hamano <junkio@cox.net>
send-pack had a confusing misfeature that "send-pack --all
master" updated all refs, while "send-pack --all" did not do
anything. Make --all and explicit refs mutually exclusive, and
make sure "send-pack --all" updates all refs.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The check that the source is ahead of the destination incorrectly expects
pop_most_recent_commit() to gracefully handle an empty list.
Fix by just checking the list itself, rather than the return value of the
pop function.
[jc: I did the test script that demonstrated the problem]
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This should make sure that if you have multiple people pushing to the
same tree, they cannot overwrite each others work, but have to merge
before being able to update the common reference tree.
This adds documentation for 'smarter push' family of commands.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Cut-and-paste dup noticed by Junio. It's not even harmless, since a
match also causes that match to be invalidated, so this made it
impossible to update an existing branch by name.
I'd only tested the case of "ref doesn't exist at all on the other end",
which worked fine.
The protocol always supported it, but send-pack didn't actually know how
to tell the other side about a new branch/tag.
NOTE! You'll have to name it explicitly on the command line: if you
don't name any branches, git-send-pack will default to the branches that
already exist.
"git_path()" returns a static pathname pointer into the git directory
using a printf-like format specifier.
"head_ref()" works like "for_each_ref()", except for just the HEAD.
This makes the receiver always send a full list of valid refs, which
will allow us to do better packs, as well as handle creation of new
refs. Eventually. Right now we just moved the matching and enabled it.
So now you can do
git-send-pack host:path branch1 branch2
to only send branches "branch1" and "branch2".
This concludes this lesson. I've actually successfully sent an update
using the git-send-pack command.
Probably tons of work still to do, and nasty debugging, but it's now
actually potentially useful.
