* sp/smart-http: (37 commits)
http-backend: Let gcc check the format of more printf-type functions.
http-backend: Fix access beyond end of string.
http-backend: Fix bad treatment of uintmax_t in Content-Length
t5551-http-fetch: Work around broken Accept header in libcurl
t5551-http-fetch: Work around some libcurl versions
http-backend: Protect GIT_PROJECT_ROOT from /../ requests
Git-aware CGI to provide dumb HTTP transport
http-backend: Test configuration options
http-backend: Use http.getanyfile to disable dumb HTTP serving
test smart http fetch and push
http tests: use /dumb/ URL prefix
set httpd port before sourcing lib-httpd
t5540-http-push: remove redundant fetches
Smart HTTP fetch: gzip requests
Smart fetch over HTTP: client side
Smart push over HTTP: client side
Discover refs via smart HTTP server when available
http-backend: more explict LocationMatch
http-backend: add example for gitweb on same URL
http-backend: use mod_alias instead of mod_rewrite
...
Conflicts:
.gitignore
remote-curl.c
The git-remote-curl backend detects if the remote server supports
the git-receive-pack service, and if so, runs git-send-pack in a
pipe to dump the command and pack data as a single POST request.
The advertisements from the server that were obtained during the
discovery are passed into git-send-pack before the POST request
starts. This permits git-send-pack to operate largely unmodified.
For smaller packs (those under 1 MiB) a HTTP/1.0 POST with a
Content-Length is used, permitting interaction with any server.
The 1 MiB limit is arbitrary, but is sufficent to fit most deltas
created by human authors against text sources with the occasional
small binary file (e.g. few KiB icon image). The configuration
option http.postBuffer can be used to increase (or shink) this
buffer if the default is not sufficient.
For larger packs which cannot be spooled entirely into the helper's
memory space (due to http.postBuffer being too small), the POST
request requires HTTP/1.1 and sets "Transfer-Encoding: chunked".
This permits the client to upload an unknown amount of data in one
HTTP transaction without needing to pregenerate the entire pack
file locally.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cmd_ls_remote() was calling transport_get() with a NULL remote and a
non-NULL url in the case where it was run outside a git
repository. This involved a bunch of ill-tested special
cases. Instead, simply get the struct remote for the URL with
remote_get(), which works fine outside a git repository, and can also
take global options into account.
This fixes a tiny and obscure bug where "git ls-remote" without a repo
didn't support global url.*.insteadOf, even though "git clone" and
"git ls-remote" in any repo did.
Also, enforce that all callers provide a struct remote to transport_get().
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The remote helper interface now supports the push capability,
which can be used to ask the implementation to push one or more
specs to the remote repository. For remote-curl we implement this
by calling the existing WebDAV based git-http-push executable.
Internally the helper interface uses the push_refs transport hook
so that the complexity of the refspec parsing and matching can be
reused between remote implementations. When possible however the
helper protocol uses source ref name rather than the source SHA-1,
thereby allowing the helper to access this name if it is useful.
>From Clemens Buchacher <drizzd@aon.at>:
update http tests according to remote-curl capabilities
o Pushing packed refs is now fixed.
o The transport helper fails if refs are already up-to-date. Add
a test for that.
o The transport helper will notice if refs are already
up-to-date. We therefore need to update server info in the
unpacked-refs test.
o The transport helper will purge deleted branches automatically.
o Use a variable ($ORIG_HEAD) instead of full SHA-1 name.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
CC: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The variable is assigned unconditionally in print_push_status, but
print_push_status is not reached by all codepaths. In particular, this
fixes a bug where "git push ... nonexisting-branch" was complaining about
non-fast forward.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* db/vcs-helper:
Makefile: remove remnant of separate http/https/ftp helpers
Use a clearer style to issue commands to remote helpers
Make the "traditionally-supported" URLs a special case
Makefile: install hardlinks for git-remote-<scheme> supported by libcurl if possible
Makefile: do not link three copies of git-remote-* programs
Makefile: git-http-fetch does not need expat
http-fetch: Fix Makefile dependancies
Add transport native helper executables to .gitignore
git-http-fetch: not a builtin
Use an external program to implement fetching with curl
Add support for external programs for handling native fetches
Instead of trying to make http://, https://, and ftp:// URLs
indicative of some sort of pattern of transport helper usage, make
them a special case which runs the "curl" helper, and leave the
mechanism by which arbitrary helpers will be chosen entirely to future
work.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This should have been part of 481c7a6, whose goal was to
make "git push -q" silent unless there is an error.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If all refs sent by the remote repo during a fetch are reachable
locally, then no further conversation is performed with the remote. This
check is skipped when the --depth argument is provided to allow the
deepening of a shallow clone which corresponding remote repo has no
changed.
However, some additional filtering was added in commit c29727d5 to
remove those refs which are equal on both sides. If the remote repo has
not changed, then the list of refs to give the remote process becomes
empty and simply attempting to deepen a shallow repo always fails.
Let's stop being smart in that case and simply send the whole list over
when that condition is met. The remote will do the right thing anyways.
Test cases for this issue are also provided.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
push: point to 'git pull' and 'git push --force' in case of non-fast forward
Documentation: add: <filepattern>... is optional
Change mentions of "git programs" to "git commands"
Documentation: merge: one <remote> is required
help.c: give correct structure's size to memset()
'git push' failing because of non-fast forward is a very common situation,
and a beginner does not necessarily understand "fast forward" immediately.
Add a new section to the git-push documentation and refer them to it.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Nanako Shiraishi <nanako3@lavabit.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* js/run-command-updates:
api-run-command.txt: describe error behavior of run_command functions
run-command.c: squelch a "use before assignment" warning
receive-pack: remove unnecessary run_status report
run_command: report failure to execute the program, but optionally don't
run_command: encode deadly signal number in the return value
run_command: report system call errors instead of returning error codes
run_command: return exit code as positive value
MinGW: simplify waitpid() emulation macros
When --quiet is given, the user generally only wants to see
errors. So let's suppress printing the ref status table
unless there is an error, in which case we print out the
whole table.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When pushing over the git protocol, pack-objects gives
progress reports about the pack being sent. If "push" is
given the --quiet flag, it now passes "-q" to pack-objects,
suppressing this output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use the transport native helper mechanism to fetch by http (and ftp, etc).
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* tr/die_errno:
Use die_errno() instead of die() when checking syscalls
Convert existing die(..., strerror(errno)) to die_errno()
die_errno(): double % in strerror() output just in case
Introduce die_errno() that appends strerror(errno) to die()
In the case where a program was not found, it was still the task of the
caller to report an error to the user. Usually, this is an interesting case
but only few callers actually reported a specific error (though many call
sites report a generic error message regardless of the cause).
With this change the error is reported by run_command, but since there is
one call site in git.c that does not want that, an option is added to
struct child_process, which is used to turn the error off.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The motivation for this change is that system call failures are serious
errors that should be reported to the user, but only few callers took the
burden to decode the error codes that the functions returned into error
messages.
If at all, then only an unspecific error message was given. A prominent
example is this:
$ git upload-pack . | :
fatal: unable to run 'git-upload-pack'
In this example, git-upload-pack, the external command invoked through the
git wrapper, dies due to SIGPIPE, but the git wrapper does not bother to
report the real cause. In fact, this very error message is copied to the
syslog if git-daemon's client aborts the connection early.
With this change, system call failures are reported immediately after the
failure and only a generic failure code is returned to the caller. In the
above example the error is now to the point:
$ git upload-pack . | :
error: git-upload-pack died of signal
Note that there is no error report if the invoked program terminated with
a non-zero exit code, because it is reasonable to expect that the invoked
program has already reported an error. (But many run_command call sites
nevertheless write a generic error message.)
There was one special return code that was used to identify the case where
run_command failed because the requested program could not be exec'd. This
special case is now treated like a system call failure with errno set to
ENOENT. No error is reported in this case, because the call site in git.c
expects this as a normal result. Therefore, the callers that carefully
decoded the return value still check for this condition.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If --porcelain is used git-push will produce machine-readable output. The
output status line for each ref will be tab-separated and sent to stdout instead
of stderr. The full symbolic names of the refs will be given. For example
$ git push --dry-run --porcelain master :foobar 2>/dev/null \
| perl -pe 's/\t/ TAB /g'
= TAB refs/heads/master:refs/heads/master TAB [up to date]
- TAB :refs/heads/foobar TAB [deleted]
Signed-off-by: Larry D'Anna <larry@elder-gods.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Lots of die() calls did not actually report the kind of error, which
can leave the user confused as to the real problem. Use die_errno()
where we check a system/library call that sets errno on failure, or
one of the following that wrap such calls:
Function Passes on error from
-------- --------------------
odb_pack_keep open
read_ancestry fopen
read_in_full xread
strbuf_read xread
strbuf_read_file open or strbuf_read_file
strbuf_readlink readlink
write_in_full xwrite
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* rc/http-push: (22 commits)
http*: add helper methods for fetching objects (loose)
http*: add helper methods for fetching packs
http: use new http API in fetch_index()
http*: add http_get_info_packs
http-push.c::fetch_symref(): use the new http API
http-push.c::remote_exists(): use the new http API
http.c::http_fetch_ref(): use the new http API
transport.c::get_refs_via_curl(): use the new http API
http.c: new functions for the http API
http: create function end_url_with_slash
http*: move common variables and macros to http.[ch]
transport.c::get_refs_via_curl(): do not leak refs_url
Don't expect verify_pack() callers to set pack_size
http-push: do not SEGV after fetching a bad pack idx file
http*: copy string returned by sha1_to_hex
http-walker: verify remote packs
http-push, http-walker: style fixes
t5550-http-fetch: test fetching of packed objects
http-push: fix missing "#ifdef USE_CURL_MULTI" around "is_running_queue"
http-push: send out fetch requests on queue
...
Avoid code duplication by moving list tail search to match_refs().
This does not change the semantics, except for http-push, which now inserts
to the front of the ref list in order to get rid of the global remote_tail.
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ar/unlink-err:
print unlink(2) errno in copy_or_link_directory
replace direct calls to unlink(2) with unlink_or_warn
Introduce an unlink(2) wrapper which gives warning if unlink failed
In preparation to be used when the ref object is not available
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This helps to notice when something's going wrong, especially on
systems which lock open files.
I used the following criteria when selecting the code for replacement:
- it was already printing a warning for the unlink failures
- it is in a function which already printing something or is
called from such a function
- it is in a static function, returning void and the function is only
called from a builtin main function (cmd_)
- it is in a function which handles emergency exit (signal handlers)
- it is in a function which is obvously cleaning up the lockfiles
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When pulling from a remote, the full URL including username
is by default added to the commit message. Since it adds
very little value but could be used by malicious people to
glean valid usernames (with matching hostnames), we're far
better off just stripping the username before storing the
remote URL locally.
Note that this patch has no lasting visible effect when
"git pull" does not create a merge commit. It simply
alters what gets written to .git/FETCH_HEAD, which is used
by "git merge" to automagically create its messages.
Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier, the rsync tests were disabled by default, as they needed a
running rsyncd daemon. This was only due to the limitation that our
rsync transport only allowed full URLs of the form
rsync://<host>/<path>
Relaxing the URLs to allow
rsync:<path>
permitted the change in the tests to run whenever rsync is available,
without requiring a fully configured and running rsyncd.
While at it, the tests were fixed so that they run in directories with a
space in their name.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For native-protocol pushes (and other protocols as they are converted
to the new method), this moves the refspec match, tracking update, and
report message out of send-pack() and into transport_push(), where it
can be shared completely with other protocols. This also makes fetch
and push more similar in terms of what code is in what file.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A new inline function is_dot_or_dotdot is used to check if the
directory name is either "." or "..". It returns a non-zero value if
the given string is "." or "..". It's applicable to a lot of Git
source code.
Signed-off-by: Alexander Potashev <aspotashev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With all calls to alloc_ref() gone, we can remove it and then we're free
to give alloc_ref_from_str() the shorter name. It's a much nicer
interface, as the callers always need to have a name string when they
allocate a ref anyway and don't need to calculate and pass its length+1
any more.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace pairs of alloc_ref() and strcpy() with alloc_ref_from_str(),
simplifying the code.
In connect.c, also a pair of alloc_ref() and memcpy() is replaced --
the additional cost of a strlen() call should not have too much of an
impact. Consistency and simplicity are more important.
In remote.c, the code was allocating 11 bytes more than needed for
the name part, but I couldn't see them being used for anything.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The new -v option forces the progressbar, even in case the output
is not a terminal. This can be useful if the caller is an IDE or
wrapper which wants to scrape the progressbar from stderr and show
its information in a different format.
Signed-off-by: Miklos Vajna <vmiklos@frugalware.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* maint:
Update release notes for 1.6.0.3
Teach rebase -i to honor pre-rebase hook
docs: describe pre-rebase hook
do not segfault if make_cache_entry failed
make prefix_path() never return NULL
fix bogus "diff --git" header from "diff --no-index"
Fix fetch/clone --quiet when stdout is connected
builtin-blame: Fix blame -C -C with submodules.
bash: remove fetch, push, pull dashed form leftovers
Conflicts:
diff.c
Fixes the `git clone --quiet` issue raised by Dave Jones in
http://marc.info/?l=git&m=121529226023180&w=2
With this simple patch applied we no longer see the following remote
messages as no-progress is correctly sent to the remote site:
remote: Counting objects: 84102, done.
remote: Compressing objects: 100% (24720/24720), done.
remote: Total 84102 (delta 60949), reused 80810 (delta 57900)
Signed-off-by: Tuncer Ayaz <tuncer.ayaz@gmail.com>
Acked-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
"git push" enhancement allows the receiving end to report not only its own
refs but refs in repositories it borrows from via the alternate object
store mechanism. By telling the sender that objects reachable from these
extra refs are already complete in the receiving end, the number of
objects that need to be transfered can be cut down.
These entries are sent over the wire with string ".have", instead of the
actual names of the refs. This string was chosen so that they are ignored
by older programs at the sending end. If we sent some random but valid
looking refnames for these entries, "matching refs" rule (triggered when
running "git push" without explicit refspecs, where the sender learns what
refs the receiver has, and updates only the ones with the names of the
refs the sender also has) and "delete missing" rule (triggered when "git
push --mirror" is used, where the sender tells the receiver to delete the
refs it itself does not have) would try to update/delete them, which is
not what we want.
This prepares the send-pack (and "push" that runs native protocol) to
accept extended existing ref information and make use of it. The ".have"
entries are excluded from ref matching rules, and are exempt from deletion
rule while pushing with --mirror option, but are still used for pack
generation purposes by providing more "bottom" range commits.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, when cloning from invalid HTTP URL, git clone will possibly
return curl error, then a confusing message about remote HEAD and then
return success and leave an empty repository behind, confusing either
the end-user or the automated service calling it (think repo.or.cz).
This patch changes the error() calls in get_refs_via_curl() to die()s,
akin to the other get_refs_*() functions.
Cc: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Petr Baudis <pasky@suse.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
