Commit Graph

23 Commits

Author SHA1 Message Date
Sergey Vlasov
bc8f265270 git-http-fetch: Remove size limit for objects/info/{packs,alternates}
git-http-fetch received objects/info/packs into a fixed-size buffer
and started to fail when this file became larger than the buffer.
Change it to grow the buffer dynamically, and do the same thing for
objects/info/alternates.  Also add missing free() calls for these
buffers.

Signed-off-by: Sergey Vlasov <vsu@altlinux.ru>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-14 17:15:46 -07:00
Junio C Hamano
94fa447ace Quote reference names while fetching with curl.
curl_escape ought to do this, but we should not let it quote
slashes (nobody said refs/tags cannot have subdirectories), so
we roll our own safer version.  With this, the last part of
git-clone from Martin's moodle repository that used to fail now
works, which reads:

 $ git-http-fetch -v -a -w 'tags/MOODLE_15_MERGED **INVALID**' \
   'tags/MOODLE_15_MERGED **INVALID**' \
   http://locke.catalyst.net.nz/git/moodle.git/

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-11 22:05:09 -07:00
Junio C Hamano
dc1b5ea878 Remove unused 'got_alternates' variable.
The function fetch_alternates() is called only once from the
main now.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-10 23:22:02 -07:00
Junio C Hamano
b721e01f6e Use the same move_temp_to_file in git-http-fetch.
The http commit walker cannot use the same temporary file
creation code because it needs to use predictable temporary
filename for partial fetch continuation purposes, but the code
to move the temporary file to the final location should be
usable from the ordinary object creation codepath.

Export move_temp_to_file from sha1_file.c and use it, while
losing the custom relink_or_rename function from http-fetch.c.

Also the temporary object file creation part needs to make sure
the leading path exists, in preparation of the really lazy
fan-out directory creation.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-10 23:22:01 -07:00
Nick Hengeveld
0312600648 Restore functionality to allow proxies to cache objects
The parallel request changes didn't properly implement the previous patch to
allow caching of retrieved objects by proxy servers.  Restore the previous
functionality such that by default requests include the "Pragma: no-cache"
header, and this header is removed on requests for pack indexes, packs, and
objects.

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-10 23:22:01 -07:00
Nick Hengeveld
11f0dafe2b [PATCH] Don't fetch objects that exist in the local repository
Be sure not to fetch objects that already exist in the local repository.
The main process loop no longer performs this check, http-fetch now checks
prior to starting a new request queue entry and when fetch_object() is called,
and local-fetch now checks when fetch_object() is called.

As discussed in this thread: http://marc.theaimsgroup.com/?t=112854890500001

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
2005-10-10 23:22:01 -07:00
Nick Hengeveld
380792390e Set the parallel HTTP request limit via an environment variable
Use an environment variable rather than a command-line argument to set the
parallel HTTP request limit.  This allows the setting to work whether
git-http-fetch is run directly or via git-fetch.

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-10 23:22:01 -07:00
Nick Hengeveld
a7a8d3786e Only compile parallel HTTP support with CURL >= 7.9.8
Only compile parallel HTTP support with CURL >= 7.9.8

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-10 23:22:01 -07:00
Nick Hengeveld
1d389ab65d Add support for parallel HTTP transfers
Add support for parallel HTTP transfers.  Prefetch populates a queue of
objects to transfer and starts feeding requests to an active request
queue for processing; fetch_object keeps the active queue moving
while the specified object is being transferred.  The size of the active
queue can be restricted using -r and defaults to 5 concurrent transfers.
Requests for objects that are not prefetched are also processed via the
active queue.

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-10 23:22:01 -07:00
Nick Hengeveld
4fa2197e61 [PATCH] HTTP partial transfer support fix.
Don't unlink the temp file when an object transfer fails, so next attempt
will pick up where the failed transfer left off

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-01 23:17:15 -07:00
Junio C Hamano
271421cd34 Update partial HTTP transfers.
Add the sanity checks discussed on the list with Nick Hengeveld in
<20050927000931.GA15615@reactrix.com>.

 * unlink of previous and rename from temp to previous can fail for
   reasons other than benign ones (missing previous and missing temp).
   Report these failures when we encounter them, to make diagnosing
   problems easier.

 * when rewinding the partially written result, make sure to
   truncate the file.

Also verify the pack after downloading by calling
verify_packfile().

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-01 23:17:14 -07:00
Nick Hengeveld
49a0f240f7 [PATCH] HTTP partial transfer support for object, pack, and index transfers
HTTP partial transfer support for object, pack, and index transfers

[jc: this should not be placed in "master" -- it does not have any
 fixes requested on the list.]

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-01 23:17:14 -07:00
Johannes Schindelin
7d167feb8b [PATCH] Old curl does not know about CURLOPT_SSLKEY
... so try to set it only in later versions.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-29 22:47:37 -07:00
Peter Hagervall
a7928f8ec7 [PATCH] Make some needlessly global stuff static
Insert 'static' where appropriate.

Signed-off-by: Peter Hagervall <hager@cs.umu.se>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-28 16:38:52 -07:00
Nick Hengeveld
5acb6de13d [PATCH] Support for more CURL SSL settings via environment variables
Added support for additional CURL SSL settings via environment variables.
Client certificate/key files can be specified as well as alternate CA
information.

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-28 16:38:51 -07:00
Nick Hengeveld
1ddea77e44 [PATCH] Return CURL error message when object transfer fails
Return CURL error message when object transfer fails

[jc: added similar curl_errorstr errors to places where we
 use curl_easy_perform() to run fetch that _must_ succeed.]

Signed-off-by: Nick Hengeveld <nickh@reactrix.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-27 00:19:18 -07:00
Daniel Barkalow
820eca68c2 [PATCH] Implement --recover for git-*-fetch
With the --recover option, we verify that we have absolutely
everything reachable from the target, not assuming that things
reachable from refs will be complete.

Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-27 00:16:40 -07:00
Junio C Hamano
09d920831e Make object creation in http fetch a bit safer.
Unlike write_sha1_file() that tries to create the object file in a
temporary location and then move it to the final location, fetch_object
could have been interrupted in the middle, leaving a corrupt file.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-21 02:47:16 -07:00
Daniel Barkalow
1b0c1e672a [PATCH] Support alternates and http-alternates in http-fetch
This allows the remote repository to refer to additional repositories
in a file objects/info/http-alternates or
objects/info/alternates. Each line may be:

 a relative path, starting with ../, to get from the objects directory
  of the starting repository to the objects directory of the added
  repository.

 an absolute path of the objects directory of the added repository (on
  the same server).

 (only in http-alternates) a full URL of the objects directory of the
  added repository.

Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-18 14:14:52 -07:00
Daniel Barkalow
b3661567cf [PATCH] Add support for alternates in HTTP
This tries .../objects/info/http-alternates and then
.../objects/info/alternates, looking for a file which specifies where
else to download objects and packs from.

It currently only supports absolute paths, and doesn't support full URLs.

Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-17 11:57:50 -07:00
Sergey Vlasov
1db69b571c [PATCH] git-http-fetch: Allow caching of retrieved objects by proxy servers
By default the curl library adds "Pragma: no-cache" header to all
requests, which disables caching by proxy servers.  However, most
files in a GIT repository are immutable, and caching them is safe and
could be useful.

This patch removes the "Pragma: no-cache" header from requests for all
files except the pack list (objects/info/packs) and references
(refs/*), which are really mutable and should not be cached.

Signed-off-by: Sergey Vlasov <vsu@altlinux.ru>
Signed-off-by: Junio C Hamano <junkio@cox.net>
(cherry picked from 3b2a4c46fd5093ec79fb60e1b14b8d4a58c74612 commit)
2005-09-15 12:46:30 -07:00
Junio C Hamano
a9ab586a5d Retire support for old environment variables.
We have deprecated the old environment variable names for quite a
while and now it's time to remove them.  Gone are:

    SHA1_FILE_DIRECTORIES AUTHOR_DATE AUTHOR_EMAIL AUTHOR_NAME
    COMMIT_AUTHOR_EMAIL COMMIT_AUTHOR_NAME SHA1_FILE_DIRECTORY

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-09 14:48:54 -07:00
Junio C Hamano
215a7ad1ef Big tool rename.
As promised, this is the "big tool rename" patch.  The primary differences
since 0.99.6 are:

  (1) git-*-script are no more.  The commands installed do not
      have any such suffix so users do not have to remember if
      something is implemented as a shell script or not.

  (2) Many command names with 'cache' in them are renamed with
      'index' if that is what they mean.

There are backward compatibility symblic links so that you and
Porcelains can keep using the old names, but the backward
compatibility support  is expected to be removed in the near
future.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-07 17:45:20 -07:00