git-clone: aggressively optimize local clone behaviour.
This changes the behaviour of cloning from a repository on the local machine, by defaulting to "-l" (use hardlinks to share files under .git/objects) and making "-l" a no-op. A new option, --no-hardlinks, is also added to cause file-level copy of files under .git/objects while still avoiding the normal "pack to pipe, then receive and index pack" network transfer overhead. The old behaviour of local cloning without -l nor -s is availble by specifying the source repository with the newly introduced file:///path/to/repo.git/ syntax (i.e. "same as network" cloning). * With --no-hardlinks (i.e. have all .git/objects/ copied via cpio) would not catch the source repository corruption, and also risks corrupted recipient repository if an alpha-particle hits memory cell while indexing and resolving deltas. As long as the recipient is created uncorrupted, you have a good back-up. * same-as-network is expensive, but it would catch the breakage of the source repository. It still risks corrupted recipient repository due to hardware failure. As long as the recipient is created uncorrupted, you have a good back-up. * The new default on the same filesystem, as long as the source repository is healthy, it is very likely that the recipient would be, too. Also it is very cheap. You do not get any back-up benefit, though. None of the method is resilient against the source repository corruption, so let's discount that from the comparison. Then the difference with and without --no-hardlinks matters primarily if you value the back-up benefit or not. If you want to use the cloned repository as a back-up, then it is cheaper to do a clone with --no-hardlinks and two git-fsck (source before clone, recipient after clone) than same-as-network clone, especially as you are likely to do a git-fsck on the recipient if you are so paranoid anyway. Which leads me to believe that being able to use file:/// is probably a good idea, if only for testability, but probably of little practical value. We default to hardlinked clone for everyday use, and paranoids can use --no-hardlinks as a way to make a back-up. Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit is contained in:
Junio C Hamano
parent
72a4f4b657
commit
3d5c418ff5
@ -9,7 +9,8 @@ git-clone - Clone a repository into a new directory
|
||||
SYNOPSIS
|
||||
--------
|
||||
[verse]
|
||||
'git-clone' [--template=<template_directory>] [-l [-s]] [-q] [-n] [--bare]
|
||||
'git-clone' [--template=<template_directory>]
|
||||
[-l] [-s] [--no-hardlinks] [-q] [-n] [--bare]
|
||||
[-o <name>] [-u <upload-pack>] [--reference <repository>]
|
||||
[--depth <depth>] <repository> [<directory>]
|
||||
|
||||
@ -40,8 +41,19 @@ OPTIONS
|
||||
this flag bypasses normal "git aware" transport
|
||||
mechanism and clones the repository by making a copy of
|
||||
HEAD and everything under objects and refs directories.
|
||||
The files under .git/objects/ directory are hardlinked
|
||||
to save space when possible.
|
||||
The files under `.git/objects/` directory are hardlinked
|
||||
to save space when possible. This is now the default when
|
||||
the source repository is specified with `/path/to/repo`
|
||||
syntax, so it essentially is a no-op option. To force
|
||||
copying instead of hardlinking (which may be desirable
|
||||
if you are trying to make a back-up of your repository),
|
||||
but still avoid the usual "git aware" transport
|
||||
mechanism, `--no-hardlinks` can be used.
|
||||
|
||||
--no-hardlinks::
|
||||
Optimize the cloning process from a repository on a
|
||||
local filesystem by copying files under `.git/objects`
|
||||
directory.
|
||||
|
||||
--shared::
|
||||
-s::
|
||||
|
||||
@ -15,11 +15,11 @@ to name the remote repository:
|
||||
- ssh://{startsb}user@{endsb}host.xz/~/path/to/repo.git
|
||||
===============================================================
|
||||
|
||||
SSH is the default transport protocol. You can optionally specify
|
||||
which user to log-in as, and an alternate, scp-like syntax is also
|
||||
supported. Both syntaxes support username expansion,
|
||||
as does the native git protocol. The following three are
|
||||
identical to the last three above, respectively:
|
||||
SSH is the default transport protocol over the network. You can
|
||||
optionally specify which user to log-in as, and an alternate,
|
||||
scp-like syntax is also supported. Both syntaxes support
|
||||
username expansion, as does the native git protocol. The following
|
||||
three are identical to the last three above, respectively:
|
||||
|
||||
===============================================================
|
||||
- {startsb}user@{endsb}host.xz:/path/to/repo.git/
|
||||
@ -27,8 +27,12 @@ identical to the last three above, respectively:
|
||||
- {startsb}user@{endsb}host.xz:path/to/repo.git
|
||||
===============================================================
|
||||
|
||||
To sync with a local directory, use:
|
||||
To sync with a local directory, you can use:
|
||||
|
||||
===============================================================
|
||||
- /path/to/repo.git/
|
||||
- file:///path/to/repo.git/
|
||||
===============================================================
|
||||
|
||||
They are mostly equivalent, except when cloning. See
|
||||
gitlink:git-clone[1] for details.
|
||||
|
||||
64
git-clone.sh
64
git-clone.sh
@ -87,7 +87,7 @@ Perhaps git-update-server-info needs to be run there?"
|
||||
|
||||
quiet=
|
||||
local=no
|
||||
use_local=no
|
||||
use_local_hardlink=yes
|
||||
local_shared=no
|
||||
unset template
|
||||
no_checkout=
|
||||
@ -108,9 +108,13 @@ while
|
||||
no_checkout=yes ;;
|
||||
*,--na|*,--nak|*,--nake|*,--naked|\
|
||||
*,-b|*,--b|*,--ba|*,--bar|*,--bare) bare=yes ;;
|
||||
*,-l|*,--l|*,--lo|*,--loc|*,--loca|*,--local) use_local=yes ;;
|
||||
*,-l|*,--l|*,--lo|*,--loc|*,--loca|*,--local)
|
||||
use_local_hardlink=yes ;;
|
||||
*,--no-h|*,--no-ha|*,--no-har|*,--no-hard|*,--no-hardl|\
|
||||
*,--no-hardli|*,--no-hardlin|*,--no-hardlink|*,--no-hardlinks)
|
||||
use_local_hardlink=no ;;
|
||||
*,-s|*,--s|*,--sh|*,--sha|*,--shar|*,--share|*,--shared)
|
||||
local_shared=yes; use_local=yes ;;
|
||||
local_shared=yes; ;;
|
||||
1,--template) usage ;;
|
||||
*,--template)
|
||||
shift; template="--template=$1" ;;
|
||||
@ -249,34 +253,36 @@ fi
|
||||
rm -f "$GIT_DIR/CLONE_HEAD"
|
||||
|
||||
# We do local magic only when the user tells us to.
|
||||
case "$local,$use_local" in
|
||||
yes,yes)
|
||||
case "$local" in
|
||||
yes)
|
||||
( cd "$repo/objects" ) ||
|
||||
die "-l flag seen but repository '$repo' is not local."
|
||||
die "cannot chdir to local '$repo/objects'."
|
||||
|
||||
case "$local_shared" in
|
||||
no)
|
||||
# See if we can hardlink and drop "l" if not.
|
||||
sample_file=$(cd "$repo" && \
|
||||
find objects -type f -print | sed -e 1q)
|
||||
|
||||
# objects directory should not be empty since we are cloning!
|
||||
test -f "$repo/$sample_file" || exit
|
||||
|
||||
l=
|
||||
if ln "$repo/$sample_file" "$GIT_DIR/objects/sample" 2>/dev/null
|
||||
then
|
||||
l=l
|
||||
fi &&
|
||||
rm -f "$GIT_DIR/objects/sample" &&
|
||||
cd "$repo" &&
|
||||
find objects -depth -print | cpio -pumd$l "$GIT_DIR/" || exit 1
|
||||
;;
|
||||
yes)
|
||||
mkdir -p "$GIT_DIR/objects/info"
|
||||
echo "$repo/objects" >> "$GIT_DIR/objects/info/alternates"
|
||||
;;
|
||||
esac
|
||||
if test "$local_shared" = yes
|
||||
then
|
||||
mkdir -p "$GIT_DIR/objects/info"
|
||||
echo "$repo/objects" >>"$GIT_DIR/objects/info/alternates"
|
||||
else
|
||||
l= &&
|
||||
if test "$use_local_hardlink" = yes
|
||||
then
|
||||
# See if we can hardlink and drop "l" if not.
|
||||
sample_file=$(cd "$repo" && \
|
||||
find objects -type f -print | sed -e 1q)
|
||||
# objects directory should not be empty because
|
||||
# we are cloning!
|
||||
test -f "$repo/$sample_file" || exit
|
||||
if ln "$repo/$sample_file" "$GIT_DIR/objects/sample" 2>/dev/null
|
||||
then
|
||||
rm -f "$GIT_DIR/objects/sample"
|
||||
l=l
|
||||
else
|
||||
echo >&2 "Warning: -l asked but cannot hardlink to $repo"
|
||||
fi
|
||||
fi &&
|
||||
cd "$repo" &&
|
||||
find objects -depth -print | cpio -pumd$l "$GIT_DIR/" || exit 1
|
||||
fi
|
||||
git-ls-remote "$repo" >"$GIT_DIR/CLONE_HEAD" || exit 1
|
||||
;;
|
||||
*)
|
||||
|
||||
@ -129,7 +129,7 @@ pull_to_client 2nd "B" $((64*3))
|
||||
|
||||
pull_to_client 3rd "A" $((1*3)) # old fails
|
||||
|
||||
test_expect_success "clone shallow" "git-clone --depth 2 . shallow"
|
||||
test_expect_success "clone shallow" "git-clone --depth 2 file://`pwd`/. shallow"
|
||||
|
||||
(cd shallow; git count-objects -v) > count.shallow
|
||||
|
||||
|
||||
@ -51,7 +51,7 @@ diff expected current'
|
||||
cd "$base_dir"
|
||||
|
||||
test_expect_success 'cloning with reference (no -l -s)' \
|
||||
'git clone --reference B A D'
|
||||
'git clone --reference B file://`pwd`/A D'
|
||||
|
||||
cd "$base_dir"
|
||||
|
||||
|
||||
@ -43,4 +43,21 @@ test_expect_success 'local clone from x.git that does not exist' '
|
||||
fi
|
||||
'
|
||||
|
||||
test_expect_success 'With -no-hardlinks, local will make a copy' '
|
||||
cd "$D" &&
|
||||
git clone --bare --no-hardlinks x w &&
|
||||
cd w &&
|
||||
linked=$(find objects -type f ! -links 1 | wc -l) &&
|
||||
test "$linked" = 0
|
||||
'
|
||||
|
||||
test_expect_success 'Even without -l, local will make a hardlink' '
|
||||
cd "$D" &&
|
||||
rm -fr w &&
|
||||
git clone -l --bare x w &&
|
||||
cd w &&
|
||||
copied=$(find objects -type f -links 1 | wc -l) &&
|
||||
test "$copied" = 0
|
||||
'
|
||||
|
||||
test_done
|
||||
|
||||
Loading…
Reference in New Issue
Block a user