When 1c192f3 (gc --aggressive: make it really aggressive - 2007-12-06)
made --depth=250 the default value, it didn't really explain the
reason behind, especially the pros and cons of --depth=250.
An old mail from Linus below explains it at length. Long story short,
--depth=250 is a disk saver and a performance killer. Not everybody
agrees on that aggressiveness. Let the user configure it.
From: Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: [PATCH] gc --aggressive: make it really aggressive
Date: Thu, 6 Dec 2007 08:19:24 -0800 (PST)
Message-ID: <alpine.LFD.0.9999.0712060803430.13796@woody.linux-foundation.org>
Gmane-URL: http://article.gmane.org/gmane.comp.gcc.devel/94637
On Thu, 6 Dec 2007, Harvey Harrison wrote:
>
> 7:41:25elapsed 86%CPU
Heh. And this is why you want to do it exactly *once*, and then just
export the end result for others ;)
> -r--r--r-- 1 hharrison hharrison 324094684 2007-12-06 07:26 pack-1d46...pack
But yeah, especially if you allow longer delta chains, the end result can
be much smaller (and what makes the one-time repack more expensive is the
window size, not the delta chain - you could make the delta chains longer
with no cost overhead at packing time)
HOWEVER.
The longer delta chains do make it potentially much more expensive to then
use old history. So there's a trade-off. And quite frankly, a delta depth
of 250 is likely going to cause overflows in the delta cache (which is
only 256 entries in size *and* it's a hash, so it's going to start having
hash conflicts long before hitting the 250 depth limit).
So when I said "--depth=250 --window=250", I chose those numbers more as
an example of extremely aggressive packing, and I'm not at all sure that
the end result is necessarily wonderfully usable. It's going to save disk
space (and network bandwidth - the delta's will be re-used for the network
protocol too!), but there are definitely downsides too, and using long
delta chains may simply not be worth it in practice.
(And some of it might just want to have git tuning, ie if people think
that long deltas are worth it, we could easily just expand on the delta
hash, at the cost of some more memory used!)
That said, the good news is that working with *new* history will not be
affected negatively, and if you want to be _really_ sneaky, there are ways
to say "create a pack that contains the history up to a version one year
ago, and be very aggressive about those old versions that we still want to
have around, but do a separate pack for newer stuff using less aggressive
parameters"
So this is something that can be tweaked, although we don't really have
any really nice interfaces for stuff like that (ie the git delta cache
size is hardcoded in the sources and cannot be set in the config file, and
the "pack old history more aggressively" involves some manual scripting
and knowing how "git pack-objects" works rather than any nice simple
command line switch).
So the thing to take away from this is:
- git is certainly flexible as hell
- .. but to get the full power you may need to tweak things
- .. happily you really only need to have one person to do the tweaking,
and the tweaked end results will be available to others that do not
need to know/care.
And whether the difference between 320MB and 500MB is worth any really
involved tweaking (considering the potential downsides), I really don't
know. Only testing will tell.
Linus
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* sr/add--interactive-term-readkey:
git-add--interactive: warn if module for interactive.singlekey is missing
git-config: document interactive.singlekey requires Term::ReadKey
A warning from "git pack-objects" were generated by referring to an
incorrect variable when forming the filename that we had trouble
with.
* sh/write-pack-file-warning-message-fix:
write_pack_file: use correct variable in diagnostic
* mh/replace-refs-variable-rename:
Document some functions defined in object.c
Add docstrings for lookup_replace_object() and do_lookup_replace_object()
rename read_replace_refs to check_replace_refs
Allow loosening remote "git archive" invocation security check that
refuses to serve tree-ish not at the tip of any ref.
* sg/archive-restrict-remote:
add uploadarchive.allowUnreachable option
docs: clarify remote restrictions for git-upload-archive
"git difftool" misbehaved when the repository is bound to the
working tree with the ".git file" mechanism, where a textual
file ".git" tells us where it is.
* da/difftool-git-files:
t7800: add a difftool test for .git-files
difftool: support repositories with .git-files
* tg/index-v4-format:
read-cache: add index.version config variable
test-lib: allow setting the index format version
introduce GIT_INDEX_VERSION environment variable
* mh/object-code-cleanup:
sha1_file.c: document a bunch of functions defined in the file
sha1_file_name(): declare to return a const string
find_pack_entry(): document last_found_pack
replace_object: use struct members instead of an array
"git push" did not pay attention to branch.*.pushremote if it is
defined earlier than remote.pushdefault; the order of these two
variables in the configuration file should not matter, but it did by
mistake.
* jk/remote-pushremote-config-reading:
remote: handle pushremote config in any order
Tighten codepaths that parse timestamps in commit objects.
* jk/commit-dates-parsing-fix:
show_ident_date: fix tz range check
log: do not segfault on gmtime errors
log: handle integer overflow in timestamps
date: check date overflow against time_t
fsck: report integer overflow in author timestamps
t4212: test bogus timestamps with git-log
We started using wildmatch() in place of fnmatch(3); complete the
process and stop using fnmatch(3).
* nd/no-more-fnmatch:
actually remove compat fnmatch source code
stop using fnmatch (either native or compat)
Revert "test-wildmatch: add "perf" command to compare wildmatch and fnmatch"
use wildmatch() directly without fnmatch() wrapper
Instead of allowing an <img> to be shown in whatever size, force
scaling it to fit on the page with max-height/max-width css style
attributes.
* ak/gitweb-fit-image:
gitweb: Avoid overflowing page body frame with large images
"git diff --external-diff" incorrectly fed the submodule directory
in the working tree to the external diff driver when it knew it is
the same as one of the versions being compared.
* tr/diff-submodule-no-reuse-worktree:
diff: do not reuse_worktree_file for submodules
"git reset" needs to refresh the index when working in a working
tree (it can also be used to match the index to the HEAD in an
otherwise bare repository), but it failed to set up the working
tree properly, causing GIT_WORK_TREE to be ignored.
* nd/reset-setup-worktree:
reset: optionally setup worktree and refresh index on --mixed
"git config" learned to read from the standard input when "-" is
given as the value to its "--file" parameter (attempting an
operation to update the configuration in the standard input of
course is rejected).
* ks/config-file-stdin:
config: teach "git config --file -" to read from the standard input
config: change git_config_with_options() interface
builtin/config.c: rename check_blob_write() -> check_write()
config: disallow relative include paths from blobs
* jk/janitorial-fixes:
open_istream(): do not dereference NULL in the error case
builtin/mv: don't use memory after free
utf8: use correct type for values in interval table
utf8: fix iconv error detection
notes-utils: handle boolean notes.rewritemode correctly
Uses of curl's "multi" interface and "easy" interface do not mix
well when we attempt to reuse outgoing connections. Teach the RPC
over http code, used in the smart HTTP transport, not to use the
"easy" interface.
* jk/http-no-curl-easy:
http: never use curl_easy_perform
Trailing whitespaces in .gitignore files, unless they are quoted for
fnmatch(3), e.g. "path\ ", are warned and ignored.
Strictly speaking, this is a backward incompatible change, but very
unlikely to bite any sane user and adjusting should be obvious and
easy.
* nd/gitignore-trailing-whitespace:
t0008: skip trailing space test on Windows
dir: ignore trailing spaces in exclude patterns
dir: warn about trailing spaces in exclude patterns
"git check-attr" when (trying to) work on a repository with a
working tree did not work well when the working tree was specified
via --work-tree (and obviously with --git-dir).
The command also works in a bare repository but it reads from the
(possibly stale, irrelevant and/or nonexistent) index, which may
need to be fixed to read from HEAD, but that is a completely
separate issue. As a related tangent to this separate issue, we
may want to also fix "check-ignore", which refuses to work in a
bare repository, to also operate in a bare one.
* jc/check-attr-honor-working-tree:
check-attr: move to the top of working tree when in non-bare repository
t0003: do not chdir the whole test process
The Windows API does not preserve file names with trailing spaces (and
dots), but rather strips them. Our tools (MSYS bash, git) base the POSIX
emulation on the Windows API. As a consequence, it is impossible for bash
on Windows to allocate a file whose name has trailing spaces, and for git
to stat such a file. Both operate on a file whose name has the spaces
stripped. Skip the test that needs such a file name.
Note that we do not use (another incarnation of) prerequisite FUNNYNAMES.
The reason is that FUNNYNAMES is intended to represent a property of the
file system. But the inability to have trailing spaces in a file name is
a property of the Windows API. The file system (NTFS) does not have this
limitation.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 1dca155fe3 (log: handle integer overflow in
timestamps, 2014-02-24) tried to catch integer overflow
coming from strtol() on the timezone field by comparing against
LONG_MIN/LONG_MAX. However, the intermediate "tz" variable
is an "int", which means it can never be LONG_MAX on LP64
systems; we would truncate the output from strtol before the
comparison.
Clang's -Wtautological-constant-out-of-range-compare notices
this and rightly complains.
Let's instead store the result of strtol in a long, and then
compare it against INT_MIN/INT_MAX. This will catch overflow
from strtol, and also overflow when we pass the result as an
int to show_date.
Reported-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allow running "gc --auto" in the background.
* nd/daemonize-gc:
gc: config option for running --auto in background
daemon: move daemonize() to libgit.a
Teach combine-diff to honour the path-output-order imposed by
diffcore-order, and optimize how matching paths are found in
the N-way diffs made with parents.
* ks/combine-diff:
tests: add checking that combine-diff emits only correct paths
combine-diff: simplify intersect_paths() further
combine-diff: combine_diff_path.len is not needed anymore
combine-diff: optimize combine_diff_path sets intersection
diff test: add tests for combine-diff with orderfile
diffcore-order: export generic ordering interface
Add missing leading dash to proposed commands in french output when
using the command:
git branch --set-upstream remotename/branchname
and when upstream is gone
Signed-off-by: Sandy Carter <sandy.carter@savoirfairelinux.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In record_author_date() & parse_gpg_output(), the callers of
starts_with() not just want to know if the string starts with the
prefix, but also can benefit from knowing the string that follows
the prefix.
By using skip_prefix(), we can do both at the same time.
Helped-by: Max Horn <max@quendi.de>
Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
