With large amount of objects, check_object() is really trashing the pack
sliding map and the filesystem cache. It has a completely random access
pattern especially with old objects where delta replay jumps back and
forth all over the pack.
This patch improves things by:
1) sorting objects by their offset in pack before calling check_object()
so the pack access pattern is linear;
2) recording the object type at add_object_entry() time since it is
already known in most cases;
3) recording the pack offset even for preferred_base objects;
4) avoid calling sha1_object_info() if all possible.
This limits pack accesses to the bare minimum and makes them perfectly
linear.
In the process check_object() was made more clear (to me at least).
Note: I thought about walking the sorted_by_offset list backward in
get_object_details() so if a pack happens to be larger than the available
file cache, then the cache would have been populated with useful data from
the beginning of the pack already when find_deltas() is called. Strangely,
testing (on Linux) showed absolutely no performance difference.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
... which consists of existing code split out of packed_delta_info()
for other callers to use it as well.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
It currently aliases delta_size on the principle that reused deltas won't
go through the whole delta matching loop hence delta_size was unused.
This is not true if given delta doesn't find its base in the pack though.
But we need that information even for whole object data reuse.
Well in short the current state looks awful and is prone to bugs. It just
works fine now because try_delta() tests trg_entry->delta before using
trg_entry->delta_size, but that is a bit subtle and I was wondering for a
while why things just worked fine... even if I'm guilty of having
introduced this abomination myself in the first place.
Let's do the sensible thing instead with no ambiguity, which is to have
a separate variable for in_pack_header_size. This might even help future
optimizations.
While at it, let's reorder some struct object_entry members so they all
align well with their own width, regardless of the architecture or the
size of off_t. Some memory saving is to be expected with this alone.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Because we don't have to know the SHA1 h(hence the name) of the pack
up front anymore, let's get rid of yet another global sorted object list
and sort them only in write_index_file(), then compute the object list
SHA1 on the fly.
This has the advantage of saving another chunk of memory, and the sorted
list SHA1 won't be computed needlessly on servers during a fetch.
Of course the cunning plan is also to make write_index_file() much like
the function with the same name in index-pack.c for an eventual easy
sharing.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This capability is practically never useful, and therefore never tested,
because it is fairly unlikely that the requested pack will be already
available. Furthermore it is of little gain over the ability to reuse
existing pack data.
In fact the ability to change delta type on the fly when reusing delta
data is a nice thing that has almost no cost and allows greater backward
compatibility with a client's capabilities than if the client is blindly
sent a whole pack without any discrimination.
And this "feature" is simply in the way of other cleanups.
Let's get rid of it.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Get rid of sort_comparator() as it impose a run time double indirect
function call for little compile time type checking gain.
Also get rid of create_sorted_list() as it only has one user which would
as well be just fine doing its sorting locally. Eventually the list of
deltifiable objects might be shorter than the whole object list.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Objects that have delta "children" from pack data reuse must consider the
depth of their deepest child when they try to deltify themselves for those
children not to become too deep.
However, in the context of a "thin" pack, the delta children depth was
skipped entirely on the presumption that the pack was always going to be
exploded on the receiving end, hence the delta length wasn't an issue.
Now that we keep received packs as is and reuse pack data when repacking,
those packs do contain delta chains that are longer than expected. Worse,
those delta chain may even grow longer when the pack is further repacked
into another thin pack for a subsequent transmission.
So this patch restores strict delta length even for thin packs, and it
moves check_delta_limit() usage directly in the delta loop where it is
needed. This way the delta_limit can be removed from struct object_entry
as well. Oh and the initial value was wrong too.
The progress_interval() function was moved to a more logical location in
the process.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Before finding best delta combinations, we sort objects by name hash,
then by size, then by their position in memory. Then we walk the list
backwards to test delta candidates.
We hope that a bigger size usually means a newer objects. But a bigger
address in memory does not mean a newer object. So the last comparison
must be reversed.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Let's avoid some cycles when there is no base to test against, and avoid
unnecessary object lookups.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* js/wrap-log:
Fix permissions on test scripts
Fix t4201: accidental arithmetic expansion
shortlog -w: make wrap-line behaviour optional.
Use print_wrapped_text() in shortlog
Make every test executable. Remove exec-attribute from included shell files,
they can't used standalone anyway.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
instead of embedded subshell. It actually breaks here (dash as /bin/sh):
t4201-shortlog.sh: 27: Syntax error: Missing '))'
FATAL: Unexpected exit with code 2
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds "--decorate" as a log option, which prints out the ref names
of any commits that are shown.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This allows you to add an arbitrary "decoration" of your choice to any
object. It's a space- and time-efficient way to add information to
arbitrary objects, especially if most objects probably do not have the
decoration.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* maint:
Have sample update hook not refuse deleting a branch through push.
variable $projectdesc needs to be set before checking against unchanged default.
Update git-annotate/git-blame documentation
Update git-apply documentation
Update git-applymbox documentation
Update git-am documentation
user-manual: use detached head when rewriting history
user-manual: start revising "internals" chapter
user-manual: detached HEAD
user-manual: fix discussion of default clone
Documentation: clarify track/no-track option.
Documentation: clarify git-checkout -f, minor editing
Documentation: minor edits of git-lost-found manpage
source ref might be 0000...0000 to delete a branch through git-push,
'git <remote> push :<branch>'. The update hook should not decline this.
Signed-off-by: Gerrit Pape <pape@smarden.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Moved options that pertained to both git-blame and git-annotate to a
common file blame-options.txt.
builtin-blame.c: Removed --compatibility, --long, --time from the
short usage as they are not handled in the code.
Documentation/git-blame.txt: Removed common options to git-annotate.
Added documentation for --score-debug. Removed --compatibility.
Adjusted usage at top to not wrap on 80 columns.
Documentation/git-annotate.txt: Using common options blame-options.txt.
Documentation/blame-options.txt: Added -b note about associated config
option, added --root note about associated config option, added
documentation for --show-stats. Removed --long, --time, --rev-file as
those options do not really exist. Added documentation for -M/-C taking
an optional score argument for detection of moved lines.
Signed-off-by: Andrew Ruder <andy@aeruder.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Document -v (short form of --verbose). Redo usage
to not wrap on 80 column terminal with typical
settings.
Signed-off-by: Andrew Ruder <andy@aeruder.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Documentation/git-applymbox.txt: updating -u documentation to include
fact that it encodes to the i18n.commitencoding setting, not just utf-8.
Added documentation of -n option to pass -n to git-mailinfo.
Signed-off-by: Andrew Ruder <andy@aeruder.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Documentation/git-am.txt missing several short versions
of options. Added documentation for --resolvemsg=<msg>
command-line option.
Signed-off-by: Andrew Ruder <andy@aeruder.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is slightly simpler if we use a detached head. And it's probably
good to have another example that uses this feature.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Add a brief mention of detached HEADs and .git/HEAD.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The name "master" isn't actually quite so special. Also, fix some bad
grammar.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Fix the description of the --no-track option so it no longer says the
opposite of what was intended. Also mention branch.autosetupmerge
explicitly.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
"Force a re-read of everything" doesn't mean much to me.
Also some minor grammar fixes.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Minor improvements to grammar and clarity of lost-found manpage.
Signed-off-by: "J. Bruce Fields" <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Useful e.g. to figure out what I did from screen history,
or to make sure subject line is short enough and makes sense
on its own.
Signed-off-by: Michael S. Tsirkin <mst@dev.mellanox.co.il>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Documentation/SubmittingPatches: Add note that all user interface changes
should include associated documentation updates.
Signed-off-by: Andrew Ruder <andy@aeruder.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* maint:
Document -g (--walk-reflogs) option of git-log
sscanf/strtoul: parse integers robustly
git-blame: Fix overrun in fake_working_tree_commit()
[PATCH] Improve look-and-feel of the gitk tool.
[PATCH] Teach gitk to use the user-defined UI font everywhere.
At the same time, we do not want to allow arbitrary strings for
attribute names, as we are likely to want to extend the syntax
later. Allow only alnum, dash, underscore and dot for now.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds "attribute macros" (for lack of better name). So far,
we have low-level attributes such as crlf and diff, which are
defined in operational terms --- setting or unsetting them on a
particular path directly affects what is done to the path. For
example, in order to decline diffs or crlf conversions on a
binary blob, no diffs on PostScript files, and treat all other
files normally, you would have something like these:
* diff crlf
*.ps !diff
proprietary.o !diff !crlf
That is fine as the operation goes, but gets unwieldy rather
rapidly, when we start adding more low-level attributes that are
defined in operational terms. A near-term example of such an
attribute would be 'merge-3way' which would control if git
should attempt the usual 3-way file-level merge internally, or
leave merging to a specialized external program of user's
choice. When it is added, we do _not_ want to force the users
to update the above to:
* diff crlf merge-3way
*.ps !diff
proprietary.o !diff !crlf !merge-3way
The way this patch solves this issue is to realize that the
attributes the user is assigning to paths are not defined in
terms of operations but in terms of what they are.
All of the three low-level attributes usually make sense for
most of the files that sane SCM users have git operate on (these
files are typically called "text'). Only a few cases, such as
binary blob, need exception to decline the "usual treatment
given to text files" -- and people mark them as "binary".
So this allows the $GIT_DIR/info/alternates and .gitattributes
at the toplevel of the project to also specify attributes that
assigns other attributes. The syntax is '[attr]' followed by an
attribute name followed by a list of attribute names:
[attr] binary !diff !crlf !merge-3way
When "binary" attribute is set to a path, if the path has not
got diff/crlf/merge-3way attribute set or unset by other rules,
this rule unsets the three low-level attributes.
It is expected that the user level .gitattributes will be
expressed mostly in terms of attributes based on what the files
are, and the above sample would become like this:
(built-in attribute configuration)
[attr] binary !diff !crlf !merge-3way
* diff crlf merge-3way
(project specific .gitattributes)
proprietary.o binary
(user preference $GIT_DIR/info/attributes)
*.ps !diff
There are a few caveats.
* As described above, you can define these macros only in
$GIT_DIR/info/attributes and toplevel .gitattributes.
* There is no attempt to detect circular definition of macro
attributes, and definitions are evaluated from bottom to top
as usual to fill in other attributes that have not yet got
values. The following would work as expected:
[attr] text diff crlf
[attr] ps text !diff
*.ps ps
while this would most likely not (I haven't tried):
[attr] ps text !diff
[attr] text diff crlf
*.ps ps
* When a macro says "[attr] A B !C", saying that a path does
not have attribute A does not let you tell anything about
attributes B or C. That is, given this:
[attr] text diff crlf
[attr] ps text !diff
*.txt !ps
path hello.txt, which would match "*.txt" pattern, would have
"ps" attribute set to zero, but that does not make text
attribute of hello.txt set to false (nor diff attribute set to
true).
Signed-off-by: Junio C Hamano <junkio@cox.net>
I lost it by mistake while shuffling the gitattributes series which
originally was on top of the subproject topic onto the master branch.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is in the same spirit as the previous one. Earlier 'diff'
meant 'do the built-in binary heuristics and disable patch text
generation based on it' while '!diff' meant 'do not guess, do
not generate patch text'. There was no way to say 'do generate
patch text even when the heuristics says it has NUL in it'.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Earlier we said 'crlf lets the path go through core.autocrlf
process while !crlf disables it altogether'. This fixes the
semantics to:
- Lack of 'crlf' attribute makes core.autocrlf to apply
(i.e. we guess based on the contents and if platform
expresses its desire to have CRLF line endings via
core.autocrlf, we do so).
- Setting 'crlf' attribute to true forces CRLF line endings in
working tree files, even if blob does not look like text
(e.g. contains NUL or other bytes we consider binary).
- Setting 'crlf' attribute to false disables conversion.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The same way we generate diffs on symlinks as the the diff of text of the
symlink, we can generate subproject diffs (when not recursing into them!)
as the diff of the text that describes the subproject.
Of course, since what descibes a subproject is just the SHA1, that's what
we'll use. Add some pretty-printing to make it a bit more obvious what is
going on, and we're done.
So with this, we can get both raw diffs and "textual" diffs of subproject
changes:
- git diff --raw:
:160000 160000 2de597b5ad348b7db04bd10cdd38cd81cbc93ab5 0000000... M sub-A
- git diff:
diff --git a/sub-A b/sub-A
index 2de597b..e8f11a4 160000
--- a/sub-A
+++ b/sub-A
@@ -1 +1 @@
-Subproject commit 2de597b5ad348b7db04bd10cdd38cd81cbc93ab5
+Subproject commit e8f11a45c5c6b9e2fec6d136d3fb5aff75393d42
NOTE! We'll also want to have the ability to recurse into the subproject
and actually diff it recursively, but that will involve a new command line
option (I'd suggest "--subproject" and "-S", but the latter is in use by
pickaxe), and some very different code.
But regardless of ay future recursive behaviour, we need the non-recursive
version too (and it should be the default, at least in the absense of
config options, so that large superprojects don't default to something
extremely expensive).
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
By showing the basename of the directory very early in the
title bar I can more easily locate a particular git-gui
session when I have 8 open at once and my Windows taskbar
is overflowing with items.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* er/ui:
Always bind the return key to the default button
Do not break git-gui messages into multiple lines.
Improve look-and-feel of the git-gui tool.
Teach git-gui to use the user-defined UI font everywhere.
Allow wish interpreter to be defined with TCLTK_PATH
* builtin-grep.c (strtoul_ui): Move function definition from here, to...
* git-compat-util.h (strtoul_ui): ...here, with an added "base" parameter.
* builtin-grep.c (cmd_grep): Update use of strtoul_ui to include base, "10".
* builtin-update-index.c (read_index_info): Diagnose an invalid mode integer
that is out of range or merely larger than INT_MAX.
(cmd_update_index): Use strtoul_ui, not sscanf.
* convert-objects.c (write_subdirectory): Likewise.
Signed-off-by: Jim Meyering <jim@meyering.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* git://git2.kernel.org/pub/scm/gitk/gitk:
[PATCH] Improve look-and-feel of the gitk tool.
[PATCH] Teach gitk to use the user-defined UI font everywhere.
