Commit Graph

254 Commits

Author SHA1 Message Date
Junio C Hamano
bb0cebd7d0 Merge branch 'jc/maint-1.6.0-pack-directory'
* jc/maint-1.6.0-pack-directory:
  Make sure objects/pack exists before creating a new pack
2009-02-25 14:50:05 -08:00
Junio C Hamano
6e180cdcec Make sure objects/pack exists before creating a new pack
In a repository created with git older than f49fb35 (git-init-db: create
"pack" subdirectory under objects, 2005-06-27), objects/pack/ directory is
not created upon initialization.  It was Ok because subdirectories are
created as needed inside directories init-db creates, and back then,
packfiles were recent invention.

After the said commit, new codepaths started relying on the presense of
objects/pack/ directory in the repository.  This was exacerbated with
8b4eb6b (Do not perform cross-directory renames when creating packs,
2008-09-22) that moved the location temporary pack files are created from
objects/ directory to objects/pack/ directory, because moving temporary to
the final location was done carefully with lazy leading directory creation.

Many packfile related operations in such an old repository can fail
mysteriously because of this.

This commit introduces two helper functions to make things work better.

 - odb_mkstemp() is a specialized version of mkstemp() to refactor the
   code and teach it to create leading directories as needed;

 - odb_pack_keep() refactors the code to create a ".keep" file while
   create leading directories as needed.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-25 14:39:42 -08:00
Junio C Hamano
141b6b83d7 Merge branch 'lt/maint-wrap-zlib' into maint
* lt/maint-wrap-zlib:
  Wrap inflate and other zlib routines for better error reporting

Conflicts:
	http-push.c
	http-walker.c
	sha1_file.c
2009-02-05 18:01:00 -08:00
Junio C Hamano
36dd939393 Merge branch 'lt/maint-wrap-zlib'
* lt/maint-wrap-zlib:
  Wrap inflate and other zlib routines for better error reporting

Conflicts:
	http-push.c
	http-walker.c
	sha1_file.c
2009-01-21 16:55:17 -08:00
Linus Torvalds
39c68542fc Wrap inflate and other zlib routines for better error reporting
R. Tyler Ballance reported a mysterious transient repository corruption;
after much digging, it turns out that we were not catching and reporting
memory allocation errors from some calls we make to zlib.

This one _just_ wraps things; it doesn't do the "retry on low memory
error" part, at least not yet. It is an independent issue from the
reporting.  Some of the errors are expected and passed back to the caller,
but we die when zlib reports it failed to allocate memory for now.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-11 02:13:06 -08:00
Nicolas Pitre
bf87489624 pack-objects: don't use too many threads with few objects
If there are few objects to deltify, they might be split amongst threads
so that there is simply no other objects left to delta against within
the same thread.  Let's use the same 2*window treshold as used for the
final load balancing to allow extra threads to be created.

This fixes the benign t5300 test failure.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Tested-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-12-13 18:55:55 -08:00
Nicolas Pitre
43cc2b4266 autodetect number of CPUs by default when using threads
... and display the actual number of threads used when locally
repacking.  A remote server still won't tell you how many threads it
uses during a fetch though.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-12-12 19:47:36 -08:00
Junio C Hamano
de0db42278 Merge branch 'maint'
* maint:
  fsck: reduce stack footprint
  make sure packs to be replaced are closed beforehand
2008-12-11 00:36:31 -08:00
Nicolas Pitre
c74faea19e make sure packs to be replaced are closed beforehand
Especially on Windows where an opened file cannot be replaced, make
sure pack-objects always close packs it is about to replace. Even on
non Windows systems, this could save potential bad results if ever
objects were to be read from the new pack file using offset from the old
index.

This should fix t5303 on Windows.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Tested-by: Johannes Sixt <j6t@kdbg.org> (MinGW)
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-12-10 17:56:05 -08:00
Junio C Hamano
0fd9d7e66d Merge branch 'bc/maint-keep-pack' into maint
* bc/maint-keep-pack:
  repack: only unpack-unreachable if we are deleting redundant packs
  t7700: test that 'repack -a' packs alternate packed objects
  pack-objects: extend --local to mean ignore non-local loose objects too
  sha1_file.c: split has_loose_object() into local and non-local counterparts
  t7700: demonstrate mishandling of loose objects in an alternate ODB
  builtin-gc.c: use new pack_keep bitfield to detect .keep file existence
  repack: do not fall back to incremental repacking with [-a|-A]
  repack: don't repack local objects in packs with .keep file
  pack-objects: new option --honor-pack-keep
  packed_git: convert pack_local flag into a bitfield and add pack_keep
  t7700: demonstrate mishandling of objects in packs with a .keep file
2008-12-02 23:00:04 -08:00
Junio C Hamano
7b51b77dbc Merge branch 'np/pack-safer'
* np/pack-safer:
  t5303: fix printf format string for portability
  t5303: work around printf breakage in dash
  pack-objects: don't leak pack window reference when splitting packs
  extend test coverage for latest pack corruption resilience improvements
  pack-objects: allow "fixing" a corrupted pack without a full repack
  make find_pack_revindex() aware of the nasty world
  make check_object() resilient to pack corruptions
  make packed_object_info() resilient to pack corruptions
  make unpack_object_header() non fatal
  better validation on delta base object offsets
  close another possibility for propagating pack corruption
2008-11-12 22:26:35 -08:00
Junio C Hamano
ecbbfb15a4 Merge branch 'bc/maint-keep-pack'
* bc/maint-keep-pack:
  t7700: test that 'repack -a' packs alternate packed objects
  pack-objects: extend --local to mean ignore non-local loose objects too
  sha1_file.c: split has_loose_object() into local and non-local counterparts
  t7700: demonstrate mishandling of loose objects in an alternate ODB
  builtin-gc.c: use new pack_keep bitfield to detect .keep file existence
  repack: do not fall back to incremental repacking with [-a|-A]
  repack: don't repack local objects in packs with .keep file
  pack-objects: new option --honor-pack-keep
  packed_git: convert pack_local flag into a bitfield and add pack_keep
  t7700: demonstrate mishandling of objects in packs with a .keep file
2008-11-12 22:00:43 -08:00
Junio C Hamano
6cd3729eae Merge branch 'maint'
* maint:
  Start 1.6.0.5 cycle
  Fix pack.packSizeLimit and --max-pack-size handling
  checkout: Fix "initial checkout" detection
  Remove the period after the git-check-attr summary

Conflicts:
	RelNotes
2008-11-12 15:03:57 -08:00
Nicolas Pitre
a1e4760fcf Fix pack.packSizeLimit and --max-pack-size handling
If the limit was sufficiently low, having a single object written
could bust the limit (by design), but caused the remaining allowed
size to go negative for subsequent objects, which for an unsigned
variable is a rather huge limit.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-12 14:55:03 -08:00
Brandon Casey
daae062595 pack-objects: extend --local to mean ignore non-local loose objects too
With this patch, --local means pack only local objects that are not already
packed.

Additionally, this fixes t7700 testing whether loose objects in an alternate
object database are repacked.

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-12 10:29:22 -08:00
Brandon Casey
e96fb9b8f9 pack-objects: new option --honor-pack-keep
This adds a new option to pack-objects which will cause it to ignore an
object which appears in a local pack which has a .keep file, even if it
was specified for packing.

This option will be used by the porcelain repack.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-12 10:28:10 -08:00
Nicolas Pitre
59dd9ed183 pack-objects: don't leak pack window reference when splitting packs
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-02 15:22:35 -08:00
Nicolas Pitre
64bd76b1de pack-objects: allow "fixing" a corrupted pack without a full repack
When the pack data to be reused is found to be bad, let's fall back to
full object access through the generic path which has its own strategies
to find alternate object sources in that case.  This allows for "fixing"
a corrupted pack simply by copying either another pack containing the
object(s) found to be bad, or the loose object itself, into the object
store and launch a repack without the need for -f.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-02 15:22:35 -08:00
Nicolas Pitre
08698b1e32 make find_pack_revindex() aware of the nasty world
It currently calls die() whenever given offset is not found thinking
that such thing should never happen.  But this offset may come from a
corrupted pack whych _could_ happen and not be found.  Callers should
deal with this possibility gracefully instead.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-02 15:22:35 -08:00
Nicolas Pitre
03d660150c make check_object() resilient to pack corruptions
The check_object() function tries to get away with the least amount of
pack access possible when it already has partial information on given
object rather than calling the more costly packed_object_info().

When things don't look right, it should just give up and fall back to
packed_object_info() directly instead of die()'ing.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-02 15:22:35 -08:00
Nicolas Pitre
09ded04b7e make unpack_object_header() non fatal
It is possible to have pack corruption in the object header.  Currently
unpack_object_header() simply die() on them instead of letting the caller
deal with that gracefully.

So let's have unpack_object_header() return an error instead, and find
a better name for unpack_object_header_gently() in that context.  All
callers of unpack_object_header() are ready for it.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-02 15:22:34 -08:00
Nicolas Pitre
d8f325563d better validation on delta base object offsets
In one case, it was possible to have a bad offset equal to 0 effectively
pointing a delta onto itself and crashing git after too many recursions.
In the other cases, a negative offset could result due to off_t being
signed.  Catch those.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-02 15:22:34 -08:00
Nicolas Pitre
0e8189e270 close another possibility for propagating pack corruption
Abstract
--------

With index v2 we have a per object CRC to allow quick and safe reuse of
pack data when repacking.  This, however, doesn't currently prevent a
stealth corruption from being propagated into a new pack when _not_
reusing pack data as demonstrated by the modification to t5302 included
here.

The Context
-----------

The Git database is all checksummed with SHA1 hashes.  Any kind of
corruption can be confirmed by verifying this per object hash against
corresponding data.  However this can be costly to perform systematically
and therefore this check is often not performed at run time when
accessing the object database.

First, the loose object format is entirely compressed with zlib which
already provide a CRC verification of its own when inflating data.  Any
disk corruption would be caught already in this case.

Then, packed objects are also compressed with zlib but only for their
actual payload.  The object headers and delta base references are not
deflated for obvious performance reasons, however this leave them
vulnerable to potentially undetected disk corruptions.  Object types
are often validated against the expected type when they're requested,
and deflated size must always match the size recorded in the object header,
so those cases are pretty much covered as well.

Where corruptions could go unnoticed is in the delta base reference.
Of course, in the OBJ_REF_DELTA case,  the odds for a SHA1 reference to
get corrupted so it actually matches the SHA1 of another object with the
same size (the delta header stores the expected size of the base object
to apply against) are virtually zero.  In the OBJ_OFS_DELTA case, the
reference is a pack offset which would have to match the start boundary
of a different base object but still with the same size, and although this
is relatively much more "probable" than in the OBJ_REF_DELTA case, the
probability is also about zero in absolute terms.  Still, the possibility
exists as demonstrated in t5302 and is certainly greater than a SHA1
collision, especially in the OBJ_OFS_DELTA case which is now the default
when repacking.

Again, repacking by reusing existing pack data is OK since the per object
CRC provided by index v2 guards against any such corruptions. What t5302
failed to test is a full repack in such case.

The Solution
------------

As unlikely as this kind of stealth corruption can be in practice, it
certainly isn't acceptable to propagate it into a freshly created pack.
But, because this is so unlikely, we don't want to pay the run time cost
associated with extra validation checks all the time either.  Furthermore,
consequences of such corruption in anything but repacking should be rather
visible, and even if it could be quite unpleasant, it still has far less
severe consequences than actively creating bad packs.

So the best compromize is to check packed object CRC when unpacking
objects, and only during the compression/writing phase of a repack, and
only when not streaming the result.  The cost of this is minimal (less
than 1% CPU time), and visible only with a full repack.

Someone with a stats background could provide an objective evaluation of
this, but I suspect that it's bad RAM that has more potential for data
corruptions at this point, even in those cases where this extra check
is not performed.  Still, it is best to prevent a known hole for
corruption when recreating object data into a new pack.

What about the streamed pack case?  Well, any client receiving a pack
must always consider that pack as untrusty and perform full validation
anyway, hence no such stealth corruption could be propagated to remote
repositoryes already.  It is therefore worthless doing local validation
in that case.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-02 15:22:15 -08:00
Junio C Hamano
aebd173ffa Merge branch 'maint'
* maint:
  Start 1.6.0.4 cycle
  add instructions on how to send patches to the mailing list with Gmail
  Documentation/gitattributes: Add subsection header for each attribute
  git send-email: avoid leaking directory file descriptors.
  send-pack: do not send out single-level refs such as refs/stash
  fix overlapping memcpy in normalize_absolute_path
  pack-objects: avoid reading uninitalized data
  correct cache_entry allocation

Conflicts:
	RelNotes
2008-11-02 00:15:22 -07:00
Jeff King
421b488a58 pack-objects: avoid reading uninitalized data
In the main loop of find_deltas, we do:

  struct object_entry *entry = *list++;
  ...
  if (!*list_size)
	  ...
	  break

Because we look at and increment *list _before_ the check of
list_size, in the very last iteration of the loop we will
look at uninitialized data, and increment the pointer beyond
one past the end of the allocated space. Since we don't
actually do anything with the data until after the check,
this is not a problem in practice.

But since it technically violates the C standard, and
because it provokes a spurious valgrind warning, let's just
move the initialization of entry to a safe place.

This fixes valgrind errors in t5300, t5301, t5302, t303, and
t9400.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-01 23:46:40 -07:00
Brandon Casey
19d4b416f4 Replace xmalloc/memset(0) pairs with xcalloc
Many call sites immediately initialize allocated memory with zero after
calling xmalloc. A single call to xcalloc can replace this two-call
sequence.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2008-10-08 07:30:59 -07:00
Junio C Hamano
c4275591fb Merge branch 'maint'
* maint:
  builtin-prune.c: prune temporary packs in <object_dir>/pack directory
  Do not perform cross-directory renames when creating packs
2008-09-23 02:05:35 -07:00
Petr Baudis
8b4eb6b6cd Do not perform cross-directory renames when creating packs
A comment on top of create_tmpfile() describes caveats ('can have
problems on various systems (FAT, NFS, Coda)') that should apply
in this situation as well.  This in the end did not end up solving
any of my personal problems, but it might be a useful cleanup patch
nevertheless.

Signed-off-by: Petr Baudis <pasky@suse.cz>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-09-22 12:19:14 -07:00
Junio C Hamano
c4398286fa Merge branch 'np/pack'
* np/pack:
  t5300: improve SHA1 collision test
  pack-objects: don't include missing preferred base objects
  sha1write: don't copy full sized buffers

Conflicts:
	t/t5300-pack-object.sh
2008-09-18 20:19:30 -07:00
Heikki Orsila
f18d244a63 Start conforming code to "git subcmd" style part 3
User notifications are presented as 'git cmd', and code comments
are presented as '"cmd"' or 'git's cmd', rather than 'git-cmd'.

Signed-off-by: Heikki Orsila <heikki.orsila@iki.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-09-15 23:11:35 -07:00
Junio C Hamano
9f35e5fd54 Merge branch 'np/maint-safer-pack' into maint
* np/maint-safer-pack:
  fixup_pack_header_footer(): use nicely aligned buffer sizes
  index-pack: use fixup_pack_header_footer()'s validation mode
  pack-objects: use fixup_pack_header_footer()'s validation mode
  improve reliability of fixup_pack_header_footer()
  pack-objects: improve returned information from write_one()
2008-09-10 02:12:47 -07:00
Nicolas Pitre
eede9f42b5 pack-objects: don't include missing preferred base objects
This improves commit 6d6f9cddbe a bit by simply not including missing
bases in the list of objects to process at all.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-09-02 17:05:54 -07:00
Junio C Hamano
59d94bc9c8 Merge branch 'np/maint-safer-pack' into np/pack
* np/maint-safer-pack:
  fixup_pack_header_footer(): use nicely aligned buffer sizes
  index-pack: use fixup_pack_header_footer()'s validation mode
  pack-objects: use fixup_pack_header_footer()'s validation mode
  improve reliability of fixup_pack_header_footer()
  pack-objects: improve returned information from write_one()
2008-09-02 17:05:44 -07:00
Junio C Hamano
df85f7855d Merge branch 'sp/missing-thin-base' into maint
* sp/missing-thin-base:
  pack-objects: Allow missing base objects when creating thin packs
2008-08-30 08:38:19 -07:00
Nicolas Pitre
ac0463ed44 pack-objects: use fixup_pack_header_footer()'s validation mode
When limiting the pack size, a new header has to be written to the
pack and a new SHA1 computed.  Make sure that the SHA1 of what is being
read back matches the SHA1 of what was written.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-29 21:51:27 -07:00
Nicolas Pitre
abeb40e5aa improve reliability of fixup_pack_header_footer()
Currently, this function has the potential to read corrupted pack data
from disk and give it a valid SHA1 checksum.  Let's add the ability to
validate SHA1 checksum of existing data along the way, including before
and after any arbitrary point in the pack.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-29 21:51:27 -07:00
Nicolas Pitre
6ed7f25e95 pack-objects: improve returned information from write_one()
This function returns 0 when the current object couldn't be written
due to the pack size limit, otherwise the current offset in the pack.
There is a problem with this approach however, since current object
could be a delta and its delta base might just have been written in
the same write_one() call, but those successfully written objects are
not accounted in the offset variable tracked by the caller. Currently
this is not an issue but a subsequent patch will need this.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-29 21:51:27 -07:00
Linus Torvalds
0c68d386da index-pack: be careful after fixing up the header/footer
The index-pack command, when processing a thin pack, fixed up the pack
after-the-fact.  It forgets to fsync the result, because it only did that
in one path rather in all cases of fixup.

This moves the fsync_or_die() to the fix-up routine itself, rather than
doing it in one of the callers, so that all cases are covered.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-27 13:33:56 -07:00
Shawn O. Pearce
6d6f9cddbe pack-objects: Allow missing base objects when creating thin packs
If we are building a thin pack and one of the base objects we would
consider for deltification is missing its OK, the other side already
has that base object.  We may be able to get a delta from another
object, or we can simply send the new object whole (no delta).

This change allows a shallow clone to store only the objects which
are unique to it, as well as the boundary commit and its trees, but
avoids storing the boundary blobs.  This special form of a shallow
clone is able to represent just the difference between two trees.

Pack objects change suggested by Nicolas Pitre.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Acked-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-12 15:39:46 -07:00
Ramsay Jones
6e1c23442a Fix some warnings (on cygwin) to allow -Werror
When printing valuds of type uint32_t, we should use PRIu32, and should
not assume that it is unsigned int.  On 32-bit platforms, it could be
defined as unsigned long. The same caution applies to ntohl().

Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-05 17:26:29 -07:00
Nicolas Pitre
c41a4a9468 verify-pack: check packed object CRC when using index version 2
To do so, check_pack_crc() moved from builtin-pack-objects.c to
pack-check.c where it is more logical to share.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-24 23:58:57 -07:00
Nicolas Pitre
1f5c74f6cf call init_pack_revindex() lazily
This makes life much easier for next patch, as well as being more efficient
when the revindex is actually not used.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-23 21:25:20 -07:00
Linus Torvalds
54352bb274 Remove now unnecessary 'sync()' calls
Since the pack-files are now always created stably on disk, there is no
need to sync() before pruning lose objects or old stale pack-files.

[jc: with Nico's clean-up]

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-31 14:49:29 -07:00
Linus Torvalds
4c81b03e30 Make pack creation always fsync() the result
This means that we can depend on packs always being stable on disk,
simplifying a lot of the object serialization worries.  And unlike loose
objects, serializing pack creation IO isn't going to be a performance
killer.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-31 14:46:57 -07:00
Junio C Hamano
9bd81e4249 Merge branch 'js/config-cb'
* js/config-cb:
  Provide git_config with a callback-data parameter

Conflicts:

	builtin-add.c
	builtin-cat-file.c
2008-05-25 14:25:02 -07:00
Junio C Hamano
e5e9714a10 Merge branch 'bc/repack'
* bc/repack:
  Documentation/git-repack.txt: document new -A behaviour
  let pack-objects do the writing of unreachable objects as loose objects
  add a force_object_loose() function
  builtin-gc.c: deprecate --prune, it now really has no effect
  git-gc: always use -A when manually repacking
  repack: modify behavior of -A option to leave unreferenced objects unpacked

Conflicts:

	builtin-pack-objects.c
2008-05-23 16:06:01 -07:00
Johannes Schindelin
ef90d6d420 Provide git_config with a callback-data parameter
git_config() only had a function parameter, but no callback data
parameter.  This assumes that all callback functions only modify
global variables.

With this patch, every callback gets a void * parameter, and it is hoped
that this will help the libification effort.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-14 12:34:44 -07:00
Nicolas Pitre
ca11b212eb let pack-objects do the writing of unreachable objects as loose objects
Commit ccc1297226 changed the behavior
of 'git repack -A' so unreachable objects are stored as loose objects.
However it did so in a naive and inn efficient way by making packs
about to be deleted inaccessible and feeding their content through
'git unpack-objects'.  While this works, there are major flaws with
this approach:

- It is unacceptably sloooooooooooooow.

  In the Linux kernel repository with no actual unreachable objects,
  doing 'git repack -A -d' before:

	real    2m33.220s
	user    2m21.675s
	sys     0m3.510s

  And with this change:

	real    0m36.849s
	user    0m24.365s
	sys     0m1.950s

  For reference, here's the timing for 'git repack -a -d':

	real    0m35.816s
	user    0m22.571s
	sys     0m2.011s

  This is explained by the fact that 'git unpack-objects' was used to
  unpack _every_ objects even if (almost) 100% of them were thrown away.

- There is a black out period.

  Between the removal of the .idx file for the redundant pack and the
  completion of its unpacking, the unreachable objects become completely
  unaccessible.  This is not a big issue as we're talking about unreachable
  objects, but some consistency is always good.

- There is no way to easily set a sensible mtime for the newly created
  unreachable loose objects.

So, while having a command called "pack-objects" to perform object
unpacking looks really odd, this is probably the best compromize to be
able to solve the above issues in an efficient way.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-13 22:45:44 -07:00
Nicolas Pitre
70baf5d41a pack-objects: fix early eviction for max depth delta objects
The 'depth' variable doesn't reflect the actual maximum depth used
when other objects already depend on the current one.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-03 18:35:38 -07:00
Nicolas Pitre
ed4a9031ea pack-objects: allow for early delta deflating
When the delta data is cached in memory until it is written to a pack
file on disk, it is best to compress it right away in find_deltas() for
the following reasons:

  - we have to compress that data anyway;

  - this allows for caching more deltas with the same cache size limit;

  - compression is potentially threaded.

This last point is especially relevant for SMP run time.  For example,
repacking the Linux repo on a quad core processor using 4 threads with
all default settings produce the following results before this change:

	real    2m27.929s
	user    4m36.492s
	sys     0m3.091s

And with this change applied:

	real    2m13.787s
	user    4m37.486s
	sys     0m3.159s

So the actual execution time stayed more or less the same but the
wall clock time is shorter.

This is however not a good thing to do when generating a pack for
network transmission.  In that case, the network is most likely to
throttle the data throughput, so it is best to make find_deltas()
faster in order to start writing data ASAP since we can afford
spending more time between writes to compress the data
at that point.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-03 18:35:38 -07:00
Nicolas Pitre
30ebb40aa1 pack-objects: move compression code in a separate function
A later patch will make use of that code too.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-03 18:35:38 -07:00
Nicolas Pitre
2c5ef82463 pack-objects: clean up write_object() a bit
... for improved readability.  No functional changes.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-03 18:35:38 -07:00
Nicolas Pitre
bcd7954e21 pack-objects: simplify the condition associated with --all-progress
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-03 18:35:38 -07:00
Nicolas Pitre
a7de713089 pack-objects: remove some double negative logic
Parsing !no_reuse_delta everywhere makes my brain spend extra
cycles wondering each time.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-03 18:35:37 -07:00
Nicolas Pitre
3613f9b4c0 pack-objects: small cleanup
Better encapsulate delta creation for writing.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-03 18:35:37 -07:00
Nicolas Pitre
f746bae84e pack-objects: proper pack time stamping with --max-pack-size
Runtime pack access is done in the pack file mtime order since recent
packs are more likely to contain frequently used objects than old packs.
However the --max-pack-size option can produce multiple packs with mtime
in the reversed order as newer objects are always written first.

Let's modify mtime of later pack files (when any) so they appear older
than preceding ones when a repack creates multiple packs.

Signed-off-by: Nicolas Pitre <nico@cam.org>
2008-03-13 22:51:30 -07:00
Shawn O. Pearce
f0a24aa56e git-pack-objects: Automatically pack annotated tags if object was packed
The new option "--include-tag" allows the caller to request that
any annotated tag be included into the packfile if the object the tag
references was also included as part of the packfile.

This option can be useful on the server side of a native git transport,
where the server knows what commits it is including into a packfile to
update the client.  If new annotated tags have been introduced then we
can also include them in the packfile, saving the client from needing
to request them through a second connection.

This change only introduces the backend option and provides a test.
Protocol extensions to make this useful in fetch-pack/upload-pack
are still necessary to activate the logic during transport.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-03-04 23:28:14 -08:00
Junio C Hamano
c0b48ad777 Merge branch 'np/verify-pack'
* np/verify-pack:
  add storage size output to 'git verify-pack -v'
  fix unimplemented packed_object_info_detail() features
  make verify_one_pack() a bit less wrong wrt packed_git structure
  factorize revindex code out of builtin-pack-objects.c

Conflicts:

	Makefile
2008-03-02 16:07:30 -08:00
Junio C Hamano
eadbcd498a Merge branch 'mk/maint-parse-careful'
* mk/maint-parse-careful:
  receive-pack: use strict mode for unpacking objects
  index-pack: introduce checking mode
  unpack-objects: prevent writing of inconsistent objects
  unpack-object: cache for non written objects
  add common fsck error printing function
  builtin-fsck: move common object checking code to fsck.c
  builtin-fsck: reports missing parent commits
  Remove unused object-ref code
  builtin-fsck: move away from object-refs to fsck_walk
  add generic, type aware object chain walker

Conflicts:

	Makefile
	builtin-fsck.c
2008-03-02 15:11:07 -08:00
Nicolas Pitre
3449f8c4cb factorize revindex code out of builtin-pack-objects.c
No functional change. This is needed to fix verify-pack in a later patch.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-03-01 01:44:45 -08:00
Junio C Hamano
3d0a936f63 Merge branch 'jm/free'
* jm/free:
  Avoid unnecessary "if-before-free" tests.

Conflicts:

	builtin-branch.c
2008-02-27 13:03:50 -08:00
Junio C Hamano
392b78ca42 Revert "pack-objects: Print a message describing the number of threads for packing"
This reverts commit 6c723f5e6b.
The additional message may be interesting for git developers,
but not useful for the end users, and clutters the output.
2008-02-26 23:27:31 -08:00
Martin Koegler
7914053ba9 Remove unused object-ref code
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-25 23:57:35 -08:00
Brandon Casey
6c723f5e6b pack-objects: Print a message describing the number of threads for packing
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-23 12:00:32 -08:00
Andreas Ericsson
833e3df171 pack-objects: Add runtime detection of online CPU's
Packing objects can be done in parallell nowadays, but it's
only done if the config option pack.threads is set to a value
above 1. Because of that, the code-path used is often not the
most optimal one.

This patch adds a routine to detect the number of online CPU's
at runtime (online_cpus()). When pack.threads (or --threads=) is
given a value of 0, the number of threads is set to the number of
online CPU's. This feature is also documented.

As per Nicolas Pitre's recommendations, the default is still to
run pack-objects single-threaded unless explicitly activated,
either by configuration or by command line parameter.

The routine online_cpus() is a rework of "numcpus.c", written by
one Philip Willoughby <pgw99@doc.ic.ac.uk>. numcpus.c is in the
public domain and can presently be downloaded from
http://csgsoft.doc.ic.ac.uk/numcpus/

Signed-off-by: Andreas Ericsson <ae@op5.se>
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-23 12:00:32 -08:00
Jim Meyering
8e0f70033b Avoid unnecessary "if-before-free" tests.
This change removes all obvious useless if-before-free tests.
E.g., it replaces code like this:

        if (some_expression)
                free (some_expression);

with the now-equivalent:

        free (some_expression);

It is equivalent not just because POSIX has required free(NULL)
to work for a long time, but simply because it has worked for
so long that no reasonable porting target fails the test.
Here's some evidence from nearly 1.5 years ago:

    http://www.winehq.org/pipermail/wine-patches/2006-October/031544.html

FYI, the change below was prepared by running the following:

  git ls-files -z | xargs -0 \
  perl -0x3b -pi -e \
    's/\bif\s*\(\s*(\S+?)(?:\s*!=\s*NULL)?\s*\)\s+(free\s*\(\s*\1\s*\))/$2/s'

Note however, that it doesn't handle brace-enclosed blocks like
"if (x) { free (x); }".  But that's ok, since there were none like
that in git sources.

Beware: if you do use the above snippet, note that it can
produce syntactically invalid C code.  That happens when the
affected "if"-statement has a matching "else".
E.g., it would transform this

  if (x)
    free (x);
  else
    foo ();

into this:

  free (x);
  else
    foo ();

There were none of those here, either.

If you're interested in automating detection of the useless
tests, you might like the useless-if-before-free script in gnulib:
[it *does* detect brace-enclosed free statements, and has a --name=S
 option to make it detect free-like functions with different names]

  http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=blob;f=build-aux/useless-if-before-free

Addendum:
  Remove one more (in imap-send.c), spotted by Jean-Luc Herren <jlh@gmx.ch>.

Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-22 14:14:40 -08:00
Martin Koegler
3d51e1b5b8 check return code of prepare_revision_walk
A failure in prepare_revision_walk can be caused by
a not parseable object.

Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-17 23:51:12 -08:00
Junio C Hamano
aa8d53ec38 Merge branch 'maint'
* maint:
  config: add test cases for empty value and no value config variables.
  cvsimport: have default merge regex also match beginning of commit message
  git clone -s documentation: force a new paragraph for the NOTE
  status: suggest "git rm --cached" to unstage for initial commit
  Protect get_author_ident_from_commit() from filenames in work tree
  upload-pack: Initialize the exec-path.
  bisect: use verbatim commit subject in the bisect log
  git-cvsimport.txt: fix '-M' description.
  Revert "pack-objects: only throw away data during memory pressure"
2008-02-13 14:33:19 -08:00
Junio C Hamano
75ad235c2e Revert "pack-objects: only throw away data during memory pressure"
This reverts commit 9c2174350c.

Nico analyzed and found out that this does not really help, and
I agree with it.

By the time this gets into action and data is actively thrown
away, performance simply goes down the drain due to the data
constantly being reloaded over and over and over and over and
over and over again, to the point of virtually making no
relative progress at all.  The previous behavior of enforcing
the memory limit by dynamically shrinking the window size at
least had the effect of allowing some kind of progress, even if
the end result wouldn't be optimal.

And that's the whole point behind this memory limiting feature:
allowing some progress to be made when resources are too limited
to let the repack go unbounded.
2008-02-12 23:39:03 -08:00
Junio C Hamano
04f32cf1b3 Merge branch 'maint'
* maint: (35 commits)
  config.c: guard config parser from value=NULL
  builtin-log.c: guard config parser from value=NULL
  imap-send.c: guard config parser from value=NULL
  wt-status.c: guard config parser from value=NULL
  setup.c: guard config parser from value=NULL
  remote.c: guard config parser from value=NULL
  merge-recursive.c: guard config parser from value=NULL
  http.c: guard config parser from value=NULL
  help.c: guard config parser from value=NULL
  git.c: guard config parser from value=NULL
  diff.c: guard config parser from value=NULL
  convert.c: guard config parser from value=NULL
  connect.c: guard config parser from value=NULL
  builtin-tag.c: guard config parser from value=NULL
  builtin-show-branch.c: guard config parser from value=NULL
  builtin-reflog.c: guard config parser from value=NULL
  builtin-log.c: guard config parser from value=NULL
  builtin-config.c: guard config parser from value=NULL
  builtin-commit.c: guard config parser from value=NULL
  builtin-branch.c: guard config parser from value=NULL
  ...
2008-02-11 13:23:06 -08:00
Martin Koegler
9c2174350c pack-objects: only throw away data during memory pressure
If pack-objects hit the memory limit, it deletes objects from the delta
window.

This patch make it only delete the data, which is recomputed, if needed again.

Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Acked-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-11 12:24:33 -08:00
Johannes Schindelin
2b84b5a874 Introduce the config variable pack.packSizeLimit
"git pack-objects" has the option --max-pack-size to limit the file
size of the packs to a certain amount of bytes.  On platforms where
the pack file size is limited by filesystem constraints, it is easy
to forget this option, and this option does not exist for "git gc"
to begin with.

So introduce a config variable to set the default maximum, but make
this overrideable by the command line.

Suggested by Tor Arvid Lund.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-09 23:41:34 -08:00
Nicolas Pitre
6fc74703de pack-objects: Fix segfault when object count is less than thread count
When partitioning the work amongst threads, dividing the number of
objects by the number of threads may return 0 when there are less
objects than threads; this will cause the subsequent code to segfault
when accessing list[sub_size-1].  Allow some threads to have
zero objects to work on instead of barfing, while letting others
to have more.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-21 17:24:12 -08:00
Junio C Hamano
8c3c7b2adb pack-objects: remove redundant and wrong call to deflateEnd()
We somehow called deflateEnd() on a stream that we have called
deflateEnd() on already.

In fact, the second deflateEnd() has always been returning
Z_STREAM_ERROR.  We just never checked the error return from
that particular deflateEnd().

The first one returns 0 for success.  We might want to tighten
the check even more to check that.

Noticed by Marco.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-10 23:27:39 -08:00
Jim Meyering
872c930dcb Don't access line[-1] for a zero-length "line" from fgets.
A NUL byte at beginning of file, or just after a newline
would provoke an invalid buf[-1] access in a few places.

* builtin-grep.c (cmd_grep): Don't access buf[-1].
* builtin-pack-objects.c (get_object_list): Likewise.
* builtin-rev-list.c (read_revisions_from_stdin): Likewise.
* bundle.c (read_bundle_header): Likewise.
* server-info.c (read_pack_info_file): Likewise.
* transport.c (insert_packed_refs): Likewise.

Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-04 12:28:58 -08:00
Johannes Sixt
68e6a4f80d Plug a resource leak in threaded pack-objects code.
A mutex and a condition variable is allocated for each thread and torn
down when the thread terminates. However, for certain workloads it can
happen that some threads are actually not started at all. In this case
we would leak the mutex and condition variable. Now we allocate them only
for those threads that are actually started.

Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-17 16:08:40 -08:00
Johannes Sixt
50f22ada52 threaded pack-objects: Use condition variables for thread communication.
In the threaded pack-objects code the main thread and the worker threads
must mutually signal that they have assigned a new pack of work or have
completed their work, respectively. Previously, the code used mutexes that
were locked in one thread and unlocked from a different thread, which is
bogus (and happens to work on Linux).

Here we rectify the implementation by using condition variables: There is
one condition variable on which the main thread waits until a thread
requests new work; and each worker thread has its own condition variable
on which it waits until it is assigned new work or signaled to terminate.

As a cleanup, the worker threads are spawned only after the initial work
packages have been assigned.

Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Acked-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-16 19:26:12 -08:00
Nicolas Pitre
eb9688ff65 pack-objects: more threaded load balancing fix with often changed paths
The code that splits the object list amongst work threads tries to do so
on "path" boundaries not to prevent good delta matches.  However, in
some cases, a few paths may largely dominate the hash distribution and
it is not possible to have good load balancing without ignoring those
boundaries.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-10 17:10:16 -08:00
Nicolas Pitre
384b32c09b pack-objects: fix threaded load balancing
The current method consists of a master thread serving chunks of objects
to work threads when they're done with their previous chunk.  The issue
is to determine the best chunk size: making it too large creates poor
load balancing, while making it too small has a negative effect on pack
size because of the increased number of chunk boundaries and poor delta
window utilization.

This patch implements a completely different approach by initially
splitting the work in large chunks uniformly amongst all threads, and
whenever a thread is done then it steals half of the remaining work from
another thread with the largest amount of unprocessed objects.

This has the advantage of greatly reducing the number of chunk boundaries
with an almost perfect load balancing.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-08 03:38:36 -08:00
Nicolas Pitre
b904166ccb pack-objects: reverse the delta search sort list
It is currently sorted and then walked backward.  Not only this doesn't
feel natural for my poor brain, but it would make the next patch less
obvious as well.

So reverse the sort order, and reverse the list walking direction,
which effectively produce the exact same end result as before.

Also bring the relevant comment nearer the actual code and adjust it
accordingly, with minor additional clarifications.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-08 03:38:35 -08:00
Nicolas Pitre
b7a28f7827 pack-objects: fix delta cache size accounting
The wrong value was substracted from delta_cache_size when replacing
a cached delta, as trg_entry->delta_size was used after the old size
had been replaced by the new size.

Noticed by Linus.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-08 03:38:35 -08:00
Linus Torvalds
4d1012c370 Fix rev-list when showing objects involving submodules
The function mark_tree_uninteresting() assumed that the tree entries
are blob when they are not trees.  This is not so.  Since we do
not traverse into submodules (yet), the gitlinks should be ignored.

In general, we should try to start moving away from using the
"S_ISLNK()" like things for internal git state. It was a mistake to
just assume the numbers all were same across all systems in the first
place.  This implementation converts to the "object_type", and then
uses a case statement.

Noticed by Ilari on IRC.
Test script taken from an earlier version by Dscho.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-14 03:44:22 -08:00
Junio C Hamano
e091653951 Merge branch 'np/pack'
* np/pack:
  pack-objects: get rid of an ugly cast
  make the pack index version configurable

Conflicts:

	builtin-pack-objects.c
2007-11-04 01:11:17 -07:00
Nicolas Pitre
79814f425c pack-objects: get rid of an ugly cast
... when calling write_idx_file().

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-02 01:41:04 -07:00
Nicolas Pitre
4d00bda2aa make the pack index version configurable
It is a good idea to use pack index version 2 all the time since it has
proper protection against propagation of certain pack corruptions when
repacking which is not possible with index version 1, as demonstrated
in test t5302.

Hence this config option.

The default is still pack index version 1.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-02 01:32:02 -07:00
Nicolas Pitre
2a128d63dc add throughput display to git-push
This one triggers only when git-pack-objects is called with
--all-progress and --stdout which is the combination used by
git-push.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30 16:08:40 -07:00
Nicolas Pitre
4d4fcc5451 relax usage of the progress API
Since it is now OK to pass a null pointer to display_progress() and
stop_progress() resulting in a no-op, then we can simplify the code
and remove a bunch of lines by not making those calls conditional all
the time.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30 16:08:40 -07:00
Nicolas Pitre
dc6a0757c4 make struct progress an opaque type
This allows for better management of progress "object" existence,
as well as making the progress display implementation more independent
from its callers.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30 16:08:40 -07:00
Shawn O. Pearce
9c60a966f2 Change 'Deltifying objects' to 'Compressing objects'
Recently I was referred to the Grammar Police as the git-pack-objects
progress message 'Deltifying %u objects' is considered to be not
proper English to at least some small but vocal segment of the
English speaking population.  Techncially we are applying delta
compression to these objects at this stage, so the new term is
slightly more acceptable to the Grammar Police but is also just
as correct.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-18 23:34:09 -04:00
Nicolas Pitre
4049b9cfc0 fix const issues with some functions
Two functions, namely write_idx_file() and open_pack_file(), currently
return a const pointer.  However that pointer is either a copy of the
first argument, or set to a malloc'd buffer when that first argument
is null.  In the later case it is wrong to qualify that pointer as const
since ownership of the buffer is transferred to the caller to dispose of,
and obviously the free() function is not meant to be passed const
pointers.

Making the return pointer not const causes a warning when the first
argument is returned since that argument is also marked const.

The correct thing to do is therefore to remove the const qualifiers,
avoiding the need for ugly casts only to silence some warnings.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-17 02:54:57 -04:00
Nicolas Pitre
7ba502c47b pack-objects.c: fix some global variable abuse and memory leaks
To keep things well layered, sha1close() now returns the file descriptor
when it doesn't close the file.

An ugly cast was added to the return of write_idx_file() to avoid a
warning.  A proper fix will come separately.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-17 02:54:56 -04:00
Nicolas Pitre
2f8b89472c pack-objects: no delta possible with only one object in the list
... so don't even try in that case, and save another useless line of
progress display.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-17 02:54:56 -04:00
Nicolas Pitre
42e18fbf5f more compact progress display
Each progress can be on a single line instead of two.

[sp: Changed "Checking files out" to "Checking out files" at
     Johannes Sixt's suggestion as it better explains the
	 action that is taking place]

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-17 02:54:55 -04:00
Junio C Hamano
0341091a9e Merge branch 'jc/autogc'
* jc/autogc:
  git-gc --auto: run "repack -A -d -l" as necessary.
  git-gc --auto: restructure the way "repack" command line is built.
  git-gc --auto: protect ourselves from accumulated cruft
  git-gc --auto: add documentation.
  git-gc --auto: move threshold check to need_to_gc() function.
  repack -A -d: use --keep-unreachable when repacking
  pack-objects --keep-unreachable
  Export matches_pack_name() and fix its return value
  Invoke "git gc --auto" from commit, merge, am and rebase.
  Implement git gc --auto
2007-10-03 03:05:32 -07:00
Junio C Hamano
08cdfb1337 pack-objects --keep-unreachable
This new option is meant to be used in conjunction with the
options "git repack -a -d" usually invokes the underlying
pack-objects with.  When this option is given, objects unreachable
from the refs in packs named with --unpacked= option are added
to the resulting pack, in addition to the reachable objects that
are not in packs marked with *.keep files.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-17 12:25:26 -07:00
Junio C Hamano
e1ef867328 builtin-pack-objects.c: avoid bogus gcc warnings
These empty statement marcos can solicit bogus "statement with no effect"
warnings; squelch them.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-14 22:30:20 -07:00
Nicolas Pitre
3c70183918 threaded delta search: proper locking for cache accounting
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-12 12:56:09 -07:00
Nicolas Pitre
693b86fffb threaded delta search: add pack.threads config variable
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-10 10:50:21 -07:00
Nicolas Pitre
b81d9af71e fix threaded delta search locking
Found by Jeff King.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-10 10:49:11 -07:00
Nicolas Pitre
367f4a4343 threaded delta search: specify number of threads at run time
This adds a --threads=<n> parameter to 'git pack-objects' with
documentation.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-09 22:26:06 -07:00