Commit Graph

50 Commits

Author SHA1 Message Date
Junio C Hamano
88a516aca0 Merge branch 'ds/repack-fixlets'
Two fixes around "git repack".

* ds/repack-fixlets:
  repack: make '--quiet' disable progress
  repack: respect kept objects with '--write-midx -b'
2022-01-05 14:01:30 -08:00
Derrick Stolee
47ca93d071 repack: make '--quiet' disable progress
While testing some ideas in 'git repack', I ran it with '--quiet' and
discovered that some progress output was still shown. Specifically, the
output for writing the multi-pack-index showed the progress.

The 'show_progress' variable in cmd_repack() is initialized with
isatty(2) and is not modified at all by the '--quiet' flag. The
'--quiet' flag modifies the po_args.quiet option which is translated
into a '--quiet' flag for the 'git pack-objects' child process. However,
'show_progress' is used to directly send progress information to the
multi-pack-index writing logic which does not use a child process.

The fix here is to modify 'show_progress' to be false if po_opts.quiet
is true, and isatty(2) otherwise. This new expectation simplifies a
later condition that checks both.

Update the documentation to make it clear that '-q' will disable all
progress in addition to ensuring the 'git pack-objects' child process
will receive the flag.

Use 'test_terminal' to check that this works to get around the isatty(2)
check.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-20 11:59:17 -08:00
Derrick Stolee
e4d0c11c04 repack: respect kept objects with '--write-midx -b'
Historically, we needed a single packfile in order to have reachability
bitmaps. This introduced logic that when 'git repack' had a '-b' option
that we should stop sending the '--honor-pack-keep' option to the 'git
pack-objects' child process, ensuring that we create a packfile
containing all reachable objects.

In the world of multi-pack-index bitmaps, we no longer need to repack
all objects into a single pack to have valid bitmaps. Thus, we should
continue sending the '--honor-pack-keep' flag to 'git pack-objects'.

The fix is very simple: only disable the flag when writing bitmaps but
also _not_ writing the multi-pack-index.

This opens the door to new repacking strategies that might want to keep
some historical set of objects in a stable pack-file while only
repacking more recent objects.

To test, create a new 'test_subcommand_inexact' helper that is more
flexible than 'test_subcommand'. This allows us to look for the
--honor-pack-keep flag without over-indexing on the exact set of
arguments.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-20 11:58:31 -08:00
Eric Sunshine
0c51d6b4ae t6000-t9999: detect and signal failure within loop
Failures within `for` and `while` loops can go unnoticed if not detected
and signaled manually since the loop itself does not abort when a
contained command fails, nor will a failure necessarily be detected when
the loop finishes since the loop returns the exit code of the last
command it ran on the final iteration, which may not be the command
which failed. Therefore, detect and signal failures manually within
loops using the idiom `|| return 1` (or `|| exit 1` within subshells).

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-13 10:29:48 -08:00
Taylor Blau
324efc90d1 builtin/repack.c: pass --refs-snapshot when writing bitmaps
To prevent the race described in an earlier patch, generate and pass a
reference snapshot to the multi-pack bitmap code, if we are writing one
from `git repack`.

This patch is mostly limited to creating a temporary file, and then
calling for_each_ref(). Except we try to minimize duplicates, since
doing so can drastically reduce the size in network-of-forks style
repositories. In the kernel's fork network (the repository containing
all objects from the kernel and all its forks), deduplicating the
references drops the snapshot size from 934 MB to just 12 MB.

But since we're handling duplicates in this way, we have to make sure
that we preferred references (those listed in pack.preferBitmapTips)
before non-preferred ones (to avoid recording an object which is pointed
at by a preferred tip as non-preferred).

We accomplish this by doing separate passes over the references: first
visiting each prefix in pack.preferBitmapTips, and then over the rest of
the references.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-01 16:40:09 -07:00
Taylor Blau
1d89d88d37 builtin/repack.c: support writing a MIDX while repacking
Teach `git repack` a new `--write-midx` option for callers that wish to
persist a multi-pack index in their repository while repacking.

There are two existing alternatives to this new flag, but they don't
cover our particular use-case. These alternatives are:

  - Call 'git multi-pack-index write' after running 'git repack', or

  - Set 'GIT_TEST_MULTI_PACK_INDEX=1' in your environment when running
    'git repack'.

The former works, but introduces a gap in bitmap coverage between
repacking and writing a new MIDX (since the repack may have deleted a
pack included in the existing MIDX, invalidating it altogether).

Setting the 'GIT_TEST_' environment variable is obviously unsupported.
In fact, even if it were supported officially, it still wouldn't work,
because it generates the MIDX *after* redundant packs have been dropped,
leading to the same issue as above.

Introduce a new option which eliminates this race by teaching `git
repack` to generate the MIDX at the critical point: after the new packs
have been written and moved into place, but before the redundant packs
have been removed.

This option is compatible with `git repack`'s '--bitmap' option (it
changes the interpretation to be: "write a bitmap corresponding to the
MIDX after one has been generated").

There is a little bit of additional noise in the patch below to avoid
repeating ourselves when selecting which packs to delete. Instead of a
single loop as before (where we iterate over 'existing_packs', decide if
a pack is worth deleting, and if so, delete it), we have two loops (the
first where we decide which ones are worth deleting, and the second
where we actually do the deleting). This makes it so we have a single
check we can make consistently when (1) telling the MIDX which packs we
want to exclude, and (2) actually unlinking the redundant packs.

There is also a tiny change to short-circuit the body of
write_midx_included_packs() when no packs remain in the case of an empty
repository. The MIDX code does not handle this, so avoid trying to
generate a MIDX covering zero packs in the first place.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28 21:20:56 -07:00
Taylor Blau
4b58b6f7b7 t7700: update to work with MIDX bitmap test knob
A number of these tests are focused only on pack-based bitmaps and need
to be updated to disable 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' where
necessary.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01 13:56:43 -07:00
Denton Liu
d9b31db2c4 t7700: stop losing return codes of git commands
In a pipe, only the return code of the last command is used. Thus, all
other commands will have their return codes masked. Rewrite pipes so
that there are no git commands upstream so that we will know if a
command fails.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-04 14:25:05 -08:00
Denton Liu
3699d69df0 t7700: make references to SHA-1 generic
Make the test more hash-agnostic by renaming variables from "sha1" to
some variation of "oid" or "packid". Also, replace the regex,
`[0-9a-f]\{40\}` with `$OID_REGEX`.

A better name for "incrpackid" (incremental pack-id) might have been
just "packid". However, later in the test suite, we have other uses of
"packid". Although the scopes of these variables don't conflict, a
future developer may think that commit_and_pack() and
test_has_duplicate_object() are semantically related somehow since they
share the same variable name. Give them distinct names so that it's
clear these uses are unrelated.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-04 14:25:05 -08:00
Denton Liu
dcf9a748ca t7700: replace egrep with grep
The egrep expressions in this test suite were of the form `^$variable`.
Although egrep works just fine, it's overkill since we're not using any
extended regex. Replace egrep invocations with grep so that we aren't
swatting flies with a sledgehammer.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-04 14:25:05 -08:00
Denton Liu
cfe5eda02a t7700: consolidate code into test_has_duplicate_object()
The code to test that objects were not duplicated from the packfile was
duplicated many times. Extract the duplicated code into
test_has_duplicate_object() and use that instead.

Refactor the resulting extraction so that if the git command fails,
the return code is not silently lost.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-04 14:25:05 -08:00
Denton Liu
ae475afc0f t7700: consolidate code into test_no_missing_in_packs()
The code to test that objects were not missing from the packfile was
duplicated many times. Extract the duplicated code into
test_no_missing_in_packs() and use that instead.

Refactor the resulting extraction so that if any git commands fail,
their return codes are not silently lost.

Instead of verifying each file of `alt_objects/pack/*.idx` individually
in a for-loop, batch them together into one verification step.

The original testing construct was O(n^2): it used a grep in a loop to
test whether any objects were missing in the packfile. Rewrite this to
extract the hash using sed or cut, sort the files, then use `comm -23`
so that finding missing lines from the original file is done more
efficiently.

While we're at it, add a space to `commit_and_pack ()` for style.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-04 14:25:05 -08:00
Denton Liu
17a4ae92ea t7700: s/test -f/test_path_is_file/
Since we have debugging-friendly alternatives to `test -f`, replace
instances of `test -f` with `test_path_is_file` so that if a command
ever fails, we get better debugging information.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-29 13:20:15 -08:00
Denton Liu
d2eee32a89 t7700: move keywords onto their own line
The code style for tests is to have statements on their own line if
possible. Move keywords onto their own line so that they conform with
the test style.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-29 13:20:15 -08:00
Denton Liu
7a1c8c2346 t7700: remove spaces after redirect operators
For shell scripts, the usual convention is for there to be no space
after redirection operators, (e.g. `>file`, not `> file`). Remove these
spaces wherever they appear.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-29 13:20:15 -08:00
Denton Liu
09279086e8 t7700: drop redirections to /dev/null
Since output is silenced when running without `-v` and debugging output
is useful with `-v`, remove redirections to /dev/null as it is not
useful.

In one case where the output of stdout is consumed, redirect the output
of test_commit to stderr.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-29 13:20:15 -08:00
Jeff King
7ff024e7b3 repack: simplify handling of auto-bitmaps and .keep files
Commit 7328482253 (repack: disable bitmaps-by-default if .keep files
exist, 2019-06-29) taught repack to prefer disabling bitmaps to
duplicating objects (unless bitmaps were asked for explicitly).

But there's an easier way to do this: if we keep passing the
--honor-pack-keep flag to pack-objects when auto-enabling bitmaps, then
pack-objects already makes the same decision (it will disable bitmaps
rather than duplicate). Better still, pack-objects can actually decide
to do so based not just on the presence of a .keep file, but on whether
that .keep file actually impacts the new pack we're making (so if we're
racing with a push or fetch, for example, their temporary .keep file
will not block us from generating bitmaps if they haven't yet updated
their refs).

And because repack uses the --write-bitmap-index-quiet flag, we don't
have to worry about pack-objects generating confusing warnings when it
does see a .keep file. We can confirm this by tweaking the .keep test to
check repack's stderr.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-31 13:26:25 -07:00
Jeff King
25575015ca repack: silence warnings when auto-enabled bitmaps cannot be built
Depending on various config options, a full repack may not be able to
build a reachability bitmap index (e.g., if pack.packSizeLimit forces us
to write multiple packs). In these cases pack-objects may write a
warning to stderr.

Since 36eba0323d (repack: enable bitmaps by default on bare repos,
2019-03-14), we may generate these warnings even when the user did not
explicitly ask for bitmaps. This has two downsides:

  - it can be confusing, if they don't know what bitmaps are

  - a daemonized auto-gc will write this to its log file, and the
    presence of the warning may suppress further auto-gc (until
    gc.logExpiry has elapsed)

Let's have repack communicate to pack-objects that the choice to turn on
bitmaps was not made explicitly by the user, which in turn allows
pack-objects to suppress these warnings.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-31 13:15:51 -07:00
Jeff King
cc2649ae2d t7700: clean up .keep file in bitmap-writing test
After our test snippet finishes, the .keep file is left in place, making
it hard to do further tests of the auto-bitmap-writing code (since it
suppresses the feature completely). Let's clean it up.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-31 13:14:04 -07:00
Eric Wong
7328482253 repack: disable bitmaps-by-default if .keep files exist
Bitmaps aren't useful with multiple packs, and users with
.keep files ended up with redundant packs when bitmaps
got enabled by default in bare repos.

So detect when .keep files exist and stop enabling bitmaps
by default in that case.

Wasteful (but otherwise harmless) race conditions with .keep files
documented by Jeff King still apply and there's a chance we'd
still end up with redundant data on the FS:

  https://public-inbox.org/git/20190623224244.GB1100@sigill.intra.peff.net/

v2: avoid subshell in test case, be multi-index aware

Fixes: 36eba0323d ("repack: enable bitmaps by default on bare repos")
Signed-off-by: Eric Wong <e@80x24.org>
Helped-by: Jeff King <peff@peff.net>
Reported-by: Janos Farkas <chexum@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-01 10:29:08 -07:00
Eric Wong
36eba0323d repack: enable bitmaps by default on bare repos
A typical use case for bare repos is for serving clones and
fetches to clients.  Enable bitmaps by default on bare repos to
make it easier for admins to host git repos in a performant way.

Signed-off-by: Eric Wong <e@80x24.org>
Helped-by: Jeff King <peff@peff.net>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-18 14:09:54 +09:00
Nguyễn Thái Ngọc Duy
ed7e5fc3a2 repack: add --keep-pack option
We allow to keep existing packs by having companion .keep files. This
is helpful when a pack is permanently kept. In the next patch, git-gc
just wants to keep a pack temporarily, for one pack-objects
run. git-gc can use --keep-pack for this use case.

A note about why the pack_keep field cannot be reused and
pack_keep_in_core has to be added. This is about the case when
--keep-pack is specified together with either --keep-unreachable or
--unpack-unreachable, but --honor-pack-keep is NOT specified.

In this case, we want to exclude objects from the packs specified on
command line, not from ones with .keep files. If only one bit flag is
used, we have to clear pack_keep on pack files with the .keep file.

But we can't make any assumption about unreachable objects in .keep
packs. If "pack_keep" field is false for .keep packs, we could
potentially pull lots of unreachable objects into the new pack, or
unpack them loose. The safer approach is ignore all packs with either
.keep file or --keep-pack.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-16 13:52:29 +09:00
Nguyễn Thái Ngọc Duy
e9e33ab0fb t7700: have closing quote of a test at the beginning of line
The closing quote of a test body by convention is always at the start
of line.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-16 13:52:29 +09:00
Elia Pinto
aa14a3c105 t/t7700-repack.sh: use the $( ... ) construct for command substitution
The Git CodingGuidelines prefer the $(...) construct for command
substitution instead of using the backquotes `...`.

The backquoted form is the traditional method for command
substitution, and is supported by POSIX.  However, all but the
simplest uses become complicated quickly.  In particular, embedded
command substitutions and/or the use of double quotes require
careful escaping with the backslash character.

The patch was generated by:

for _f in $(find . -name "*.sh")
do
	perl -i -pe 'BEGIN{undef $/;} s/`(.+?)`/\$(\1)/smg'  "${_f}"
done

and then carefully proof-read.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-08 12:54:07 -08:00
Junio C Hamano
25f3119000 Merge branch 'jk/repack-pack-writebitmaps-config'
* jk/repack-pack-writebitmaps-config:
  t7700: drop explicit --no-pack-kept-objects from .keep test
  repack: introduce repack.writeBitmaps config option
  repack: simplify handling of --write-bitmap-index
  pack-objects: stop respecting pack.writebitmaps
2014-06-25 12:23:19 -07:00
Junio C Hamano
b7ce583682 Merge branch 'jk/repack-pack-keep-objects'
Recent updates to "git repack" started to duplicate objects that
are in packfiles marked with .keep flag into the new packfile by
mistake.

* jk/repack-pack-keep-objects:
  repack: s/write_bitmap/&s/ in code
  repack: respect pack.writebitmaps
  repack: do not accidentally pack kept objects by default
2014-06-25 12:21:51 -07:00
Jeff King
2d0174e38e t7700: drop explicit --no-pack-kept-objects from .keep test
We want to make sure that the default behavior of git-repack,
without any options, continues to treat .keep files as it
always has. Adding an explicit --no-pack-kept-objects, as
ee34a2b did, is a much less interesting test, and prevented
us from noticing the bug fixed by 64d3dc9 (repack: do not
accidentally pack kept objects by default, 2014-06-10).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-12 13:53:45 -07:00
Jeff King
71d76cb480 repack: introduce repack.writeBitmaps config option
We currently have pack.writeBitmaps, which originally
operated at the pack-objects level. This should really have
been a repack.* option from day one. Let's give it the more
sensible name, but keep the old version as a deprecated
synonym.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10 14:05:19 -07:00
Jeff King
3198b89fb2 repack: respect pack.writebitmaps
The config option to turn on bitmaps is read all the way
down in the plumbing of pack-objects. This makes it hard for
other options in the porcelain of repack to make decisions
based on the bitmap setting. For example,
repack.packKeptObjects tries to kick in by default only when
bitmaps are turned on. But it can't do so reliably because
it doesn't yet know whether we are using bitmaps.

This patch teaches repack to respect pack.writebitmaps. It
means we pass a redundant command-line flag to pack-objects,
but that's OK; it shouldn't affect the outcome.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10 14:01:08 -07:00
Jeff King
64d3dc9468 repack: do not accidentally pack kept objects by default
Commit ee34a2b (repack: add `repack.packKeptObjects` config
var, 2014-03-03) added a flag which could duplicate kept
objects, but did not mean to turn it on by default. Instead,
the option is tied by default to the decision to write
bitmaps, like:

  if (pack_kept_objects < 0)
	  pack_kept_objects = write_bitmap;

after which we expect pack_kept_objects to be a boolean 0 or
1.  However, that assignment neglects that write_bitmap is
_also_ a tri-state with "-1" as the default, and with
neither option given, we accidentally turn the option on.

This patch is the minimal fix to restore the desired
behavior for the default state. Further patches will fix the
more complicated cases.

Note the update to t7700. It failed to turn on bitmaps,
meaning we were actually confirming the wrong behavior!

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10 13:58:43 -07:00
Junio C Hamano
249d54b231 Merge branch 'jk/repack-pack-keep-objects'
* jk/repack-pack-keep-objects:
  repack: add `repack.packKeptObjects` config var
2014-03-18 13:50:29 -07:00
Jeff King
ee34a2bead repack: add repack.packKeptObjects config var
The git-repack command always passes `--honor-pack-keep`
to pack-objects. This has traditionally been a good thing,
as we do not want to duplicate those objects in a new pack,
and we are not going to delete the old pack.

However, when bitmaps are in use, it is important for a full
repack to include all reachable objects, even if they may be
duplicated in a .keep pack. Otherwise, we cannot generate
the bitmaps, as the on-disk format requires the set of
objects in the pack to be fully closed.

Even if the repository does not generally have .keep files,
a simultaneous push could cause a race condition in which a
.keep file exists at the moment of a repack. The repack may
try to include those objects in one of two situations:

  1. The pushed .keep pack contains objects that were
     already in the repository (e.g., blobs due to a revert of
     an old commit).

  2. Receive-pack updates the refs, making the objects
     reachable, but before it removes the .keep file, the
     repack runs.

In either case, we may prefer to duplicate some objects in
the new, full pack, and let the next repack (after the .keep
file is cleaned up) take care of removing them.

This patch introduces both a command-line and config option
to disable the `--honor-pack-keep` option.  By default, it
is triggered when pack.writeBitmaps (or `--write-bitmap-index`
is turned on), but specifying it explicitly can override the
behavior (e.g., in cases where you prefer .keep files to
bitmaps, but only when they are present).

Note that this option just disables the pack-objects
behavior. We still leave packs with a .keep in place, as we
do not necessarily know that we have duplicated all of their
objects.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-03 12:21:49 -08:00
Jeff King
a0332337be t7700: do not use "touch" unnecessarily
Some versions of touch (such as /usr/ucb/touch on Solaris)
do not know about the "-r" option. This would make sense as
a feature of test-chmtime, but fortunately this fix is even
easier.

The test does not care about the timestamp of the .keep file it
creates at all, only that it exists. For such a use case, with or
without portability issues around "-r", "touch" should not be used
in the first place.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-23 13:13:20 -08:00
Jonathan Nieder
2dec68cf8f tests: add missing &&, batch 2
Same rules as before: this patch only adds " &&" to the end of
some lines in the test suite.

Intended to be applied on top of or squashed with the last
batch if they look okay.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-09 14:45:22 -08:00
Junio C Hamano
713c79e84b more war on "sleep" in tests
Two more tests that sleep only to waste tick can be converted to use
test_tick and take expiry parameters relative to $test_tick.  The basic
idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass.

These tests are interested in expiring things with "now" as the timestamp,
soo use a timestamp relative to $test_tick to give them more stability and
reproducibility.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-17 18:20:23 -07:00
Johannes Schindelin
7f3140cd23 git repack: keep commits hidden by a graft
When you have grafts that pretend that a given commit has different
parents than the ones recorded in the commit object, it is dangerous
to let 'git repack' remove those hidden parents, as you can easily
remove the graft and end up with a broken repository.

So let's play it safe and keep those parent objects and everything
that is reachable by them, in addition to the grafted parents.

As this behavior can only be triggered by git pack-objects, and as that
command handles duplicate parents gracefully, we do not bother to cull
duplicated parents that may result by using both true and grafted
parents.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-07-24 09:10:16 -07:00
Björn Steinbrink
1ec648278e Add a test showing that 'git repack' throws away grafted-away parents
Signed-off-by: Björn Steinbrink <B.Steinbrink@gmx.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-07-23 22:07:51 -07:00
Brandon Casey
1ef2d5a640 t7700-repack: repack -a now works properly, expect success from test
Since the recent rework of the object listing mechanism of
pack-objects/rev-list, git-repack now properly packs objects from alternate
repositories even when the local repository contains packs.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-04-25 08:54:18 -07:00
Junio C Hamano
3c91bf6805 Merge branch 'jc/maint-1.6.0-keep-pack'
* jc/maint-1.6.0-keep-pack:
  pack-objects: don't loosen objects available in alternate or kept packs
  t7700: demonstrate repack flaw which may loosen objects unnecessarily
  Remove --kept-pack-only option and associated infrastructure
  pack-objects: only repack or loosen objects residing in "local" packs
  git-repack.sh: don't use --kept-pack-only option to pack-objects
  t7700-repack: add two new tests demonstrating repacking flaws

Conflicts:
	t/t7700-repack.sh
2009-04-01 22:34:19 -07:00
Brandon Casey
094085e336 pack-objects: don't loosen objects available in alternate or kept packs
If pack-objects is called with the --unpack-unreachable option then it
will unpack (i.e. loosen) all unreferenced objects from local not-kept
packs, including those that also exist in packs residing in an alternate
object database or a locally kept pack.  The only user of this option is
git-repack.

In this case, repack will follow the call to pack-objects with a call to
prune-packed, which will delete these newly loosened objects, making the
act of loosening a waste of time.  The unnecessary loosening can be
avoided by checking whether an object exists in a non-local pack or a
locally kept pack before loosening it.

This fixes the 'local packed unreachable obs that exist in alternate ODB
are not loosened' test in t7700.

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-21 21:58:44 -07:00
Brandon Casey
869a3d34c1 t7700: demonstrate repack flaw which may loosen objects unnecessarily
If an unreferenced object exists in both a local pack and in either a pack
residing in an alternate object database or a local kept pack, then the
pack-objects call made by repack will loosen that object only to have it
immediately pruned by repack's call to prune-packed.

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-21 21:31:24 -07:00
Brandon Casey
79bc4c7155 pack-objects: only repack or loosen objects residing in "local" packs
These two features were invented for use by repack when repack will delete
the local packs that have been made redundant.  The packs accessible
through alternates are not deleted by repack, so the objects contained in
them are still accessible after the local packs are deleted.  They do not
need to be repacked into the new pack or loosened.  For the case of
loosening they would immediately be deleted by the subsequent prune-packed
that is called by repack anyway.

This fixes the test
'packed unreachable obs in alternate ODB are not loosened' in t7700.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-20 13:32:33 -07:00
Brandon Casey
171110a4a6 git-repack.sh: don't use --kept-pack-only option to pack-objects
The --kept-pack-only option to pack-objects treats all kept packs as equal.
This results in objects that reside in an alternate pack that has a .keep
file, not being packed into a newly created pack when the user specifies the
-a option to repack.  Since the user may not have any control over the
alternate database, git should not refrain from repacking those objects
even though they are in a pack with a .keep file.

This fixes the 'packed obs in alternate ODB kept pack are repacked' test in
t7700.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-20 13:32:33 -07:00
Brandon Casey
92cd872202 t7700-repack: add two new tests demonstrating repacking flaws
1) The new --kept-pack-only mechansim of rev-list/pack-objects has
     replaced --unpacked=.  This new mechansim does not operate solely on
     "local" packs now.  The result is that objects residing in an alternate
     pack which has a .keep file will not be repacked with repack -a.

     This flaw is only apparent when a commit object is the one residing in
     an alternate kept pack.

  2) The 'repack unpacked objects' and 'loosen unpacked objects' mechanisms
     of pack-objects, i.e. --keep-unreachable and --unpack-unreachable,
     now do not operate solely on local packs.  The --keep-unreachable
     option no longer has any callers, but --unpack-unreachable is used when
     repack is called with '-A -d' and the local repo has existing packs.
     In this case, objects residing in alternate, not-kept packs will be
     loosened, and then immediately deleted by repack's call to
     prune-packed.

     The test must manually call pack-objects to avoid the call to
     prune-packed that is made by repack when -d is used.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-20 13:32:33 -07:00
Brandon Casey
a83c88525e t7700: demonstrate misbehavior of 'repack -a' when local packs exist
The ability to "...fatten [the] local repository by packing everything that
is needed by the local ref into a single new pack, including things that are
borrowed from alternates"[1] is supposed to be provided by the '-a' or '-A'
options to repack when '-l' is not used, but there is a flaw.  For each
pack in the local repository without a .keep file, repack supplies a
--unpacked=<pack> argument to pack-objects.

The --unpacked option to pack-objects, with or without an argument, causes
pack-objects to ignore any object which is packed in a pack not mentioned
in an argument to --unpacked=.  So, if there are local packs, and
'repack -a' is called, then any objects which reside in packs accessible
through alternates will _not_ be packed.  If there are no local packs, then
no --unpacked argument will be supplied, and repack will behave as expected.

[1] http://mid.gmane.org/7v8wrwidi3.fsf@gitster.siamese.dyndns.org

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-17 23:44:33 -08:00
Brandon Casey
3289b9dec5 t7700: test that 'repack -a' packs alternate packed objects
Previously, when 'repack -a' was called and there were no packs in the local
repository without a .keep file, the repack would fall back to calling
pack-objects with '--unpacked --incremental'. This resulted in the created
pack file, if any, to be missing the packed objects in the alternate object
store. Test that this specific case has been fixed.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-12 17:29:41 -08:00
Brandon Casey
daae062595 pack-objects: extend --local to mean ignore non-local loose objects too
With this patch, --local means pack only local objects that are not already
packed.

Additionally, this fixes t7700 testing whether loose objects in an alternate
object database are repacked.

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-12 10:29:22 -08:00
Brandon Casey
3c3df42910 t7700: demonstrate mishandling of loose objects in an alternate ODB
Loose objects residing in an alternate object database should not be packed
when the -l option to repack is used.

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-12 10:29:22 -08:00
Brandon Casey
dd718365cc repack: don't repack local objects in packs with .keep file
If the user created a .keep file for a local pack, then it can be inferred
that the user does not want those objects repacked.

This fixes the repack bug tested by t7700.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-12 10:28:12 -08:00
Brandon Casey
9245ddd515 t7700: demonstrate mishandling of objects in packs with a .keep file
Objects residing in pack files that have an associated .keep file are not
supposed to be repacked into new pack files, but they are.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-12 10:28:06 -08:00