197443e80a
`git repack` supports a `--pack-kept-objects` flag which more or less translates to whether or not we pass `--honor-pack-keep` down to `git pack-objects` when assembling a new pack. This behavior has existed sinceee34a2bead(repack: add `repack.packKeptObjects` config var, 2014-03-03). In that commit, the documentation was extended to say: [...] Note that we still do not delete `.keep` packs after `pack-objects` finishes. Unfortunately, this is not the case when `--pack-kept-objects` is combined with a `--geometric` repack. When doing a geometric repack, we include `.keep` packs when enumerating available packs only when `pack_kept_objects` is set. So this all works fine when `--no-pack-kept-objects` (or similar) is given. Kept packs are excluded from the geometric roll-up, so when we go to delete redundant packs (with `-d`), no `.keep` packs appear "below the split" in our geometric progression. But when `--pack-kept-objects` is given, things can go awry. Namely, when a kept pack is included in the list of packs tracked by the `pack_geometry` struct *and* part of the pack roll-up, we will delete the `.keep` pack when we shouldn't. Note that this *doesn't* result in object corruption, since the `.keep` pack's objects are still present in the new pack. But the `.keep` pack itself is removed, which violates our promise from back inee34a2bead. But there's more. Because `repack` computes the geometric roll-up independently from selecting which packs belong in a MIDX (with `--write-midx`), this can lead to odd behavior. Consider when a `.keep` pack appears below the geometric split (ie., its objects will be part of the new pack we generate). We'll write a MIDX containing the new pack along with the existing `.keep` pack. But because the `.keep` pack appears below the geometric split line, we'll (incorrectly) try to remove it. While this doesn't corrupt the repository, it does cause us to remove the MIDX we just wrote, since removing that pack would invalidate the new MIDX. Funny enough, this behavior became far less noticeable aftere4d0c11c04(repack: respect kept objects with '--write-midx -b', 2021-12-20), which made `pack_kept_objects` be enabled by default only when we were writing a non-MIDX bitmap. Bute4d0c11c04didn't resolve this bug, it just made it harder to notice unless callers explicitly passed `--pack-kept-objects`. The solution is to avoid trying to remove `.keep` packs during `--geometric` repacks, even when they appear below the geometric split line, which is the approach this patch implements. Co-authored-by: Victoria Dye <vdye@github.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
285 lines
7.6 KiB
Bash
Executable File
285 lines
7.6 KiB
Bash
Executable File
#!/bin/sh
|
|
|
|
test_description='git repack --geometric works correctly'
|
|
|
|
. ./test-lib.sh
|
|
|
|
GIT_TEST_MULTI_PACK_INDEX=0
|
|
|
|
objdir=.git/objects
|
|
packdir=$objdir/pack
|
|
midx=$objdir/pack/multi-pack-index
|
|
|
|
test_expect_success '--geometric with no packs' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
git repack --write-midx --geometric 2 >out &&
|
|
test_i18ngrep "Nothing new to pack" out
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric with one pack' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
test_commit "base" &&
|
|
git repack -d &&
|
|
|
|
git repack --geometric 2 >out &&
|
|
|
|
test_i18ngrep "Nothing new to pack" out
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric with an intact progression' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
# These packs already form a geometric progression.
|
|
test_commit_bulk --start=1 1 && # 3 objects
|
|
test_commit_bulk --start=2 2 && # 6 objects
|
|
test_commit_bulk --start=4 4 && # 12 objects
|
|
|
|
find $objdir/pack -name "*.pack" | sort >expect &&
|
|
git repack --geometric 2 -d &&
|
|
find $objdir/pack -name "*.pack" | sort >actual &&
|
|
|
|
test_cmp expect actual
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric with loose objects' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
# These packs already form a geometric progression.
|
|
test_commit_bulk --start=1 1 && # 3 objects
|
|
test_commit_bulk --start=2 2 && # 6 objects
|
|
# The loose objects are packed together, breaking the
|
|
# progression.
|
|
test_commit loose && # 3 objects
|
|
|
|
find $objdir/pack -name "*.pack" | sort >before &&
|
|
git repack --geometric 2 -d &&
|
|
find $objdir/pack -name "*.pack" | sort >after &&
|
|
|
|
comm -13 before after >new &&
|
|
comm -23 before after >removed &&
|
|
|
|
test_line_count = 1 new &&
|
|
test_must_be_empty removed &&
|
|
|
|
git repack --geometric 2 -d &&
|
|
find $objdir/pack -name "*.pack" | sort >after &&
|
|
|
|
# The progression (3, 3, 6) is combined into one new pack.
|
|
test_line_count = 1 after
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric with small-pack rollup' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
test_commit_bulk --start=1 1 && # 3 objects
|
|
test_commit_bulk --start=2 1 && # 3 objects
|
|
find $objdir/pack -name "*.pack" | sort >small &&
|
|
test_commit_bulk --start=3 4 && # 12 objects
|
|
test_commit_bulk --start=7 8 && # 24 objects
|
|
find $objdir/pack -name "*.pack" | sort >before &&
|
|
|
|
git repack --geometric 2 -d &&
|
|
|
|
# Three packs in total; two of the existing large ones, and one
|
|
# new one.
|
|
find $objdir/pack -name "*.pack" | sort >after &&
|
|
test_line_count = 3 after &&
|
|
comm -3 small before | tr -d "\t" >large &&
|
|
grep -qFf large after
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric with small- and large-pack rollup' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
# size(small1) + size(small2) > size(medium) / 2
|
|
test_commit_bulk --start=1 1 && # 3 objects
|
|
test_commit_bulk --start=2 1 && # 3 objects
|
|
test_commit_bulk --start=2 3 && # 7 objects
|
|
test_commit_bulk --start=6 9 && # 27 objects &&
|
|
|
|
find $objdir/pack -name "*.pack" | sort >before &&
|
|
|
|
git repack --geometric 2 -d &&
|
|
|
|
find $objdir/pack -name "*.pack" | sort >after &&
|
|
comm -12 before after >untouched &&
|
|
|
|
# Two packs in total; the largest pack from before running "git
|
|
# repack", and one new one.
|
|
test_line_count = 1 untouched &&
|
|
test_line_count = 2 after
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric ignores kept packs' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
test_commit kept && # 3 objects
|
|
test_commit pack && # 3 objects
|
|
|
|
KEPT=$(git pack-objects --revs $objdir/pack/pack <<-EOF
|
|
refs/tags/kept
|
|
EOF
|
|
) &&
|
|
PACK=$(git pack-objects --revs $objdir/pack/pack <<-EOF
|
|
refs/tags/pack
|
|
^refs/tags/kept
|
|
EOF
|
|
) &&
|
|
|
|
# neither pack contains more than twice the number of objects in
|
|
# the other, so they should be combined. but, marking one as
|
|
# .kept on disk will "freeze" it, so the pack structure should
|
|
# remain unchanged.
|
|
touch $objdir/pack/pack-$KEPT.keep &&
|
|
|
|
find $objdir/pack -name "*.pack" | sort >before &&
|
|
git repack --geometric 2 -d &&
|
|
find $objdir/pack -name "*.pack" | sort >after &&
|
|
|
|
# both packs should still exist
|
|
test_path_is_file $objdir/pack/pack-$KEPT.pack &&
|
|
test_path_is_file $objdir/pack/pack-$PACK.pack &&
|
|
|
|
# and no new packs should be created
|
|
test_cmp before after &&
|
|
|
|
# Passing --pack-kept-objects causes packs with a .keep file to
|
|
# be repacked, too.
|
|
git repack --geometric 2 -d --pack-kept-objects &&
|
|
|
|
# After repacking, two packs remain: one new one (containing the
|
|
# objects in both the .keep and non-kept pack), and the .keep
|
|
# pack (since `--pack-kept-objects -d` does not actually delete
|
|
# the kept pack).
|
|
find $objdir/pack -name "*.pack" >after &&
|
|
test_line_count = 2 after
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric ignores --keep-pack packs' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
# Create two equal-sized packs
|
|
test_commit kept && # 3 objects
|
|
git repack -d &&
|
|
test_commit pack && # 3 objects
|
|
git repack -d &&
|
|
|
|
find $objdir/pack -type f -name "*.pack" | sort >packs.before &&
|
|
git repack --geometric 2 -dm \
|
|
--keep-pack="$(basename "$(head -n 1 packs.before)")" >out &&
|
|
find $objdir/pack -type f -name "*.pack" | sort >packs.after &&
|
|
|
|
# Packs should not have changed (only one non-kept pack, no
|
|
# loose objects), but $midx should now exist.
|
|
grep "Nothing new to pack" out &&
|
|
test_path_is_file $midx &&
|
|
|
|
test_cmp packs.before packs.after &&
|
|
|
|
git fsck
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric chooses largest MIDX preferred pack' '
|
|
git init geometric &&
|
|
test_when_finished "rm -fr geometric" &&
|
|
(
|
|
cd geometric &&
|
|
|
|
# These packs already form a geometric progression.
|
|
test_commit_bulk --start=1 1 && # 3 objects
|
|
test_commit_bulk --start=2 2 && # 6 objects
|
|
ls $objdir/pack/pack-*.idx >before &&
|
|
test_commit_bulk --start=4 4 && # 12 objects
|
|
ls $objdir/pack/pack-*.idx >after &&
|
|
|
|
git repack --geometric 2 -dbm &&
|
|
|
|
comm -3 before after | xargs -n 1 basename >expect &&
|
|
test-tool read-midx --preferred-pack $objdir >actual &&
|
|
|
|
test_cmp expect actual
|
|
)
|
|
'
|
|
|
|
test_expect_success '--geometric with pack.packSizeLimit' '
|
|
git init pack-rewrite &&
|
|
test_when_finished "rm -fr pack-rewrite" &&
|
|
(
|
|
cd pack-rewrite &&
|
|
|
|
test-tool genrandom foo 1048576 >foo &&
|
|
test-tool genrandom bar 1048576 >bar &&
|
|
|
|
git add foo bar &&
|
|
test_tick &&
|
|
git commit -m base &&
|
|
|
|
git rev-parse HEAD:foo HEAD:bar >p1.objects &&
|
|
git rev-parse HEAD HEAD^{tree} >p2.objects &&
|
|
|
|
# These two packs each contain two objects, so the following
|
|
# `--geometric` repack will try to combine them.
|
|
p1="$(git pack-objects $packdir/pack <p1.objects)" &&
|
|
p2="$(git pack-objects $packdir/pack <p2.objects)" &&
|
|
|
|
# Remove any loose objects in packs, since we do not want extra
|
|
# copies around (which would mask over potential object
|
|
# corruption issues).
|
|
git prune-packed &&
|
|
|
|
# Both p1 and p2 will be rolled up, but pack-objects will write
|
|
# three packs:
|
|
#
|
|
# - one containing object "foo",
|
|
# - another containing object "bar",
|
|
# - a final pack containing the commit and tree objects
|
|
# (identical to p2 above)
|
|
git repack --geometric 2 -d --max-pack-size=1048576 &&
|
|
|
|
# Ensure `repack` can detect that the third pack it wrote
|
|
# (containing just the tree and commit objects) was identical to
|
|
# one that was below the geometric split, so that we can save it
|
|
# from deletion.
|
|
#
|
|
# If `repack` fails to do that, we will incorrectly delete p2,
|
|
# causing object corruption.
|
|
git fsck
|
|
)
|
|
'
|
|
|
|
test_done
|