undef/git-commit-vandalism
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
0fabafd0b9
git-commit-vandalism/t/t7703-repack-geometric.sh

138 lines
3.5 KiB
Bash
Raw Normal View History

builtin/repack.c: add '--geometric' option Often it is useful to both: - have relatively few packfiles in a repository, and - avoid having so few packfiles in a repository that we repack its entire contents regularly This patch implements a '--geometric=<n>' option in 'git repack'. This allows the caller to specify that they would like each pack to be at least a factor times as large as the previous largest pack (by object count). Concretely, say that a repository has 'n' packfiles, labeled P1, P2, ..., up to Pn. Each packfile has an object count equal to 'objects(Pn)'. With a geometric factor of 'r', it should be that: objects(Pi) > r*objects(P(i-1)) for all i in [1, n], where the packs are sorted by objects(P1) <= objects(P2) <= ... <= objects(Pn). Since finding a true optimal repacking is NP-hard, we approximate it along two directions: 1. We assume that there is a cutoff of packs _before starting the repack_ where everything to the right of that cut-off already forms a geometric progression (or no cutoff exists and everything must be repacked). 2. We assume that everything smaller than the cutoff count must be repacked. This forms our base assumption, but it can also cause even the "heavy" packs to get repacked, for e.g., if we have 6 packs containing the following number of objects: 1, 1, 1, 2, 4, 32 then we would place the cutoff between '1, 1' and '1, 2, 4, 32', rolling up the first two packs into a pack with 2 objects. That breaks our progression and leaves us: 2, 1, 2, 4, 32 ^ (where the '^' indicates the position of our split). To restore a progression, we move the split forward (towards larger packs) joining each pack into our new pack until a geometric progression is restored. Here, that looks like: 2, 1, 2, 4, 32 ~> 3, 2, 4, 32 ~> 5, 4, 32 ~> ... ~> 9, 32 ^ ^ ^ ^ This has the advantage of not repacking the heavy-side of packs too often while also only creating one new pack at a time. Another wrinkle is that we assume that loose, indexed, and reflog'd objects are insignificant, and lump them into any new pack that we create. This can lead to non-idempotent results. Suggested-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-23 03:25:27 +01:00
#!/bin/sh
test_description='git repack --geometric works correctly'
. ./test-lib.sh
GIT_TEST_MULTI_PACK_INDEX=0
objdir=.git/objects
midx=$objdir/pack/multi-pack-index
test_expect_success '--geometric with no packs' '
git init geometric &&
test_when_finished "rm -fr geometric" &&
(
cd geometric &&
git repack --geometric 2 >out &&
test_i18ngrep "Nothing new to pack" out
)
'
test_expect_success '--geometric with an intact progression' '
git init geometric &&
test_when_finished "rm -fr geometric" &&
(
cd geometric &&
# These packs already form a geometric progression.
test_commit_bulk --start=1 1 && # 3 objects
test_commit_bulk --start=2 2 && # 6 objects
test_commit_bulk --start=4 4 && # 12 objects
find $objdir/pack -name "*.pack" | sort >expect &&
git repack --geometric 2 -d &&
find $objdir/pack -name "*.pack" | sort >actual &&
test_cmp expect actual
)
'
test_expect_success '--geometric with small-pack rollup' '
git init geometric &&
test_when_finished "rm -fr geometric" &&
(
cd geometric &&
test_commit_bulk --start=1 1 && # 3 objects
test_commit_bulk --start=2 1 && # 3 objects
find $objdir/pack -name "*.pack" | sort >small &&
test_commit_bulk --start=3 4 && # 12 objects
test_commit_bulk --start=7 8 && # 24 objects
find $objdir/pack -name "*.pack" | sort >before &&
git repack --geometric 2 -d &&
# Three packs in total; two of the existing large ones, and one
# new one.
find $objdir/pack -name "*.pack" | sort >after &&
test_line_count = 3 after &&
comm -3 small before | tr -d "\t" >large &&
grep -qFf large after
)
'
test_expect_success '--geometric with small- and large-pack rollup' '
git init geometric &&
test_when_finished "rm -fr geometric" &&
(
cd geometric &&
# size(small1) + size(small2) > size(medium) / 2
test_commit_bulk --start=1 1 && # 3 objects
test_commit_bulk --start=2 1 && # 3 objects
test_commit_bulk --start=2 3 && # 7 objects
test_commit_bulk --start=6 9 && # 27 objects &&
find $objdir/pack -name "*.pack" | sort >before &&
git repack --geometric 2 -d &&
find $objdir/pack -name "*.pack" | sort >after &&
comm -12 before after >untouched &&
# Two packs in total; the largest pack from before running "git
# repack", and one new one.
test_line_count = 1 untouched &&
test_line_count = 2 after
)
'
test_expect_success '--geometric ignores kept packs' '
git init geometric &&
test_when_finished "rm -fr geometric" &&
(
cd geometric &&
test_commit kept && # 3 objects
test_commit pack && # 3 objects
KEPT=$(git pack-objects --revs $objdir/pack/pack <<-EOF
refs/tags/kept
EOF
) &&
PACK=$(git pack-objects --revs $objdir/pack/pack <<-EOF
refs/tags/pack
^refs/tags/kept
EOF
) &&
# neither pack contains more than twice the number of objects in
# the other, so they should be combined. but, marking one as
# .kept on disk will "freeze" it, so the pack structure should
# remain unchanged.
touch $objdir/pack/pack-$KEPT.keep &&
find $objdir/pack -name "*.pack" | sort >before &&
git repack --geometric 2 -d &&
find $objdir/pack -name "*.pack" | sort >after &&
# both packs should still exist
test_path_is_file $objdir/pack/pack-$KEPT.pack &&
test_path_is_file $objdir/pack/pack-$PACK.pack &&
# and no new packs should be created
test_cmp before after &&
# Passing --pack-kept-objects causes packs with a .keep file to
# be repacked, too.
git repack --geometric 2 -d --pack-kept-objects &&
find $objdir/pack -name "*.pack" >after &&
test_line_count = 1 after
)
'
test_done
Reference in New Issue Copy Permalink