undef/git-commit-vandalism
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3084a01e5e
git-commit-vandalism/builtin
History
Sun Chao 3084a01e5e pack-redundant: new algorithm to find min packs 
When calling `git pack-redundant --all`, if there are too many local
packs and too many redundant objects within them, the too deep iteration
of `get_permutations` will exhaust all the resources, and the process of
`git pack-redundant` will be killed.

The following script could create a repository with too many redundant
packs, and running `git pack-redundant --all` in the `test.git` repo
will die soon.

    #!/bin/sh

    repo="$(pwd)/test.git"
    work="$(pwd)/test"
    i=1
    max=199

    if test -d "$repo" || test -d "$work"; then
    	echo >&2 "ERROR: '$repo' or '$work' already exist"
    	exit 1
    fi

    git init -q --bare "$repo"
    git --git-dir="$repo" config gc.auto 0
    git --git-dir="$repo" config transfer.unpackLimit 0
    git clone -q "$repo" "$work" 2>/dev/null

    while :; do
        cd "$work"
        echo "loop $i: $(date +%s)" >$i
        git add $i
        git commit -q -sm "loop $i"
        git push -q origin HEAD:master
        printf "\rCreate pack %4d/%d\t" $i $max
        if test $i -ge $max; then break; fi

        cd "$repo"
        git repack -q
        if test $(($i % 2)) -eq 0; then
            git repack -aq
            pack=$(ls -t $repo/objects/pack/*.pack | head -1)
            touch "${pack%.pack}.keep"
        fi
        i=$((i+1))
    done
    printf "\ndone\n"

To get the `min` unique pack list, we can replace the iteration in
`minimize` function with a new algorithm, and this could solve this
issue:

1. Get the unique and non_uniqe packs, add the unique packs to the
   `min` list.

2. Remove the objects of unique packs from non_unique packs, then each
   object left in the non_unique packs will have at least two copies.

3. Sort the non_unique packs by the objects' size, more objects first,
   and add the first non_unique pack to `min` list.

4. Drop the duplicated objects from other packs in the ordered
   non_unique pack list, and repeat step 3.

Some test cases will fail on Mac OS X. Mark them and will resolve in
later commit.

Original PR and discussions: https://github.com/jiangxin/git/pull/25

Signed-off-by: Sun Chao <sunchao9@huawei.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:24 -08:00
..
add.c Merge branch 'nd/attr-pathspec-fix' into maint 2018-11-21 22:57:45 +09:00
am.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
annotate.c
apply.c apply.c: make init_apply_state() take a struct repository 2018-08-13 14:14:44 -07:00
archive.c archive: initialize archivers earlier 2018-10-26 10:17:59 +09:00
bisect--helper.c bisect.c: remove the_repository reference 2018-11-12 14:50:05 +09:00
blame.c Merge branch 'nd/show-gitcomp-compilation-fix' into maint 2018-12-15 12:24:33 +09:00
branch.c Merge branch 'tq/branch-style-fix' into maint 2018-11-21 22:58:06 +09:00
bundle.c bundle.c: remove the_repository references 2018-11-12 14:50:06 +09:00
cat-file.c Merge branch 'tb/print-size-t-with-uintmax-format' 2018-11-19 16:24:41 +09:00
check-attr.c Make git_check_attr() a void function 2018-09-12 15:15:34 -07:00
check-ignore.c check-ignore: fix mix of directories and other file types 2018-02-12 13:09:35 -08:00
check-mailmap.c config: don't include config.h by default 2017-06-15 12:56:22 -07:00
check-ref-format.c Merge branch 'jc/check-ref-format-oor' into maint 2017-11-15 12:04:57 +09:00
checkout-index.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
checkout.c Merge branch 'nd/checkout-dwim-fix' 2019-01-04 13:33:34 -08:00
clean.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
clone.c Merge branch 'sg/clone-initial-fetch-configuration' 2019-01-04 13:33:34 -08:00
column.c column: fix off-by-one default width 2018-05-13 10:45:05 +09:00
commit-graph.c Merge branch 'ds/commit-graph-leakfix' 2018-10-19 13:34:07 +09:00
commit-tree.c commit: add repository argument to lookup_commit 2018-06-29 10:43:39 -07:00
commit.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
config.c worktree: add per-worktree config files 2018-10-22 13:17:04 +09:00
count-objects.c rename "alternate_object_database" to "object_directory" 2018-11-13 14:22:02 +09:00
credential.c
describe.c refresh_index: remove unnecessary calls to preload_index() 2018-11-06 12:49:59 +09:00
diff-files.c revision.c: remove implicit dependency on the_index 2018-09-21 09:51:19 -07:00
diff-index.c revision.c: remove implicit dependency on the_index 2018-09-21 09:51:19 -07:00
diff-tree.c revision.c: remove implicit dependency on the_index 2018-09-21 09:51:19 -07:00
diff.c Merge branch 'nd/the-index' 2018-10-19 13:34:02 +09:00
difftool.c Merge branch 'nd/the-index' 2018-10-19 13:34:02 +09:00
fast-export.c Merge branch 'en/fast-export-import' 2019-01-04 13:33:33 -08:00
fetch-pack.c builtin/fetch-pack: remove constants with parse_oid_hex 2018-10-15 12:53:15 +09:00
fetch.c Merge branch 'jk/unused-parameter-fixes' 2018-11-18 18:23:53 +09:00
fmt-merge-msg.c Merge branch 'nd/the-index' 2018-10-19 13:34:02 +09:00
for-each-ref.c provide --color option for all ref-filter users 2017-10-04 11:35:29 +09:00
fsck.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
gc.c Merge branch 'jn/gc-auto' 2018-10-16 16:16:02 +09:00
get-tar-commit-id.c distinguish error versus short read from read_in_full() 2017-09-27 15:45:24 +09:00
grep.c Merge branch 'jk/loose-object-cache' 2019-01-04 13:33:32 -08:00
hash-object.c sha1-file.c: remove implicit dependency on the_index 2018-09-21 09:48:11 -07:00
help.c Merge branch 'rv/alias-help' 2018-10-26 14:22:13 +09:00
index-pack.c Merge branch 'tb/print-size-t-with-uintmax-format' 2018-11-19 16:24:41 +09:00
init-db.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
interpret-trailers.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
log.c range-diff: always pass at least minimal diff options 2018-12-04 10:36:14 +09:00
ls-files.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
ls-remote.c Merge branch 'jk/proto-v2-ref-prefix-fix' 2018-11-13 22:37:17 +09:00
ls-tree.c Upcast size_t variables to uintmax_t when printing 2018-11-12 16:43:52 +09:00
mailinfo.c
mailsplit.c
merge-base.c Merge branch 'pk/rebase-in-c-4-opts' 2018-11-02 11:04:55 +09:00
merge-file.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
merge-index.c
merge-ours.c diff-lib.c: remove the_repository references 2018-11-12 14:50:06 +09:00
merge-recursive.c builtin/merge-recursive: make hash independent 2018-07-16 14:27:39 -07:00
merge-tree.c Merge branch 'jk/xdiff-interface' 2018-11-13 22:37:27 +09:00
merge.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
mktag.c object-store: move object access functions to object-store.h 2018-05-16 11:42:03 +09:00
mktree.c builtin/mktree: remove hard-coded constant 2018-10-15 12:53:15 +09:00
multi-pack-index.c multi-pack-index: add 'verify' verb 2018-09-17 13:49:38 -07:00
mv.c Merge branch 'ma/lockfile-cleanup' 2018-05-30 14:04:05 +09:00
name-rev.c tag: add repository argument to deref_tag 2018-06-29 10:43:39 -07:00
notes.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
pack-objects.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
pack-redundant.c pack-redundant: new algorithm to find min packs 2019-02-04 14:18:24 -08:00
pack-refs.c refs: add repository argument to get_main_ref_store 2018-04-12 11:38:56 +09:00
patch-id.c
prune-packed.c for_each_*_object: move declarations to object-store.h 2018-08-14 12:29:57 -07:00
prune.c Merge branch 'md/exclude-promisor-objects-fix' into maint 2018-11-21 22:57:52 +09:00
pull.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
push.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
range-diff.c Merge branch 'nd/the-index' 2018-10-19 13:34:02 +09:00
read-tree.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
rebase--interactive.c rebase-interactive.c: remove the_repository references 2018-11-12 14:50:06 +09:00
rebase.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
receive-pack.c Merge branch 'jk/check-everything-connected-is-long-gone' into maint 2018-11-21 22:58:00 +09:00
reflog.c Merge branch 'nd/i18n' 2019-01-04 13:33:31 -08:00
remote-ext.c consistently use "fallthrough" comments in switches 2017-09-22 12:49:57 +09:00
remote-fd.c
remote.c Merge branch 'ms/remote-error-message-update' 2018-09-24 10:30:52 -07:00
repack.c Merge branch 'nd/i18n' 2019-01-04 13:33:31 -08:00
replace.c Merge branch 'ab/replace-graft-with-replace-advice' 2018-12-01 21:41:42 +09:00
rerere.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
reset.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
rev-list.c Merge branch 'md/exclude-promisor-objects-fix' into maint 2018-11-21 22:57:52 +09:00
rev-parse.c Merge branch 'ag/rev-parse-all-exclude-fix' 2018-11-13 22:37:23 +09:00
revert.c Merge branch 'nd/the-index' 2019-01-04 13:33:33 -08:00
rm.c convert "oidcmp() != 0" to "!oideq()" 2018-08-29 11:32:49 -07:00
send-pack.c Merge branch 'rs/opt-updates' 2018-08-27 14:33:43 -07:00
serve.c serve: introduce git-serve 2018-03-15 12:01:08 -07:00
shortlog.c Merge branch 'nd/show-gitcomp-compilation-fix' into maint 2018-12-15 12:24:33 +09:00
show-branch.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
show-index.c make show-index a builtin 2018-05-29 00:28:22 +09:00
show-ref.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
stripspace.c
submodule--helper.c Merge branch 'jk/loose-object-cache' 2019-01-04 13:33:32 -08:00
symbolic-ref.c refs: rename constant REF_NODEREF to REF_NO_DEREF 2017-11-06 10:31:08 +09:00
tag.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
unpack-file.c object-store: move object access functions to object-store.h 2018-05-16 11:42:03 +09:00
unpack-objects.c convert "hashcmp() != 0" to "!hasheq()" 2018-08-29 11:32:49 -07:00
update-index.c Merge branch 'nd/show-gitcomp-compilation-fix' into maint 2018-12-15 12:24:33 +09:00
update-ref.c update-ref: allow --no-deref with --stdin 2018-09-12 15:17:17 -07:00
update-server-info.c parse-options: let OPT__FORCE take optional flags argument 2018-02-09 10:24:50 -08:00
upload-archive.c archive: initialize archivers earlier 2018-10-26 10:17:59 +09:00
upload-pack.c check_replace_refs: rename to read_replace_refs 2018-07-18 15:45:14 -07:00
var.c
verify-commit.c commit: add repository argument to lookup_commit 2018-06-29 10:43:39 -07:00
verify-pack.c config: don't include config.h by default 2017-06-15 12:56:22 -07:00
verify-tag.c ref-filter: use "struct object_id" consistently 2018-04-09 06:14:45 +09:00
worktree.c worktree: rename is_worktree_locked to worktree_lock_reason 2018-10-31 11:41:47 +09:00
write-tree.c Merge branch 'rs/parse-opt-lithelp' 2018-08-17 13:09:56 -07:00