Merge branch 'ds/maintenance-part-2'
"git maintenance", an extended big brother of "git gc", continues to evolve. * ds/maintenance-part-2: maintenance: add incremental-repack auto condition maintenance: auto-size incremental-repack batch maintenance: add incremental-repack task midx: use start_delayed_progress() midx: enable core.multiPackIndex by default maintenance: create auto condition for loose-objects maintenance: add loose-objects task maintenance: add prefetch task
This commit is contained in:
Junio C Hamano
commit
52b8c8c716
@ -606,8 +606,8 @@ core.useReplaceRefs::
|
||||
|
||||
core.multiPackIndex::
|
||||
Use the multi-pack-index file to track multiple packfiles using a
|
||||
single index. See link:technical/multi-pack-index.html[the
|
||||
multi-pack-index design document].
|
||||
single index. See linkgit:git-multi-pack-index[1] for more
|
||||
information. Defaults to true.
|
||||
|
||||
core.sparseCheckout::
|
||||
Enable "sparse checkout" feature. See linkgit:git-sparse-checkout[1]
|
||||
|
||||
@ -14,3 +14,21 @@ maintenance.commit-graph.auto::
|
||||
reachable commits that are not in the commit-graph file is at least
|
||||
the value of `maintenance.commit-graph.auto`. The default value is
|
||||
100.
|
||||
|
||||
maintenance.loose-objects.auto::
|
||||
This integer config option controls how often the `loose-objects` task
|
||||
should be run as part of `git maintenance run --auto`. If zero, then
|
||||
the `loose-objects` task will not run with the `--auto` option. A
|
||||
negative value will force the task to run every time. Otherwise, a
|
||||
positive value implies the command should run when the number of
|
||||
loose objects is at least the value of `maintenance.loose-objects.auto`.
|
||||
The default value is 100.
|
||||
|
||||
maintenance.incremental-repack.auto::
|
||||
This integer config option controls how often the `incremental-repack`
|
||||
task should be run as part of `git maintenance run --auto`. If zero,
|
||||
then the `incremental-repack` task will not run with the `--auto`
|
||||
option. A negative value will force the task to run every time.
|
||||
Otherwise, a positive value implies the command should run when the
|
||||
number of pack-files not in the multi-pack-index is at least the value
|
||||
of `maintenance.incremental-repack.auto`. The default value is 10.
|
||||
|
||||
@ -47,6 +47,21 @@ commit-graph::
|
||||
`commit-graph-chain` file. They will be deleted by a later run based
|
||||
on the expiration delay.
|
||||
|
||||
prefetch::
|
||||
The `prefetch` task updates the object directory with the latest
|
||||
objects from all registered remotes. For each remote, a `git fetch`
|
||||
command is run. The refmap is custom to avoid updating local or remote
|
||||
branches (those in `refs/heads` or `refs/remotes`). Instead, the
|
||||
remote refs are stored in `refs/prefetch/<remote>/`. Also, tags are
|
||||
not updated.
|
||||
+
|
||||
This is done to avoid disrupting the remote-tracking branches. The end users
|
||||
expect these refs to stay unmoved unless they initiate a fetch. With prefetch
|
||||
task, however, the objects necessary to complete a later real fetch would
|
||||
already be obtained, so the real fetch would go faster. In the ideal case,
|
||||
it will just become an update to bunch of remote-tracking branches without
|
||||
any object transfer.
|
||||
|
||||
gc::
|
||||
Clean up unnecessary files and optimize the local repository. "GC"
|
||||
stands for "garbage collection," but this task performs many
|
||||
@ -55,6 +70,39 @@ gc::
|
||||
be disruptive in some situations, as it deletes stale data. See
|
||||
linkgit:git-gc[1] for more details on garbage collection in Git.
|
||||
|
||||
loose-objects::
|
||||
The `loose-objects` job cleans up loose objects and places them into
|
||||
pack-files. In order to prevent race conditions with concurrent Git
|
||||
commands, it follows a two-step process. First, it deletes any loose
|
||||
objects that already exist in a pack-file; concurrent Git processes
|
||||
will examine the pack-file for the object data instead of the loose
|
||||
object. Second, it creates a new pack-file (starting with "loose-")
|
||||
containing a batch of loose objects. The batch size is limited to 50
|
||||
thousand objects to prevent the job from taking too long on a
|
||||
repository with many loose objects. The `gc` task writes unreachable
|
||||
objects as loose objects to be cleaned up by a later step only if
|
||||
they are not re-added to a pack-file; for this reason it is not
|
||||
advisable to enable both the `loose-objects` and `gc` tasks at the
|
||||
same time.
|
||||
|
||||
incremental-repack::
|
||||
The `incremental-repack` job repacks the object directory
|
||||
using the `multi-pack-index` feature. In order to prevent race
|
||||
conditions with concurrent Git commands, it follows a two-step
|
||||
process. First, it calls `git multi-pack-index expire` to delete
|
||||
pack-files unreferenced by the `multi-pack-index` file. Second, it
|
||||
calls `git multi-pack-index repack` to select several small
|
||||
pack-files and repack them into a bigger one, and then update the
|
||||
`multi-pack-index` entries that refer to the small pack-files to
|
||||
refer to the new pack-file. This prepares those small pack-files
|
||||
for deletion upon the next run of `git multi-pack-index expire`.
|
||||
The selection of the small pack-files is such that the expected
|
||||
size of the big pack-file is at least the batch size; see the
|
||||
`--batch-size` option for the `repack` subcommand in
|
||||
linkgit:git-multi-pack-index[1]. The default batch-size is zero,
|
||||
which is a special case that attempts to repack all pack-files
|
||||
into a single pack-file.
|
||||
|
||||
OPTIONS
|
||||
-------
|
||||
--auto::
|
||||
|
||||
326
builtin/gc.c
326
builtin/gc.c
@ -29,6 +29,8 @@
|
||||
#include "tree.h"
|
||||
#include "promisor-remote.h"
|
||||
#include "refs.h"
|
||||
#include "remote.h"
|
||||
#include "object-store.h"
|
||||
|
||||
#define FAILED_RUN "failed to run %s"
|
||||
|
||||
@ -816,6 +818,51 @@ static int maintenance_task_commit_graph(struct maintenance_run_opts *opts)
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int fetch_remote(const char *remote, struct maintenance_run_opts *opts)
|
||||
{
|
||||
struct child_process child = CHILD_PROCESS_INIT;
|
||||
|
||||
child.git_cmd = 1;
|
||||
strvec_pushl(&child.args, "fetch", remote, "--prune", "--no-tags",
|
||||
"--no-write-fetch-head", "--recurse-submodules=no",
|
||||
"--refmap=", NULL);
|
||||
|
||||
if (opts->quiet)
|
||||
strvec_push(&child.args, "--quiet");
|
||||
|
||||
strvec_pushf(&child.args, "+refs/heads/*:refs/prefetch/%s/*", remote);
|
||||
|
||||
return !!run_command(&child);
|
||||
}
|
||||
|
||||
static int append_remote(struct remote *remote, void *cbdata)
|
||||
{
|
||||
struct string_list *remotes = (struct string_list *)cbdata;
|
||||
|
||||
string_list_append(remotes, remote->name);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int maintenance_task_prefetch(struct maintenance_run_opts *opts)
|
||||
{
|
||||
int result = 0;
|
||||
struct string_list_item *item;
|
||||
struct string_list remotes = STRING_LIST_INIT_DUP;
|
||||
|
||||
if (for_each_remote(append_remote, &remotes)) {
|
||||
error(_("failed to fill remotes"));
|
||||
result = 1;
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
for_each_string_list_item(item, &remotes)
|
||||
result |= fetch_remote(item->string, opts);
|
||||
|
||||
cleanup:
|
||||
string_list_clear(&remotes, 0);
|
||||
return result;
|
||||
}
|
||||
|
||||
static int maintenance_task_gc(struct maintenance_run_opts *opts)
|
||||
{
|
||||
struct child_process child = CHILD_PROCESS_INIT;
|
||||
@ -834,6 +881,268 @@ static int maintenance_task_gc(struct maintenance_run_opts *opts)
|
||||
return run_command(&child);
|
||||
}
|
||||
|
||||
static int prune_packed(struct maintenance_run_opts *opts)
|
||||
{
|
||||
struct child_process child = CHILD_PROCESS_INIT;
|
||||
|
||||
child.git_cmd = 1;
|
||||
strvec_push(&child.args, "prune-packed");
|
||||
|
||||
if (opts->quiet)
|
||||
strvec_push(&child.args, "--quiet");
|
||||
|
||||
return !!run_command(&child);
|
||||
}
|
||||
|
||||
struct write_loose_object_data {
|
||||
FILE *in;
|
||||
int count;
|
||||
int batch_size;
|
||||
};
|
||||
|
||||
static int loose_object_auto_limit = 100;
|
||||
|
||||
static int loose_object_count(const struct object_id *oid,
|
||||
const char *path,
|
||||
void *data)
|
||||
{
|
||||
int *count = (int*)data;
|
||||
if (++(*count) >= loose_object_auto_limit)
|
||||
return 1;
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int loose_object_auto_condition(void)
|
||||
{
|
||||
int count = 0;
|
||||
|
||||
git_config_get_int("maintenance.loose-objects.auto",
|
||||
&loose_object_auto_limit);
|
||||
|
||||
if (!loose_object_auto_limit)
|
||||
return 0;
|
||||
if (loose_object_auto_limit < 0)
|
||||
return 1;
|
||||
|
||||
return for_each_loose_file_in_objdir(the_repository->objects->odb->path,
|
||||
loose_object_count,
|
||||
NULL, NULL, &count);
|
||||
}
|
||||
|
||||
static int bail_on_loose(const struct object_id *oid,
|
||||
const char *path,
|
||||
void *data)
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
|
||||
static int write_loose_object_to_stdin(const struct object_id *oid,
|
||||
const char *path,
|
||||
void *data)
|
||||
{
|
||||
struct write_loose_object_data *d = (struct write_loose_object_data *)data;
|
||||
|
||||
fprintf(d->in, "%s\n", oid_to_hex(oid));
|
||||
|
||||
return ++(d->count) > d->batch_size;
|
||||
}
|
||||
|
||||
static int pack_loose(struct maintenance_run_opts *opts)
|
||||
{
|
||||
struct repository *r = the_repository;
|
||||
int result = 0;
|
||||
struct write_loose_object_data data;
|
||||
struct child_process pack_proc = CHILD_PROCESS_INIT;
|
||||
|
||||
/*
|
||||
* Do not start pack-objects process
|
||||
* if there are no loose objects.
|
||||
*/
|
||||
if (!for_each_loose_file_in_objdir(r->objects->odb->path,
|
||||
bail_on_loose,
|
||||
NULL, NULL, NULL))
|
||||
return 0;
|
||||
|
||||
pack_proc.git_cmd = 1;
|
||||
|
||||
strvec_push(&pack_proc.args, "pack-objects");
|
||||
if (opts->quiet)
|
||||
strvec_push(&pack_proc.args, "--quiet");
|
||||
strvec_pushf(&pack_proc.args, "%s/pack/loose", r->objects->odb->path);
|
||||
|
||||
pack_proc.in = -1;
|
||||
|
||||
if (start_command(&pack_proc)) {
|
||||
error(_("failed to start 'git pack-objects' process"));
|
||||
return 1;
|
||||
}
|
||||
|
||||
data.in = xfdopen(pack_proc.in, "w");
|
||||
data.count = 0;
|
||||
data.batch_size = 50000;
|
||||
|
||||
for_each_loose_file_in_objdir(r->objects->odb->path,
|
||||
write_loose_object_to_stdin,
|
||||
NULL,
|
||||
NULL,
|
||||
&data);
|
||||
|
||||
fclose(data.in);
|
||||
|
||||
if (finish_command(&pack_proc)) {
|
||||
error(_("failed to finish 'git pack-objects' process"));
|
||||
result = 1;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
static int maintenance_task_loose_objects(struct maintenance_run_opts *opts)
|
||||
{
|
||||
return prune_packed(opts) || pack_loose(opts);
|
||||
}
|
||||
|
||||
static int incremental_repack_auto_condition(void)
|
||||
{
|
||||
struct packed_git *p;
|
||||
int enabled;
|
||||
int incremental_repack_auto_limit = 10;
|
||||
int count = 0;
|
||||
|
||||
if (git_config_get_bool("core.multiPackIndex", &enabled) ||
|
||||
!enabled)
|
||||
return 0;
|
||||
|
||||
git_config_get_int("maintenance.incremental-repack.auto",
|
||||
&incremental_repack_auto_limit);
|
||||
|
||||
if (!incremental_repack_auto_limit)
|
||||
return 0;
|
||||
if (incremental_repack_auto_limit < 0)
|
||||
return 1;
|
||||
|
||||
for (p = get_packed_git(the_repository);
|
||||
count < incremental_repack_auto_limit && p;
|
||||
p = p->next) {
|
||||
if (!p->multi_pack_index)
|
||||
count++;
|
||||
}
|
||||
|
||||
return count >= incremental_repack_auto_limit;
|
||||
}
|
||||
|
||||
static int multi_pack_index_write(struct maintenance_run_opts *opts)
|
||||
{
|
||||
struct child_process child = CHILD_PROCESS_INIT;
|
||||
|
||||
child.git_cmd = 1;
|
||||
strvec_pushl(&child.args, "multi-pack-index", "write", NULL);
|
||||
|
||||
if (opts->quiet)
|
||||
strvec_push(&child.args, "--no-progress");
|
||||
|
||||
if (run_command(&child))
|
||||
return error(_("failed to write multi-pack-index"));
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int multi_pack_index_expire(struct maintenance_run_opts *opts)
|
||||
{
|
||||
struct child_process child = CHILD_PROCESS_INIT;
|
||||
|
||||
child.git_cmd = 1;
|
||||
strvec_pushl(&child.args, "multi-pack-index", "expire", NULL);
|
||||
|
||||
if (opts->quiet)
|
||||
strvec_push(&child.args, "--no-progress");
|
||||
|
||||
close_object_store(the_repository->objects);
|
||||
|
||||
if (run_command(&child))
|
||||
return error(_("'git multi-pack-index expire' failed"));
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
#define TWO_GIGABYTES (INT32_MAX)
|
||||
|
||||
static off_t get_auto_pack_size(void)
|
||||
{
|
||||
/*
|
||||
* The "auto" value is special: we optimize for
|
||||
* one large pack-file (i.e. from a clone) and
|
||||
* expect the rest to be small and they can be
|
||||
* repacked quickly.
|
||||
*
|
||||
* The strategy we select here is to select a
|
||||
* size that is one more than the second largest
|
||||
* pack-file. This ensures that we will repack
|
||||
* at least two packs if there are three or more
|
||||
* packs.
|
||||
*/
|
||||
off_t max_size = 0;
|
||||
off_t second_largest_size = 0;
|
||||
off_t result_size;
|
||||
struct packed_git *p;
|
||||
struct repository *r = the_repository;
|
||||
|
||||
reprepare_packed_git(r);
|
||||
for (p = get_all_packs(r); p; p = p->next) {
|
||||
if (p->pack_size > max_size) {
|
||||
second_largest_size = max_size;
|
||||
max_size = p->pack_size;
|
||||
} else if (p->pack_size > second_largest_size)
|
||||
second_largest_size = p->pack_size;
|
||||
}
|
||||
|
||||
result_size = second_largest_size + 1;
|
||||
|
||||
/* But limit ourselves to a batch size of 2g */
|
||||
if (result_size > TWO_GIGABYTES)
|
||||
result_size = TWO_GIGABYTES;
|
||||
|
||||
return result_size;
|
||||
}
|
||||
|
||||
static int multi_pack_index_repack(struct maintenance_run_opts *opts)
|
||||
{
|
||||
struct child_process child = CHILD_PROCESS_INIT;
|
||||
|
||||
child.git_cmd = 1;
|
||||
strvec_pushl(&child.args, "multi-pack-index", "repack", NULL);
|
||||
|
||||
if (opts->quiet)
|
||||
strvec_push(&child.args, "--no-progress");
|
||||
|
||||
strvec_pushf(&child.args, "--batch-size=%"PRIuMAX,
|
||||
(uintmax_t)get_auto_pack_size());
|
||||
|
||||
close_object_store(the_repository->objects);
|
||||
|
||||
if (run_command(&child))
|
||||
return error(_("'git multi-pack-index repack' failed"));
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int maintenance_task_incremental_repack(struct maintenance_run_opts *opts)
|
||||
{
|
||||
prepare_repo_settings(the_repository);
|
||||
if (!the_repository->settings.core_multi_pack_index) {
|
||||
warning(_("skipping incremental-repack task because core.multiPackIndex is disabled"));
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (multi_pack_index_write(opts))
|
||||
return 1;
|
||||
if (multi_pack_index_expire(opts))
|
||||
return 1;
|
||||
if (multi_pack_index_repack(opts))
|
||||
return 1;
|
||||
return 0;
|
||||
}
|
||||
|
||||
typedef int maintenance_task_fn(struct maintenance_run_opts *opts);
|
||||
|
||||
/*
|
||||
@ -854,6 +1163,9 @@ struct maintenance_task {
|
||||
};
|
||||
|
||||
enum maintenance_task_label {
|
||||
TASK_PREFETCH,
|
||||
TASK_LOOSE_OBJECTS,
|
||||
TASK_INCREMENTAL_REPACK,
|
||||
TASK_GC,
|
||||
TASK_COMMIT_GRAPH,
|
||||
|
||||
@ -862,6 +1174,20 @@ enum maintenance_task_label {
|
||||
};
|
||||
|
||||
static struct maintenance_task tasks[] = {
|
||||
[TASK_PREFETCH] = {
|
||||
"prefetch",
|
||||
maintenance_task_prefetch,
|
||||
},
|
||||
[TASK_LOOSE_OBJECTS] = {
|
||||
"loose-objects",
|
||||
maintenance_task_loose_objects,
|
||||
loose_object_auto_condition,
|
||||
},
|
||||
[TASK_INCREMENTAL_REPACK] = {
|
||||
"incremental-repack",
|
||||
maintenance_task_incremental_repack,
|
||||
incremental_repack_auto_condition,
|
||||
},
|
||||
[TASK_GC] = {
|
||||
"gc",
|
||||
maintenance_task_gc,
|
||||
|
||||
21
midx.c
21
midx.c
@ -10,6 +10,7 @@
|
||||
#include "progress.h"
|
||||
#include "trace2.h"
|
||||
#include "run-command.h"
|
||||
#include "repository.h"
|
||||
|
||||
#define MIDX_SIGNATURE 0x4d494458 /* "MIDX" */
|
||||
#define MIDX_VERSION 1
|
||||
@ -398,15 +399,9 @@ int prepare_multi_pack_index_one(struct repository *r, const char *object_dir, i
|
||||
{
|
||||
struct multi_pack_index *m;
|
||||
struct multi_pack_index *m_search;
|
||||
int config_value;
|
||||
static int env_value = -1;
|
||||
|
||||
if (env_value < 0)
|
||||
env_value = git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0);
|
||||
|
||||
if (!env_value &&
|
||||
(repo_config_get_bool(r, "core.multipackindex", &config_value) ||
|
||||
!config_value))
|
||||
prepare_repo_settings(r);
|
||||
if (!r->settings.core_multi_pack_index)
|
||||
return 0;
|
||||
|
||||
for (m_search = r->objects->multi_pack_index; m_search; m_search = m_search->next)
|
||||
@ -850,7 +845,7 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *
|
||||
|
||||
packs.pack_paths_checked = 0;
|
||||
if (flags & MIDX_PROGRESS)
|
||||
packs.progress = start_progress(_("Adding packfiles to multi-pack-index"), 0);
|
||||
packs.progress = start_delayed_progress(_("Adding packfiles to multi-pack-index"), 0);
|
||||
else
|
||||
packs.progress = NULL;
|
||||
|
||||
@ -987,7 +982,7 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *
|
||||
}
|
||||
|
||||
if (flags & MIDX_PROGRESS)
|
||||
progress = start_progress(_("Writing chunks to multi-pack-index"),
|
||||
progress = start_delayed_progress(_("Writing chunks to multi-pack-index"),
|
||||
num_chunks);
|
||||
for (i = 0; i < num_chunks; i++) {
|
||||
if (written != chunk_offsets[i])
|
||||
@ -1129,7 +1124,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag
|
||||
}
|
||||
|
||||
if (flags & MIDX_PROGRESS)
|
||||
progress = start_progress(_("Looking for referenced packfiles"),
|
||||
progress = start_delayed_progress(_("Looking for referenced packfiles"),
|
||||
m->num_packs);
|
||||
for (i = 0; i < m->num_packs; i++) {
|
||||
if (prepare_midx_pack(r, m, i))
|
||||
@ -1250,7 +1245,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla
|
||||
count = xcalloc(m->num_packs, sizeof(uint32_t));
|
||||
|
||||
if (flags & MIDX_PROGRESS)
|
||||
progress = start_progress(_("Counting referenced objects"),
|
||||
progress = start_delayed_progress(_("Counting referenced objects"),
|
||||
m->num_objects);
|
||||
for (i = 0; i < m->num_objects; i++) {
|
||||
int pack_int_id = nth_midxed_pack_int_id(m, i);
|
||||
@ -1260,7 +1255,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla
|
||||
stop_progress(&progress);
|
||||
|
||||
if (flags & MIDX_PROGRESS)
|
||||
progress = start_progress(_("Finding and deleting unreferenced packfiles"),
|
||||
progress = start_delayed_progress(_("Finding and deleting unreferenced packfiles"),
|
||||
m->num_packs);
|
||||
for (i = 0; i < m->num_packs; i++) {
|
||||
char *pack_name;
|
||||
|
||||
@ -1,6 +1,7 @@
|
||||
#include "cache.h"
|
||||
#include "config.h"
|
||||
#include "repository.h"
|
||||
#include "midx.h"
|
||||
|
||||
#define UPDATE_DEFAULT_BOOL(s,v) do { if (s == -1) { s = v; } } while(0)
|
||||
|
||||
@ -52,6 +53,11 @@ void prepare_repo_settings(struct repository *r)
|
||||
r->settings.pack_use_sparse = value;
|
||||
UPDATE_DEFAULT_BOOL(r->settings.pack_use_sparse, 1);
|
||||
|
||||
value = git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0);
|
||||
if (value || !repo_config_get_bool(r, "core.multipackindex", &value))
|
||||
r->settings.core_multi_pack_index = value;
|
||||
UPDATE_DEFAULT_BOOL(r->settings.core_multi_pack_index, 1);
|
||||
|
||||
if (!repo_config_get_bool(r, "feature.manyfiles", &value) && value) {
|
||||
UPDATE_DEFAULT_BOOL(r->settings.index_version, 4);
|
||||
UPDATE_DEFAULT_BOOL(r->settings.core_untracked_cache, UNTRACKED_CACHE_WRITE);
|
||||
|
||||
@ -39,6 +39,8 @@ struct repo_settings {
|
||||
|
||||
int pack_use_sparse;
|
||||
enum fetch_negotiation_setting fetch_negotiation_algorithm;
|
||||
|
||||
int core_multi_pack_index;
|
||||
};
|
||||
|
||||
struct repository {
|
||||
|
||||
@ -3,6 +3,7 @@
|
||||
test_description='multi-pack-indexes'
|
||||
. ./test-lib.sh
|
||||
|
||||
GIT_TEST_MULTI_PACK_INDEX=0
|
||||
objdir=.git/objects
|
||||
|
||||
HASH_LEN=$(test_oid rawsz)
|
||||
@ -173,12 +174,12 @@ test_expect_success 'write progress off for redirected stderr' '
|
||||
'
|
||||
|
||||
test_expect_success 'write force progress on for stderr' '
|
||||
git multi-pack-index --object-dir=$objdir --progress write 2>err &&
|
||||
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir --progress write 2>err &&
|
||||
test_file_not_empty err
|
||||
'
|
||||
|
||||
test_expect_success 'write with the --no-progress option' '
|
||||
git multi-pack-index --object-dir=$objdir --no-progress write 2>err &&
|
||||
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir --no-progress write 2>err &&
|
||||
test_line_count = 0 err
|
||||
'
|
||||
|
||||
@ -368,17 +369,17 @@ test_expect_success 'git-fsck incorrect offset' '
|
||||
'
|
||||
|
||||
test_expect_success 'repack progress off for redirected stderr' '
|
||||
git multi-pack-index --object-dir=$objdir repack 2>err &&
|
||||
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir repack 2>err &&
|
||||
test_line_count = 0 err
|
||||
'
|
||||
|
||||
test_expect_success 'repack force progress on for stderr' '
|
||||
git multi-pack-index --object-dir=$objdir --progress repack 2>err &&
|
||||
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir --progress repack 2>err &&
|
||||
test_file_not_empty err
|
||||
'
|
||||
|
||||
test_expect_success 'repack with the --no-progress option' '
|
||||
git multi-pack-index --object-dir=$objdir --no-progress repack 2>err &&
|
||||
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir --no-progress repack 2>err &&
|
||||
test_line_count = 0 err
|
||||
'
|
||||
|
||||
@ -562,7 +563,7 @@ test_expect_success 'expire progress off for redirected stderr' '
|
||||
test_expect_success 'expire force progress on for stderr' '
|
||||
(
|
||||
cd dup &&
|
||||
git multi-pack-index --progress expire 2>err &&
|
||||
GIT_PROGRESS_DELAY=0 git multi-pack-index --progress expire 2>err &&
|
||||
test_file_not_empty err
|
||||
)
|
||||
'
|
||||
@ -570,7 +571,7 @@ test_expect_success 'expire force progress on for stderr' '
|
||||
test_expect_success 'expire with the --no-progress option' '
|
||||
(
|
||||
cd dup &&
|
||||
git multi-pack-index --no-progress expire 2>err &&
|
||||
GIT_PROGRESS_DELAY=0 git multi-pack-index --no-progress expire 2>err &&
|
||||
test_line_count = 0 err
|
||||
)
|
||||
'
|
||||
|
||||
@ -5,6 +5,7 @@ test_description='git maintenance builtin'
|
||||
. ./test-lib.sh
|
||||
|
||||
GIT_TEST_COMMIT_GRAPH=0
|
||||
GIT_TEST_MULTI_PACK_INDEX=0
|
||||
|
||||
test_expect_success 'help text' '
|
||||
test_expect_code 129 git maintenance -h 2>err &&
|
||||
@ -62,4 +63,188 @@ test_expect_success 'run --task duplicate' '
|
||||
test_i18ngrep "cannot be selected multiple times" err
|
||||
'
|
||||
|
||||
test_expect_success 'run --task=prefetch with no remotes' '
|
||||
git maintenance run --task=prefetch 2>err &&
|
||||
test_must_be_empty err
|
||||
'
|
||||
|
||||
test_expect_success 'prefetch multiple remotes' '
|
||||
git clone . clone1 &&
|
||||
git clone . clone2 &&
|
||||
git remote add remote1 "file://$(pwd)/clone1" &&
|
||||
git remote add remote2 "file://$(pwd)/clone2" &&
|
||||
git -C clone1 switch -c one &&
|
||||
git -C clone2 switch -c two &&
|
||||
test_commit -C clone1 one &&
|
||||
test_commit -C clone2 two &&
|
||||
GIT_TRACE2_EVENT="$(pwd)/run-prefetch.txt" git maintenance run --task=prefetch 2>/dev/null &&
|
||||
fetchargs="--prune --no-tags --no-write-fetch-head --recurse-submodules=no --refmap= --quiet" &&
|
||||
test_subcommand git fetch remote1 $fetchargs +refs/heads/\\*:refs/prefetch/remote1/\\* <run-prefetch.txt &&
|
||||
test_subcommand git fetch remote2 $fetchargs +refs/heads/\\*:refs/prefetch/remote2/\\* <run-prefetch.txt &&
|
||||
test_path_is_missing .git/refs/remotes &&
|
||||
git log prefetch/remote1/one &&
|
||||
git log prefetch/remote2/two &&
|
||||
git fetch --all &&
|
||||
test_cmp_rev refs/remotes/remote1/one refs/prefetch/remote1/one &&
|
||||
test_cmp_rev refs/remotes/remote2/two refs/prefetch/remote2/two
|
||||
'
|
||||
|
||||
test_expect_success 'loose-objects task' '
|
||||
# Repack everything so we know the state of the object dir
|
||||
git repack -adk &&
|
||||
|
||||
# Hack to stop maintenance from running during "git commit"
|
||||
echo in use >.git/objects/maintenance.lock &&
|
||||
|
||||
# Assuming that "git commit" creates at least one loose object
|
||||
test_commit create-loose-object &&
|
||||
rm .git/objects/maintenance.lock &&
|
||||
|
||||
ls .git/objects >obj-dir-before &&
|
||||
test_file_not_empty obj-dir-before &&
|
||||
ls .git/objects/pack/*.pack >packs-before &&
|
||||
test_line_count = 1 packs-before &&
|
||||
|
||||
# The first run creates a pack-file
|
||||
# but does not delete loose objects.
|
||||
git maintenance run --task=loose-objects &&
|
||||
ls .git/objects >obj-dir-between &&
|
||||
test_cmp obj-dir-before obj-dir-between &&
|
||||
ls .git/objects/pack/*.pack >packs-between &&
|
||||
test_line_count = 2 packs-between &&
|
||||
ls .git/objects/pack/loose-*.pack >loose-packs &&
|
||||
test_line_count = 1 loose-packs &&
|
||||
|
||||
# The second run deletes loose objects
|
||||
# but does not create a pack-file.
|
||||
git maintenance run --task=loose-objects &&
|
||||
ls .git/objects >obj-dir-after &&
|
||||
cat >expect <<-\EOF &&
|
||||
info
|
||||
pack
|
||||
EOF
|
||||
test_cmp expect obj-dir-after &&
|
||||
ls .git/objects/pack/*.pack >packs-after &&
|
||||
test_cmp packs-between packs-after
|
||||
'
|
||||
|
||||
test_expect_success 'maintenance.loose-objects.auto' '
|
||||
git repack -adk &&
|
||||
GIT_TRACE2_EVENT="$(pwd)/trace-lo1.txt" \
|
||||
git -c maintenance.loose-objects.auto=1 maintenance \
|
||||
run --auto --task=loose-objects 2>/dev/null &&
|
||||
test_subcommand ! git prune-packed --quiet <trace-lo1.txt &&
|
||||
printf data-A | git hash-object -t blob --stdin -w &&
|
||||
GIT_TRACE2_EVENT="$(pwd)/trace-loA" \
|
||||
git -c maintenance.loose-objects.auto=2 \
|
||||
maintenance run --auto --task=loose-objects 2>/dev/null &&
|
||||
test_subcommand ! git prune-packed --quiet <trace-loA &&
|
||||
printf data-B | git hash-object -t blob --stdin -w &&
|
||||
GIT_TRACE2_EVENT="$(pwd)/trace-loB" \
|
||||
git -c maintenance.loose-objects.auto=2 \
|
||||
maintenance run --auto --task=loose-objects 2>/dev/null &&
|
||||
test_subcommand git prune-packed --quiet <trace-loB &&
|
||||
GIT_TRACE2_EVENT="$(pwd)/trace-loC" \
|
||||
git -c maintenance.loose-objects.auto=2 \
|
||||
maintenance run --auto --task=loose-objects 2>/dev/null &&
|
||||
test_subcommand git prune-packed --quiet <trace-loC
|
||||
'
|
||||
|
||||
test_expect_success 'incremental-repack task' '
|
||||
packDir=.git/objects/pack &&
|
||||
for i in $(test_seq 1 5)
|
||||
do
|
||||
test_commit $i || return 1
|
||||
done &&
|
||||
|
||||
# Create three disjoint pack-files with size BIG, small, small.
|
||||
echo HEAD~2 | git pack-objects --revs $packDir/test-1 &&
|
||||
test_tick &&
|
||||
git pack-objects --revs $packDir/test-2 <<-\EOF &&
|
||||
HEAD~1
|
||||
^HEAD~2
|
||||
EOF
|
||||
test_tick &&
|
||||
git pack-objects --revs $packDir/test-3 <<-\EOF &&
|
||||
HEAD
|
||||
^HEAD~1
|
||||
EOF
|
||||
rm -f $packDir/pack-* &&
|
||||
rm -f $packDir/loose-* &&
|
||||
ls $packDir/*.pack >packs-before &&
|
||||
test_line_count = 3 packs-before &&
|
||||
|
||||
# the job repacks the two into a new pack, but does not
|
||||
# delete the old ones.
|
||||
git maintenance run --task=incremental-repack &&
|
||||
ls $packDir/*.pack >packs-between &&
|
||||
test_line_count = 4 packs-between &&
|
||||
|
||||
# the job deletes the two old packs, and does not write
|
||||
# a new one because the batch size is not high enough to
|
||||
# pack the largest pack-file.
|
||||
git maintenance run --task=incremental-repack &&
|
||||
ls .git/objects/pack/*.pack >packs-after &&
|
||||
test_line_count = 2 packs-after
|
||||
'
|
||||
|
||||
test_expect_success EXPENSIVE 'incremental-repack 2g limit' '
|
||||
for i in $(test_seq 1 5)
|
||||
do
|
||||
test-tool genrandom foo$i $((512 * 1024 * 1024 + 1)) >>big ||
|
||||
return 1
|
||||
done &&
|
||||
git add big &&
|
||||
git commit -m "Add big file (1)" &&
|
||||
|
||||
# ensure any possible loose objects are in a pack-file
|
||||
git maintenance run --task=loose-objects &&
|
||||
|
||||
rm big &&
|
||||
for i in $(test_seq 6 10)
|
||||
do
|
||||
test-tool genrandom foo$i $((512 * 1024 * 1024 + 1)) >>big ||
|
||||
return 1
|
||||
done &&
|
||||
git add big &&
|
||||
git commit -m "Add big file (2)" &&
|
||||
|
||||
# ensure any possible loose objects are in a pack-file
|
||||
git maintenance run --task=loose-objects &&
|
||||
|
||||
# Now run the incremental-repack task and check the batch-size
|
||||
GIT_TRACE2_EVENT="$(pwd)/run-2g.txt" git maintenance run \
|
||||
--task=incremental-repack 2>/dev/null &&
|
||||
test_subcommand git multi-pack-index repack \
|
||||
--no-progress --batch-size=2147483647 <run-2g.txt
|
||||
'
|
||||
|
||||
test_expect_success 'maintenance.incremental-repack.auto' '
|
||||
git repack -adk &&
|
||||
git config core.multiPackIndex true &&
|
||||
git multi-pack-index write &&
|
||||
GIT_TRACE2_EVENT="$(pwd)/midx-init.txt" git \
|
||||
-c maintenance.incremental-repack.auto=1 \
|
||||
maintenance run --auto --task=incremental-repack 2>/dev/null &&
|
||||
test_subcommand ! git multi-pack-index write --no-progress <midx-init.txt &&
|
||||
test_commit A &&
|
||||
git pack-objects --revs .git/objects/pack/pack <<-\EOF &&
|
||||
HEAD
|
||||
^HEAD~1
|
||||
EOF
|
||||
GIT_TRACE2_EVENT=$(pwd)/trace-A git \
|
||||
-c maintenance.incremental-repack.auto=2 \
|
||||
maintenance run --auto --task=incremental-repack 2>/dev/null &&
|
||||
test_subcommand ! git multi-pack-index write --no-progress <trace-A &&
|
||||
test_commit B &&
|
||||
git pack-objects --revs .git/objects/pack/pack <<-\EOF &&
|
||||
HEAD
|
||||
^HEAD~1
|
||||
EOF
|
||||
GIT_TRACE2_EVENT=$(pwd)/trace-B git \
|
||||
-c maintenance.incremental-repack.auto=2 \
|
||||
maintenance run --auto --task=incremental-repack 2>/dev/null &&
|
||||
test_subcommand git multi-pack-index write --no-progress <trace-B
|
||||
'
|
||||
|
||||
test_done
|
||||
|
||||
Loading…
Reference in New Issue
Block a user