f57a739691
Opening multiple instance of the same MIDX can lead to problems like two
separate packed_git structures which represent the same pack being added
to the repository's object store.
The above scenario can happen because prepare_midx_pack() checks if
`m->packs[pack_int_id]` is NULL in order to determine if a pack has been
opened and installed in the repository before. But a caller can
construct two copies of the same MIDX by calling get_multi_pack_index()
and load_multi_pack_index() since the former manipulates the
object store directly but the latter is a lower-level routine which
allocates a new MIDX for each call.
So if prepare_midx_pack() is called on multiple MIDXs with the same
pack_int_id, then that pack will be installed twice in the object
store's packed_git pointer.
This can lead to problems in, for e.g., the pack-bitmap code, which does
something like the following (in pack-bitmap.c:open_pack_bitmap()):
struct bitmap_index *bitmap_git = ...;
for (p = get_all_packs(r); p; p = p->next) {
if (open_pack_bitmap_1(bitmap_git, p) == 0)
ret = 0;
}
which is a problem if two copies of the same pack exist in the
packed_git list because pack-bitmap.c:open_pack_bitmap_1() contains a
conditional like the following:
if (bitmap_git->pack || bitmap_git->midx) {
/* ignore extra bitmap file; we can only handle one */
warning("ignoring extra bitmap file: %s", packfile->pack_name);
close(fd);
return -1;
}
Avoid this scenario by not letting write_midx_internal() open a MIDX
that isn't also pointed at by the object store. So long as this is the
case, other routines should prefer to open MIDXs with
get_multi_pack_index() or reprepare_packed_git() instead of creating
instances on their own. Because get_multi_pack_index() returns
`r->object_store->multi_pack_index` if it is non-NULL, we'll only have
one instance of a MIDX open at one time, avoiding these problems.
To encourage this, drop the `struct multi_pack_index *` parameter from
`write_midx_internal()`, and rely instead on the `object_dir` to find
(or initialize) the correct MIDX instance.
Likewise, replace the call to `close_midx()` with
`close_object_store()`, since we're about to replace the MIDX with a new
one and should invalidate the object store's memory of any MIDX that
might have existed beforehand.
Note that this now forbids passing object directories that don't belong
to alternate repositories over `--object-dir`, since before we would
have happily opened a MIDX in any directory, but now restrict ourselves
to only those reachable by `r->objects->multi_pack_index` (and alternate
MIDXs that we can see by walking the `next` pointer).
As far as I can tell, supporting arbitrary directories with
`--object-dir` was a historical accident, since even the documentation
says `<alt>` when referring to the value passed to this option.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
109 lines
3.3 KiB
Plaintext
109 lines
3.3 KiB
Plaintext
git-multi-pack-index(1)
|
|
=======================
|
|
|
|
NAME
|
|
----
|
|
git-multi-pack-index - Write and verify multi-pack-indexes
|
|
|
|
|
|
SYNOPSIS
|
|
--------
|
|
[verse]
|
|
'git multi-pack-index' [--object-dir=<dir>] [--[no-]progress]
|
|
[--preferred-pack=<pack>] <subcommand>
|
|
|
|
DESCRIPTION
|
|
-----------
|
|
Write or verify a multi-pack-index (MIDX) file.
|
|
|
|
OPTIONS
|
|
-------
|
|
|
|
--object-dir=<dir>::
|
|
Use given directory for the location of Git objects. We check
|
|
`<dir>/packs/multi-pack-index` for the current MIDX file, and
|
|
`<dir>/packs` for the pack-files to index.
|
|
+
|
|
`<dir>` must be an alternate of the current repository.
|
|
|
|
--[no-]progress::
|
|
Turn progress on/off explicitly. If neither is specified, progress is
|
|
shown if standard error is connected to a terminal.
|
|
|
|
The following subcommands are available:
|
|
|
|
write::
|
|
Write a new MIDX file. The following options are available for
|
|
the `write` sub-command:
|
|
+
|
|
--
|
|
--preferred-pack=<pack>::
|
|
Optionally specify the tie-breaking pack used when
|
|
multiple packs contain the same object. `<pack>` must
|
|
contain at least one object. If not given, ties are
|
|
broken in favor of the pack with the lowest mtime.
|
|
--
|
|
|
|
verify::
|
|
Verify the contents of the MIDX file.
|
|
|
|
expire::
|
|
Delete the pack-files that are tracked by the MIDX file, but
|
|
have no objects referenced by the MIDX. Rewrite the MIDX file
|
|
afterward to remove all references to these pack-files.
|
|
|
|
repack::
|
|
Create a new pack-file containing objects in small pack-files
|
|
referenced by the multi-pack-index. If the size given by the
|
|
`--batch-size=<size>` argument is zero, then create a pack
|
|
containing all objects referenced by the multi-pack-index. For
|
|
a non-zero batch size, Select the pack-files by examining packs
|
|
from oldest-to-newest, computing the "expected size" by counting
|
|
the number of objects in the pack referenced by the
|
|
multi-pack-index, then divide by the total number of objects in
|
|
the pack and multiply by the pack size. We select packs with
|
|
expected size below the batch size until the set of packs have
|
|
total expected size at least the batch size, or all pack-files
|
|
are considered. If only one pack-file is selected, then do
|
|
nothing. If a new pack-file is created, rewrite the
|
|
multi-pack-index to reference the new pack-file. A later run of
|
|
'git multi-pack-index expire' will delete the pack-files that
|
|
were part of this batch.
|
|
+
|
|
If `repack.packKeptObjects` is `false`, then any pack-files with an
|
|
associated `.keep` file will not be selected for the batch to repack.
|
|
|
|
|
|
EXAMPLES
|
|
--------
|
|
|
|
* Write a MIDX file for the packfiles in the current .git folder.
|
|
+
|
|
-----------------------------------------------
|
|
$ git multi-pack-index write
|
|
-----------------------------------------------
|
|
|
|
* Write a MIDX file for the packfiles in an alternate object store.
|
|
+
|
|
-----------------------------------------------
|
|
$ git multi-pack-index --object-dir <alt> write
|
|
-----------------------------------------------
|
|
|
|
* Verify the MIDX file for the packfiles in the current .git folder.
|
|
+
|
|
-----------------------------------------------
|
|
$ git multi-pack-index verify
|
|
-----------------------------------------------
|
|
|
|
|
|
SEE ALSO
|
|
--------
|
|
See link:technical/multi-pack-index.html[The Multi-Pack-Index Design
|
|
Document] and link:technical/pack-format.html[The Multi-Pack-Index
|
|
Format] for more information on the multi-pack-index feature.
|
|
|
|
|
|
GIT
|
|
---
|
|
Part of the linkgit:git[1] suite
|