git-commit-vandalism/builtin
Taylor Blau f57a739691 midx: avoid opening multiple MIDXs when writing
Opening multiple instance of the same MIDX can lead to problems like two
separate packed_git structures which represent the same pack being added
to the repository's object store.

The above scenario can happen because prepare_midx_pack() checks if
`m->packs[pack_int_id]` is NULL in order to determine if a pack has been
opened and installed in the repository before. But a caller can
construct two copies of the same MIDX by calling get_multi_pack_index()
and load_multi_pack_index() since the former manipulates the
object store directly but the latter is a lower-level routine which
allocates a new MIDX for each call.

So if prepare_midx_pack() is called on multiple MIDXs with the same
pack_int_id, then that pack will be installed twice in the object
store's packed_git pointer.

This can lead to problems in, for e.g., the pack-bitmap code, which does
something like the following (in pack-bitmap.c:open_pack_bitmap()):

    struct bitmap_index *bitmap_git = ...;
    for (p = get_all_packs(r); p; p = p->next) {
      if (open_pack_bitmap_1(bitmap_git, p) == 0)
        ret = 0;
    }

which is a problem if two copies of the same pack exist in the
packed_git list because pack-bitmap.c:open_pack_bitmap_1() contains a
conditional like the following:

    if (bitmap_git->pack || bitmap_git->midx) {
      /* ignore extra bitmap file; we can only handle one */
      warning("ignoring extra bitmap file: %s", packfile->pack_name);
      close(fd);
      return -1;
    }

Avoid this scenario by not letting write_midx_internal() open a MIDX
that isn't also pointed at by the object store. So long as this is the
case, other routines should prefer to open MIDXs with
get_multi_pack_index() or reprepare_packed_git() instead of creating
instances on their own. Because get_multi_pack_index() returns
`r->object_store->multi_pack_index` if it is non-NULL, we'll only have
one instance of a MIDX open at one time, avoiding these problems.

To encourage this, drop the `struct multi_pack_index *` parameter from
`write_midx_internal()`, and rely instead on the `object_dir` to find
(or initialize) the correct MIDX instance.

Likewise, replace the call to `close_midx()` with
`close_object_store()`, since we're about to replace the MIDX with a new
one and should invalidate the object store's memory of any MIDX that
might have existed beforehand.

Note that this now forbids passing object directories that don't belong
to alternate repositories over `--object-dir`, since before we would
have happily opened a MIDX in any directory, but now restrict ourselves
to only those reachable by `r->objects->multi_pack_index` (and alternate
MIDXs that we can see by walking the `next` pointer).

As far as I can tell, supporting arbitrary directories with
`--object-dir` was a historical accident, since even the documentation
says `<alt>` when referring to the value passed to this option.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01 13:56:43 -07:00
..
add.c dir.[ch]: replace dir_init() with DIR_INIT 2021-07-01 12:32:22 -07:00
am.c *.c static functions: add missing __attribute__((format)) 2021-07-13 15:20:20 -07:00
annotate.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
apply.c
archive.c
bisect--helper.c *.c static functions: add missing __attribute__((format)) 2021-07-13 15:20:20 -07:00
blame.c Merge branch 'rs/blame-optim' 2021-02-25 16:43:29 -08:00
branch.c ref-filter: reuse output buffer 2021-04-20 11:09:50 -07:00
bugreport.c builtin/bugreport: don't leak prefixed filename 2021-04-28 09:25:45 +09:00
bundle.c bundle: remove "ref_list" in favor of string-list.c API 2021-07-06 12:10:17 -07:00
cat-file.c cat-file: merge two block into one 2021-06-04 07:50:26 +09:00
check-attr.c
check-ignore.c dir.[ch]: replace dir_init() with DIR_INIT 2021-07-01 12:32:22 -07:00
check-mailmap.c shortlog: remove unused(?) "repo-abbrev" feature 2021-01-12 14:04:42 -08:00
check-ref-format.c
checkout--worker.c builtin/checkout--worker: zero-initialise struct to avoid MSAN complaints 2021-06-15 12:07:56 +09:00
checkout-index.c Merge branch 'mt/parallel-checkout-part-3' 2021-05-16 21:05:23 +09:00
checkout.c checkout: stop expanding sparse indexes 2021-07-14 15:05:53 -07:00
clean.c dir.[ch]: replace dir_init() with DIR_INIT 2021-07-01 12:32:22 -07:00
clone.c Merge branch 'jk/clone-clean-upon-transport-error' 2021-06-14 13:33:26 +09:00
column.c column, range-diff: downcase option description 2021-03-29 14:06:08 -07:00
commit-graph.c midx: avoid opening multiple MIDXs when writing 2021-09-01 13:56:43 -07:00
commit-tree.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
commit.c Merge branch 'ds/commit-and-checkout-with-sparse-index' 2021-08-04 13:28:53 -07:00
config.c config: unify code paths to get global config paths 2021-04-19 14:16:59 -07:00
count-objects.c
credential-cache--daemon.c unix-socket: add backlog size option to unix_stream_listen() 2021-03-15 14:32:51 -07:00
credential-cache.c unix-socket: disallow chdir() when creating unix domain sockets 2021-03-15 14:32:51 -07:00
credential-store.c crendential-store: use timeout when locking file 2020-11-25 12:30:18 -08:00
credential.c credential: load default config 2020-10-16 12:30:45 -07:00
describe.c hash: provide per-algorithm null OIDs 2021-04-27 16:31:39 +09:00
diff-files.c Merge branch 'jc/diffcore-rotate' 2021-02-25 16:43:30 -08:00
diff-index.c diff-merges: move specific diff-index "-m" handling to diff-index 2021-05-21 09:24:14 +09:00
diff-tree.c Merge branch 'jc/diffcore-rotate' 2021-02-25 16:43:30 -08:00
diff.c Merge branch 'dl/diff-merge-base' 2021-07-28 13:17:59 -07:00
difftool.c Merge branch 'ab/cmd-foo-should-return' 2021-07-08 13:15:04 -07:00
env--helper.c assert PARSE_OPT_NONEG in parse-options callbacks 2020-09-30 12:53:47 -07:00
fast-export.c hash: provide per-algorithm null OIDs 2021-04-27 16:31:39 +09:00
fast-import.c Use the final_oid_fn to finalize hashing of object IDs 2021-04-27 16:31:38 +09:00
fetch-pack.c connect, transport: encapsulate arg in struct 2021-02-05 13:49:54 -08:00
fetch.c Merge branch 'ab/fetch-negotiate-segv-fix' 2021-07-16 17:42:48 -07:00
fmt-merge-msg.c Lib-ify fmt-merge-msg 2020-03-24 15:04:43 -07:00
for-each-ref.c Merge branch 'ah/plugleaks' 2021-05-07 12:47:41 +09:00
for-each-repo.c builtin/for-each-repo: remove unnecessary argv copy to plug leak 2021-07-26 12:19:20 -07:00
fsck.c Merge branch 'ab/fsck-api-cleanup' 2021-06-02 07:34:27 +09:00
gc.c maintenance: fix two memory leaks 2021-05-12 07:00:45 +09:00
get-tar-commit-id.c
grep.c dir.[ch]: replace dir_init() with DIR_INIT 2021-07-01 12:32:22 -07:00
hash-object.c
help.c help: convert git_cmd to page in one place 2021-07-06 13:09:20 -07:00
index-pack.c *.c static functions: don't forward-declare __attribute__ 2021-07-12 12:09:53 -07:00
init-db.c Merge branch 'mt/init-template-userpath-fix' 2021-05-25 16:21:20 +09:00
interpret-trailers.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
log.c Merge branch 'jk/log-decorate-optim' 2021-07-28 13:17:58 -07:00
ls-files.c dir.[ch]: replace dir_init() with DIR_INIT 2021-07-01 12:32:22 -07:00
ls-remote.c Merge branch 'ah/plugleaks' 2021-04-07 16:54:08 -07:00
ls-tree.c tree.h API: simplify read_tree_recursive() signature 2021-03-20 16:09:26 -07:00
mailinfo.c mailinfo: allow squelching quoted CRLF warning 2021-05-10 15:06:22 +09:00
mailsplit.c
merge-base.c rebase: --fork-point regression fix 2020-02-11 09:59:39 -08:00
merge-file.c
merge-index.c merge-index: ensure full index 2021-04-14 13:47:21 -07:00
merge-ours.c builtins + test helpers: use return instead of exit() in cmd_* 2021-06-09 09:15:58 +09:00
merge-recursive.c
merge-tree.c xdiff users: use designated initializers for out_line 2021-05-11 12:47:31 +09:00
merge.c Merge branch 'pb/merge-autostash-more' 2021-08-04 13:28:54 -07:00
mktag.c fsck.c: add an fsck_set_msg_type() API that takes enums 2021-03-28 19:03:10 -07:00
mktree.c builtins + test helpers: use return instead of exit() in cmd_* 2021-06-09 09:15:58 +09:00
multi-pack-index.c multi-pack-index: fix potential segfault without sub-command 2021-07-19 15:24:01 -07:00
mv.c builtin/mv: free or UNLEAK multiple pointers at end of cmd_mv 2021-07-26 12:19:20 -07:00
name-rev.c oid_pos(): access table through const pointers 2021-01-28 12:03:26 -08:00
notes.c use CALLOC_ARRAY 2021-03-13 16:00:09 -08:00
pack-objects.c pack-bitmap-write.c: gracefully fail to write non-closed bitmaps 2021-08-24 13:21:13 -07:00
pack-redundant.c builtin/pack-redundant: avoid casting buffers to struct object_id 2021-04-27 16:31:38 +09:00
pack-refs.c
patch-id.c patch-id: use oid_to_hex() to print multiple object IDs 2019-12-09 12:26:40 -08:00
prune-packed.c Lib-ify prune-packed 2020-03-24 15:04:44 -07:00
prune.c Merge branch 'tb/shallow-cleanup' 2020-05-13 12:19:18 -07:00
pull.c pull: trivial whitespace style fix 2021-06-19 16:36:17 +09:00
push.c push: don't get a full remote object 2021-06-02 10:12:03 +09:00
range-diff.c column, range-diff: downcase option description 2021-03-29 14:06:08 -07:00
read-tree.c Use OPT_CALLBACK and OPT_CALLBACK_F 2020-04-28 10:47:10 -07:00
rebase.c builtin/rebase: fix options.strategy memory lifecycle 2021-07-26 12:19:21 -07:00
receive-pack.c *.c static functions: don't forward-declare __attribute__ 2021-07-12 12:09:53 -07:00
reflog.c reflog expire --stale-fix: be generous about missing objects 2021-02-11 09:21:52 -08:00
remote-ext.c strvec: convert builtin/ callers away from argv_array name 2020-07-28 15:02:18 -07:00
remote-fd.c
remote.c Merge branch 'ah/plugleaks' 2021-04-07 16:54:08 -07:00
repack.c repack: avoid loosening promisor objects in partial clones 2021-04-28 13:36:13 +09:00
replace.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
rerere.c xdiff users: use designated initializers for out_line 2021-05-11 12:47:31 +09:00
reset.c reset: free instead of leaking unneeded ref 2021-03-14 15:57:59 -07:00
rev-list.c rev-list: add option for --pretty=format without header 2021-07-12 10:12:31 -07:00
rev-parse.c rev-parse: mark die() messages for translation 2021-05-17 18:39:53 +09:00
revert.c sequencer: fix edit handling for cherry-pick and revert messages 2021-03-31 14:10:50 -07:00
rm.c Merge branch 'ah/plugleaks' 2021-05-07 12:47:41 +09:00
send-pack.c push: parse and set flag for "--force-if-includes" 2020-10-03 09:59:19 -07:00
shortlog.c Merge branch 'ab/mailmap' 2021-01-25 14:19:19 -08:00
show-branch.c show-branch: don't <COLOR></RESET> for space characters 2021-06-28 09:33:06 -07:00
show-index.c builtin/show-index: set the algorithm for object IDs 2021-04-27 16:31:39 +09:00
show-ref.c refs: switch peel_ref() to peel_iterated_oid() 2021-01-21 15:51:31 -08:00
sparse-checkout.c use fspathhash() everywhere 2021-07-30 12:14:27 -07:00
stash.c Merge branch 'ab/struct-init' 2021-07-16 17:42:53 -07:00
stripspace.c
submodule--helper.c Merge branch 'ah/plugleaks' 2021-08-04 13:28:52 -07:00
symbolic-ref.c symbolic-ref: don't leak shortened refname in check_symref() 2021-03-14 15:57:59 -07:00
tag.c ref-filter: reuse output buffer 2021-04-20 11:09:50 -07:00
unpack-file.c
unpack-objects.c hash: provide per-algorithm null OIDs 2021-04-27 16:31:39 +09:00
update-index.c update-index: ensure full index 2021-04-14 13:47:29 -07:00
update-ref.c update-ref: disallow "start" for ongoing transactions 2020-11-16 13:44:01 -08:00
update-server-info.c
upload-archive.c strvec: rename struct fields 2020-07-30 19:18:06 -07:00
upload-pack.c
var.c
verify-commit.c
verify-pack.c Merge branch 'bc/sha-256-part-3' 2020-08-11 18:04:11 -07:00
verify-tag.c
worktree.c worktree: teach add to accept --reason <string> with --lock 2021-07-15 13:30:59 -07:00
write-tree.c