git-commit-vandalism/builtin
Jeff King 9cf10d8786 repack: use tempfiles for signal cleanup
When git-repack exits due to a signal, it tries to clean up by calling
its remove_temporary_files() function, which walks through the packs dir
looking for ".tmp-$$-pack-*" files to delete (where "$$" is the pid of
the current process).

The biggest problem here is that remove_temporary_files() is not safe to
call in a signal handler. It uses opendir(), which isn't on the POSIX
async-signal-safe list. The details will be platform-specific, but a
likely issue is that it needs to allocate memory; if we receive a signal
while inside malloc(), etc, we'll conflict on the allocator lock and
deadlock with ourselves.

We can fix this by just cleaning up the files directly, without walking
the directory. We already know the complete list of .tmp-* files that
were generated, because we recorded them via populate_pack_exts(). When
we find files there, we can use register_tempfile() to record the
filenames. If we receive a signal, then the tempfile API will clean them
up for us, and it's async-safe and pretty battle-tested.

Note that this is slightly racier than the existing scheme. We don't
record the filenames until pack-objects tells us the hash over stdout.
So during the period between it generating the file and reporting the
hash, we'd fail to clean up. However, that period is very small. During
most of the pack generation process pack-objects is using its own
internal tempfiles. It's only at the very end that it moves them into
the names git-repack expects, and then it immediately reports the name
to us. Given that cleanup like this is best effort (after all, we may
get SIGKILL), this level of race is acceptable.

When we register the tempfiles, we'll record them locally and use the
result to call rename_tempfile(), rather than renaming by hand.  This
isn't strictly necessary, as once we've renamed the files they're gone,
and the tempfile API's cleanup unlink() would simply become a pointless
noop. But managing the lifetimes of the tempfile objects is the cleanest
thing to do, and the tempfile pointers naturally fill the same role as
the old booleans.

This patch also fixes another small problem. We only hook signals, and
don't set up an atexit handler. So if we see an error that causes us to
die(), we'll leave the .tmp-* files in place. But since the tempfile API
handles this for us, this is now fixed for free. The new test covers
this by stimulating a failure of pack-objects when generating a cruft
pack. Before this patch, the .tmp-* file for the main pack would have
been left, but now we correctly clean it up.

Two small subtleties on the implementation:

  - in the renaming loop, we can stop re-constructing fname_old; we only
    use it when we have a tempfile to rename, so we can just ask the
    tempfile for its path (which, barring bugs, should be identical)

  - when renaming fails, our error message mentions fname_old. But since
    a failed rename_tempfile() invalidates the tempfile struct, we'll
    lose access to that string. Instead, let's mention the destination
    filename, which is what most other callers do.

Reported-by: Jan Pokorný <poki@fnusa.cz>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-21 18:03:52 -07:00
..
add.c Merge branch 'ab/plug-leak-in-revisions' 2022-06-07 14:10:56 -07:00
am.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
annotate.c
apply.c apply.c: remove unnecessary include 2022-04-06 09:42:14 -07:00
archive.c parse-options: PARSE_OPT_KEEP_UNKNOWN only applies to --options 2022-08-19 11:13:14 -07:00
bisect--helper.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
blame.c parse-options: add support for parsing subcommands 2022-08-19 11:13:14 -07:00
branch.c branch: drop unused worktrees variable 2022-06-21 08:52:37 -07:00
bugreport.c builtin/bugreport.c: create '--diagnose' option 2022-08-12 13:20:02 -07:00
bundle.c builtin/bundle.c: let parse-options parse subcommands 2022-08-19 11:13:15 -07:00
cat-file.c Merge branch 'tb/cat-file-z' 2022-08-05 15:52:14 -07:00
check-attr.c
check-ignore.c dir.[ch]: replace dir_init() with DIR_INIT 2021-07-01 12:32:22 -07:00
check-mailmap.c
check-ref-format.c check-ref-format: fix trivial memory leak 2022-07-01 11:43:42 -07:00
checkout--worker.c pkt-line.[ch]: remove unused packet_read_line_buf() 2021-10-15 13:09:40 -07:00
checkout-index.c checkout-index: integrate with sparse index 2022-01-13 13:49:45 -08:00
checkout.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
clean.c Merge branch 'vd/sparse-clean-etc' 2022-02-17 16:25:05 -08:00
clone.c Sync with 2.37.4 2022-10-06 20:00:04 -04:00
column.c column: fix parsing of the '--nl' option 2021-08-26 14:36:27 -07:00
commit-graph.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
commit-tree.c use xopen() to handle fatal open(2) failures 2021-08-25 14:39:08 -07:00
commit.c Merge branch 'ab/plug-leak-in-revisions' 2022-06-07 14:10:56 -07:00
config.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
count-objects.c i18n: remove from i18n strings that do not hold translatable parts 2022-02-04 13:58:28 -08:00
credential-cache--daemon.c
credential-cache.c credential-cache: check for windows specific errors 2021-09-14 09:30:54 -07:00
credential-store.c Use a better name for the function interpolating paths 2021-07-26 12:17:16 -07:00
credential.c doc: fix git credential synopsis 2021-10-28 09:57:09 -07:00
describe.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
diagnose.c builtin/diagnose.c: don't translate the two mode values 2022-09-21 11:53:35 -07:00
diff-files.c diff-files: move misplaced cleanup label 2022-07-12 07:17:28 -07:00
diff-index.c revisions API: call diff_free(&revs->pruning) in revisions_release() 2022-04-13 23:56:10 -07:00
diff-tree.c 2.36 gitk/diff-tree --stdin regression fix 2022-04-26 09:26:35 -07:00
diff.c Merge branch 'ab/plug-leak-in-revisions' 2022-06-07 14:10:56 -07:00
difftool.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
env--helper.c parse-options: PARSE_OPT_KEEP_UNKNOWN only applies to --options 2022-08-19 11:13:14 -07:00
fast-export.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
fast-import.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
fetch-pack.c list-objects-filter: add and use initializers 2022-09-12 08:38:59 -07:00
fetch.c Merge branch 'jk/list-objects-filter-cleanup' 2022-09-19 14:35:24 -07:00
fmt-merge-msg.c merge: allow to pretend a merge is made into a different branch 2021-12-20 14:55:02 -08:00
for-each-ref.c for-each-ref: delay parsing of --sort=<atom> options 2021-10-20 14:33:07 -07:00
for-each-repo.c builtin/for-each-repo: remove unnecessary argv copy to plug leak 2021-07-26 12:19:20 -07:00
fsck.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
fsmonitor--daemon.c fsmonitor--daemon: stub in health thread 2022-05-26 15:59:27 -07:00
gc.c Merge branch 'sg/parse-options-subcommand' 2022-09-21 15:27:03 -07:00
get-tar-commit-id.c
grep.c grep: add --max-count command line option 2022-06-22 13:23:29 -07:00
hash-object.c Merge branch 'ab/object-file-api-updates' 2022-03-16 17:53:08 -07:00
help.c git help: special-case scalar 2022-09-02 10:02:56 -07:00
hook.c builtin/hook.c: let parse-options parse subcommands 2022-08-19 11:13:15 -07:00
index-pack.c i18n: fix mismatched camelCase config variables 2022-06-17 10:38:26 -07:00
init-db.c i18n: refactor "foo and bar are mutually exclusive" 2022-01-05 13:29:23 -08:00
interpret-trailers.c
log.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
ls-files.c ls-files: fix black space in error message 2022-09-12 09:25:40 -07:00
ls-remote.c Merge branch 'ep/maint-equals-null-cocci' 2022-05-20 15:26:59 -07:00
ls-tree.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
mailinfo.c mailinfo: allow squelching quoted CRLF warning 2021-05-10 15:06:22 +09:00
mailsplit.c Merge branch 'ep/maint-equals-null-cocci' 2022-05-20 15:26:59 -07:00
merge-base.c merge-base: free() allocated "struct commit **" list 2022-03-04 13:24:17 -08:00
merge-file.c merge-file: fix memory leaks on error path 2022-07-01 11:43:43 -07:00
merge-index.c merge-index: ensure full index 2021-04-14 13:47:21 -07:00
merge-ours.c builtins + test helpers: use return instead of exit() in cmd_* 2021-06-09 09:15:58 +09:00
merge-recursive.c gettext API users: don't explicitly cast ngettext()'s "n" 2022-03-07 11:57:52 -08:00
merge-tree.c merge-tree: add a --allow-unrelated-histories flag 2022-06-22 16:10:06 -07:00
merge.c Merge branch 'en/merge-unstash-only-on-clean-merge' into maint 2022-09-13 12:21:11 -07:00
mktag.c Merge branch 'ab/object-file-api-updates' 2022-03-16 17:53:08 -07:00
mktree.c mktree: do not check type of remote objects 2022-06-21 10:12:15 -07:00
multi-pack-index.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
mv.c Merge branch 'sy/mv-out-of-cone' 2022-09-19 14:35:23 -07:00
name-rev.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
notes.c notes, remote: show unknown subcommands between `' 2022-09-07 12:06:12 -07:00
pack-objects.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
pack-redundant.c tree-wide: apply equals-null.cocci 2022-05-02 09:50:37 -07:00
pack-refs.c
patch-id.c patch-id: fix scan_hunk_header on diffs with 1 line of before/after 2022-02-02 11:24:23 -08:00
prune-packed.c i18n: remove from i18n strings that do not hold translatable parts 2022-02-04 13:58:28 -08:00
prune.c revisions API users: add straightforward release_revisions() 2022-04-13 23:56:08 -07:00
pull.c pull: fix a "struct oid_array" memory leak 2022-07-01 11:43:43 -07:00
push.c push: fix capitalisation of the option name autoSetupMerge 2022-06-15 11:45:46 -07:00
range-diff.c range-diff: optionally accept pathspecs 2022-08-26 09:49:26 -07:00
read-tree.c read-tree: make three-way merge sparse-aware 2022-03-01 12:36:01 -08:00
rebase.c rebase: add rebase.updateRefs config option 2022-07-19 12:49:04 -07:00
receive-pack.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
reflog.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
remote-ext.c
remote-fd.c
remote.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
repack.c repack: use tempfiles for signal cleanup 2022-10-21 18:03:52 -07:00
replace.c refs: use ref_namespaces for replace refs base 2022-08-05 14:13:12 -07:00
rerere.c xdiff users: use designated initializers for out_line 2021-05-11 12:47:31 +09:00
reset.c pathspec.h: move pathspec_needs_expanded_index() from reset.c to here 2022-08-08 13:23:26 -07:00
rev-list.c rev-list: support human-readable output for --disk-usage 2022-08-11 13:45:23 -07:00
rev-parse.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
revert.c parse-options: PARSE_OPT_KEEP_UNKNOWN only applies to --options 2022-08-19 11:13:14 -07:00
rm.c rm: integrate with sparse-index 2022-08-08 13:23:26 -07:00
send-pack.c i18n: factorize "invalid value" messages 2022-02-04 13:58:28 -08:00
shortlog.c parse-options: add support for parsing subcommands 2022-08-19 11:13:14 -07:00
show-branch.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
show-index.c builtin/show-index: set the algorithm for object IDs 2021-04-27 16:31:39 +09:00
show-ref.c git-compat-util.h: use "UNUSED", not "UNUSED(var)" 2022-09-01 10:49:48 -07:00
sparse-checkout.c pass subcommand "prefix" arguments to parse_options() 2022-08-25 09:43:29 -07:00
stash.c Merge branch 'ab/unused-annotation' 2022-09-14 12:56:39 -07:00
stripspace.c i18n: remove from i18n strings that do not hold translatable parts 2022-02-04 13:58:28 -08:00
submodule--helper.c Merge branch 'jk/list-objects-filter-cleanup' 2022-09-19 14:35:24 -07:00
symbolic-ref.c symbolic-ref: refuse to set syntactically invalid target 2022-08-01 12:17:13 -07:00
tag.c Merge branch 'ep/maint-equals-null-cocci' 2022-05-20 15:26:59 -07:00
unpack-file.c
unpack-objects.c unpack-objects: use stream_loose_object() to unpack large objects 2022-06-13 10:22:36 -07:00
update-index.c Merge branch 'jh/builtin-fsmonitor-part3' 2022-06-10 15:04:15 -07:00
update-ref.c update-ref: fix streaming of status updates 2021-09-03 11:35:15 -07:00
update-server-info.c i18n: remove from i18n strings that do not hold translatable parts 2022-02-04 13:58:28 -08:00
upload-archive.c upload-archive: use regular "struct child_process" pattern 2021-11-25 22:15:07 -08:00
upload-pack.c upload-pack: document and rename --advertise-refs 2021-08-05 08:59:37 -07:00
var.c var: add GIT_DEFAULT_BRANCH variable 2021-11-03 13:25:36 -07:00
verify-commit.c
verify-pack.c
verify-tag.c
worktree.c builtin/worktree.c: let parse-options parse subcommands 2022-08-19 11:13:16 -07:00
write-tree.c