* js/maint-1.6.6-send-pack-stateless-rpc-deadlock-fix:
send-pack: avoid deadlock when pack-object dies early
Evil merge to adjust the way the use of pthreads in sideband-demultiplexor
was decided (earlier it was "if we are not on Windows", now it is "if we
are not using pthreads").
Send-pack deadlocks in two ways when pack-object dies early (for example,
because there is some repo corruption).
The first deadlock happens with the smart push protocol (--stateless-rpc).
After the initial rev-exchange, the remote is waiting for the pack data
to arrive, and the sideband demuxer at the local side continues trying to
stream data from the remote repository until it gets EOF. Meanwhile,
send-pack (in function pack_objects()) has noticed that pack-objects did
not produce output and died. Back in send_pack(), it now tries to clean
up the sideband demuxer using finish_async(). The demuxer, however, waits
for the remote end to close down, the remote waits for pack data, and
the reason that it still waits is that send-pack forgot to close the
outgoing channel. Add the missing close() in pack_objects().
The second deadlock happens in a similar constellation when the sideband
demuxer runs in a forked process (rather than in a thread). Again, the
remote end waits for pack data to arrive, the sideband demuxer waits for
the remote to shut down, and send-pack (in the regular clean-up) waits for
the demuxer to terminate. This time, the send-pack parent process closes
the writable end of the outgoing channel (in start_command() that spawned
pack-objects) so that after the death of the pack-objects process all
writable ends should have been closed and the remote repo should see EOF.
This does not happen, however, because when the sideband demuxer was forked
earlier, it also inherited a writable end; it remains open and keeps the
remote repo from seeing EOF. To break this deadlock, close the writable end
in the demuxer.
Analyzed-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Async procedures are intended as helpers that perform a very restricted
task, and the caller usually has to manage them in a larger context.
Conceptually, the async procedure is not concerned with the "bigger
picture" in whose context it is run. When it dies, it is not supposed
to destroy this "bigger picture", but rather only its own limit view
of the world. On POSIX, the async procedure is run in its own process,
and exiting this process naturally had only these limited effects.
On Windows (or when ASYNC_AS_THREAD is set), calling die() exited the
whole process, destroying the caller (the "big picture") as well.
This fixes it to exit only the thread.
Without ASYNC_AS_THREAD, one particular effect of exiting the async
procedure process is that it automatically closes file descriptors, most
notably the writable end of the pipe that the async procedure writes to.
The async API already requires that the async procedure closes the pipe
ends when it exits normally. But for calls to die() no requirements are
imposed. In the non-threaded case the pipe ends are closed implicitly
by the exiting process, but in the threaded case, the die routine must
take care of closing them.
Now t5530-upload-pack-error.sh passes on Windows.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Windows, async procedures have always been run in threads, and the
implementation used Windows specific APIs. Rewrite the code to use pthreads.
A new configuration option is introduced so that the threaded implementation
can also be used on POSIX systems. Since this option is intended only as
playground on POSIX, but is mandatory on Windows, the option is not
documented.
One detail is that on POSIX it is necessary to set FD_CLOEXEC on the pipe
handles. On Windows, this is not needed because pipe handles are not
inherited to child processes, and the new calls to set_cloexec() are
effectively no-ops.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This adds:
pthread_self
pthread_equal
pthread_exit
pthread_key_create
pthread_setspecific
pthread_getspecific
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Obviously, this function was never called with two arguments in Windows
code sections, but this will be the case in a subsequent patch.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There exist already a number of static functions named 'report', therefore,
the function name was changed.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/maint-fix-pager:
tests: Fix race condition in t7006-pager
t7006-pager: if stdout is not a terminal, make a new one
tests: Add tests for automatic use of pager
am: Fix launching of pager
git svn: Fix launching of pager
git.1: Clarify the behavior of the --paginate option
Make 'git var GIT_PAGER' always print the configured pager
Fix 'git var' usage synopsis
* np/compress-loose-object-memsave:
sha1_file: be paranoid when creating loose objects
sha1_file: don't malloc the whole compressed result when writing out objects
All tests in t9119 were disabled for subversion versions other than
1.[45].*. Make the test script run with subversion 1.[456].*.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Acked-by: Eric Wong <normalperson@yhbt.net>
The delayed loading of SVN missed a place where SVN::Core is used. Make
sure to load the package before trying to use it.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Acked-by: Eric Wong <normalperson@yhbt.net>
When "git submodule summary" is run without any argument, we default to
compare the state of index with the HEAD, but tried to shift out $1 that
does not exist (and worse yet, we didn't use it).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* sp/maint-push-sideband:
receive-pack: Send internal errors over side-band #2
t5401: Use a bare repository for the remote peer
receive-pack: Send hook output over side band #2
receive-pack: Wrap status reports inside side-band-64k
receive-pack: Refactor how capabilities are shown to the client
send-pack: demultiplex a sideband stream with status data
run-command: support custom fd-set in async
run-command: Allow stderr to be a caller supplied pipe
Conflicts:
builtin-receive-pack.c
run-command.c
t/t5401-update-hooks.sh
* np/fast-import-idx-v2:
fast-import: use the diff_delta() max_delta_size argument
fast-import: honor pack.indexversion and pack.packsizelimit config vars
fast-import: make default pack size unlimited
fast-import: use write_idx_file() instead of custom code
fast-import: use sha1write() for pack data
fast-import: start using struct pack_idx_entry
* jn/maint-fix-pager:
tests: Fix race condition in t7006-pager
t7006-pager: if stdout is not a terminal, make a new one
tests: Add tests for automatic use of pager
am: Fix launching of pager
git svn: Fix launching of pager
git.1: Clarify the behavior of the --paginate option
Make 'git var GIT_PAGER' always print the configured pager
Fix 'git var' usage synopsis
* jc/for-each-ref:
for-each-ref --format='%(flag)'
for-each-ref --format='%(symref) %(symref:short)'
builtin-for-each-ref.c: check if we need to peel onion while parsing the format
builtin-for-each-ref.c: comment fixes
* np/compress-loose-object-memsave:
sha1_file: be paranoid when creating loose objects
sha1_file: don't malloc the whole compressed result when writing out objects
