Async processes can be implemented as separate forked
processes, or as threads (depending on the NO_PTHREADS
setting). In the latter case, if an async thread gets
SIGPIPE, it takes down the whole process. This is obviously
bad if the main process was not otherwise going to die, but
even if we were going to die, it means the main process does
not have a chance to report a useful error message.
There's also the small matter that forked async processes
will not take the main process down on a signal, meaning git
will behave differently depending on the NO_PTHREADS
setting.
This patch fixes it by adding a new flag to "struct async"
to block SIGPIPE just in the async thread. In theory, this
should always be on (which makes async threads behave more
like async processes), but we would first want to make sure
that each async process we spawn is careful about checking
return codes from write() and would not spew endlessly into
a dead pipe. So let's start with it as optional, and we can
enable it for specific sites in future patches.
The natural name for this option would be "ignore_sigpipe",
since that's what it does for the threaded case. But since
that name might imply that we are ignoring it in all cases
(including the separate-process one), let's call it
"isolate_sigpipe". What we are really asking for is
isolation. I.e., not to have our main process taken down by
signals spawned by the async process. How that is
implemented is up to the run-command code.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This fixes a deadlock on the client side when pushing a
large number of refs from a corrupted repo. There's a
reproduction script below, but let's start with a
human-readable explanation.
The client side of a push goes something like this:
1. Start an async process to demux sideband coming from
the server.
2. Run pack-objects to send the actual pack, and wait for
its status via finish_command().
3. If pack-objects failed, abort immediately.
4. If pack-objects succeeded, read the per-ref status from
the server, which is actually coming over a pipe from
the demux process started in step 1.
We run finish_async() to wait for and clean up the demux
process in two places. In step 3, if we see an error, we
want it to end early. And after step 4, it should be done
writing any data and we are just cleaning it up.
Let's focus on the error case first. We hand the output
descriptor to the server over to pack-objects. So by the
time it has returned an error to us, it has closed the
descriptor and the server has gotten EOF. The server will
mark all refs as failed with "unpacker error" and send us
back the status for each (followed by EOF).
This status goes to the demuxer thread, which relays it over
a pipe to the main thread. But the main thread never even
tries reading the status. It's trying to bail because of the
pack-objects error, and is waiting for the demuxer thread to
finish. If there are a small number of refs, that's OK; the
demuxer thread writes into the pipe buffer, sees EOF from
the server, and quits. But if there are a large number of
refs, it may block on write() back to the main thread,
leading to a deadlock (the main thread is waiting for the
demuxer to finish, the demuxer is waiting for the main
thread to read).
We can break this deadlock by closing the pipe between the
demuxer and the main thread before calling finish_async().
Then the demuxer gets a write() error and exits.
The non-error case usually just works, because we will have
read all of the data from the other side. We do close
demux.out already, but we only do so _after_ calling
finish_async(). This is OK because there shouldn't be any
more data coming from the server. But technically we've only
read to a flush packet, and a broken or malicious server
could be sending more cruft. In such a case, we would hit
the same deadlock. Closing the pipe first doesn't affect the
normal case, and means that for a cruft-sending server,
we'll notice a write() error rather than deadlocking.
Note that when write() sees this error, we'll actually
deliver SIGPIPE to the thread, which will take down the
whole process (unless we're compiled with NO_PTHREADS). This
isn't ideal, but it's an improvement over the status quo,
which is deadlocking. And SIGPIPE handling in async threads
is a bigger problem that we can deal with separately.
A simple reproduction for the error case is below. It's
technically racy (we could exit the main process and take
down the async thread with us before it even reads the
status), though in practice it seems to fail pretty
consistently.
git init repo &&
cd repo &&
# make some commits; we need two so we can simulate corruption
# in the history later.
git commit --allow-empty -m one &&
one=$(git rev-parse HEAD) &&
git commit --allow-empty -m two &&
two=$(git rev-parse HEAD) &&
# now make a ton of refs; our goal here is to overflow the pipe buffer
# when reporting the ref status, which will cause the demuxer to block
# on write()
for i in $(seq 20000); do
echo "create refs/heads/this-is-a-really-long-branch-name-$i $two"
done |
git update-ref --stdin &&
# now make a corruption in the history such that pack-objects will fail
rm -vf .git/objects/$(echo $one | sed 's}..}&/}') &&
# and then push the result
git init --bare dst.git &&
git push --mirror dst.git
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We simply need to read the config, is all.
This fixes https://github.com/git-for-windows/git/issues/733
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When directories are moved using `git mv` all files in the directory
have been just moved, but no further action was taken on them. This
was done by assigning the mode = WORKING_DIRECTORY to the files
inside a moved directory.
submodules however need to update their link to the git directory as
well as updates to the .gitmodules file. By removing the condition of
`mode != INDEX` (the remaining modes are BOTH and WORKING_DIRECTORY) for
the required submodule actions, we perform these for submodules in a
moved directory.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git merge FETCH_HEAD" dereferenced NULL pointer when merging
nothing into an unborn history (which is arguably unusual usage,
which perhaps was the reason why nobody noticed it).
* jv/merge-nothing-into-void:
merge: fix NULL pointer dereference when merging nothing into void
When "git merge --squash" stopped due to conflict, the concluding
"git commit" failed to read in the SQUASH_MSG that shows the log
messages from all the squashed commits.
* ss/commit-squash-msg:
commit: do not lose SQUASH_MSG contents
"git send-email" had trouble parsing alias file in mailrc format
when lines in it had trailing whitespaces on them.
* jk/send-email-rtrim-mailrc-alias:
send-email: ignore trailing whitespace in mailrc alias file
"git mergetool" did not work well with conflicts that both sides
deleted.
* da/mergetool-delete-delete-conflict:
mergetool: honor tempfile configuration when resolving delete conflicts
mergetool: support delete/delete conflicts
The startup_info data, which records if we are working inside a
repository (among other things), are now uniformly available to Git
subcommand implementations, and Git avoids attempting to touch
references when we are not in a repository.
* jk/startup-info:
use setup_git_directory() in test-* programs
grep: turn off gitlink detection for --no-index
mailmap: do not resolve blobs in a non-repository
remote: don't resolve HEAD in non-repository
setup: set startup_info->have_repository more reliably
setup: make startup_info available everywhere
strbuf_getwholeline() did not NUL-terminate the buffer on certain
corner cases in its error codepath.
* jk/getwholeline-getdelim-empty:
strbuf_getwholeline: NUL-terminate getdelim buffer on error
A small memory leak in an error codepath has been plugged in xdiff
code.
* rj/xdiff-prepare-plug-leak-on-error-codepath:
xdiff/xprepare: fix a memory leak
xdiff/xprepare: use the XDF_DIFF_ALG() macro to access flag bits
Fetching of history by naming a commit object name directly didn't
work across remote-curl transport.
* gf/fetch-pack-direct-object-fetch:
fetch-pack: update the documentation for "<refs>..." arguments
fetch-pack: fix object_id of exact sha1
The "--local-env-vars" and "--resolve-git-dir" options of "git
rev-parse" failed to work outside a repository when the command's
option parsing was rewritten in 1.8.5 era.
* jk/rev-parse-local-env-vars:
rev-parse: let some options run outside repository
t1515: add tests for rev-parse out-of-repo helpers
"git config --get-urlmatch", unlike other variants of the "git
config --get" family, did not signal error with its exit status
when there was no matching configuration.
* jk/config-get-urlmatch:
Documentation/git-config: fix --get-all description
Documentation/git-config: use bulleted list for exit codes
config: fail if --get-urlmatch finds no value
The tests that involve running httpd leaked the system-wide
configuration in /etc/gitconfig to the tested environment.
* jk/test-httpd-config-nosystem:
t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM env
The embedded args argv-array in the child process is used to build
the command line to run pack-objects instead of using a separate
array of strings.
* mp/upload-pack-use-embedded-args:
upload-pack: use argv_array for pack_objects
Remove extra + 1 from resp_len, the length of the byte sequence to be
Base64 encoded and passed to the server as the response. Or the response
incorrectly contains an extra \0.
Signed-off-by: Kazuki Yamaguchi <k@rhe.jp>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Don't check for NOLOGIN (LOGINDISABLED) capability when imap.authMethod
is specified.
LOGINDISABLED capability doesn't forbid using AUTHENTICATE, so it should
be allowed, or we can't connect to IMAP servers which only accepts
AUTHENTICATE command.
Signed-off-by: Kazuki Yamaguchi <k@rhe.jp>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test uses hardcoded #!/bin/sh to create a pre-commit hook
script. Because the generated script uses $(command substitution),
which is not supported by /bin/sh on some platforms (e.g. Solaris),
the resulting pre-commit always fails.
Which is not noticeable as the test that uses the hook is about
checking the behaviour of the command when the hook fails ;-), but
nevertheless it is not testing what we wanted to test.
Use write_script so that the resulting script is run under the same
shell our scripted Porcelain commands are run, which must support
the necessary $(construct).
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test prepares a sample file "dir/two" with a single incomplete
line in it with "printf", and also prepares a small helper script
"diff" to create a file with a single incomplete line in it, again
with "printf". The output from the latter is compared with an
expected output, again prepared with "printf" hence lacking the
final LF. There is no reason for this test to be using files with
an incomplete line at the end, and these look more like a mistake
of not using
printf "%s\n" "string to be written"
and using
printf "string to be written"
Depending on what would be in $GIT_PREFIX, using the latter form
could be a bug waiting to happen. Correct them.
Also, the test uses hardcoded #!/bin/sh to create a small helper
script. For a small task like what the generated script does, it
does not matter too much in that what appears as /bin/sh would not
be _so_ broken, but while we are at it, use write_script instead,
which happens to make the result easier to read by reducing need
of one level of quoting.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The recent cleanup in b7cbbff switched t5532's use of
backticks to $(). This matches our normal shell style, which
is good. But it also breaks the test on Solaris, where
/bin/sh does not understand $().
Our normal shell style assumes a modern-ish shell which
knows about $(). However, some tests create small helper
scripts and just write "#!/bin/sh" into them. These scripts
either need to go back to using backticks, or they need to
respect $SHELL_PATH. The easiest way to do the latter is to
use write_script.
While we're at it, let's also stick the script creation
inside a test_expect block (our usual style), and split the
perl snippet into its own script (to prevent quoting
madness).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We pass off to the "_gently" form to do the real work, and
just die() if it returned an error. However, our die message
de-references "value", which may be NULL if the request was
to unset a variable. Nobody using glibc noticed, because it
simply prints "(null)", which is good enough for the test
suite (and presumably very few people run across this in
practice). But other libc implementations (like Solaris) may
segfault.
Let's not only fix that, but let's make the message more
clear about what is going on in the "unset" case.
Reported-by: "Tom G. Christensen" <tgc@jupiterrise.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This function is just a thin wrapper for the "_gently" form
of the function. But the gently form is designed to feed
builtin/config.c, which passes our return code directly to
its exit status, and thus uses positive error values for
some cases. We check only negative values, meaning we would
fail to die in some cases (e.g., a malformed key).
This may or may not be triggerable in practice; we tend to
use this non-gentle form only when setting internal
variables, which would not have malformed keys.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This follows our usual style (both throughout git, and
throughout the rest of this file).
This covers the whole file, but note that I left the capitalization in
the multi-sentence:
error: malformed value...
error: Must be one of ...
because it helps make it clear that we are starting a new sentence in
the second one.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Felix Ruess <felix.ruess@gmail.com> noticed that with configuration
$ git config --global 'http.proxy=socks5h://127.0.0.1:1080'
connections to remote sites time out, waiting for DNS resolution.
The logic to detect various flavours of SOCKS proxy and ask the
libcurl layer to use appropriate one understands the proxy string
that begin with socks5, socks4a, etc., but does not know socks5h,
and we end up using CURLPROXY_SOCKS5. The correct one to use is
CURLPROXY_SOCKS5_HOSTNAME.
https://curl.haxx.se/libcurl/c/CURLOPT_PROXY.html says
..., socks5h:// (the last one to enable socks5 and asking the
proxy to do the resolving, also known as CURLPROXY_SOCKS5_HOSTNAME
type).
which is consistent with the way the breakage was reported.
Tested-by: Felix Ruess <felix.ruess@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We don't need it, as we no longer use HMAC_CTX_cleanup() directly.
Signed-off-by: Kazuki Yamaguchi <k@rhe.jp>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use SSLv23_method always and disable SSL if needed.
TLSv1_method() function is deprecated in OpenSSL 1.1.0 and the compiler
emits a warning.
SSLv23_method() is also deprecated, but the alternative, TLS_method(),
is new in OpenSSL 1.1.0 so requires checking by configure. Stick to
SSLv23_method() for now (this is aliased to TLS_method()).
Signed-off-by: Kazuki Yamaguchi <k@rhe.jp>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix compile errors with OpenSSL 1.1.0.
HMAC_CTX is made opaque and HMAC_CTX_cleanup is removed in OpenSSL
1.1.0. But since we just want to calculate one HMAC, we can use HMAC()
here, which exists since OpenSSL 0.9.6 at least.
Signed-off-by: Kazuki Yamaguchi <k@rhe.jp>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Emit an informative error when failed to hold lock of HEAD.
2233066e (refs: add a new function set_worktree_head_symref,
2016-03-27) added set_worktree_head_symref(), but this is missing a
call to unable_to_lock_message() after hold_lock_file_for_update()
fails, so it emits an empty error message:
% git branch -m oldname newname
error:
error: HEAD of working tree /path/to/wt is not updated
fatal: Branch renamed to newname, but HEAD is not updated!
Thanks to Eric Sunshine for pointing this out.
Signed-off-by: Kazuki Yamaguchi <k@rhe.jp>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When f9568530 (builtin-commit: resurrect behavior for multiple -m
options, 2007-11-11) converted a "char *message" to "struct strbuf
message" to hold the messages given with the "-m" option, it
incorrectly changed the checks "did we get a message with the -m
option?" to "is message.len 0?". Later, we noticed one breakage
from this change and corrected it with 25206778 (commit: don't start
editor if empty message is given with -m, 2013-05-25).
However, "we got a message with -m, even though an empty one, so we
shouldn't be launching an editor" was not the only breakage.
* "git commit --amend -m '' --allow-empty", even though it looks
strange, is a valid request to amend the commit to have no
message at all. Due to the misdetection of the presence of -m on
the command line, we ended up keeping the log messsage from the
original commit.
* "git commit -m "$msg" -F file" should be rejected whether $msg is
an empty string or not, but due to the same bug, was not rejected
when $msg is empty.
* "git -c template=file -m "$msg"" should ignore the template even
when $msg is empty, but it didn't and instead used the contents
from the template file.
Correct these by checking have_option_m, which the earlier 25206778
introduced to fix the same bug.
Reported-by: Adam Dinwoodie <adam@dinwoodie.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`git commit --amend -m ''` seems to be an unambiguous request to blank a
commit message, but it actually leaves the commit message as-is. That's
the case regardless of whether `--allow-empty-message` is specified, and
doesn't so much as drop a non-zero return code.
Add failing tests to show this behaviour.
Signed-off-by: Adam Dinwoodie <adam@dinwoodie.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A recent update 75faa45a (replace trivial malloc + sprintf / strcpy
calls with xstrfmt, 2015-09-24) rewrote
prepare an empty buffer
if (len)
append the first len bytes of "prefix" to the buffer
append "path" to the buffer
that computed "path", optionally prefixed by "prefix", into
xstrfmt("%.*s%s", len, prefix, path);
However, passing a NULL pointer to the printf(3) family of functions
to format it with %s conversion, even with the precision set to 0,
i.e.
xstrfmt("%.*s", 0, NULL)
yields undefined results, at least on some platforms.
Avoid this problem by substituting prefix with "" when len==0, as
prefix can legally be NULL in that case. This would mimick the
intent of the original code better.
Reported-by: Tom G. Christensen <tgc@jupiterrise.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We never used Data::Dumper in this script. The only reference
of it was always commented out and removed over a decade ago in
commit 4bc87a28be
("send-email: Change from Mail::Sendmail to Net::SMTP")
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using a YYYYmmddHHMMSS date representation is more meaningful to
humans, especially when used for lookups on NNTP servers or linking
to archive sites via Message-ID (e.g. mid.gmane.org or
mid.mail-archive.com). This timestamp format more easily gives a
reader of the URL itself a rough date of a linked message compared
to having them calculate the seconds since the Unix epoch.
Furthermore, having the MUA name in the Message-ID seems to be a
rare oddity I haven't noticed outside of git-send-email. We
already have an optional X-Mailer header field to advertise for
us, so extending the Message-ID by 15 characters can make for
unpleasant Message-ID-based URLs to archive sites.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git blame reports lines as not "Not Committed Yet" when they have
CRLF in the index, CRLF in the worktree and core.autocrlf is true.
Since commit c4805393 (autocrlf: Make it work also for un-normalized
repositories, 2010-05-12), files that have CRLF in the index are not
normalized at commit when core.autocrl is set.
Add a call to read_cache() early in fake_working_tree_commit(),
before calling convert_to_git().
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The correct api is trace_printf_key(), not trace_print_key().
Also do not throw a random string at printf(3)-like function;
instead, feed it as a parameter that is fed to a "%s" conversion
specifier.
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In general "echo 2>&1 $msg" to redirect a possible error message
that comes from 'echo' itself into the same standard output stream
$msg is getting written to does not make any sense; it is not like
we are expecting to see any errors out of 'echo' in these statements,
and even if it were the case, there is no reason to prevent the
error messages from being sent to the standard error stream.
These are clearly meant to send the argument given to echo to the
standard error stream as error messages. Correctly redirect by
saying "send what is written to the standard output to the standard
error", i.e. "1>&2" aka ">&2".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
