Commit Graph

10938 Commits

Author SHA1 Message Date
Jeff King
cb4d2d35c4 http: treat http-alternates like redirects
The previous commit made HTTP redirects more obvious and
tightened up the default behavior. However, there's another
way for a server to ask a git client to fetch arbitrary
content: by having an http-alternates file (or a regular
alternates file, which is used as a backup).

Similar to the HTTP redirect case, a malicious server can
claim to have refs pointing at object X, return a 404 when
the client asks for X, but point to some other URL via
http-alternates, which the client will transparently fetch.
The end result is that it looks from the user's perspective
like the objects came from the malicious server, as the
other URL is not mentioned at all.

Worse, because we feed the new URL to curl ourselves, the
usual protocol restrictions do not kick in (neither curl's
default of disallowing file://, nor the protocol
whitelisting in f4113cac0 (http: limit redirection to
protocol-whitelist, 2015-09-22).

Let's apply the same rules here as we do for HTTP redirects.
Namely:

  - unless http.followRedirects is set to "always", we will
    not follow remote redirects from http-alternates (or
    alternates) at all

  - set CURLOPT_PROTOCOLS alongside CURLOPT_REDIR_PROTOCOLS
    restrict ourselves to a known-safe set and respect any
    user-provided whitelist.

  - mention alternate object stores on stderr so that the
    user is aware another source of objects may be involved

The first item may prove to be too restrictive. The most
common use of alternates is to point to another path on the
same server. While it's possible for a single-server
redirect to be an attack, it takes a fairly obscure setup
(victim and evil repository on the same host, host speaks
dumb http, and evil repository has access to edit its own
http-alternates file).

So we could make the checks more specific, and only cover
cross-server redirects. But that means parsing the URLs
ourselves, rather than letting curl handle them. This patch
goes for the simpler approach. Given that they are only used
with dumb http, http-alternates are probably pretty rare.
And there's an escape hatch: the user can allow redirects on
a specific server by setting http.<url>.followRedirects to
"always".

Reported-by: Jann Horn <jannh@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06 12:32:48 -08:00
Jeff King
50d3413740 http: make redirects more obvious
We instruct curl to always follow HTTP redirects. This is
convenient, but it creates opportunities for malicious
servers to create confusing situations. For instance,
imagine Alice is a git user with access to a private
repository on Bob's server. Mallory runs her own server and
wants to access objects from Bob's repository.

Mallory may try a few tricks that involve asking Alice to
clone from her, build on top, and then push the result:

  1. Mallory may simply redirect all fetch requests to Bob's
     server. Git will transparently follow those redirects
     and fetch Bob's history, which Alice may believe she
     got from Mallory. The subsequent push seems like it is
     just feeding Mallory back her own objects, but is
     actually leaking Bob's objects. There is nothing in
     git's output to indicate that Bob's repository was
     involved at all.

     The downside (for Mallory) of this attack is that Alice
     will have received Bob's entire repository, and is
     likely to notice that when building on top of it.

  2. If Mallory happens to know the sha1 of some object X in
     Bob's repository, she can instead build her own history
     that references that object. She then runs a dumb http
     server, and Alice's client will fetch each object
     individually. When it asks for X, Mallory redirects her
     to Bob's server. The end result is that Alice obtains
     objects from Bob, but they may be buried deep in
     history. Alice is less likely to notice.

Both of these attacks are fairly hard to pull off. There's a
social component in getting Mallory to convince Alice to
work with her. Alice may be prompted for credentials in
accessing Bob's repository (but not always, if she is using
a credential helper that caches). Attack (1) requires a
certain amount of obliviousness on Alice's part while making
a new commit. Attack (2) requires that Mallory knows a sha1
in Bob's repository, that Bob's server supports dumb http,
and that the object in question is loose on Bob's server.

But we can probably make things a bit more obvious without
any loss of functionality. This patch does two things to
that end.

First, when we encounter a whole-repo redirect during the
initial ref discovery, we now inform the user on stderr,
making attack (1) much more obvious.

Second, the decision to follow redirects is now
configurable. The truly paranoid can set the new
http.followRedirects to false to avoid any redirection
entirely. But for a more practical default, we will disallow
redirects only after the initial ref discovery. This is
enough to thwart attacks similar to (2), while still
allowing the common use of redirects at the repository
level. Since c93c92f30 (http: update base URLs when we see
redirects, 2013-09-28) we re-root all further requests from
the redirect destination, which should generally mean that
no further redirection is necessary.

As an escape hatch, in case there really is a server that
needs to redirect individual requests, the user can set
http.followRedirects to "true" (and this can be done on a
per-server basis via http.*.followRedirects config).

Reported-by: Jann Horn <jannh@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06 12:32:48 -08:00
Jeff King
6628eb41db http: always update the base URL for redirects
If a malicious server redirects the initial ref
advertisement, it may be able to leak sha1s from other,
unrelated servers that the client has access to. For
example, imagine that Alice is a git user, she has access to
a private repository on a server hosted by Bob, and Mallory
runs a malicious server and wants to find out about Bob's
private repository.

Mallory asks Alice to clone an unrelated repository from her
over HTTP. When Alice's client contacts Mallory's server for
the initial ref advertisement, the server issues an HTTP
redirect for Bob's server. Alice contacts Bob's server and
gets the ref advertisement for the private repository. If
there is anything to fetch, she then follows up by asking
the server for one or more sha1 objects. But who is the
server?

If it is still Mallory's server, then Alice will leak the
existence of those sha1s to her.

Since commit c93c92f30 (http: update base URLs when we see
redirects, 2013-09-28), the client usually rewrites the base
URL such that all further requests will go to Bob's server.
But this is done by textually matching the URL. If we were
originally looking for "http://mallory/repo.git/info/refs",
and we got pointed at "http://bob/other.git/info/refs", then
we know that the right root is "http://bob/other.git".

If the redirect appears to change more than just the root,
we punt and continue to use the original server. E.g.,
imagine the redirect adds a URL component that Bob's server
will ignore, like "http://bob/other.git/info/refs?dummy=1".

We can solve this by aborting in this case rather than
silently continuing to use Mallory's server. In addition to
protecting from sha1 leakage, it's arguably safer and more
sane to refuse a confusing redirect like that in general.
For example, part of the motivation in c93c92f30 is
avoiding accidentally sending credentials over clear http,
just to get a response that says "try again over https". So
even in a non-malicious case, we'd prefer to err on the side
of caution.

The downside is that it's possible this will break a
legitimate but complicated server-side redirection scheme.
The setup given in the newly added test does work, but it's
convoluted enough that we don't need to care about it. A
more plausible case would be a server which redirects a
request for "info/refs?service=git-upload-pack" to just
"info/refs" (because it does not do smart HTTP, and for some
reason really dislikes query parameters).  Right now we
would transparently downgrade to dumb-http, but with this
patch, we'd complain (and the user would have to set
GIT_SMART_HTTP=0 to fetch).

Reported-by: Jann Horn <jannh@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06 12:32:48 -08:00
Junio C Hamano
b5abd302ef Merge branch 'sg/reflog-past-root' into maint
A small test clean-up for a topic introduced in v2.9.1 and later.

* sg/reflog-past-root:
  t1410: remove superfluous 'git reflog' from the 'walk past root' test
2016-09-08 21:36:02 -07:00
Junio C Hamano
c0e8b3b444 Merge branch 'bw/mingw-avoid-inheriting-fd-to-lockfile' into maint
The tempfile (hence its user lockfile) API lets the caller to open
a file descriptor to a temporary file, write into it and then
finalize it by first closing the filehandle and then either
removing or renaming the temporary file.  When the process spawns a
subprocess after obtaining the file descriptor, and if the
subprocess has not exited when the attempt to remove or rename is
made, the last step fails on Windows, because the subprocess has
the file descriptor still open.  Open tempfile with O_CLOEXEC flag
to avoid this (on Windows, this is mapped to O_NOINHERIT).

* bw/mingw-avoid-inheriting-fd-to-lockfile:
  mingw: ensure temporary file handles are not inherited by child processes
  t6026-merge-attr: child processes must not inherit index.lock handles
2016-09-08 21:35:56 -07:00
Junio C Hamano
bde42f081e Merge branch 'jk/difftool-command-not-found' into maint
"git difftool" by default ignores the error exit from the backend
commands it spawns, because often they signal that they found
differences by exiting with a non-zero status code just like "diff"
does; the exit status codes 126 and above however are special in
that they are used to signal that the command is not executable,
does not exist, or killed by a signal.  "git difftool" has been
taught to notice these exit status codes.

* jk/difftool-command-not-found:
  difftool: always honor fatal error exit codes
2016-09-08 21:35:54 -07:00
Junio C Hamano
7c96471947 Merge branch 'sb/checkout-explit-detach-no-advice' into maint
"git checkout --detach <branch>" used to give the same advice
message as that is issued when "git checkout <tag>" (or anything
that is not a branch name) is given, but asking with "--detach" is
an explicit enough sign that the user knows what is going on.  The
advice message has been squelched in this case.

* sb/checkout-explit-detach-no-advice:
  checkout: do not mention detach advice for explicit --detach option
2016-09-08 21:35:54 -07:00
Junio C Hamano
69307312d1 Merge branch 'rs/pull-signed-tag' into maint
When "git merge-recursive" works on history with many criss-cross
merges in "verbose" mode, the names the command assigns to the
virtual merge bases could have overwritten each other by unintended
reuse of the same piece of memory.

* rs/pull-signed-tag:
  commit: use FLEX_ARRAY in struct merge_remote_desc
  merge-recursive: fix verbose output for multiple base trees
  commit: factor out set_merge_remote_desc()
  commit: use xstrdup() in get_merge_parent()
2016-09-08 21:35:54 -07:00
Junio C Hamano
86df11b1a4 Merge branch 'js/test-lint-pathname' into maint
The "t/" hierarchy is prone to get an unusual pathname; "make test"
has been taught to make sure they do not contain paths that cannot
be checked out on Windows (and the mechanism can be reusable to
catch pathnames that are not portable to other platforms as need
arises).

* js/test-lint-pathname:
  t/Makefile: ensure that paths are valid on platforms we care
2016-09-08 21:35:54 -07:00
Junio C Hamano
f34d900aa7 Merge branch 'jk/push-force-with-lease-creation' into maint
"git push --force-with-lease" already had enough logic to allow
ensuring that such a push results in creation of a ref (i.e. the
receiving end did not have another push from sideways that would be
discarded by our force-pushing), but didn't expose this possibility
to the users.  It does so now.

* jk/push-force-with-lease-creation:
  t5533: make it pass on case-sensitive filesystems
  push: allow pushing new branches with --force-with-lease
  push: add shorthand for --force-with-lease branch creation
  Documentation/git-push: fix placeholder formatting
2016-09-08 21:35:53 -07:00
Junio C Hamano
f59c6e6ccb Merge branch 'jk/reflog-date' into maint
The reflog output format is documented better, and a new format
--date=unix to report the seconds-since-epoch (without timezone)
has been added.

* jk/reflog-date:
  date: clarify --date=raw description
  date: add "unix" format
  date: document and test "raw-local" mode
  doc/pretty-formats: explain shortening of %gd
  doc/pretty-formats: describe index/time formats for %gd
  doc/rev-list-options: explain "-g" output formats
  doc/rev-list-options: clarify "commit@{Nth}" for "-g" option
2016-09-08 21:35:52 -07:00
Junio C Hamano
7f5885ad2a Merge branch 'jc/renormalize-merge-kill-safer-crlf' into maint
"git merge" with renormalization did not work well with
merge-recursive, due to "safer crlf" conversion kicking in when it
shouldn't.

* jc/renormalize-merge-kill-safer-crlf:
  merge: avoid "safer crlf" during recording of merge results
  convert: unify the "auto" handling of CRLF
2016-09-08 21:35:52 -07:00
Junio C Hamano
faacc8efe5 Merge branch 'jk/common-main' into maint
There are certain house-keeping tasks that need to be performed at
the very beginning of any Git program, and programs that are not
built-in commands had to do them exactly the same way as "git"
potty does.  It was easy to make mistakes in one-off standalone
programs (like test helpers).  A common "main()" function that
calls cmd_main() of individual program has been introduced to
make it harder to make mistakes.

* jk/common-main:
  mingw: declare main()'s argv as const
  common-main: call git_setup_gettext()
  common-main: call restore_sigpipe_to_default()
  common-main: call sanitize_stdfds()
  common-main: call git_extract_argv0_path()
  add an extra level of indirection to main()
2016-09-08 21:35:51 -07:00
Ben Wijen
05d1ed6148 mingw: ensure temporary file handles are not inherited by child processes
When the index is locked and child processes inherit the handle to
said lock and the parent process wants to remove the lock before the
child process exits, on Windows there is a problem: it won't work
because files cannot be deleted if a process holds a handle on them.
The symptom:

    Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed.
    Should I try again? (y/n)

Spawning child processes with bInheritHandles==FALSE would not work
because no file handles would be inherited, not even the hStdXxx
handles in STARTUPINFO (stdin/stdout/stderr).

Opening every file with O_NOINHERIT does not work, either, as e.g.
git-upload-pack expects inherited file handles.

This leaves us with the only way out: creating temp files with the
O_NOINHERIT flag. This flag is Windows-specific, however. For our
purposes, it is equivalent to O_CLOEXEC (which does not exist on
Windows), so let's just open temporary files with the O_CLOEXEC flag and
map that flag to O_NOINHERIT on Windows.

As Eric Wong pointed out, we need to be careful to handle the case where
the Linux headers used to compile Git support O_CLOEXEC but the Linux
kernel used to run Git does not: it returns an EINVAL.

This fixes the test that we just introduced to demonstrate the problem.

Signed-off-by: Ben Wijen <ben@wijen.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-23 09:09:55 -07:00
Ben Wijen
ad65f7e3b7 t6026-merge-attr: child processes must not inherit index.lock handles
On Windows, a file cannot be removed unless all file handles to it have
been released. Hence it is particularly important to close handles when
spawning children (which would probably not even know that they hold on
to those handles).

The example chosen for this test is a custom merge driver that indeed
has no idea that it blocks the deletion of index.lock. The full use case
is a daemon that lives on after the merge, with subsequent invocations
handing off to the daemon, thereby avoiding hefty start-up costs. We
simulate this behavior by simply sleeping one second.

Note that the test only fails on Windows, due to the file locking issue.
Since we have no way to say "expect failure with MINGW, success
otherwise", we simply skip this test on Windows for now.

Signed-off-by: Ben Wijen <ben@wijen.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-18 13:56:45 -07:00
Johannes Schindelin
c2cafd39bc t/Makefile: ensure that paths are valid on platforms we care
Some pathnames that are okay on ext4 and on HFS+ cannot be checked
out on Windows. Tests that want to see operations on such paths on
filesystems that support them must do so behind appropriate test
prerequisites, and must not include them in the source tree (instead
they should create them when they run). Otherwise, the source tree
cannot even be checked out.

Make sure that double-quotes, asterisk, colon, greater/less-than,
question-mark, backslash, tab, vertical-bar, as well as any non-ASCII
characters never appear in the pathnames with a new test-lint-* target
as part of a `make test`. To that end, we call `git ls-files` (ensuring
that the paths are quoted properly), relying on the fact that paths
containing non-ASCII characters are quoted within double-quotes.

In case that the source code does not actually live in a Git
repository (e.g. when extracted from a .zip file), or that the `git`
executable cannot be executed, we simply ignore the error for now; In
that case, our trusty Continuous Integration will be the last line of
defense and catch any problematic file name.

Noticed when a topic wanted to add a pathname with '>' in it.  A
check like this will prevent a similar problems from happening in the
future.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-16 11:56:42 -07:00
John Keeping
45a4f5d9f9 difftool: always honor fatal error exit codes
At the moment difftool's "trust exit code" logic always suppresses the
exit status of the diff utility we invoke.  This is useful because we
don't want to exit just because diff returned "1" because the files
differ, but it's confusing if the shell returns an error because the
selected diff utility is not found.

POSIX specifies 127 as the exit status for "command not found", 126 for
"command found but is not executable" and values greater than 128 if the
command terminated because it received a signal [1] and at least bash
and dash follow this specification, while diff utilities generally use
"1" for the exit status we want to ignore.

Handle any value of 126 or greater as a special value indicating that
some form of fatal error occurred.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_08_02

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-15 15:24:05 -07:00
Stefan Beller
779b88a91f checkout: do not mention detach advice for explicit --detach option
When a user asked for a detached HEAD specifically with `--detach`,
we do not need to give advice on what a detached HEAD state entails as
we can assume they know what they're getting into as they asked for it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-15 15:01:45 -07:00
SZEDER Gábor
0eb75ce827 t1410: remove superfluous 'git reflog' from the 'walk past root' test
The test added in 71abeb753f (reflog: continue walking the reflog
past root commits, 2016-06-03) contains an unnecessary 'git reflog'
execution, which was part of my debug/tracing instrumentation that I
somehow didn't manage to remove before submitting.

Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-15 09:21:39 -07:00
René Scharfe
a25716535b merge-recursive: fix verbose output for multiple base trees
One of the indirect callers of make_virtual_commit() passes the result of
oid_to_hex() as the name, i.e. a pointer to a static buffer.  Since the
function uses that string pointer directly in building a struct
merge_remote_desc, multiple entries can end up sharing the same name
inadvertently.

Fix that by calling set_merge_remote_desc(), which creates a copy of the
string, instead of building the struct by hand.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-13 19:48:04 -07:00
Junio C Hamano
9b601eafd1 Merge branch 'jk/difftool-in-subdir' into maint
"git difftool <paths>..." started in a subdirectory failed to
interpret the paths relative to that directory, which has been
fixed.

* jk/difftool-in-subdir:
  difftool: use Git::* functions instead of passing around state
  difftool: avoid $GIT_DIR and $GIT_WORK_TREE
  difftool: fix argument handling in subdirs
2016-08-12 09:16:57 -07:00
Junio C Hamano
574a31b5b7 Merge branch 'rs/use-strbuf-addstr' into maint
* rs/use-strbuf-addstr:
  use strbuf_addstr() instead of strbuf_addf() with "%s"
  use strbuf_addstr() for adding constant strings to a strbuf
2016-08-10 11:55:34 -07:00
Junio C Hamano
66d6511c53 Merge branch 'jk/t4205-cleanup' into maint
Test modernization.

* jk/t4205-cleanup:
  t4205: indent here documents
  t4205: drop top-level &&-chaining
2016-08-10 11:55:32 -07:00
Junio C Hamano
f7fb6e21b8 Merge branch 'nd/fbsd-lazy-mtime' into maint
FreeBSD can lie when asked mtime of a directory, which made the
untracked cache code to fall back to a slow-path, which in turn
caused tests in t7063 to fail because it wanted to verify the
behaviour of the fast-path.

* nd/fbsd-lazy-mtime:
  t7063: work around FreeBSD's lazy mtime update feature
2016-08-10 11:55:30 -07:00
Junio C Hamano
85b2ea29e8 Merge branch 'js/t4130-rename-without-ino' into maint
Windows port was failing some tests in t4130, due to the lack of
inum in the returned values by its lstat(2) emulation.

* js/t4130-rename-without-ino:
  t4130: work around Windows limitation
2016-08-10 11:55:30 -07:00
Junio C Hamano
7b163e9187 Merge branch 'jc/grep-commandline-vs-configuration' into maint
"git -c grep.patternType=extended log --basic-regexp" misbehaved
because the internal API to access the grep machinery was not
designed well.

* jc/grep-commandline-vs-configuration:
  grep: further simplify setting the pattern type
2016-08-10 11:55:29 -07:00
Junio C Hamano
cee6c5b47b Merge branch 'jk/diff-do-not-reuse-wtf-needs-cleaning' into maint
There is an optimization used in "git diff $treeA $treeB" to borrow
an already checked-out copy in the working tree when it is known to
be the same as the blob being compared, expecting that open/mmap of
such a file is faster than reading it from the object store, which
involves inflating and applying delta.  This however kicked in even
when the checked-out copy needs to go through the convert-to-git
conversion (including the clean filter), which defeats the whole
point of the optimization.  The optimization has been disabled when
the conversion is necessary.

* jk/diff-do-not-reuse-wtf-needs-cleaning:
  diff: do not reuse worktree files that need "clean" conversion
2016-08-10 11:55:28 -07:00
Junio C Hamano
61efc5c2d8 Merge branch 'mm/status-suggest-merge-abort' into maint
"git status" learned to suggest "merge --abort" during a conflicted
merge, just like it already suggests "rebase --abort" during a
conflicted rebase.

* mm/status-suggest-merge-abort:
  status: suggest 'git merge --abort' when appropriate
2016-08-10 11:55:19 -07:00
Junio C Hamano
f7b01d3eb7 Merge branch 'rs/submodule-config-code-cleanup' into maint
Code cleanup.

* rs/submodule-config-code-cleanup:
  submodule-config: fix test binary crashing when no arguments given
  submodule-config: combine early return code into one goto
  submodule-config: passing name reference for .gitmodule blobs
  submodule-config: use explicit empty string instead of strbuf in config_from()
2016-08-08 14:21:46 -07:00
Junio C Hamano
970994deb1 Merge branch 'nd/test-helpers' into maint
Build clean-up.

* nd/test-helpers:
  t/test-lib.sh: fix running tests with --valgrind
  Makefile: use VCSSVN_LIB to refer to svn library
  Makefile: drop extra dependencies for test helpers
2016-08-08 14:21:43 -07:00
Junio C Hamano
e69771c3af Merge branch 'ah/unpack-trees-advice-messages' into maint
Grammofix.

* ah/unpack-trees-advice-messages:
  unpack-trees: fix English grammar in do-this-before-that messages
2016-08-08 14:21:40 -07:00
Junio C Hamano
aa9136a87e Merge branch 'nd/pack-ofs-4gb-limit' into maint
"git pack-objects" and "git index-pack" mostly operate with off_t
when talking about the offset of objects in a packfile, but there
were a handful of places that used "unsigned long" to hold that
value, leading to an unintended truncation.

* nd/pack-ofs-4gb-limit:
  fsck: use streaming interface for large blobs in pack
  pack-objects: do not truncate result in-pack object size on 32-bit systems
  index-pack: correct "offset" type in unpack_entry_data()
  index-pack: report correct bad object offsets even if they are large
  index-pack: correct "len" type in unpack_data()
  sha1_file.c: use type off_t* for object_info->disk_sizep
  pack-objects: pass length to check_pack_crc() without truncation
2016-08-08 14:21:36 -07:00
Junio C Hamano
a52fb9b8f3 Merge branch 'js/ignore-space-at-eol' into maint
An age old bug that caused "git diff --ignore-space-at-eol"
misbehave has been fixed.

* js/ignore-space-at-eol:
  diff: fix a double off-by-one with --ignore-space-at-eol
  diff: demonstrate a bug with --patience and --ignore-space-at-eol
2016-08-08 14:21:35 -07:00
Junio C Hamano
71076e11cd Merge branch 'jk/push-scrub-url' into maint
"git fetch http://user:pass@host/repo..." scrubbed the userinfo
part, but "git push" didn't.

* jk/push-scrub-url:
  t5541: fix url scrubbing test when GPG is not set
  push: anonymize URL in status output
2016-08-08 14:21:34 -07:00
Junio C Hamano
880b3fee51 Merge branch 'nd/cache-tree-ita' into maint
"git add -N dir/file && git write-tree" produced an incorrect tree
when there are other paths in the same directory that sorts after
"file".

* nd/cache-tree-ita:
  cache-tree: do not generate empty trees as a result of all i-t-a subentries
  cache-tree.c: fix i-t-a entry skipping directory updates sometimes
  test-lib.sh: introduce and use $EMPTY_BLOB
  test-lib.sh: introduce and use $EMPTY_TREE
2016-08-08 14:21:33 -07:00
Junio C Hamano
327b3f8459 Merge branch 'mh/blame-worktree' into maint
"git blame file" allowed the lineage of lines in the uncommitted,
unadded contents of "file" to be inspected, but it refused when
"file" did not appear in the current commit.  When "file" was
created by renaming an existing file (but the change has not been
committed), this restriction was unnecessarily tight.

* mh/blame-worktree:
  t/t8003-blame-corner-cases.sh: Use here documents
  blame: allow to blame paths freshly added to the index
2016-08-08 14:21:32 -07:00
Johannes Schindelin
9eed4f3711 t5533: make it pass on case-sensitive filesystems
The newly-added test case wants to commit a file "c.t" (note the lower
case) when a previous test case already committed a file "C.t". This
confuses Git to the point that it thinks "c.t" was not staged when "git
add c.t" was called.

Simply make the naming of the test commits consistent with the previous
test cases: use upper-case, and advance in the alphabet.

This came up in local work to rebase the Windows-specific patches to the
current `next` branch. An identical fix was suggested by John Keeping.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-04 10:18:36 -07:00
Nguyễn Thái Ngọc Duy
6b7728db81 t7063: work around FreeBSD's lazy mtime update feature
Let's start with the commit message of [1] from freebsd.git [2]

    Sync timestamp changes for inodes of special files to disk as late
    as possible (when the inode is reclaimed).  Temporarily only do
    this if option UFS_LAZYMOD configured and softupdates aren't
    enabled.  UFS_LAZYMOD is intentionally left out of
    /sys/conf/options.

    This is mainly to avoid almost useless disk i/o on battery powered
    machines.  It's silly to write to disk (on the next sync or when
    the inode becomes inactive) just because someone hit a key or
    something wrote to the screen or /dev/null.

    PR:             5577 [3]

The short version of that, in the context of t7063, is that when a
directory is updated, its mtime may be updated later, not
immediately. This can be shown with a simple command sequence

    date; sleep 1; touch abc; rm abc; sleep 10; ls -lTd .

One would expect that the date shown in `ls` would be one second from
`date`, but it's 10 seconds later. If we put another `ls -lTd .` in
front of `sleep 10`, then the date of the last `ls` comes as
expected. The first `ls` somehow forces mtime to be updated.

t7063 is really sensitive to directory mtime. When mtime is too "new",
git code suspects racy timestamps and will not trigger the shortcut in
untracked cache, in t7063.24 and eventually be detected in t7063.27

We have two options thanks to this special FreeBSD feature:

1) Stop supporting untracked cache on FreeBSD. Skip t7063 entirely
   when running on FreeBSD

2) Work around this problem (using the same 'ls' trick) and continue
   to support untracked cache on FreeBSD

I initially wanted to go with 1) because I didn't know the exact
nature of this feature and feared that it would make untracked cache
work unreliably, using the cached version when it should not.

Since the behavior of this thing is clearer now. The picture is not
that bad. If this indeed happens often, untracked cache would assume
racy condition more often and _fall back_ to non-untracked cache code
paths. Which means it may be less effective, but it will not show
wrong things.

This patch goes with option 2.

PS. For those who want to look further in FreeBSD source code, this
flag is now called IN_LAZYMOD. I can see it's effective in ext2 and
ufs. zfs is not affected.

[1] 660e6408e6df99a20dacb070c5e7f9739efdf96d
[2] git://github.com/freebsd/freebsd.git
[3] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=5577

Reported-by: Eric Wong <e@80x24.org>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-04 09:51:42 -07:00
Johannes Sixt
54956df9bc t4130: work around Windows limitation
On Windows, it is already pretty expensive to try to recreate the stat()
data that Git assumes is cheap to obtain. To make things halfway decent
in performance, we even have to skip emulating the inode and to
determine the number of hard links.

This is not a huge problem, usually, as either the size or the mtime or
the ctime are tell-tale enough to say when a file has changed, and even
if not, those changes are typically made after the index file was
written, triggering a rehashing of the files' contents.

The t4130-apply-criss-cross-rename test case, however, requires the
inode to determine that files of equal size were swapped, as renaming
files does not update their mtime. Every once in a while, t4130 fails
on Windows because of this missing piece.

Equal file sizes are not crucial for the test cases, however. Hence,
generate files with different sizes so that there is some property that
the swapped files can be discovered reliably even on Windows.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-03 08:47:38 -07:00
René Scharfe
02962d3684 use strbuf_addstr() for adding constant strings to a strbuf
Replace uses of strbuf_addf() for adding strings with more lightweight
strbuf_addstr() calls.

In http-push.c it becomes easier to see what's going on without having
to verfiy that the definition of PROPFIND_ALL_REQUEST doesn't contain
any format specifiers.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-01 13:42:10 -07:00
David Aguilar
98f917ed42 difftool: avoid $GIT_DIR and $GIT_WORK_TREE
Environment variables are global and hard to reason about.
Use the `--git-dir` and `--work-tree` arguments when invoking `git`
instead of relying on the environment.

Add a test to ensure that difftool's dir-diff feature works when these
variables are present in the environment.

Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-28 14:01:55 -07:00
Heiko Voigt
514dea905a submodule-config: passing name reference for .gitmodule blobs
Commit 959b5455 (submodule: implement a config API for lookup of
.gitmodules values, 2015-08-18) implemented the initial version of the
submodule config cache. During development of that initial version we
extracted the function gitmodule_sha1_from_commit(). During that process
we missed that the strbuf rev was still used in config_from() and now is
left empty. Lets fix this by also returning this string.

This means that now when reading .gitmodules from revisions, the error
messages also contain a reference to the blob they are from.

Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-28 13:05:14 -07:00
Junio C Hamano
1ecc6b291c Merge branch 'ak/lazy-prereq-mktemp' into maint
A test that unconditionally used "mktemp" learned that the command
is not necessarily available everywhere.

* ak/lazy-prereq-mktemp:
  t7610: test for mktemp before test execution
2016-07-28 11:26:03 -07:00
Junio C Hamano
6cbec0da47 Merge branch 'nd/icase' into maint
"git grep -i" has been taught to fold case in non-ascii locales
correctly.

* nd/icase:
  grep.c: reuse "icase" variable
  diffcore-pickaxe: support case insensitive match on non-ascii
  diffcore-pickaxe: Add regcomp_or_die()
  grep/pcre: support utf-8
  gettext: add is_utf8_locale()
  grep/pcre: prepare locale-dependent tables for icase matching
  grep: rewrite an if/else condition to avoid duplicate expression
  grep/icase: avoid kwsset when -F is specified
  grep/icase: avoid kwsset on literal non-ascii strings
  test-regex: expose full regcomp() to the command line
  test-regex: isolate the bug test code
  grep: break down an "if" stmt in preparation for next changes
2016-07-28 11:26:03 -07:00
Junio C Hamano
e5a730a1c3 Merge branch 'jk/test-match-signal' into maint
The test framework learned a new helper test_match_signal to
check an exit code from getting killed by an expected signal.

* jk/test-match-signal:
  t/lib-git-daemon: use test_match_signal
  test_must_fail: use test_match_signal
  t0005: use test_match_signal as appropriate
  tests: factor portable signal check out of t0005
2016-07-28 11:26:00 -07:00
Junio C Hamano
8a81d5f5f8 Merge branch 'js/t3404-grammo-fix' into maint
Grammofix.

* js/t3404-grammo-fix:
  t3404: fix a grammo (commands are ran -> commands are run)
2016-07-28 11:25:59 -07:00
Junio C Hamano
ae8daba601 Merge branch 'ps/rebase-i-auto-unstash-upon-abort' into maint
"git rebase -i --autostash" did not restore the auto-stashed change
when the operation was aborted.

* ps/rebase-i-auto-unstash-upon-abort:
  rebase -i: restore autostash on abort
2016-07-28 11:25:52 -07:00
Junio C Hamano
c12c71fabb Merge branch 'nd/ita-cleanup' into maint
Git does not know what the contents in the index should be for a
path added with "git add -N" yet, so "git grep --cached" should not
show hits (or show lack of hits, with -L) in such a path, but that
logic does not apply to "git grep", i.e. searching in the working
tree files.  But we did so by mistake, which has been corrected.

* nd/ita-cleanup:
  grep: fix grepping for "intent to add" files
  t7810-grep.sh: fix a whitespace inconsistency
  t7810-grep.sh: fix duplicated test name
2016-07-28 11:25:51 -07:00
Junio C Hamano
4966b58f3e Merge branch 'js/find-commit-subject-ignore-leading-blanks' into maint
A helper function that takes the contents of a commit object and
finds its subject line did not ignore leading blank lines, as is
commonly done by other codepaths.  Make it ignore leading blank
lines to match.

* js/find-commit-subject-ignore-leading-blanks:
  reset --hard: skip blank lines when reporting the commit subject
  sequencer: use skip_blank_lines() to find the commit subject
  commit -C: skip blank lines at the beginning of the message
  commit.c: make find_commit_subject() more robust
  pretty: make the skip_blank_lines() function public
2016-07-28 11:25:50 -07:00
Junio C Hamano
053e2fb579 Merge branch 'dg/subtree-rebase-test' into maint
Add a test to specify the desired behaviour that currently is not
available in "git rebase -Xsubtree=...".

* dg/subtree-rebase-test:
  contrib/subtree: Add a test for subtree rebase that loses commits
2016-07-28 11:25:49 -07:00