Picking two versions of Git and running tests to make sure the
older one and the newer one interoperate happily has now become
possible.
* jk/interop-test:
t/interop: add test of old clients against modern git-daemon
t: add an interoperability test harness
The experimental "split index" feature has gained a few
configuration variables to make it easier to use.
* cc/split-index-config: (22 commits)
Documentation/git-update-index: explain splitIndex.*
Documentation/config: add splitIndex.sharedIndexExpire
read-cache: use freshen_shared_index() in read_index_from()
read-cache: refactor read_index_from()
t1700: test shared index file expiration
read-cache: unlink old sharedindex files
config: add git_config_get_expiry() from gc.c
read-cache: touch shared index files when used
sha1_file: make check_and_freshen_file() non static
Documentation/config: add splitIndex.maxPercentChange
t1700: add tests for splitIndex.maxPercentChange
read-cache: regenerate shared index if necessary
config: add git_config_get_max_percent_split_change()
Documentation/git-update-index: talk about core.splitIndex config var
Documentation/config: add information for core.splitIndex
t1700: add tests for core.splitIndex
update-index: warn in case of split-index incoherency
read-cache: add and then use tweak_split_index()
split-index: add {add,remove}_split_index() functions
config: add git_config_get_split_index()
...
Make git-credential-cache follow the XDG base path specification by
default. This increases consistency with other applications and helps
keep clutter out of users' home directories.
Check the old socket location, ~/.git-credential-cache/, and use
~/.git-credential-cache/socket if that directory exists rather than
forcing users who have used `git credential-cache` before to migrate to
the new XDG compliant location.
Otherwise use the socket $XDG_CACHE_HOME/git/credential/socket following
XDG base path specification. Use the subdirectory credential/ in case
other files are cached under $XDG_CACHE_HOME/git/ in the future and to
make the socket's purpose clear.
Signed-off-by: Devin Lehmacher <lehmacdj@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We used to use the SHA1 implementation from the OpenSSL library by
default. As we are trying to be careful against collision attacks
after the recent "shattered" announcement, switch the default to
encourage people to use DC_SHA1 implementation instead. Those who
want to use the implementation from OpenSSL can explicitly ask for
it by OPENSSL_SHA1=YesPlease when running "make".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We don't actually have a Git-object collision, so the best
we can do is to run one of the shattered PDFs through
test-sha1. This should trigger the collision check and die.
In a sense this isn't really checking anything that the
upstream sha1collisiondetection project doesn't cover
already. But it at least makes sure that our build correctly
uses the library.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This knob lets you use the sha1dc implementation from:
https://github.com/cr-marcstevens/sha1collisiondetection
which can detect certain types of collision attacks (even
when we only see half of the colliding pair). So it
mitigates any attack which consists of getting the "good"
half of a collision into a trusted repository, and then
later replacing it with the "bad" half. The "good" half is
rejected by the victim's version of Git (and even if they
run an old version of Git, any sha1dc-enabled git will
complain loudly if it ever has to interact with the object).
The big downside is that it's slower than either the openssl
or block-sha1 implementations.
Here are some timings based off of linux.git:
- compute sha1 over whole packfile
sha1dc: 3.580s
blk-sha1: 2.046s (-43%)
openssl: 1.335s (-62%)
- rev-list --all --objects
sha1dc: 33.512s
blk-sha1: 33.514s (+0.0%)
openssl: 33.650s (+0.4%)
- git log --no-merges -10000 -p
sha1dc: 8.124s
blk-sha1: 7.986s (-1.6%)
openssl: 8.203s (+0.9%)
- index-pack --verify
sha1dc: 4m19s
blk-sha1: 2m57s (-32%)
openssl: 2m19s (-42%)
So overall the sha1 computation with collision detection is
about 1.75x slower than block-sha1, and 2.7x slower than
sha1. But of course most operations do more than just sha1.
Normal object access isn't really slowed at all (both the
+/- changes there are well within the run-to-run noise); any
changes are drowned out by the other work Git is doing.
The most-affected operation is `index-pack --verify`, which
is essentially just computing the sha1 on every object. This
is similar to the `index-pack` invocation that the receiver
of a push or fetch would perform. So clearly there's some
extra CPU load here.
There will also be some latency for the user, though keep in
mind that such an operation will generally be network bound
(this is about a 1.2GB packfile). Some of that extra CPU is
"free" in the sense that we use it while the pack is
streaming in anyway. But most of it comes during the
delta-resolution phase, after the whole pack has been
received. So we can imagine that for this (quite large)
push, the user might have to wait an extra 100 seconds over
openssl (which is what we use now). If we assume they can
push to us at 20Mbit/s, that's 480s for a 1.2GB pack, which
is only 20% slower.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Improve readability by using the for_each_string_list_item helper
instead of manually iterating with an integer counter.
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since the conversion from shell to C in 56dc3ab04 (sequencer
(rebase -i): implement the 'edit' command, 2017-01-02),
stopping at an "edit" instruction went from:
$ git rebase -i
Stopped at 6ce6b914a... odb_pack_keep(): stop generating keepfile name
You can amend the commit now, with
[...more instructions...]
to:
$ git rebase -i
warning: stopped at 6ce6b914a... odb_pack_keep(): stop generating keepfile name
You can amend the commit now, with
[...more instructions...]
The "warning" implies that it's something unexpected, but
it's not. Let's switch back to the original message.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The safe_hash feature is designed to make sha1dc a drop-in
replacement for sha1, where colliding entries will get a
permuted hash to un-collide them. However, since we're
handling the collision case ourselves, this isn't helpful
(and is actually harmful, as it means you get the wrong
object id if you want to show it in a log message).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We can replace system includes with git-compat-util.h or
cache.h (and should make sure it is included first in all C
files). And we can drop includes from headers entirely, as
every C file should include git-compat-util.h itself.
We will add in new include guards around the header files,
though (otherwise you get into trouble including both
sha1dc/sha1.h and cache.h).
And finally, we'll use the full "sha1dc/" path for including
related files. This isn't strictly necessary, but makes the
expected resolution more obvious.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is pulled straight from:
https://github.com/cr-marcstevens/sha1collisiondetection
with no modifications yet (though I've pulled in only the
subset of files necessary for Git to use).
This is commit 007905a93c973f55b2daed6585f9f6c23545bf66.
Further updates can be done like:
git checkout -b vendor-sha1dc $this_commit
cp /path/to/sha1dc/{LICENSE.txt,lib/*} sha1dc/
git add -A sha1dc
git commit -m "update sha1dc"
git checkout -b update-sha1dc origin
git merge vendor-sha1dc
Thanks to both Marc and Dan for making the code fit our
needs by doing both optimization work, cutting down on the
object size, and doing some syntactic changes to work better
with git. And to Linus for kicking off the "diet" work that
removed some of the unused code.
The license of the sha1dc code is the MIT license, which is
obviously compatible with the GPLv2 of git.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git v2.12 was shipped with an embarrassing breakage where various
operations that verify paths given from the user stopped dying when
seeing an issue, and instead later triggering segfault.
... and then to down to 'maint'.
* js/realpath-pathdup-fix:
real_pathdup(): fix callsites that wanted it to die on error
t1501: demonstrate NULL pointer access with invalid GIT_WORK_TREE
Code clean-up and a string truncation fix.
* mm/two-more-xstrfmt:
bisect_next_all: convert xsnprintf to xstrfmt
stop_progress_msg: convert xsnprintf to xstrfmt
The command-line parsing of "git log -L" copied internal data
structures using incorrect size on ILP32 systems.
* vn/line-log-memcpy-size-fix:
line-log: use COPY_ARRAY to fix mis-sized memcpy
The code to parse "git log -L..." command line was buggy when there
are many ranges specified with -L; overrun of the allocated buffer
has been fixed.
* ax/line-log-range-merge-fix:
line-log.c: prevent crash during union of too many ranges
The patch subcommand of "git add -i" was meant to have paths
selection prompt just like other subcommand, unlike "git add -p"
directly jumps to hunk selection. Recently, this was broken and
"add -i" lost the paths selection dialog, but it now has been
fixed.
* jk/add-i-patch-do-prompt:
add--interactive: fix missing file prompt for patch mode with "-i"
When a redirected http transport gets an error during the
redirected request, we ignored the error we got from the server,
and ended up giving a not-so-useful error message.
* jt/http-base-url-update-upon-redirect:
http: attempt updating base URL only if no error
Reduce authentication round-trip over HTTP when the server supports
just a single authentication method.
* jk/http-auth:
http: add an "auto" mode for http.emptyauth
http: restrict auth methods to what the server advertises
The final() function accepts a NULL value for certain
parameters, and falls back to writing into a reusable "name"
buffer, and then either:
1. For "keep_name", requiring all uses to do "keep_name ?
keep_name : name.buf". This is awkward, and it's easy
to accidentally look at the maybe-NULL keep_name.
2. For "final_index_name" and "final_pack_name", aliasing
those pointers to the "name" buffer. This is easier to
use, but the aliased pointers become invalid after the
buffer is reused (this isn't a bug now, but it's a
potential pitfall).
One way to make this safer would be to introduce an extra
pointer to do the aliasing, and have its lifetime match the
validity of the "name" buffer. But it's still easy to
accidentally use the wrong name (i.e., to use
"final_pack_name" instead of the aliased pointer).
Instead, let's use three separate buffers that will remain
valid through the function. That makes it safe to alias the
pointers and use them consistently. The extra allocations
shouldn't matter, as this function is not performance
sensitive.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In several places we write the name of the pack filename
into a fixed-size buffer using snprintf(), but do not check
the return value. As a result, a very long object directory
could cause us to quietly truncate the pack filename
(potentially leading to a corrupted repository, as a newly
written packfile could be missing its .pack extension).
We can use odb_pack_name() to do this with a strbuf (and
shorten the code, as well).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The odb_pack_keep() function generates the name of a .keep
file and opens it. This has two problems:
1. It requires a fixed-size buffer to create the filename
and doesn't notice when the result is truncated.
2. Of the two callers, one sometimes wants to open a
filename it already has, which makes things awkward (it
has to do so manually, and skips the leading-directory
creation).
Instead, let's have odb_pack_keep() just open the file.
Generating the name isn't hard, and a future patch will
switch callers over to odb_pack_name() anyway.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We provide sha1_pack_name() and sha1_pack_index_name(), but
the more generic form (which takes its own strbuf and an
arbitrary extension) is only used to implement the other
two. Let's make it available, but clean up a few things:
1. Name it odb_pack_name(), as the original
sha1_get_pack_name() is long but not all that
descriptive.
2. Switch the strbuf argument to the beginning, so that it
matches similar path-building functions like
git_path_buf().
3. Clean up the out-dated docstring and move it to the
public declaration.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These functions were originally conceived as wrapper
functions similar to xmkstemp(). They were later moved by
463db9b10 (wrapper: move odb_* to environment.c,
2010-11-06). The more appropriate place for a declaration is
in cache.h.
While we're at it, let's add some basic docstrings.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
All of these options do the same thing "--foo" iterates over
the "foo" refs, and "--foo=<glob>" does the same with a
glob. We can factor this into its own function to avoid
repeating ourselves.
There are two subtleties to note:
- the original called for_each_branch_ref(), etc, in the
non-glob case. Now we will call for_each_ref_in("refs/heads/")
which is exactly what for_each_branch_ref() did under
the hood.
- for --glob, we'll call for_each_glob_ref_in() with a
NULL "prefix" argument. Which is exactly what
for_each_glob_ref() was doing already.
So both cases should behave identically, and it seems
reasonable to assume that this will remain the same. The
functions we are calling now are the more-generic ones, and
the ones we are dropping are just convenience wrappers.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We can't just use a bare skip_prefix() for these cases,
because we need to match both the "--foo" form and the
"--foo=<value>" form (and tell the difference between the
two in the caller).
We can wrap this in a simple helper which has two obvious
callsites, and will gain some more in the next patch.
Note that the error output for abbrev-ref changes slightly,
as we don't keep our original "arg" pointer. However, the
new output should hopefully be more clear:
[before]
fatal: unknown mode for --abbrev-ref=foo
[after]
fatal: unknown mode for --abbrev-ref: foo
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using skip_prefix lets us avoid manually-counted offsets
into the argument string. This patch converts the simple and
obvious cases.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-prompt has various describe styles, among them "describe" (by
annotated tags) and "default" (by exact match with any tag).
Add a mode "tag" that describes by any tag, annotated or not.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The cherry-pick and revert commands use OPT_INTEGER() to
parse --mainline. The stock parser is smart enough to reject
non-numeric nonsense, but it doesn't know that parent
counting starts at 1.
Worse, the value "0" is indistinguishable from the unset
case, so a user who assumes the counting is 0-based will get
a confusing message:
$ git cherry-pick -m 0 $merge
error: commit ... is a merge but no -m option was given.
Let's use a custom callback that enforces our range.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Detect the null object ID for symlinks in dir-diff so that difftool can
detect when symlinks are modified in the worktree.
Previously, a null symlink object ID would crash difftool.
Handle null object IDs as unknown content that must be read from
the worktree.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As 'var' contains the whole value we get error messages that repeat
the section and key currently:
warning: Invalid parameter 'true' for config option 'submodule.submodule.plugins/hooks.ignore.ignore'
Fix this by only giving the section name in the warning.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Many developers use functionality in their editors that allows for quick
syntax checks, including warning about questionable constructs. This
functionality allows rapid development with fewer errors. However, such
functionality generally does not allow the specification of
project-specific defines or command-line options.
Since the SHA1_HEADER include is not defined in such a case,
developers see spurious errors when using these tools. Furthermore,
there are known implementations of "cc" whose '#include' is unhappy
with this construct.
Instead of using SHA1_HEADER, create a hash.h header and use #if
and #elif to select the desired header. Have the Makefile pass an
appropriate option to help the header select the right implementation to
use.
[jc: make BLK_SHA1 the fallback default as discussed on list,
e.g. <20170314201424.vccij5z2ortq4a4o@sigill.intra.peff.net>; also
remove SHA1_HEADER and SHA1_HEADER_SQ that are no longer used].
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git branch --list" takes the "--abbrev" and "--no-abbrev" options
to control the output of the object name in its "-v"(erbose)
output, but a recent update started ignoring them; this fixes it
before the breakage reaches to any released version.
* kn/ref-filter-branch-list:
branch: honor --abbrev/--no-abbrev in --list mode
"git push" had a handful of codepaths that could lead to a deadlock
when unexpected error happened, which has been fixed.
* jk/push-deadlock-regression-fix:
send-pack: report signal death of pack-objects
send-pack: read "unpack" status even on pack-objects failure
send-pack: improve unpack-status error messages
send-pack: use skip_prefix for parsing unpack status
send-pack: extract parsing of "unpack" response
receive-pack: fix deadlock when we cannot create tmpdir
Recent versions of Git treats http alternates (used in dumb http
transport) just like HTTP redirects and requires the client to
enable following it, due to security concerns. But we forgot to
give a warning when we decide not to honor the alternates.
* ew/http-alternates-as-redirects-warning:
http: release strbuf on disabled alternates
http: inform about alternates-as-redirects behavior
"git filter-branch --prune-empty" drops a single-parent commit that
becomes a no-op, but did not drop a root commit whose tree is empty.
* dp/filter-branch-prune-empty:
p7000: add test for filter-branch with --prune-empty
filter-branch: fix --prune-empty on parentless commits
t7003: ensure --prune-empty removes entire branch when applicable
t7003: ensure --prune-empty can prune root commit
The t/perf performance test suite was not prepared to test not so
old versions of Git, but now it covers versions of Git that are not
so ancient.
* jt/perf-updates:
t/perf: add fallback for pre-bin-wrappers versions of git
t/perf: use $MODERN_GIT for all repo-copying steps
t/perf: export variable used in other blocks
"git fetch" that requests a commit by object name, when the other
side does not allow such an request, failed without much
explanation.
* mm/fetch-show-error-message-on-unadvertised-object:
fetch-pack: add specific error for fetching an unadvertised object
fetch_refs_via_pack: call report_unmatched_refs
fetch-pack: move code to report unmatched refs to a function
"git branch @" created refs/heads/@ as a branch, and in general the
code that handled @{-1} and @{upstream} was a bit too loose in
disambiguating.
* jk/interpret-branch-name:
checkout: restrict @-expansions when finding branch
strbuf_check_ref_format(): expand only local branches
branch: restrict @-expansions when deleting
t3204: test git-branch @-expansion corner cases
interpret_branch_name: allow callers to restrict expansions
strbuf_branchname: add docstring
strbuf_branchname: drop return value
interpret_branch_name: move docstring to header file
interpret_branch_name(): handle auto-namelen for @{-1}
A few tests were run conditionally under (rare) conditions where
they cannot be run (like running cvs tests under 'root' account).
* ab/cond-skip-tests:
gitweb tests: skip tests when we don't have Time::HiRes
gitweb tests: change confusing "skip_all" phrasing
cvs tests: skip tests that call "cvs commit" when running as root
During the review of the `early-config` patch series, two issues
have been identified that have been with us forever. Mark the
identified problems for later so that we do not forget them.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King came up with a couple examples that demonstrate how the new
read_early_config() that looks harder for the current .git/ directory
could die() in an undesirable way.
Let's add those cases to the test script, to document what we would like
to happen when early config encounters problems.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
