Commit Graph

15007 Commits

Author SHA1 Message Date
Johannes Schindelin
c871fbee2b t6500(mingw): use the Windows PID of the shell
In Git for Windows, we use the MSYS2 Bash which inherits a non-standard
PID model from Cygwin's POSIX emulation layer: every MSYS2 process has a
regular Windows PID, and in addition it has an MSYS2 PID (which
corresponds to a shadow process that emulates Unix-style signal
handling).

With the upgrade to the MSYS2 runtime v3.x, this shadow process cannot
be accessed via `OpenProcess()` any longer, and therefore t6500 thought
incorrectly that the process referenced in `gc.pid` (which is not
actually a real `gc` process in this context, but the current shell) no
longer exists.

Let's fix this by making sure that the Windows PID is written into
`gc.pid` in this test script so that `git.exe` is able to understand
that that process does indeed still exist.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-08 15:04:25 +09:00
Jeff King
d9ef573837 t/lib-httpd: pass LSAN_OPTIONS through apache
Just as we instruct Apache to pass through ASAN_OPTIONS (so that
server-side Git programs it spawns will respect our options while
running the tests), we should do the same with LSAN_OPTIONS. Otherwise
trying to collect a list of leaks like:

    export LSAN_OPTIONS=exitcode=0:log_path=/tmp/lsan
    make SANITIZE=leak test

won't work for http tests (the server-side programs won't log their
leaks to the right place, and they'll prematurely die, producing a
spurious test failure).

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-08 13:21:23 +09:00
Johannes Schindelin
398a3b0899 fsmonitor: force a refresh after the index was discarded
With this change, the `index_state` struct becomes the new home for the
flag that says whether the fsmonitor hook has been run, i.e. it is now
per-index.

It also gets re-set when the index is discarded, fixing the bug
demonstrated by the "test_expect_failure" test added in the preceding
commit. In that case fsmonitor-enabled Git would miss updates under
certain circumstances, see that preceding commit for details.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-08 12:03:48 +09:00
Johannes Schindelin
dc76852df2 fsmonitor: demonstrate that it is not refreshed after discard_index()
This one is tricky.

When `core.fsmonitor` is set, a `refresh_index()` will not perform a
full scan of files that might be modified, but will query the fsmonitor
and refresh only the ones that have been actually touched.

Due to implementation details, the fsmonitor is queried in
`refresh_cache_ent()`, but of course it only has to be queried once, so
we set a flag when we did that. But when the index was discarded, we did
not re-set that flag.

So far, this is only covered by our test suite when running with
GIT_TEST_FSMONITOR=$PWD/t7519/fsmonitor-all, and only due to the way the
built-in stash interacts with the recursive merge machinery.

Let's introduce a straight-forward regression test for this.

We simply extend the "read & discard index" loop in `test-tool
read-cache` to optionally refresh the index, report on a given file's
status, and then modify that file. Due to the bug described above, only
the first refresh will actually query the fsmonitor; subsequent loop
iterations will not.

This problem was reported by Ævar Arnfjörð Bjarmason.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-08 11:58:12 +09:00
Ævar Arnfjörð Bjarmason
82b7eb231d perf-lib.sh: forbid the use of GIT_TEST_INSTALLED
As noted in preceding commits setting GIT_TEST_INSTALLED has never
been supported or documented, and as noted in an earlier t/perf/README
change to the extent that it's been documented nobody's notices that
the example hasn't worked since 3c8f12c96c ("test-lib: reorder and
include GIT-BUILD-OPTIONS a lot earlier", 2012-06-24).

We could directly support GIT_TEST_INSTALLED for invocations without
the "run" script, such as:

    GIT_TEST_INSTALLED=../../ ./p0000-perf-lib-sanity.sh
    GIT_TEST_INSTALLED=/home/avar/g/git ./p0000-perf-lib-sanity.sh

But while not having this "error" will "work", it won't write the the
resulting "test-results/*" files to the right place, and thus a
subsequent call to aggregate.perl won't work as expected.

Let's just tell the user that they need to use the "run" script,
which'll correctly deal with this and set the right
PERF_RESULTS_PREFIX.

If someone's in desperate need of bypassing "run" for whatever reason
they can trivially do so by setting "PERF_SET_GIT_TEST_INSTALLED", but
not we won't have people who expect GIT_TEST_INSTALLED to just work
wondering why their aggregation doesn't work, even though they're
running the right "git".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
2019-05-08 11:00:28 +09:00
Ævar Arnfjörð Bjarmason
fab80ee79d perf tests: add "bindir" prefix to git tree test results
Change the output file names in test-results/ to be
"test-results/bindir_<munged dir>" rather than just
"test-results/<munged dir>".

This is for consistency with the "build_" directories we have for
built revisions, i.e. "test-results/build_<SHA-1>".

There's no user-visible functional changes here, it just makes it
easier to see at a glance what "test-results" files are of what "type"
as they're all explicitly grouped together now, and to grep this code
to find both the run_dirs_helper() implementation and its
corresponding aggregate.perl code.

Note that we already guarantee that the rest of the
PERF_RESULTS_PREFIX is an absolute path, and since it'll start with
e.g. "/" which we munge to "_" we'll up with a readable string like
"bindir_home_avar[...]".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
2019-05-08 11:00:28 +09:00
Ævar Arnfjörð Bjarmason
df0f502195 perf-lib.sh: remove GIT_TEST_INSTALLED from perf-lib.sh
Follow-up my preceding change which fixed the immediate "./run
<revisions>" regression in 0baf78e7bc ("perf-lib.sh: rely on
test-lib.sh for --tee handling", 2019-03-15) and entirely get rid of
GIT_TEST_INSTALLED from perf-lib.sh (and aggregate.perl).

As noted in that change the dance we're doing with GIT_TEST_INSTALLED
perf-lib.sh isn't necessary, but there I was doing the most minimal
set of changes to quickly fix a regression.

But it's much simpler to never deal with the "GIT_TEST_INSTALLED" we
were setting in perf-lib.sh at all. Instead the run_dirs_helper() sets
the previously inferred $PERF_RESULTS_PREFIX directly.

Setting this at the callsite that's already best positioned to
exhaustively know about all the different cases we need to handle
where PERF_RESULTS_PREFIX isn't what we want already (the empty
string) makes the most sense. In one-off cases like:

    ./run ./p0000-perf-lib-sanity.sh
    ./p0000-perf-lib-sanity.sh

We'll just do the right thing because PERF_RESULTS_PREFIX will be
empty, and test-lib.sh takes care of finding where our git is.

Any refactoring of this code needs to change both the shell code and
the Perl code in aggregate.perl, because when running e.g.:

    ./run ../../ -- <test>

The "../../" path to a relative bindir needs to be munged to a
filename containing the results, and critically aggregate.perl does
not get passed the path to those aggregations, just "../..".

Let's fix cases where aggregate.perl would print e.g. ".." in its
report output for this, and "git" for "/home/avar/g/git", i.e. it
would always pick the last element. Now'll always print the full path
instead.

This also makes the code sturdier, e.g. you can feed "../.."  to
"./run" and then an absolute path to the aggregate.perl script, as
long as the absolute path and "../.." resolved to the same directory
printing the aggregation will work.

Also simplify the "[_*]" on the RHS of "tr -c", we're trimming
everything to "_", so we don't need that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
2019-05-08 11:00:28 +09:00
Ævar Arnfjörð Bjarmason
90e38154ee perf-lib.sh: make "./run <revisions>" use the correct gits
Fix a really bad regression in 0baf78e7bc ("perf-lib.sh: rely on
test-lib.sh for --tee handling", 2019-03-15). Since that change all
runs of different <revisions> of git have used the git found in the
user's $PATH, e.g. /usr/bin/git instead of the <revision> we just
built and wanted to performance test.

The problem starts with GIT_TEST_INSTALLED not working like our
non-perf tests with the "run" script. I.e. you can't run performance
tests against a given installed git. Instead we expect to use it
ourselves to point GIT_TEST_INSTALLED to the <revision> we just built.

However, we had been relying on '$(cd "$GIT_TEST_INSTALLED" && pwd)'
to resolve that relative $GIT_TEST_INSTALLED to an absolute
path *before* test-lib.sh was loaded, in cases where it was
e.g. "build/<rev>/bin-wrappers" and we wanted "<abs_path>build/...".

This change post-dates another proposed solution by a few days[1], I
didn't notice that version when I initially wrote this. I'm doing the
most minimal thing to solve the regression here, a follow-up change
will move this result prefix selection logic entirely into the "run"
script.

This makes e.g. these cases all work:

    ./run . $PWD/../../ origin/master origin/next HEAD -- <tests>

As well as just a plain one-off:

    ./run <tests>

And, since we're passing down the new GIT_PERF_DIR_MYDIR_REL we make
sure the bug relating to aggregate.perl not finding our files as
described in 0baf78e7bc doesn't happen again.

What *doesn't* work is setting GIT_TEST_INSTALLED to a relative path,
this will subtly fail in test-lib.sh. This has always been the case
even before 0baf78e7bc, and as documented in t/README the
GIT_TEST_INSTALLED variable should be set to an absolute path (needs
to be set "to the bindir", which is always absolute), and the "perf"
framework expects to munge it itself.

Perhaps that should be dealt with in the future to allow manually
setting GIT_TEST_INSTALLED, but as a preceding commit showed the user
can just use the "run" script, which'll also pick the right output
directory for the test results as expected by aggregate.perl.

1. https://public-inbox.org/git/20190502222409.GA15631@sigill.intra.peff.net/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
2019-05-08 11:00:28 +09:00
Ævar Arnfjörð Bjarmason
c43b7e6089 perf aggregate: remove GIT_TEST_INSTALLED from --codespeed
Remove the setting of the "environment" from the --codespeed output. I
don't think this is useful, and it helps with a later refactoring
where we GIT_TEST_INSTALLED stop munging/reading GIT_TEST_INSTALLED in
the perf tests in so many places.

This was added in 05eb1c37ed ("perf/aggregate: implement codespeed
JSON output", 2018-01-05), but since the "run" scripts uses
"GIT_TEST_INSTALLED" internally this was only ever useful for one-off
runs of a single revision as all the "environment" values would be
ones for whatever directory the "run" script ran last.

Let's instead fall back on the "uname -r" case, which is the sort of
thing the environment should be set to, not something that duplicates
other parts of the codpseed output. For setting the "environment" to
something custom the perf.repoName variable can be used. See
19cf57a92e ("perf/run: read GIT_PERF_REPO_NAME from perf.repoName",
2018-01-05).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
2019-05-08 11:00:28 +09:00
Ævar Arnfjörð Bjarmason
9bb81452ff perf README: correct docs for 3c8f12c96c regression
Since 3c8f12c96c ("test-lib: reorder and include GIT-BUILD-OPTIONS a
lot earlier", 2012-06-24) the suggested advice of overriding
GIT_BUILD_DIR has not worked. We've printed a hard error like this
given e.g. GIT_BUILD_DIR=/home/avar/g/git:

    /bin-wrappers/git is not executable; using GIT_EXEC_PATH
    error: You haven't built things yet, have you?

Let's just suggest that the user run other gits via the "run"
script. That'll do the right thing for setting the path to the other
git, and running the "aggregate.perl" scripts afterwards will work.

As an aside, if setting GIT_BUILD_DIR had still worked, then the
MODERN_GIT feature/fix added in 1a0962dee5 ("t/perf: fix regression in
testing older versions of git", 2016-06-22) would have broke.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
2019-05-08 11:00:28 +09:00
Johannes Schindelin
397a46db78 t5580: verify that alternates can be UNC paths
On Windows, UNC paths are a very convenient way to share data, and
alternates are all about sharing data.

We fixed a bug where alternates specifying UNC paths were not handled
properly, and it is high time that we add a regression test to ensure
that this bug is not reintroduced.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-07 18:40:01 +09:00
Boxuan Li
ce4c7bfc90 t4253-am-keep-cr-dos: avoid using pipes
The exit code of the upstream in a pipe is ignored thus we should avoid
using it. By writing out the output of the git command to a file, we can
test the exit codes of both the commands.

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-07 18:04:48 +09:00
Denton Liu
e3d6539d58 branch: make create_branch accept a merge base rev
When we ran something like

    $ git checkout -b test master...

it would fail with the message

    fatal: Not a valid object name: 'master...'.

This was caused by the call to `create_branch` where `start_name` is
expected to be a valid rev. However, git-checkout allows the branch to
be a valid _merge base_ rev (i.e. with a "...") so it was possible for
an invalid rev to be passed in.

Make `create_branch` accept a merge base rev so that this case does not
error out.

As a side-effect, teach git-branch how to handle merge base revs as
well.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-07 13:23:13 +09:00
Denton Liu
27434bf08c t2018: cleanup in current test
Before, in t2018, if do_checkout failed to create `branch2`, the next
test-case would run `git branch -D branch2` but then fail because it was
expecting `branch2` to exist, even though it doesn't. As a result, an
early failure could cause a cascading failure of tests.

Make test-case responsible for cleaning up their own branches so that
future tests can start with a sane environment.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-07 13:23:10 +09:00
Nguyễn Thái Ngọc Duy
5c387428f1 parse-options: don't emit "ambiguous option" for aliases
Change the option parsing machinery so that e.g. "clone --recurs ..."
doesn't error out because "clone" understands both "--recursive" and
"--recurse-submodules" to mean the same thing.

Initially "clone" just understood --recursive until the
--recurses-submodules alias was added in ccdd3da652 ("clone: Add the
--recurse-submodules option as alias for --recursive",
2010-11-04). Since bb62e0a99f ("clone: teach --recurse-submodules to
optionally take a pathspec", 2017-03-17) the longer form has been
promoted to the default.

But due to the way the options parsing machinery works this resulted
in the rather absurd situation of:

    $ git clone --recurs [...]
    error: ambiguous option: recurs (could be --recursive or --recurse-submodules)

Add OPT_ALIAS() to express this link between two or more options and use
it in git-clone. Multiple aliases of an option could be written as

    OPT_ALIAS(0, "alias1", "original-name"),
    OPT_ALIAS(0, "alias2", "original-name"),
    ...

The current implementation is not exactly optimal in this case. But we
can optimize it when it becomes a problem. So far we don't even have two
aliases of any option.

A big chunk of code is actually from Junio C Hamano.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-07 12:23:22 +09:00
Jeff King
1bb10d4f7c t/perf: add perf script for partial clones
We don't cover the partial clone feature at all in t/perf. Let's at
least run a few basic tests so that we'll notice any regressions.

We'll do a no-blob clone, and split it into two parts: the actual object
transfer, and the subsequent checkout (which will of course require
another transfer to get the blobs). That will help us more clearly
assess the performance of each.

There are obviously a lot more possibilities besides just a no-blob
partial clone, but this should serve as a canary that alerts us to any
generic slow-downs (and we can add more tests later for cases that
aren't exercised here).

There are a few non-ideal things here that make this not an entirely
accurate test, but are probably OK for our purposes:

  1. We have to do some extra prep/cleanup work inside the timing tests,
     since they impact the on-disk state and the perf harness may run
     each one multiple times.

     In practice this is probably OK, since these bits should be much
     less expensive than the operations we are measuring.

  2. The clone time is likely to be dominated by the server's object
     enumeration. In the real world, a repo large enough to drive people
     to partial clones is likely to have reachability bitmaps enabled.

     And in the opposite direction, our object transfer is happening at
     the speed of a local pipe, whereas in the real world it would
     bottle-neck on the network.

     So any percentage speedups should be taken with a grain of salt.
     But hopefully any regressions will produce enough of an effect to
     be noticeable.

This script also demonstrates the recent improvement from dfa33a298d
(clone: do faster object check for partial clones, 2019-04-19):

  Test                          dfa33a298d^         dfa33a298d
  -------------------------------------------------------------------------
  5600.2: clone without blobs   18.41(22.72+1.09)   6.83(11.65+0.50) -62.9%
  5600.3: checkout of result    1.82(3.24+0.26)     1.84(3.24+0.26) +1.1%

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-05 14:03:57 +09:00
Junio C Hamano
5983ddc165 Merge branch 'bc/send-email-qp-cr'
"git send-email" has been taught to use quoted-printable when the
payload contains carriage-return.  The use of the mechanism is in
line with the design originally added the codepath that chooses QP
when the payload has overly long lines.

* bc/send-email-qp-cr:
  send-email: default to quoted-printable when CR is present
2019-04-25 16:41:25 +09:00
Junio C Hamano
f1c9f6ce38 Merge branch 'nd/submodule-foreach-quiet'
"git submodule foreach <command> --quiet" did not pass the option
down correctly, which has been corrected.

* nd/submodule-foreach-quiet:
  submodule foreach: fix "<command> --quiet" not being respected
2019-04-25 16:41:24 +09:00
Junio C Hamano
5c2b4ca06e Merge branch 'js/iso8895-test-on-apfs'
Test fix on APFS that is incapable of store paths in Latin-1.

* js/iso8895-test-on-apfs:
  t9822: skip tests if file names cannot be ISO-8859-1 encoded
2019-04-25 16:41:24 +09:00
Junio C Hamano
49f50f58cb Merge branch 'jc/gettext-test-fix'
The GETTEXT_POISON test option has been quite broken ever since it
was made runtime-tunable, which has been fixed.

* jc/gettext-test-fix:
  gettext tests: export the restored GIT_TEST_GETTEXT_POISON
2019-04-25 16:41:23 +09:00
Junio C Hamano
57a6b93236 Merge branch 'jk/fetch-reachability-error-fix'
Code clean-up and a fix for "git fetch" by an explicit object name
(as opposed to fetching refs by name).

* jk/fetch-reachability-error-fix:
  fetch: do not consider peeled tags as advertised tips
  remote.c: make singular free_ref() public
  fetch: use free_refs()
  pkt-line: prepare buffer before handling ERR packets
  upload-pack: send ERR packet for non-tip objects
  t5530: check protocol response for "not our ref"
  t5516: drop ok=sigpipe from unreachable-want tests
2019-04-25 16:41:23 +09:00
Junio C Hamano
449f2db75d Merge branch 'jk/xmalloc'
The code is updated to check the result of memory allocation before
it is used in more places, by using xmalloc and/or xcalloc calls.

* jk/xmalloc:
  progress: use xmalloc/xcalloc
  xdiff: use xmalloc/xrealloc
  xdiff: use git-compat-util
  test-prio-queue: use xmalloc
2019-04-25 16:41:23 +09:00
Junio C Hamano
c8e8b5c325 Merge branch 'km/t3000-retitle'
A test update.

* km/t3000-retitle:
  t3000 (ls-files -o): widen description to reflect current tests
2019-04-25 16:41:22 +09:00
Junio C Hamano
af152bd5b0 Merge branch 'js/t3301-unbreak-notes-test'
Test fix.

* js/t3301-unbreak-notes-test:
  t3301: fix false negative
2019-04-25 16:41:22 +09:00
Junio C Hamano
d8620d3ca7 Merge branch 'sg/blame-in-bare-start-at-head'
"git blame -- path" in a non-bare repository starts blaming from
the working tree, and the same command in a bare repository errors
out because there is no working tree by definition.  The command
has been taught to instead start blaming from the commit at HEAD,
which is more useful.

* sg/blame-in-bare-start-at-head:
  blame: default to HEAD in a bare repo when no start commit is given
2019-04-25 16:41:21 +09:00
Junio C Hamano
32dc15dec1 Merge branch 'jt/batch-fetch-blobs-in-diff'
While running "git diff" in a lazy clone, we can upfront know which
missing blobs we will need, instead of waiting for the on-demand
machinery to discover them one by one.  Aim to achieve better
performance by batching the request for these promised blobs.

* jt/batch-fetch-blobs-in-diff:
  diff: batch fetching of missing blobs
  sha1-file: support OBJECT_INFO_FOR_PREFETCH
2019-04-25 16:41:19 +09:00
Junio C Hamano
01f8d78887 Merge branch 'dl/submodule-set-branch'
"git submodule" learns "set-branch" subcommand that allows the
submodule.*.branch settings to be modified.

* dl/submodule-set-branch:
  submodule: teach set-branch subcommand
  submodule--helper: teach config subcommand --unset
  git-submodule.txt: "--branch <branch>" option defaults to 'master'
2019-04-25 16:41:18 +09:00
Junio C Hamano
d9d65e9f6a Merge branch 'jk/revision-rewritten-parents-in-prio-queue'
Performance fix for "rev-list --parents -- pathspec".

* jk/revision-rewritten-parents-in-prio-queue:
  revision: use a prio_queue to hold rewritten parents
2019-04-25 16:41:18 +09:00
Junio C Hamano
d4e568b2a3 Merge branch 'bc/hash-transition-16'
Conversion from unsigned char[20] to struct object_id continues.

* bc/hash-transition-16: (35 commits)
  gitweb: make hash size independent
  Git.pm: make hash size independent
  read-cache: read data in a hash-independent way
  dir: make untracked cache extension hash size independent
  builtin/difftool: use parse_oid_hex
  refspec: make hash size independent
  archive: convert struct archiver_args to object_id
  builtin/get-tar-commit-id: make hash size independent
  get-tar-commit-id: parse comment record
  hash: add a function to lookup hash algorithm by length
  remote-curl: make hash size independent
  http: replace sha1_to_hex
  http: compute hash of downloaded objects using the_hash_algo
  http: replace hard-coded constant with the_hash_algo
  http-walker: replace sha1_to_hex
  http-push: remove remaining uses of sha1_to_hex
  http-backend: allow 64-character hex names
  http-push: convert to use the_hash_algo
  builtin/pull: make hash-size independent
  builtin/am: make hash size independent
  ...
2019-04-25 16:41:17 +09:00
Junio C Hamano
dae82ecf14 Merge branch 'en/fast-import-parsing-fix'
"git fast-import" update.

* en/fast-import-parsing-fix:
  fast-import: fix erroneous handling of get-mark with empty orphan commits
  fast-import: only allow cat-blob requests where it makes sense
  fast-import: check most prominent commands first
  git-fast-import.txt: fix wording about where ls command can appear
  t9300: demonstrate bug with get-mark and empty orphan commits
2019-04-25 16:41:16 +09:00
Junio C Hamano
8baf40b5b4 Merge branch 'ab/test-lib-pass-trace2-env'
Allow tracing of Git executable while running the testsuite.

* ab/test-lib-pass-trace2-env:
  test-lib: whitelist GIT_TR2_* in the environment
2019-04-25 16:41:15 +09:00
Junio C Hamano
a5e4be2f68 Merge branch 'ab/commit-graph-fixes'
Code cleanup with more careful error checking before using data
read from the commit-graph file.

* ab/commit-graph-fixes:
  commit-graph: improve & i18n error messages
  commit-graph write: don't die if the existing graph is corrupt
  commit-graph verify: detect inability to read the graph
  commit-graph: don't pass filename to load_commit_graph_one_fd_st()
  commit-graph: don't early exit(1) on e.g. "git status"
  commit-graph: fix segfault on e.g. "git status"
  commit-graph tests: test a graph that's too small
  commit-graph tests: split up corrupt_graph_and_verify()
2019-04-25 16:41:15 +09:00
Junio C Hamano
f3c19f85c5 Merge branch 'ab/gc-reflog'
Fix various glitches in "git gc" around reflog handling.

* ab/gc-reflog:
  gc: handle & check gc.reflogExpire config
  reflog tests: assert lack of early exit with expiry="never"
  reflog tests: test for the "points nowhere" warning
  reflog tests: make use of "test_config" idiom
  gc: refactor a "call me once" pattern
  gc: convert to using the_hash_algo
  gc: remove redundant check for gc_auto_threshold
2019-04-25 16:41:14 +09:00
Junio C Hamano
4a3ed2bec6 Merge branch 'nd/checkout-m'
"git checkout -m <other>" was about carrying the differences
between HEAD and the working-tree files forward while checking out
another branch, and ignored the differences between HEAD and the
index.  The command has been taught to abort when the index and the
HEAD are different.

* nd/checkout-m:
  checkout: prevent losing staged changes with --merge
  read-tree: add --quiet
  unpack-trees: rename "gently" flag to "quiet"
  unpack-trees: keep gently check inside add_rejected_path
2019-04-25 16:41:14 +09:00
Junio C Hamano
b72e90712e Merge branch 'js/difftool-no-index'
"git difftool" can now run outside a repository.

* js/difftool-no-index:
  difftool: allow running outside Git worktrees with --no-index
  parse-options: make OPT_ARGUMENT() more useful
  difftool: remove obsolete (and misleading) comment
2019-04-25 16:41:14 +09:00
Junio C Hamano
d2dba18ced Merge branch 'pw/cherry-pick-continue'
"git cherry-pick --options A..B", after giving control back to the
user to ask help resolving a conflicted step, did not honor the
options it originally received, which has been corrected.

* pw/cherry-pick-continue:
  cherry-pick --continue: remember options
  cherry-pick: demonstrate option amnesia
  sequencer: break some long lines
2019-04-25 16:41:14 +09:00
Junio C Hamano
776f3e1fb7 Merge branch 'jk/server-info-rabbit-hole'
Code clean-up around a much-less-important-than-it-used-to-be
update_server_info() funtion.

* jk/server-info-rabbit-hole:
  update_info_refs(): drop unused force parameter
  server-info: drop objdirlen pointer arithmetic
  server-info: drop nr_alloc struct member
  server-info: use strbuf to read old info/packs file
  server-info: simplify cleanup in parse_pack_def()
  server-info: fix blind pointer arithmetic
  http: simplify parsing of remote objects/info/packs
  packfile: fix pack basename computation
  midx: check both pack and index names for containment
  t5319: drop useless --buffer from cat-file
  t5319: fix bogus cat-file argument
  pack-revindex: open index if necessary
  packfile.h: drop extern from function declarations
2019-04-25 16:41:13 +09:00
Junio C Hamano
dcd6a8c09a Merge branch 'nd/diff-parseopt-4'
Fourth batch to teach the diff machinery to use the parse-options
API.

* nd/diff-parseopt-4:
  am: avoid diff_opt_parse()
  diff --no-index: use parse_options() instead of diff_opt_parse()
  range-diff: use parse_options() instead of diff_opt_parse()
  diff.c: allow --no-color-moved-ws
  diff-parseopt: convert --color-moved-ws
  diff-parseopt: convert --[no-]color-moved
  diff-parseopt: convert --inter-hunk-context
  diff-parseopt: convert --no-prefix
  diff-parseopt: convert --line-prefix
  diff-parseopt: convert --[src|dst]-prefix
  diff-parseopt: convert --[no-]abbrev
  diff-parseopt: convert --diff-filter
  diff-parseopt: convert --find-object
  diff-parseopt: convert -O
  diff-parseopt: convert --pickaxe-all|--pickaxe-regex
  diff-parseopt: convert -S|-G
  diff-parseopt: convert -l
  diff-parseopt: convert -z
  diff-parseopt: convert --ita-[in]visible-in-index
  diff-parseopt: convert --ws-error-highlight
2019-04-25 16:41:12 +09:00
Junio C Hamano
4284497396 Merge branch 'jk/unused-params-even-more'
Code cleanup.

* jk/unused-params-even-more:
  parse_opt_ref_sorting: always use with NONEG flag
  pretty: drop unused strbuf from parse_padding_placeholder()
  pretty: drop unused "type" parameter in needs_rfc2047_encoding()
  parse-options: drop unused ctx parameter from show_gitcomp()
  fetch_pack(): drop unused parameters
  report_path_error(): drop unused prefix parameter
  unpack-trees: drop unused error_type parameters
  unpack-trees: drop name_entry from traverse_by_cache_tree()
  test-date: drop unused "now" parameter from parse_dates()
  update-index: drop unused prefix_length parameter from do_reupdate()
  log: drop unused "len" from show_tagger()
  log: drop unused rev_info from early output
  revision: drop some unused "revs" parameters
2019-04-25 16:41:12 +09:00
Junio C Hamano
579b75ad95 Merge branch 'sg/test-atexit'
Test framework update to more robustly clean up leftover files and
processes after tests are done.

* sg/test-atexit:
  t9811-git-p4-label-import: fix pipeline negation
  git p4 test: disable '-x' tracing in the p4d watchdog loop
  git p4 test: simplify timeout handling
  git p4 test: clean up the p4d cleanup functions
  git p4 test: use 'test_atexit' to kill p4d and the watchdog process
  t0301-credential-cache: use 'test_atexit' to stop the credentials helper
  tests: use 'test_atexit' to stop httpd
  git-daemon: use 'test_atexit` to stop 'git-daemon'
  test-lib: introduce 'test_atexit'
  t/lib-git-daemon: make sure to kill the 'git-daemon' process
  test-lib: fix interrupt handling with 'dash' and '--verbose-log -x'
2019-04-25 16:41:12 +09:00
Junio C Hamano
5795a75f9b Merge branch 'bp/post-index-change-hook'
A new hook "post-index-change" is called when the on-disk index
file changes, which can help e.g. a virtualized working tree
implementation.

* bp/post-index-change-hook:
  read-cache: add post-index-change hook
2019-04-25 16:41:11 +09:00
Denton Liu
57d93c1d2c t7610: add mergetool --gui tests
In 063f2bdbf7 (mergetool: accept -g/--[no-]gui as arguments,
2018-10-24), mergetool was taught the --gui option but no tests were
added to ensure that it was working properly. Add a test to ensure that
it works.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25 11:31:49 +09:00
Denton Liu
e9d309e2fc t7610: unsuppress output
The output for commands used to be suppressed by redirecting both stdout
and stderr to /dev/null. However, this should not happen since the
output is useful for debugging and, without the "-v" flag, test scripts
don't output anyway.

Unsuppress the output by removing the redirections to /dev/null.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25 11:27:53 +09:00
Jeff King
f2e875d6df t/perf: depend on perl JSON only when using --codespeed
Commit 05eb1c37ed (perf/aggregate: implement codespeed JSON output,
2018-01-05) added a dependency on the perl JSON module to show output
from aggregate.perl, but we only need it when the user asks for
--codespeed output. While the module is pretty common, it's not part of
the base system, and this dependency can get in the way of producing the
default human-readable output.

Let's bump the "use" down to a "require" in the code path that needs it,
which will be interpreted at run-time instead of compile-time. People
not using "--codespeed" won't even load the module, and anybody using it
should see the same results (including the same perl error if they don't
have it).

Note that this skips the importing step, so we'll have to fully qualify
our function call. We could accomplish the same thing in other ways.
E.g., calling JSON->import() ourselves, or wrapping "use JSON" in an
eval. Since there's only one such call, this seems like the
least-magical way of doing it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-24 10:21:06 +09:00
Jeff King
775c71e16d p5302: create the repo in each index-pack test
The p5302 script runs "index-pack --stdin" in each timing test. It does
two things to try to get good timings:

  1. we do the repo creation in a separate (non-timed) setup test, so
     that our timing is purely the index-pack run

  2. we use a separate repo for each test; this is important because the
     presence of existing objects in the repo influences the result
     (because we'll end up doing collision checks against them)

But this forgets one thing: we generally run each timed test multiple
times to reduce the impact of noise. Which means that repeats of each
test after the first will be subject to the collision slowdown from
point 2, and we'll generally just end up taking the first time anyway.

Instead, let's create the repo in the test (effectively undoing point
1). That does add a constant amount of extra work to each iteration, but
it's quite small compared to the actual effects we're interested in
measuring.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-23 09:56:44 +09:00
Junio C Hamano
39e4773daa Merge branch 'js/spell-out-options-in-tests'
The tests have been updated not to rely on the abbreviated option
names the parse-options API offers, to protect us from an
abbreviated form of an option that used to be unique within the
command getting non-unique when a new option that share the same
prefix is added.

* js/spell-out-options-in-tests:
  tests: disallow the use of abbreviated options (by default)
  tests (pack-objects): use the full, unabbreviated `--revs` option
  tests (status): spell out the `--find-renames` option in full
  tests (push): do not abbreviate the `--follow-tags` option
  t5531: avoid using an abbreviated option
  t7810: do not abbreviate `--no-exclude-standard` nor `--invert-match`
  tests (rebase): spell out the `--force-rebase` option
  tests (rebase): spell out the `--keep-empty` option
2019-04-22 11:14:47 +09:00
Junio C Hamano
078b254deb Merge branch 'nd/include-if-wildmatch'
A buglet in configuration parser has been fixed.

* nd/include-if-wildmatch:
  config: correct '**' matching in includeIf patterns
2019-04-22 11:14:46 +09:00
Junio C Hamano
8aed8034be Merge branch 'tg/stash-in-c-show-default-to-p-fix'
A regression fix.

* tg/stash-in-c-show-default-to-p-fix:
  stash: setup default diff output format if necessary
2019-04-22 11:14:45 +09:00
Junio C Hamano
0ba1ba4846 Merge branch 'js/stash-in-c-pathspec-fix'
Further fixes to "git stash" reimplemented in C.

* js/stash-in-c-pathspec-fix:
  stash: pass pathspec as pointer
  built-in stash: handle :(glob) pathspecs again
  legacy stash: fix "rudimentary backport of -q"
2019-04-22 11:14:44 +09:00
Junio C Hamano
e36adf7122 Merge branch 'ps/stash-in-c'
"git stash" rewritten in C.

* ps/stash-in-c: (28 commits)
  tests: add a special setup where stash.useBuiltin is off
  stash: optionally use the scripted version again
  stash: add back the original, scripted `git stash`
  stash: convert `stash--helper.c` into `stash.c`
  stash: replace all `write-tree` child processes with API calls
  stash: optimize `get_untracked_files()` and `check_changes()`
  stash: convert save to builtin
  stash: make push -q quiet
  stash: convert push to builtin
  stash: convert create to builtin
  stash: convert store to builtin
  stash: convert show to builtin
  stash: convert list to builtin
  stash: convert pop to builtin
  stash: convert branch to builtin
  stash: convert drop and clear to builtin
  stash: convert apply to builtin
  stash: mention options in `show` synopsis
  stash: add tests for `git stash show` config
  stash: rename test cases to be more descriptive
  ...
2019-04-22 11:14:43 +09:00
Jonathan Tan
f3534c98e4 worktree: update is_bare heuristics
When "git branch -D <name>" is run, Git usually first checks if that
branch is currently checked out. But this check is not performed if the
Git directory of that repository is not at "<repo>/.git", which is the
case if that repository is a submodule that has its Git directory stored
as "super/.git/modules/<repo>", for example. This results in the branch
being deleted even though it is checked out.

This is because get_main_worktree() in worktree.c sets is_bare on a
worktree only using the heuristic that a repo is bare if the worktree's
path does not end in "/.git", and not bare otherwise. This is_bare code
was introduced in 92718b7438 ("worktree: add details to the worktree
struct", 2015-10-08), following a pre-core.bare heuristic. This patch
does 2 things:

 - Teach get_main_worktree() to use is_bare_repository() instead,
   introduced in 7d1864ce67 ("Introduce is_bare_repository() and
   core.bare configuration variable", 2007-01-07) and updated in
   e90fdc39b6 ("Clean up work-tree handling", 2007-08-01). This solves
   the "git branch -D <name>" problem described above. However...

 - If a repository has core.bare=1 but the "git" command is being run
   from one of its secondary worktrees, is_bare_repository() returns
   false (which is fine, since there is a worktree available). However,
   treating the main worktree as non-bare when it is bare causes issues:
   for example, failure to delete a branch from a secondary worktree
   that is referred to by a main worktree's HEAD, even if that main
   worktree is bare.

   In order to avoid that, also check core.bare when setting is_bare. If
   core.bare=1, trust it, and otherwise, use is_bare_repository().

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-21 13:58:55 +09:00
Jeff King
fe6f2b081f t5304: add a test for pruning with bitmaps
Commit fde67d6896 (prune: use bitmaps for reachability traversal,
2019-02-13) uses bitmaps for pruning when they're available, but only
covers this functionality in the t/perf tests. This makes a kind of
sense, since the point is that the behaviour is indistinguishable before
and after the patch, just faster.

But since the bitmap code path is not exercised at all in the regular
test suite, it leaves us open to a regression where the behavior does in
fact change. The most thorough way to test that would be running the
whole suite with bitmaps enabled. But we don't yet have a way to do
that, and anyway it's expensive to do so. Let's at least add a basic
test that exercises this path and make sure we prune an object we should
(and not one that we shouldn't).

That would hopefully catch the most obvious breakages early.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:22:32 +09:00
Johannes Schindelin
b7ce24d095 Turn git serve into a test helper
The `git serve` built-in was introduced in ed10cb952d (serve:
introduce git-serve, 2018-03-15) as a backend to serve Git protocol v2,
probably originally intended to be spawned by `git upload-pack`.

However, in the version that the protocol v2 patches made it into core
Git, `git upload-pack` calls the `serve()` function directly instead of
spawning `git serve`; The only reason in life for `git serve` to survive
as a built-in command is to provide a way to test the protocol v2
functionality.

Meaning that it does not even have to be a built-in that is installed
with end-user facing Git installations, but it can be a test helper
instead.

Let's make it so.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:03:24 +09:00
Johannes Schindelin
6ea18fffb0 test-tool: handle the -C <directory> option just like git
In preparation for moving `git serve` into `test-tool` (because it
really is only used by the test suite), we teach the `test-tool` the
useful trick to change the working directory before running the test
command, which will avoid introducing subshells in the test code.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:03:24 +09:00
Denton Liu
1a2b985fb3 cherry-pick/revert: add scissors line on merge conflict
Fix a bug where the scissors line is placed after the Conflicts:
section, in the case where a merge conflict occurs and
commit.cleanup = scissors.

Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 12:05:36 +09:00
Denton Liu
1055997e2f merge: add scissors line on merge conflict
This fixes a bug where the scissors line is placed after the Conflicts:
section, in the case where a merge conflict occurs and
commit.cleanup = scissors.

Next, if commit.cleanup = scissors is specified, don't produce a
scissors line in commit if one already exists in the MERGE_MSG file.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 13:49:29 +09:00
Denton Liu
d540b70c85 merge: cleanup messages like commit
This change allows git-merge messages to be cleaned up with the
commit.cleanup configuration or --cleanup option, just like how
git-commit does it.

We also give git-pull the option of --cleanup so that it can also take
advantage of this change.

Finally, add testing to ensure that messages are properly cleaned up.
Note that some newlines that were added to the commit message were
removed so that if a file were read via -F, it would be copied
faithfully.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 13:49:29 +09:00
Denton Liu
94ca361bfb t7502: clean up style
Refactor out Git commands that were upstream of a pipe. Remove spaces
after "> ". Indent here-docs appropriately. Convert echo chains to use
the test_write_lines function. Refactor 'sign off' test to use test_cmp
instead of comparing variables.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 13:49:28 +09:00
Denton Liu
b720e1e3c3 t7604: clean up style
Before, we had some Git commands which were upstream of the pipe. This
meant that if it produced an error, it would've gone unnoticed. Refactor
to place Git commands on their own.

Also, while we're at it, remove spaces after redirection operators.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 13:49:28 +09:00
Denton Liu
5caab8de8a t3507: clean up style
Remove space after redirection operators for style. Also, remove a git
command which was upstream of a pipe. Finally, let grep and sed open
their own input instead of letting the shell redirect the input.

Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 13:49:28 +09:00
Denton Liu
b510f0beab t7600: clean up style
Clean up the 'merge --squash c3 with c7' test by removing some
unnecessary braces and removing a pipe.

Also, generally cleanup style by unindenting a here-doc, removing stray
spaces after a redirection operator and allowing sed to open its own
input instead of redirecting input from the shell.

Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 13:49:28 +09:00
Phillip Wood
d74f3e5811 sequencer: fix cleanup with --signoff and -x
Before commit 356ee4659b ("sequencer: try to commit without forking 'git
commit'", 2017-11-24) when --signoff or -x were given on the command
line the commit message was cleaned up with --cleanup=space or
commit.cleanup if it was set. Unfortunately this behavior was lost when
I implemented committing without forking. Fix this and add some tests to
catch future regressions.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 13:48:42 +09:00
Jonathan Tan
6e98305985 clone: send server options when using protocol v2
Commit 5e3548ef16 ("fetch: send server options when using protocol v2",
2018-04-24) taught "fetch" the ability to send server options when using
protocol v2, but not "clone". This ability is triggered by "-o" or
"--server-option".

Teach "clone" the same ability, except that because "clone" already
has "-o" for another parameter, teach "clone" only to receive
"--server-option".

Explain in the documentation, both for clone and for fetch, that server
handling of server options are server-specific. This is similar to
receive-pack's handling of push options - currently, they are just sent
to hooks to interpret as they see fit.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 10:53:51 +09:00
Jonathan Tan
35eb8240b0 transport: die if server options are unsupported
Server options were added in commit 5e3548ef16 ("fetch: send server
options when using protocol v2", 2018-04-24), supported only for
protocol version 2. But if the user specifies server options, and the
protocol version being used doesn't support them, the server options are
silently ignored.

Teach any transport users to die instead in this situation, just like
how "push" dies if push options are provided when the server doesn't
support them.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 10:53:51 +09:00
Damien Robert
c646d0934e ref-filter: use correct branch for %(push:track)
In ref-filter.c, when processing the atom %(push:track), the
ahead/behind values are computed using `stat_tracking_info` which refers
to the upstream branch.

Fix that by introducing a new flag `for_push` in `stat_tracking_info`
in remote.c, which does the same thing but for the push branch.
Update the few callers of `stat_tracking_info` to handle this flag. This
ensure that whenever we use this function in the future, we are careful
to specify is this should apply to the upstream or the push branch.

This bug was not detected in t/t6300-for-each-ref.sh because in the test
for push:track, both the upstream and the push branches were behind by 1
from the local branch. Change the test so that the upstream branch is
behind by 1 while the push branch is ahead by 1. This allows us to test
that %(push:track) refers to the correct branch.

This changes the expected value of some following tests (by introducing
new references), so update them too.

Signed-off-by: Damien Robert <damien.olivier.robert+git@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18 09:17:41 +09:00
Phillip Wood
4a72486de9 fix cherry-pick/revert status after commit
If the user commits a conflict resolution using `git commit` in the
middle of a sequence of cherry-picks/reverts then `git status` missed
the fact that a cherry-pick/revert is still in progress.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-17 11:31:39 +09:00
Phillip Wood
b07d9bfd17 commit/reset: try to clean up sequencer state
When cherry-picking or reverting a sequence of commits and if the final
pick/revert has conflicts and the user uses `git commit` to commit the
conflict resolution and does not run `git cherry-pick --continue` then
the sequencer state is left behind. This can cause problems later. In my
case I cherry-picked a sequence of commits the last one of which I
committed with `git commit` after resolving some conflicts, then a while
later, on a different branch I aborted a revert which rewound my HEAD to
the end of the cherry-pick sequence on the previous branch. Avoid this
potential problem by removing the sequencer state if we're committing or
resetting the final pick in a sequence.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-17 11:31:37 +09:00
Junio C Hamano
5866e58666 Merge branch 'sg/t5318-cleanup'
Code cleanup.

* sg/t5318-cleanup:
  t5318-commit-graph: remove unused variable
2019-04-16 19:28:12 +09:00
Junio C Hamano
d11650dcbd Merge branch 'jt/t5551-protocol-v2-does-not-have-half-auth'
Test update.

* jt/t5551-protocol-v2-does-not-have-half-auth:
  t5551: mark half-auth no-op fetch test as v0-only
2019-04-16 19:28:12 +09:00
Junio C Hamano
3151a5fc45 Merge branch 'jk/http-walker-status-fix'
dumb-http walker has been updated to share more error recovery
strategy with the normal codepath.

* jk/http-walker-status-fix:
  http: use normalize_curl_result() instead of manual conversion
  http: normalize curl results for dumb loose and alternates fetches
  http: factor out curl result code normalization
2019-04-16 19:28:11 +09:00
Junio C Hamano
2e08c892a7 Merge branch 'jk/refs-double-abort'
A corner case bug in the refs API has been corrected.

* jk/refs-double-abort:
  refs/files-backend: don't look at an aborted transaction
  refs/files-backend: handle packed transaction prepare failure
2019-04-16 19:28:11 +09:00
Junio C Hamano
2850232a21 Merge branch 'tz/completion'
The completion helper code now pays attention to repository-local
configuration (when available), which allows --list-cmds to honour
a repository specific setting of completion.commands, for example.

* tz/completion:
  completion: use __git when calling --list-cmds
  completion: fix multiple command removals
  t9902: test multiple removals via completion.commands
  git: read local config in --list-cmds
2019-04-16 19:28:09 +09:00
Junio C Hamano
3221177559 Merge branch 'tz/t4038-bash-redirect-target-workaround'
Work-around extra warning from bash in our tests.

* tz/t4038-bash-redirect-target-workaround:
  t4038-diff-combined: quote paths with whitespace
2019-04-16 19:28:09 +09:00
Junio C Hamano
4f3036cfa1 Merge branch 'ab/drop-scripted-rebase'
Retire scripted "git rebase" implementation.

* ab/drop-scripted-rebase:
  rebase: remove the rebase.useBuiltin setting
2019-04-16 19:28:09 +09:00
Junio C Hamano
67e3ec10d0 Merge branch 'jk/perf-lib-tee'
Code cleanup in the test framework.

* jk/perf-lib-tee:
  perf-lib.sh: rely on test-lib.sh for --tee handling
2019-04-16 19:28:09 +09:00
Junio C Hamano
04f879a7fb Merge branch 'nd/checkout-f-while-conflicted-fix'
"git checkout -f <branch>" while the index has an unmerged path
incorrectly left some paths in an unmerged state, which has been
corrected.

* nd/checkout-f-while-conflicted-fix:
  unpack-trees: fix oneway_merge accidentally carry over stage index
2019-04-16 19:28:07 +09:00
Junio C Hamano
17c8f1f2ed Merge branch 'jc/format-patch-error-check'
"git format-patch" used overwrite an existing patch/cover-letter
file.  A new "--no-clobber" option stops it.

* jc/format-patch-error-check:
  format-patch: notice failure to open cover letter for writing
  builtin/log: downcase the beginning of error messages
2019-04-16 19:28:06 +09:00
Junio C Hamano
e197980eb2 Merge branch 'js/get-short-oid-drop-cache'
A corner-case object name ambiguity while the sequencer machinery
is working (e.g. "rebase -i -x") has been (half) fixed.

* js/get-short-oid-drop-cache:
  get_oid(): when an object was not found, try harder
  sequencer: move stale comment into correct location
  sequencer: improve error message when an OID could not be parsed
  rebase -i: demonstrate obscure loose object cache bug
2019-04-16 19:28:06 +09:00
Junio C Hamano
63643865d2 Merge branch 'js/init-db-update-for-mingw'
"git init" forgot to read platform-specific repository
configuration, which made Windows port to ignore settings of
core.hidedotfiles, for example.

* js/init-db-update-for-mingw:
  mingw: respect core.hidedotfiles = false in git-init again
2019-04-16 19:28:05 +09:00
Junio C Hamano
24d73d2a4c Merge branch 'jt/test-protocol-version'
Help developers by making it easier to run most of the tests under
different versions of over-the-wire protocols.

* jt/test-protocol-version:
  t5552: compensate for v2 filtering ref adv.
  tests: fix protocol version for overspecifications
  t5700: only run with protocol version 1
  t5512: compensate for v0 only sending HEAD symrefs
  t5503: fix overspecification of trace expectation
  tests: always test fetch of unreachable with v0
  t5601: check ssh command only with protocol v0
  tests: define GIT_TEST_PROTOCOL_VERSION
2019-04-16 19:28:03 +09:00
Nguyễn Thái Ngọc Duy
d9b8b8f896 submodule-config.c: use repo_get_oid for reading .gitmodules
Since 76e9bdc437 (submodule: support reading .gitmodules when it's not
in the working tree - 2018-10-25), every time you do

    git grep --recurse-submodules

you are likely to see one warning line per submodule (unless all those
submodules also have submodules). On a superproject with plenty of
submodules (I've seen one with 67) this is really annoying.

The warning was there because we could not resolve extended SHA-1
syntax on a submodule. We can now. Make use of the new API and get rid
of the warning.

It would be even better if config_with_options() supports multiple
repositories too. But one step at a time.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 18:56:53 +09:00
Jeff King
fc78915674 packfile: fix pack basename computation
When we have a multi-pack-index that covers many packfiles, we try to
avoid opening the .idx for those packfiles. To do that we feed the pack
name to midx_contains_pack(). But that function wants to see only the
basename, which we compute using strrchr() to find the final slash. But
that leaves an extra "/" at the start of our string.

We can fix this by incrementing the pointer. That also raises the
question of what to do when the name does not have a '/' at all. This
should generally not happen (we always find files in "pack/"), but it
doesn't hurt to be defensive here.

Let's wrap all of that up in a helper function and make it publicly
available, since a later patch will need to use it, too.

The tests don't notice because there's nothing about opening those .idx
files that would cause us to give incorrect output. It's just a little
slower. The new test checks this case by corrupting the covered .idx,
and then making sure we don't complain about it.

We also have to tweak t5570, which intentionally corrupts a .idx file
and expects us to notice it. When run with GIT_TEST_MULTI_PACK_INDEX,
this will fail since we now will (correctly) not bother opening the .idx
at all. We can fix that by unconditionally dropping any midx that's
there, which ensures we'll have to read the .idx.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 16:58:21 +09:00
Jeff King
5670ad98a8 t5319: drop useless --buffer from cat-file
The cat-file --buffer option is the default already when using
--batch-all-objects. It doesn't hurt to specify it, but it's nice for
the test scripts to model good usage.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 16:58:21 +09:00
Jeff King
b4a14394af t5319: fix bogus cat-file argument
There's no such argument as "--unsorted"; it's spelled "--unordered".
But our test failed to notice that cat-file didn't run at all because:

  1. It lost the exit code of git on the left-hand side of a pipe.

  2. It was comparing two runs of the broken invocation with and without
     a particular config variable (and indeed, both cases produced no
     output!).

Let's fix the option, but also tweak the helper function to check the
exit code.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 16:58:21 +09:00
Jeff Hostetler
bce9db6de9 trace2: use system/global config for default trace2 settings
Teach git to read the system and global config files for
default Trace2 settings.  This allows system-wide Trace2 settings to
be installed and inherited to make it easier to manage a collection of
systems.

The original GIT_TR2* environment variables are loaded afterwards and
can be used to override the system settings.

Only the system and global config files are used.  Repo and worktree
local config files are ignored.  Likewise, the "-c" command line
arguments are also ignored.  These limits are for performance reasons.

(1) For users not using Trace2, there should be minimal overhead to
detect that Trace2 is not enabled.  In particular, Trace2 should not
allocate lots of otherwise unused data strucutres.

(2) For accurate performance measurements, Trace2 should be initialized
as early in the git process as possible, and before most of the normal
git process initialization (which involves discovering the .git directory
and reading a hierarchy of config files).

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:06 +09:00
Jeff Hostetler
39f4317744 trace2: add absolute elapsed time to start event
Add elapsed process time to "start" event to measure
the performance of early process startup.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:06 +09:00
Johannes Schindelin
5afb2ce4cd remote-testgit: move it into the support directory for t5801
The `git-remote-testgit` script is really only used in
`t5801-remote-helpers.sh`. It does not even contain any `@@<MAGIC>@@`
placeholders that would need to be interpolated via `make
git-remote-testgit`.

Let's just move it to a new home, decluttering the top-level directory
and clarifying that this is just a test helper, not an official Git
command that we would want to ever support.

Suggested by Ævar Arnfjörð Bjarmason.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 14:30:04 +09:00
Jeff King
34066f0661 fetch: do not consider peeled tags as advertised tips
Our filter_refs() function accidentally considers the target of a peeled
tag to be advertised by the server, even though upload-pack on the
server side does not consider it so. This can result in the client
making a bogus fetch to the server, which will end with the server
complaining "not our ref". Whereas the correct behavior is for the
client to notice that the server will not allow the request and error
out immediately.

So as bugs go, this is not very serious (the outcome is the same either
way -- the fetch fails). But it's worth making the logic here correct
and consistent with other related cases (e.g., fetching an oid that the
server did not mention at all).

The crux of the issue comes from fdb69d33c4 (fetch-pack: always allow
fetching of literal SHA1s, 2017-05-15). After that, the strategy of
filter_refs() is basically:

  - for each advertised ref, try to match it with a "sought" ref
    provided by the user. Skip any malformed refs (which includes
    peeled values like "refs/tags/foo^{}"), and place any unmatched
    items onto the unmatched list.

  - if there are unmatched sought refs, then put all of the advertised
    tips into an oidset, including the unmatched ones.

  - for each sought ref, see if it's in the oidset, in which case it's
    legal for us to ask the server for it

The problem is in the second step. Our list of unmatched refs includes
the peeled refs, even though upload-pack does not allow them to be
directly fetched. So the simplest fix would be to exclude them during
that step.

However, we can observe that the unmatched list isn't used for anything
else, and is freed at the end. We can just free those malformed refs
immediately. That saves us having to check each ref a second time to see
if it's malformed.

Note that this code only kicks in when "strict" is in effect. I.e., if
we are using the v0 protocol and uploadpack.allowReachableSHA1InWant is
not in effect. With v2, all oids are allowed, and we do not bother
creating or consulting the oidset at all. To future-proof our test
against the upcoming GIT_TEST_PROTOCOL_VERSION flag, we'll manually mark
it as a v0-only test.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 14:00:52 +09:00
Jeff King
533ddba47e pkt-line: prepare buffer before handling ERR packets
Since 2d103c31c2 (pack-protocol.txt: accept error packets in any
context, 2018-12-29), the pktline code will detect an ERR packet and die
automatically, saving the caller from dealing with it. But we do so too
early in the function, before we have terminated the buffer with a NUL.

As a result, passing the ERR message to die() may result in us printing
random cruft from a previous packet. This doesn't trigger memory tools
like ASan because we reuse the same buffer over and over (so the
contents are valid and initialized; they're just stale).

We can see demonstrate this by tightening the regex we use to match the
error message in t5516; without this patch, git-fetch will accidentally
print the capabilities from the (much longer) initial packet we
received.

By moving the ERR code later in the function we get a few other
benefits, too:

  - we'll now chomp any newline sent by the other side (which is what we
    want, since die() will add its own newline)

  - we'll now mention the ERR packet with GIT_TRACE_PACKET

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 14:00:51 +09:00
Jeff King
014ade7484 upload-pack: send ERR packet for non-tip objects
Commit bdb31eada7 (upload-pack: report "not our ref" to client,
2017-02-23) catches the case where a client asks for an object we don't
have, and issues a message that the client can show to the user (in
addition to dying and writing to stderr).

There's a similar case (with the same message) when the client asks for
an object which we _do_ have, but which isn't a ref tip (or isn't
reachable, when uploadpack.allowReachableSHA1InWant is true). Let's give
that one the same treatment, for the same reason (namely that it's more
informative to the client than just hanging up, since they won't see our
stderr over some protocols).

There are two tests here. We cover it most directly in t5530 by invoking
upload-pack, which matches the existing "not our ref" test.

But a more end-to-end check is that "git fetch" actually shows the
message to the client. We're already checking in t5516 that this case
fails, so we can just check stderr there, too. Note that even after we
started ignoring SIGPIPE in 8bf4becf0c, this could in theory still be
racy as described in that commit (because we die() on write failures
before pumping the connection for any ERR packets).

In practice this should be OK for this case. The server will not
actually check reachability until it has received our whole group of
"want" lines. And since we have no objects in the repository, we won't
send any "have" lines, meaning we're always waiting to read the server
response.

Note also that this case cannot happen in the v2 protocol, since it
allows any available object to be requested. However, we don't have to
take any steps to protect against the upcoming GIT_TEST_PROTOCOL_VERSION
in our tests:

  - the tests in t5516 would already need to be skipped under v2, and
    that is covered by ab0c5f5096 (tests: always test fetch of
    unreachable with v0, 2019-02-25)

  - the tests in t5530 invoke upload-pack directly, which will continue
    to default to v0. Eventually we may have a test setting which uses
    v2 even for bare upload-pack calls, but we can't override it here
    until we know what the setting looks like.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 14:00:51 +09:00
Jeff King
6963a4e4e1 t5530: check protocol response for "not our ref"
Back in 9f9aa76130 (upload-pack: Improve error message when bad ref
requested, 2010-07-31), we added a test to make sure that we die with a
sensible message when the client asks for an object we don't have.

Much later, in bdb31eada7 (upload-pack: report "not our ref" to client,
2017-02-23), we started reporting that information via an "ERR" line in
the protocol. Let's check that part, as well.

While we're touching this test, let's drop the "-q" on the grep calls.
Our usual test style just relies on --verbose to control output.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 14:00:51 +09:00
Jeff King
98024d1cb6 t5516: drop ok=sigpipe from unreachable-want tests
We annotated our test_must_fail calls in 8bf4becf0c (add "ok=sigpipe" to
test_must_fail and use it to fix flaky tests, 2015-11-27) because the
abrupt hangup of the server meant that we'd sometimes fail on read() and
sometimes get SIGPIPE on write().

But since 143588949c (fetch: ignore SIGPIPE during network operation,
2019-03-03), we make sure that we end up with a real die(), and our
tests no longer need to work around the race.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 14:00:51 +09:00
Junio C Hamano
f88b9cb603 gettext tests: export the restored GIT_TEST_GETTEXT_POISON
6cdccfce ("i18n: make GETTEXT_POISON a runtime option", 2018-11-08)
made the gettext-poison test a runtime option (which was a good
move) and adjusted the test framework so that Git commands we run as
part of the framework, as opposed to the ones that are part of the
test proper, are not affected by the setting.  The original value
for the GIT_TEST_GETTEXT_POISON environment variable is saved away
in another variable and gets unset, and then later the saved value
is restored to the environment variable.

But the code forgot to export the variable again, which is necessary
to restore the "export" bit that was lost when the variable was unset.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 13:57:07 +09:00
Johannes Schindelin
3889149619 t9822: skip tests if file names cannot be ISO-8859-1 encoded
Most notably, it seems that macOS' APFS does not allow that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 12:03:14 +09:00
Nguyễn Thái Ngọc Duy
a282f5a906 submodule foreach: fix "<command> --quiet" not being respected
Robin reported that

    git submodule foreach --quiet git pull --quiet origin

is not really quiet anymore [1]. "git pull" behaves as if --quiet is not
given.

This happens because parseopt in submodule--helper will try to parse
both --quiet options as if they are foreach's options, not git-pull's.
The parsed options are removed from the command line. So when we do
pull later, we execute just this

    git pull origin

When calling submodule helper, adding "--" in front of "git pull" will
stop parseopt for parsing options that do not really belong to
submodule--helper foreach.

PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should
never see unknown options or something has gone wrong. There are also
a couple usage string update while I'm looking at them.

While at it, I also add "--" to other subcommands that pass "$@" to
submodule--helper. "$@" in these cases are paths and less likely to be
--something-like-this. But the point still stands, git-submodule has
parsed and classified what are options, what are paths. submodule--helper
should never consider paths passed by git-submodule to be options even
if they look like one.

The test case is also contributed by Robin.

[1] it should be quiet before fc1b9243cd (submodule: port submodule
    subcommand 'foreach' from shell to C, 2018-05-10) because parseopt
    can't accidentally eat options then.

Reported-by: Robin H. Johnson <robbat2@gentoo.org>
Tested-by: Robin H. Johnson <robbat2@gentoo.org>
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 11:58:42 +09:00
Johannes Schindelin
b02e7d5d70 tests: disallow the use of abbreviated options (by default)
Git's command-line parsers support uniquely abbreviated options, e.g.
`git init --ba` would automatically expand `--ba` to `--bare`.

This is a very convenient feature in every day life for Git users, in
particular when tab completion is not available.

However, it is not a good idea to rely on that in Git's test suite, as
something that is a unique abbreviation of a command line option today
might no longer be a unique abbreviation tomorrow.

For example, if a future contribution added a new mode
`git init --babyproofing` and a previously-introduced test case used the
fact that `git init --ba` expanded to `git init --bare`, that future
contribution would now have to touch seemingly unrelated tests just to
keep the test suite from failing.

So let's disallow abbreviated options in the test suite by default.

Note: for ease of implementation, this patch really only touches the
`parse-options` machinery: more and more hand-rolled option parsers are
converted to use that internal API, and more and more scripts are
converted to built-ins (naturally using the parse-options API, too), so
in practice this catches most issues, and is definitely the biggest bang
for the buck.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 11:54:04 +09:00
Christian Couder
ee521ec4cb replace: peel tag when passing a tag first to --graft
When passing a tag as the first argument to `git replace --graft`,
it can be useful to accept it and use the underlying commit as a
the commit that will be replaced.

This already works for lightweight tags, but unfortunately
for annotated tags we have been using the hash of the tag object
instead of the hash of the underlying commit.

Especially we would pass the hash of the tag object to
replace_object_oid() where we would likely fail with an error
like:

"error: Objects must be of the same type.
'annotated_replaced_object' points to a replaced object of type 'tag'
while 'replacement' points to a replacement object of type 'commit'."

This patch fixes that by using the hash of the underlying commit
when an annotated tag is passed.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 11:33:23 +09:00
Christian Couder
f8e44a81bf replace: peel tag when passing a tag as parent to --graft
When passing a tag as a parent argument to `git replace --graft`,
it can be useful to accept it and use the underlying commit as a
parent.

This already works for lightweight tags, but unfortunately
for annotated tags we have been using the hash of the tag object
instead of the hash of the underlying commit as a parent in the
replacement object we create.

This created invalid objects, but the replace succeeded even if
it showed an error like:

error: object A is a tag, not a commit

This patch fixes that by using the hash of the underlying commit
when an annotated tag is passed.

While at it, let's also update an error message to make it
clearer.

Reviewed-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 11:31:42 +09:00
brian m. carlson
74d76a1701 send-email: default to quoted-printable when CR is present
In 7a36987fff ("send-email: add an auto option for transfer encoding",
2018-07-08), git send-email learned how to automatically determine the
transfer encoding for a patch. However, the only criterion considered
was the length of the lines.

Another case we need to consider is that of carriage returns. Because
emails have CRLF endings when canonicalized, we don't want to write raw
carriage returns into a patch, lest they be stripped off as an artifact
of the transport. Ensure that we choose quoted-printable encoding if the
patch we're sending contains carriage returns.

Note that we are guaranteed to always correctly encode carriage returns
when writing quoted-printable since we explicitly specify the line
ending as "\n", forcing MIME::QuotedPrint to encode our carriage return
as "=0D".

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-14 11:47:03 +09:00
Jeff King
73a5faf017 test-prio-queue: use xmalloc
test-prio-queue.c doesn't check the return value of malloc, and could
segfault.

It's unlikely for this to matter in practice; it's a small allocation,
and this code isn't even installed alongside the rest of Git. But let's
use xmalloc(), which makes auditing for other accidental uses of bare
malloc() easier.

Reported-by: 王健强 <jianqiang.wang@securitygossip.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-12 13:34:17 +09:00