Commit Graph

14739 Commits

Author SHA1 Message Date
Kevin Daudt
99e9ab54ab t0028: fix wrong octal values for BOM in setup
The setup code uses octal values with printf to generate a BOM for
UTF-16/32 BE/LE. It specifically uses '\777' to emit a 0xff byte. This
relies on the fact that most shells truncate the value above 0o377.

Ash however interprets '\777' as '\77' + a literal '7', resulting in an
invalid BOM.

Fix this by using the proper value of 0xff: '\377'.

Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11 14:46:36 -08:00
SZEDER Gábor
7d661e5ed1 test-lib: fix non-portable pattern bracket expressions
Use a '!' character to start a non-matching pattern bracket
expression, as specified by POSIX in Shell Command Language section
2.13.1 Patterns Matching a Single Character [1].

I used '^' instead in three places in the previous three commits, to
verify that the arguments of the '--stress=' and '--stress-limit='
options and the values of various '*_PORT' environment variables are
valid numbers.  With certain shells, at least with dash (upstream and
in Ubuntu 14.04) and mksh, this led to various undesired behaviors:

  # error message in case of a valid number
  $ ~/src/dash/src/dash ./t3903-stash.sh --stress=8
  error: --stress=<N> requires the number of jobs to run

  # not the expected error message
  $ ~/src/dash/src/dash ./t3903-stash.sh --stress=foo
  ./t3903-stash.sh: 238: test: Illegal number: foo

  # no error message at all?!
  $ mksh ./t3903-stash.sh --stress=foo
  $ echo $?
  0

Some other shells, e.g. Bash (even in posix mode), ksh, dash in Ubuntu
16.04 or later, are apparently happy to accept '^' just as well.

[1] http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_13

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11 14:34:36 -08:00
Junio C Hamano
11f470aee7 test: caution on our version of 'yes'
During a review of a patch, we noticed that we use our own imitation
of 'yes' with the limit of 99 lines.  It is very tempting to lift this
arbitrary limit, but the limit is there for a reason.

Add an in-code comment to prevent future developers from wasting
their time.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11 12:31:19 -08:00
Junio C Hamano
0be2f36fdf Merge branch 'ld/git-p4-remove-flakey-test'
A flakey "p4" test has been removed.

* ld/git-p4-remove-flakey-test:
  git-p4: remove ticket expiry test
2019-02-08 20:44:52 -08:00
Junio C Hamano
b966813e71 Merge branch 'js/rebase-i-redo-exec-fix'
For "rebase -i --reschedule-failed-exec", we do not want the "-y"
shortcut after all.

* js/rebase-i-redo-exec-fix:
  Revert "rebase: introduce a shortcut for --reschedule-failed-exec"
2019-02-08 20:44:52 -08:00
Junio C Hamano
fd357c4270 Merge branch 'js/smart-http-detect-remote-error'
Some errors from the other side coming over smart HTTP transport
were not noticed, which has been corrected.

* js/smart-http-detect-remote-error:
  t5551: test server-side ERR packet
  remote-curl: tighten "version 2" check for smart-http
  remote-curl: refactor smart-http discovery
2019-02-08 20:44:51 -08:00
Junio C Hamano
154f22014b Merge branch 'tz/gpg-test-fix'
Test fix.

* tz/gpg-test-fix:
  t/lib-gpg: drop redundant killing of gpg-agent
  t/lib-gpg: quote path to ${GNUPGHOME}/trustlist.txt
2019-02-08 20:44:51 -08:00
Junio C Hamano
d379d46419 Merge branch 'os/rebase-runs-post-checkout-hook'
Test fix.

* os/rebase-runs-post-checkout-hook:
  t5403: correct bash ambiguous redirect error in subtest 8 by quoting $GIT_DIR
2019-02-08 20:44:50 -08:00
SZEDER Gábor
76e27fbfd9 test-lib: make '--stress' more bisect-friendly
Let's suppose that a test somehow becomes flaky between 'master' and
'pu', and tends to fail within the first 50 repetitions when run with
'--stress'.  In such a case we could use 'git bisect' to find the
culprit: if the test script fails with '--stress', then the commit is
definitely bad, but if it survives, say, 300 repetitions, then we could
consider it good with reasonable confidence.

Unfortunately, all this could only be done manually, because
'--stress' would run the test script repeatedly for all eternity on a
good commit, and it would exit with success even when it found a
failure on a bad commit.

So let's make '--stress' usable with 'git bisect run':

  - Make it exit with failure if a failure is found.

  - Add the '--stress-limit=<N>' option to repeat the test script
    at most N times in each of the parallel jobs, and exit with
    success when the limit is reached.

And then we could simply run something like:

  $ git bisect start origin/pu master
  $ git bisect run sh -c 'make && cd t &&
                          ./t1234-foo.sh --stress --stress-limit=300'

Sure, as a brand new feature it won't be any useful right now, but in
a release or three most cooking topics will already contain this, so
we could automatically bisect at least newly introduced flakiness.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08 11:57:59 -08:00
Randall S. Becker
a250d418c0 t5403: correct bash ambiguous redirect error in subtest 8 by quoting $GIT_DIR
The embedded blanks in the full path of the test git repository cased bash
to generate an ambugious redirect error.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08 11:53:49 -08:00
Todd Zullinger
2e285e7803 t/lib-gpg: drop redundant killing of gpg-agent
In 53fc999306 ("gpg-interface t: extend the existing GPG tests with
GPGSM", 2018-07-20), the gpgconf call which kills gpg-agent was copied
from the existing gpg setup code.

The reason for killing gpg-agent is given in 29ff1f8f74 ("t: lib-gpg:
flush gpg agent on startup", 2017-07-20):

  When running gpg-relevant tests, a gpg-daemon is spawned for each
  GNUPGHOME used. This daemon may stay running after the test and cache
  file descriptors for the trash directories, even after the trash
  directory is removed. This leads to ENOENT errors when attempting to
  create files if tests are run multiple times.

  Add a cleanup script to force flushing the gpg-agent for that GNUPGHOME
  (if any) before setting up the GPG relevant-environment.

Killing gpg-agent once per test is sufficient.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08 10:01:46 -08:00
Todd Zullinger
ddf3a1152d t/lib-gpg: quote path to ${GNUPGHOME}/trustlist.txt
When gpgsm is installed, lib-gpg.sh attempts to update trustlist.txt to
relax the checking of some root certificate requirements.  The path to
"${GNUPGHOME}" contains spaces which cause an "ambiguous redirect"
warning when bash is used to run the tests:

  $ bash t7030-verify-tag.sh
  /git/t/lib-gpg.sh: line 66: ${GNUPGHOME}/trustlist.txt: ambiguous redirect
  ok 1 - create signed tags
  ok 2 # skip create signed tags x509  (missing GPGSM)
  ...

No warning is issued when using bash called as /bin/sh, dash, or mksh.

Quote the path to ensure the redirect works as intended and sets the
GPGSM prereq.  While we're here, drop the space after ">>".

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08 10:01:39 -08:00
Elijah Newren
d76ce4f734 log,diff-tree: add --combined-all-paths option
The combined diff format for merges will only list one filename, even if
rename or copy detection is active.  For example, with raw format one
might see:

  ::100644 100644 100644 fabadb8 cc95eb0 4866510 MM	describe.c
  ::100755 100755 100755 52b7a2d 6d1ac04 d2ac7d7 RM	bar.sh
  ::100644 100644 100644 e07d6c5 9042e82 ee91881 RR	phooey.c

This doesn't let us know what the original name of bar.sh was in the
first parent, and doesn't let us know what either of the original names
of phooey.c were in either of the parents.  In contrast, for non-merge
commits, raw format does provide original filenames (and a rename score
to boot).  In order to also provide original filenames for merge
commits, add a --combined-all-paths option (which must be used with
either -c or --cc, and is likely only useful with rename or copy
detection active) so that we can print tab-separated filenames when
renames are involved.  This transforms the above output to:

  ::100644 100644 100644 fabadb8 cc95eb0 4866510 MM	desc.c	desc.c	desc.c
  ::100755 100755 100755 52b7a2d 6d1ac04 d2ac7d7 RM	foo.sh	bar.sh	bar.sh
  ::100644 100644 100644 e07d6c5 9042e82 ee91881 RR	fooey.c	fuey.c	phooey.c

Further, in patch format, this changes the from/to headers so that
instead of just having one "from" header, we get one for each parent.
For example, instead of having

  --- a/phooey.c
  +++ b/phooey.c

we would see

  --- a/fooey.c
  --- a/fuey.c
  +++ b/phooey.c

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-07 20:15:25 -08:00
Junio C Hamano
a1e19004e1 Merge branch 'ss/describe-dirty-in-the-right-directory'
"git --work-tree=$there --git-dir=$here describe --dirty" did not
work correctly as it did not pay attention to the location of the
worktree specified by the user by mistake, which has been
corrected.

* ss/describe-dirty-in-the-right-directory:
  t6120: test for describe with a bare repository
  describe: setup working tree for --dirty
2019-02-06 22:05:29 -08:00
Junio C Hamano
cba595ab1a Merge branch 'jk/loose-object-cache-oid'
Code clean-up.

* jk/loose-object-cache-oid:
  prefer "hash mismatch" to "sha1 mismatch"
  sha1-file: avoid "sha1 file" for generic use in messages
  sha1-file: prefer "loose object file" to "sha1 file" in messages
  sha1-file: drop has_sha1_file()
  convert has_sha1_file() callers to has_object_file()
  sha1-file: convert pass-through functions to object_id
  sha1-file: modernize loose header/stream functions
  sha1-file: modernize loose object file functions
  http: use struct object_id instead of bare sha1
  update comment references to sha1_object_info()
  sha1-file: fix outdated sha1 comment references
2019-02-06 22:05:27 -08:00
Junio C Hamano
96e6547c2e Merge branch 'pw/rebase-x-sanity-check'
"git rebase -x $cmd" did not reject multi-line command, even though
the command is incapable of handling such a command.  It now is
rejected upfront.

* pw/rebase-x-sanity-check:
  rebase -x: sanity check command
2019-02-06 22:05:26 -08:00
Junio C Hamano
57cbc53d3e Merge branch 'js/vsts-ci'
Prepare to run test suite on Azure Pipeline.

* js/vsts-ci: (22 commits)
  test-date: drop unused parameter to getnanos()
  ci: parallelize testing on Windows
  ci: speed up Windows phase
  tests: optionally skip bin-wrappers/
  t0061: workaround issues with --with-dashes and RUNTIME_PREFIX
  tests: add t/helper/ to the PATH with --with-dashes
  mingw: try to work around issues with the test cleanup
  tests: include detailed trace logs with --write-junit-xml upon failure
  tests: avoid calling Perl just to determine file sizes
  README: add a build badge (status of the Azure Pipelines build)
  mingw: be more generous when wrapping up the setitimer() emulation
  ci: use git-sdk-64-minimal build artifact
  ci: add a Windows job to the Azure Pipelines definition
  Add a build definition for Azure DevOps
  ci/lib.sh: add support for Azure Pipelines
  tests: optionally write results as JUnit-style .xml
  test-date: add a subcommand to measure times in shell scripts
  ci: use a junction on Windows instead of a symlink
  ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
  ci/lib.sh: encapsulate Travis-specific things
  ...
2019-02-06 22:05:26 -08:00
Junio C Hamano
e7b120bfa5 Merge branch 'br/commit-tree-fully-spelled-gpg-sign-option'
The documentation of "git commit-tree" said that the command
understands "--gpg-sign" in addition to "-S", but the command line
parser did not know about the longhand, which has been corrected.

* br/commit-tree-fully-spelled-gpg-sign-option:
  commit-tree: add missing --gpg-sign flag
  t7510: invoke git as part of &&-chain
2019-02-06 22:05:25 -08:00
Junio C Hamano
5fda343321 Merge branch 'ds/push-sparse-tree-walk'
"git pack-objects" learned another algorithm to compute the set of
objects to send, that trades the resulting packfile off to save
traversal cost to favor small pushes.

* ds/push-sparse-tree-walk:
  pack-objects: create GIT_TEST_PACK_SPARSE
  pack-objects: create pack.useSparse setting
  revision: implement sparse algorithm
  list-objects: consume sparse tree walk
  revision: add mark_tree_uninteresting_sparse
2019-02-06 22:05:25 -08:00
Junio C Hamano
d8d62e6135 Merge branch 'tb/test-lint-sed-options'
The test lint learned to catch non-portable "sed" options.

* tb/test-lint-sed-options:
  test-lint: only use only sed [-n] [-e command] [-f command_file]
2019-02-06 22:05:24 -08:00
Junio C Hamano
ecbe1beb8e Merge branch 'lt/date-human'
A new date format "--date=human" that morphs its output depending
on how far the time is from the current time has been introduced.
"--date=auto" can be used to use this new format when the output is
going to the pager or to the terminal and otherwise the default
format.

* lt/date-human:
  Add `human` date format tests.
  Add `human` format to test-tool
  Add 'human' date format documentation
  Replace the proposed 'auto' mode with 'auto:'
  Add 'human' date format
2019-02-06 22:05:24 -08:00
Junio C Hamano
b2fc9d2fb0 Merge branch 'jk/unused-parameter-cleanup'
Code cleanup.

* jk/unused-parameter-cleanup:
  convert: drop path parameter from actual conversion functions
  convert: drop len parameter from conversion checks
  config: drop unused parameter from maybe_remove_section()
  show_date_relative(): drop unused "tz" parameter
  column: drop unused "opts" parameter in item_length()
  create_bundle(): drop unused "header" parameter
  apply: drop unused "def" parameter from find_name_gnu()
  match-trees: drop unused path parameter from score functions
2019-02-06 22:05:23 -08:00
Junio C Hamano
7589e63648 Merge branch 'nd/the-index-final'
The assumption to work on the single "in-core index" instance has
been reduced from the library-ish part of the codebase.

* nd/the-index-final:
  cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch
  read-cache.c: remove the_* from index_has_changes()
  merge-recursive.c: remove implicit dependency on the_repository
  merge-recursive.c: remove implicit dependency on the_index
  sha1-name.c: remove implicit dependency on the_index
  read-cache.c: replace update_index_if_able with repo_&
  read-cache.c: kill read_index()
  checkout: avoid the_index when possible
  repository.c: replace hold_locked_index() with repo_hold_locked_index()
  notes-utils.c: remove the_repository references
  grep: use grep_opt->repo instead of explict repo argument
2019-02-06 22:05:23 -08:00
Junio C Hamano
09a9c1f427 Merge branch 'tt/bisect-in-c'
More code in "git bisect" has been rewritten in C.

* tt/bisect-in-c:
  bisect--helper: `bisect_start` shell function partially in C
  bisect--helper: `get_terms` & `bisect_terms` shell function in C
  bisect--helper: `bisect_next_check` shell function in C
  bisect--helper: `check_and_set_terms` shell function in C
  wrapper: move is_empty_file() and rename it as is_empty_or_missing_file()
  bisect--helper: `bisect_write` shell function in C
  bisect--helper: `bisect_reset` shell function in C
2019-02-06 22:05:22 -08:00
Junio C Hamano
0fa3cc77ee Merge branch 'tb/utf-16-le-with-explicit-bom'
A new encoding UTF-16LE-BOM has been invented to force encoding to
UTF-16 with BOM in little endian byte order, which cannot be directly
generated by using iconv.

* tb/utf-16-le-with-explicit-bom:
  Support working-tree-encoding "UTF-16LE-BOM"
2019-02-06 22:05:21 -08:00
Junio C Hamano
cfd9167c15 Merge branch 'dt/cat-file-batch-ambiguous'
"git cat-file --batch" reported a dangling symbolic link by
mistake, when it wanted to report that a given name is ambiguous.

* dt/cat-file-batch-ambiguous:
  t1512: test ambiguous cat-file --batch and --batch-output
  Do not print 'dangling' for cat-file in case of ambiguity
2019-02-06 22:05:21 -08:00
Junio C Hamano
8fe9c3f21d Merge branch 'en/rebase-merge-on-sequencer'
"git rebase --merge" as been reimplemented by reusing the internal
machinery used for "git rebase -i".

* en/rebase-merge-on-sequencer:
  rebase: implement --merge via the interactive machinery
  rebase: define linearization ordering and enforce it
  git-legacy-rebase: simplify unnecessary triply-nested if
  git-rebase, sequencer: extend --quiet option for the interactive machinery
  am, rebase--merge: do not overlook --skip'ed commits with post-rewrite
  t5407: add a test demonstrating how interactive handles --skip differently
  rebase: fix incompatible options error message
  rebase: make builtin and legacy script error messages the same
2019-02-06 22:05:20 -08:00
Luke Diamand
3f360ded54 git-p4: remove ticket expiry test
The git-p4 login ticket expiry test causes unreliable test
runs. Since the handling of ticket expiry in git-p4 is far
from polished anyway, let's remove it for now.

A better way to actually run the test is to create a python
"fake" version of "p4" which returns whatever expiry results
the test requires.

Ideally git-p4 would look at the expiry time before starting
any long operations, and cleanup gracefully if there is not
enough time left. But that's quite hard to do.

Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06 12:58:57 -08:00
Josh Steadmon
30dea56536 t5551: test server-side ERR packet
When a smart HTTP server sends an error message via pkt-line, we detect
the error due to using PACKET_READ_DIE_ON_ERR_PACKET. This case was
added by 2d103c31c2 (pack-protocol.txt: accept error packets in any
context, 2018-12-29), but not covered by tests.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06 12:20:23 -08:00
Jeff King
ba285a712d test-date: drop unused parameter to getnanos()
The getnanos() helper always gets the current time from our
getnanotime() facility. The caller cannot override it via TEST_DATE_NOW,
and hence we simply ignore the "now" parameter to the function. Let's
remove it, as it may mislead callers into thinking it does something.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06 12:17:10 -08:00
Johannes Schindelin
e11ff8975b Revert "rebase: introduce a shortcut for --reschedule-failed-exec"
This patch was contributed only as a tentative "we could introduce a
convenient short option if we do not want to change the default behavior
in the long run" patch, opening the discussion whether other people
agree with deprecating the current behavior in favor of the rescheduling
behavior.

But the consensus on the Git mailing list was that it would make sense
to show a warning in the near future, and flip the default
rebase.rescheduleFailedExec to reschedule failed `exec` commands by
default. See e.g.
<CAGZ79kZL5CRqCDRb6B-EedUm8Z_i4JuSF2=UtwwdRXMitrrOBw@mail.gmail.com>

So let's back out that patch that added the `-y` short option that we
agreed was not necessary or desirable.

This reverts commit 81ef8ee75d.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06 11:27:17 -08:00
Junio C Hamano
69dd6e5737 Merge branch 'pw/no-editor-in-rebase-i-implicit'
When GIT_SEQUENCE_EDITOR is set, the command was incorrectly
started when modes of "git rebase" that implicitly uses the
machinery for the interactive rebase are run, which has been
corrected.

* pw/no-editor-in-rebase-i-implicit:
  implicit interactive rebase: don't run sequence editor
2019-02-05 14:26:17 -08:00
Junio C Hamano
5d2710bd3c Merge branch 'jk/diff-cc-stat-fixes'
"git diff --color-moved --cc --stat -p" did not work well due to
funny interaction between a bug in color-moved and the rest, which
has been fixed.

* jk/diff-cc-stat-fixes:
  combine-diff: treat --dirstat like --stat
  combine-diff: treat --summary like --stat
  combine-diff: treat --shortstat like --stat
  combine-diff: factor out stat-format mask
  diff: clear emitted_symbols flag after use
  t4006: resurrect commented-out tests
2019-02-05 14:26:17 -08:00
Junio C Hamano
5ad3550f02 Merge branch 'bp/checkout-new-branch-optim'
"git checkout -b <new> [HEAD]" to create a new branch from the
current commit and check it out ought to be a no-op in the index
and the working tree in normal cases, but there are corner cases
that do require updates to the index and the working tree.  Running
it immediately after "git clone --no-checkout" is one of these
cases that an earlier optimization kicked in incorrectly, which has
been fixed.

* bp/checkout-new-branch-optim:
  checkout: fix regression in checkout -b on intitial checkout
  checkout: add test demonstrating regression with checkout -b on initial commit
2019-02-05 14:26:17 -08:00
Junio C Hamano
02bf766cc5 Merge branch 'jk/attr-macro-fix'
Asking "git check-attr" about a macro (e.g. "binary") on a specific
path did not work correctly, even though "git check-attr -a" listed
such a macro correctly.  This has been corrected.

* jk/attr-macro-fix:
  attr: do not mark queried macros as unset
2019-02-05 14:26:16 -08:00
Junio C Hamano
11494daae7 Merge branch 'js/test-git-installed'
Test fix for Windows.

* js/test-git-installed:
  tests: explicitly use `test-tool.exe` on Windows
2019-02-05 14:26:15 -08:00
Junio C Hamano
ff09c9e5ba Merge branch 'js/abspath-part-inside-repo'
On a case-insensitive filesystem, we failed to compare the part of
the path that is above the worktree directory in an absolute
pathname, which has been corrected.

* js/abspath-part-inside-repo:
  abspath_part_inside_repo: respect core.ignoreCase
2019-02-05 14:26:15 -08:00
Junio C Hamano
9c96ab9872 Merge branch 'jt/namespaced-ls-refs-fix'
Fix namespace support in protocol v2.

* jt/namespaced-ls-refs-fix:
  ls-refs: filter refs using namespace-stripped name
2019-02-05 14:26:15 -08:00
Junio C Hamano
e5eac57356 Merge branch 'ab/commit-graph-write-progress'
The codepath to show progress meter while writing out commit-graph
file has been improved.

* ab/commit-graph-write-progress:
  commit-graph write: emit a percentage for all progress
  commit-graph write: add itermediate progress
  commit-graph write: remove empty line for readability
  commit-graph write: add more descriptive progress output
  commit-graph write: show progress for object search
  commit-graph write: more descriptive "writing out" output
  commit-graph write: add "Writing out" progress output
  commit-graph: don't call write_graph_chunk_extra_edges() unnecessarily
  commit-graph: rename "large edges" to "extra edges"
2019-02-05 14:26:14 -08:00
Junio C Hamano
3009c8bcca Merge branch 'js/t6042-timing-fix'
Test update.

* js/t6042-timing-fix:
  t6042: work around speed optimization on Windows
2019-02-05 14:26:14 -08:00
Junio C Hamano
1c418243a5 Merge branch 'jk/add-ignore-errors-bit-assignment-fix'
"git add --ignore-errors" did not work as advertised and instead
worked as an unintended synonym for "git add --renormalize", which
has been fixed.

* jk/add-ignore-errors-bit-assignment-fix:
  add: use separate ADD_CACHE_RENORMALIZE flag
2019-02-05 14:26:13 -08:00
Junio C Hamano
f5dd919064 Merge branch 'js/mingw-unc-path-w-backslashes'
In Git for Windows, "git clone \\server\share\path" etc. that uses
UNC paths from command line had bad interaction with its shell
emulation.

* js/mingw-unc-path-w-backslashes:
  mingw: special-case arguments to `sh`
  mingw (t5580): document bug when cloning from backslashed UNC paths
2019-02-05 14:26:13 -08:00
Junio C Hamano
e91a1b1ade Merge branch 'cc/test-ref-store-typofix'
An obvious typo in an assertion error message has been fixed.

* cc/test-ref-store-typofix:
  helper/test-ref-store: fix "new-sha1" vs "old-sha1" typo
2019-02-05 14:26:13 -08:00
Junio C Hamano
5f8b86db94 Merge branch 'jt/fetch-v2-sideband'
"git fetch" and "git upload-pack" learned to send all exchange over
the sideband channel while talking the v2 protocol.

* jt/fetch-v2-sideband:
  tests: define GIT_TEST_SIDEBAND_ALL
  {fetch,upload}-pack: sideband v2 fetch response
  sideband: reverse its dependency on pkt-line
  pkt-line: introduce struct packet_writer
  pack-protocol.txt: accept error packets in any context
  Use packet_reader instead of packet_read_line
2019-02-05 14:26:11 -08:00
Junio C Hamano
19a504d92b Merge branch 'js/commit-graph-chunk-table-fix'
The codepath to read from the commit-graph file attempted to read
past the end of it when the file's table-of-contents was corrupt.

* js/commit-graph-chunk-table-fix:
  Makefile: correct example fuzz build
  commit-graph: fix buffer read-overflow
  commit-graph, fuzz: add fuzzer for commit-graph
2019-02-05 14:26:11 -08:00
Junio C Hamano
40b8ba2c44 Merge branch 'ld/git-p4-shelve-update-fix'
"git p4" failed to update a shelved change when there were moved
files, which has been corrected.

* ld/git-p4-shelve-update-fix:
  git-p4: handle update of moved/copied files when updating a shelve
  git-p4: add failing test for shelved CL update involving move/copy
2019-02-05 14:26:10 -08:00
Junio C Hamano
073312b4c7 Merge branch 'js/filter-options-should-use-plain-int'
Update the protocol message specification to allow only the limited
use of scaled quantities.  This is ensure potential compatibility
issues will not go out of hand.

* js/filter-options-should-use-plain-int:
  filter-options: expand scaled numbers
  tree:<depth>: skip some trees even when collecting omits
  list-objects-filter: teach tree:# how to handle >0
2019-02-05 14:26:10 -08:00
Junio C Hamano
b99a579f8e Merge branch 'sb/more-repo-in-api'
The in-core repository instances are passed through more codepaths.

* sb/more-repo-in-api: (23 commits)
  t/helper/test-repository: celebrate independence from the_repository
  path.h: make REPO_GIT_PATH_FUNC repository agnostic
  commit: prepare free_commit_buffer and release_commit_memory for any repo
  commit-graph: convert remaining functions to handle any repo
  submodule: don't add submodule as odb for push
  submodule: use submodule repos for object lookup
  pretty: prepare format_commit_message to handle arbitrary repositories
  commit: prepare logmsg_reencode to handle arbitrary repositories
  commit: prepare repo_unuse_commit_buffer to handle any repo
  commit: prepare get_commit_buffer to handle any repo
  commit-reach: prepare in_merge_bases[_many] to handle any repo
  commit-reach: prepare get_merge_bases to handle any repo
  commit-reach.c: allow get_merge_bases_many_0 to handle any repo
  commit-reach.c: allow remove_redundant to handle any repo
  commit-reach.c: allow merge_bases_many to handle any repo
  commit-reach.c: allow paint_down_to_common to handle any repo
  commit: allow parse_commit* to handle any repo
  object: parse_object to honor its repository argument
  object-store: prepare has_{sha1, object}_file to handle any repo
  object-store: prepare read_object_file to deal with any repo
  ...
2019-02-05 14:26:09 -08:00
Jiang Xin
0e37abd2e8 pack-redundant: consistent sort method
SZEDER reported that test case t5323 has different test result on MacOS.
This is because `cmp_pack_list_reverse` cannot give identical result
when two pack being sorted has the same size of remaining_objects.

Changes to the sorting function will make consistent test result for
t5323.

The new algorithm to find redundant packs is a trade-off to save memory
resources, and the result of it may be different with old one, and may
be not the best result sometimes.  Update t5323 for the new algorithm.

Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:25 -08:00
Sun Chao
3084a01e5e pack-redundant: new algorithm to find min packs
When calling `git pack-redundant --all`, if there are too many local
packs and too many redundant objects within them, the too deep iteration
of `get_permutations` will exhaust all the resources, and the process of
`git pack-redundant` will be killed.

The following script could create a repository with too many redundant
packs, and running `git pack-redundant --all` in the `test.git` repo
will die soon.

    #!/bin/sh

    repo="$(pwd)/test.git"
    work="$(pwd)/test"
    i=1
    max=199

    if test -d "$repo" || test -d "$work"; then
    	echo >&2 "ERROR: '$repo' or '$work' already exist"
    	exit 1
    fi

    git init -q --bare "$repo"
    git --git-dir="$repo" config gc.auto 0
    git --git-dir="$repo" config transfer.unpackLimit 0
    git clone -q "$repo" "$work" 2>/dev/null

    while :; do
        cd "$work"
        echo "loop $i: $(date +%s)" >$i
        git add $i
        git commit -q -sm "loop $i"
        git push -q origin HEAD:master
        printf "\rCreate pack %4d/%d\t" $i $max
        if test $i -ge $max; then break; fi

        cd "$repo"
        git repack -q
        if test $(($i % 2)) -eq 0; then
            git repack -aq
            pack=$(ls -t $repo/objects/pack/*.pack | head -1)
            touch "${pack%.pack}.keep"
        fi
        i=$((i+1))
    done
    printf "\ndone\n"

To get the `min` unique pack list, we can replace the iteration in
`minimize` function with a new algorithm, and this could solve this
issue:

1. Get the unique and non_uniqe packs, add the unique packs to the
   `min` list.

2. Remove the objects of unique packs from non_unique packs, then each
   object left in the non_unique packs will have at least two copies.

3. Sort the non_unique packs by the objects' size, more objects first,
   and add the first non_unique pack to `min` list.

4. Drop the duplicated objects from other packs in the ordered
   non_unique pack list, and repeat step 3.

Some test cases will fail on Mac OS X. Mark them and will resolve in
later commit.

Original PR and discussions: https://github.com/jiangxin/git/pull/25

Signed-off-by: Sun Chao <sunchao9@huawei.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:24 -08:00
Jiang Xin
3173a94d57 t5323: test cases for git-pack-redundant
Add test cases for git pack-redundant to validate new algorithm for git
pack-redundant.

Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Reviewed-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Reviewed-by: Sun Chao <sunchao9@huawei.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 14:18:23 -08:00
Thomas Gummerer
e92aa0e4ef revert "checkout: introduce checkout.overlayMode config"
This reverts 1495ff7da5 ("checkout: introduce checkout.overlayMode
config", 2019-01-08) and thus removes the checkout.overlayMode config
option.

The option was originally introduced to give users the option to make
the new no-overlay behaviour the default.  However users may be using
'git checkout' in scripts, even though it is porcelain.  Users setting
the option to false may actually end up accidentally breaking scripts.

With the introduction of a new subcommand that will make the behaviour
the default, the config option will not be needed anymore anyway.
Revert the commit and remove the config option, so we don't risk
breaking scripts.

Suggested-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 13:30:14 -08:00
William Hubbs
39ab4d0951 config: allow giving separate author and committer idents
The author.email, author.name, committer.email and committer.name
settings are analogous to the GIT_AUTHOR_* and GIT_COMMITTER_*
environment variables, but for the git config system. This allows them
to be set separately for each repository.

Git supports setting different authorship and committer
information with environment variables. However, environment variables
are set in the shell, so if different authorship and committer
information is needed for different repositories an external tool is
required.

This adds support to git config for author.email, author.name,
committer.email and committer.name  settings so this information
can be set per repository.

Also, it generalizes the fmt_ident function so it can handle author vs
committer identification.

Signed-off-by: William Hubbs <williamh@gentoo.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 12:18:13 -08:00
Sebastian Staudt
c801170b0c t6120: test for describe with a bare repository
This ensures that nothing breaks the basic functionality of describe for
bare repositories. Please note that --broken and --dirty need a working
tree.

Signed-off-by: Sebastian Staudt <koraktor@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 10:27:57 -08:00
Sebastian Staudt
2ed5c8e174 describe: setup working tree for --dirty
We don't use NEED_WORK_TREE when running the git-describe builtin,
since you should be able to describe a commit even in a bare repository.
However, the --dirty flag does need a working tree. Since we don't call
setup_work_tree(), it uses whatever directory we happen to be in. That's
unlikely to match our index, meaning we'd say "dirty" even when the real
working tree is clean.

We can fix that by calling setup_work_tree() once we know that the user
has asked for --dirty.

The --broken option also needs a working tree. But because its
implementation calls git-diff-index we don‘t have to setup the working
tree in the git-describe process.

Signed-off-by: Sebastian Staudt <koraktor@gmail.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-04 10:27:54 -08:00
Eric Wong
df799f5d99 t1512: test ambiguous cat-file --batch and --batch-output
Test the new "ambiguous" result from cat-file --batch and
--batch-check.  This is in t1512 instead of t1006 since
we need a repo with ambiguous object_id names.

Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-31 10:52:37 -08:00
Torsten Bögershausen
aab2a1ae48 Support working-tree-encoding "UTF-16LE-BOM"
Users who want UTF-16 files in the working tree set the .gitattributes
like this:
test.txt working-tree-encoding=UTF-16

The unicode standard itself defines 3 allowed ways how to encode UTF-16.
The following 3 versions convert all back to 'g' 'i' 't' in UTF-8:

a) UTF-16, without BOM, big endian:
$ printf "\000g\000i\000t" | iconv -f UTF-16 -t UTF-8 | od -c
0000000    g   i   t

b) UTF-16, with BOM, little endian:
$ printf "\377\376g\000i\000t\000" | iconv -f UTF-16 -t UTF-8 | od -c
0000000    g   i   t

c) UTF-16, with BOM, big endian:
$ printf "\376\377\000g\000i\000t" | iconv -f UTF-16 -t UTF-8 | od -c
0000000    g   i   t

Git uses libiconv to convert from UTF-8 in the index into ITF-16 in the
working tree.
After a checkout, the resulting file has a BOM and is encoded in "UTF-16",
in the version (c) above.
This is what iconv generates, more details follow below.

iconv (and libiconv) can generate UTF-16, UTF-16LE or UTF-16BE:

d) UTF-16
$ printf 'git' | iconv -f UTF-8 -t UTF-16 | od -c
0000000  376 377  \0   g  \0   i  \0   t

e) UTF-16LE
$ printf 'git' | iconv -f UTF-8 -t UTF-16LE | od -c
0000000    g  \0   i  \0   t  \0

f)  UTF-16BE
$ printf 'git' | iconv -f UTF-8 -t UTF-16BE | od -c
0000000   \0   g  \0   i  \0   t

There is no way to generate version (b) from above in a Git working tree,
but that is what some applications need.
(All fully unicode aware applications should be able to read all 3 variants,
but in practise we are not there yet).

When producing UTF-16 as an output, iconv generates the big endian version
with a BOM. (big endian is probably chosen for historical reasons).

iconv can produce UTF-16 files with little endianess by using "UTF-16LE"
as encoding, and that file does not have a BOM.

Not all users (especially under Windows) are happy with this.
Some tools are not fully unicode aware and can only handle version (b).

Today there is no way to produce version (b) with iconv (or libiconv).
Looking into the history of iconv, it seems as if version (c) will
be used in all future iconv versions (for compatibility reasons).

Solve this dilemma and introduce a Git-specific "UTF-16LE-BOM".
libiconv can not handle the encoding, so Git pick it up, handles the BOM
and uses libiconv to convert the rest of the stream.
(UTF-16BE-BOM is added for consistency)

Rported-by: Adrián Gimeno Balaguer <adrigibal@gmail.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-31 10:27:52 -08:00
Phillip Wood
c762aada1a rebase -x: sanity check command
If the user gives an empty argument to --exec then git creates a todo
list that it cannot parse. The rebase starts to run before erroring out
with

  error: missing arguments for exec
  error: invalid line 2: exec
  You can fix this with 'git rebase --edit-todo' and then run 'git rebase --continue'.
  Or you can abort the rebase with 'git rebase --abort'.

Instead check for empty commands before starting the rebase.

Also check that the command does not contain any newlines as the
todo-list format is unable to cope with multiline commands. Note that
this changes the behavior, before this change one could do

git rebase --exec='echo one
exec echo two'

and it would insert two exec lines in the todo list, now it will error
out.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 13:34:51 -08:00
Junio C Hamano
a562a11983 Merge branch 'it/log-format-source'
Custom userformat "log --format" learned %S atom that stands for
the tip the traversal reached the commit from, i.e. --source.

* it/log-format-source:
  log: add %S option (like --source) to log --format
2019-01-29 12:47:57 -08:00
Junio C Hamano
a6e3839976 Merge branch 'jt/upload-pack-deepen-relative-proto-v2'
"git fetch --deepen=<more>" has been corrected to work over v2
protocol.

* jt/upload-pack-deepen-relative-proto-v2:
  upload-pack: teach deepen-relative in protocol v2
  fetch-pack: do not take shallow lock unnecessarily
2019-01-29 12:47:55 -08:00
Junio C Hamano
99c0bdd09d Merge branch 'ms/http-no-more-failonerror'
Debugging help for http transport.

* ms/http-no-more-failonerror:
  test: test GIT_CURL_VERBOSE=1 shows an error
  remote-curl: unset CURLOPT_FAILONERROR
  remote-curl: define struct for CURLOPT_WRITEFUNCTION
  http: enable keep_error for HTTP requests
  http: support file handles for HTTP_KEEP_ERROR
2019-01-29 12:47:55 -08:00
Junio C Hamano
d94ade7f1f Merge branch 'os/rebase-runs-post-checkout-hook'
"git rebase" internally runs "checkout" to switch between branches,
and the command used to call the post-checkout hook, but the
reimplementation stopped doing so, which is getting fixed.

* os/rebase-runs-post-checkout-hook:
  rebase: run post-checkout hook on checkout
  t5403: simplify by using a single repository
2019-01-29 12:47:55 -08:00
Junio C Hamano
33e4ae9c50 Merge branch 'bc/sha-256'
Add sha-256 hash and plug it through the code to allow building Git
with the "NewHash".

* bc/sha-256:
  hash: add an SHA-256 implementation using OpenSSL
  sha256: add an SHA-256 implementation using libgcrypt
  Add a base implementation of SHA-256 support
  commit-graph: convert to using the_hash_algo
  t/helper: add a test helper to compute hash speed
  sha1-file: add a constant for hash block size
  t: make the sha1 test-tool helper generic
  t: add basic tests for our SHA-1 implementation
  cache: make hashcmp and hasheq work with larger hashes
  hex: introduce functions to print arbitrary hashes
  sha1-file: provide functions to look up hash algorithms
  sha1-file: rename algorithm to "sha1"
2019-01-29 12:47:55 -08:00
Junio C Hamano
5d3635db19 Merge branch 'sb/submodule-recursive-fetch-gets-the-tip'
"git fetch --recurse-submodules" may not fetch the necessary commit
that is bound to the superproject, which is getting corrected.

* sb/submodule-recursive-fetch-gets-the-tip:
  fetch: ensure submodule objects fetched
  submodule.c: fetch in submodules git directory instead of in worktree
  submodule: migrate get_next_submodule to use repository structs
  repository: repo_submodule_init to take a submodule struct
  submodule: store OIDs in changed_submodule_names
  submodule.c: tighten scope of changed_submodule_names struct
  submodule.c: sort changed_submodule_names before searching it
  submodule.c: fix indentation
  sha1-array: provide oid_array_filter
2019-01-29 12:47:54 -08:00
Junio C Hamano
f33989464e Merge branch 'jt/fetch-pack-v2'
"git fetch-pack" now can talk the version 2 protocol.

* jt/fetch-pack-v2:
  fetch-pack: support protocol version 2
2019-01-29 12:47:54 -08:00
Junio C Hamano
d3b017822d Merge branch 'jk/proto-v2-hidden-refs-fix'
The v2 upload-pack protocol implementation failed to honor
hidden-ref configuration, which has been corrected.
An earlier attempt reverted out of 'next'.

* jk/proto-v2-hidden-refs-fix:
  upload-pack: support hidden refs with protocol v2
2019-01-29 12:47:54 -08:00
Junio C Hamano
15b07cba0b Merge branch 'pw/diff-color-moved-ws-fix'
"git diff --color-moved-ws" updates.

* pw/diff-color-moved-ws-fix:
  diff --color-moved-ws: handle blank lines
  diff --color-moved-ws: modify allow-indentation-change
  diff --color-moved-ws: optimize allow-indentation-change
  diff --color-moved=zebra: be stricter with color alternation
  diff --color-moved-ws: fix false positives
  diff --color-moved-ws: demonstrate false positives
  diff: allow --no-color-moved-ws
  Use "whitespace" consistently
  diff: document --no-color-moved
2019-01-29 12:47:53 -08:00
Junio C Hamano
d9d9ab0876 Merge branch 'js/rebase-i-redo-exec'
"git rebase -i" learned to re-execute a command given with 'exec'
to run after it failed the last time.

* js/rebase-i-redo-exec:
  rebase: introduce a shortcut for --reschedule-failed-exec
  rebase: add a config option to default to --reschedule-failed-exec
  rebase: introduce --reschedule-failed-exec
2019-01-29 12:47:53 -08:00
Anders Waldenborg
0b691d8685 pretty: add support for separator option in %(trailers)
By default trailer lines are terminated by linebreaks ('\n'). By
specifying the new 'separator' option they will instead be separated by
user provided string and have separator semantics rather than terminator
semantics. The separator string can contain the literal formatting codes
%n and %xNN allowing it to be things that are otherwise hard to type
such as %x00, or comma and end-parenthesis which would break parsing.

E.g:
 $ git log --pretty='%(trailers:key=Reviewed-by,valueonly,separator=%x00)'

Signed-off-by: Anders Waldenborg <anders@0x63.nu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 10:03:32 -08:00
Anders Waldenborg
d9b936db52 pretty: add support for "valueonly" option in %(trailers)
With the new "key=" option to %(trailers) it often makes little sense to
show the key, as it by definition already is knows which trailer is
printed there. This new "valueonly" option makes it omit the key when
printing trailers.

E.g.:
 $ git show -s --pretty='%s%n%(trailers:key=Signed-off-by,valueonly)' aaaa88182
will show:
 > upload-pack: fix broken if/else chain in config callback
 > Jeff King <peff@peff.net>
 > Junio C Hamano <gitster@pobox.com>

Signed-off-by: Anders Waldenborg <anders@0x63.nu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 10:03:32 -08:00
Anders Waldenborg
250bea0c16 pretty: allow showing specific trailers
Adds a new "key=X" option to "%(trailers)" which will cause it to only
print trailer lines which match any of the specified keys.

Signed-off-by: Anders Waldenborg <anders@0x63.nu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 10:03:32 -08:00
Anders Waldenborg
4f732e0fd7 pretty: allow %(trailers) options with explicit value
In addition to old %(trailers:only) it is now allowed to write
%(trailers:only=yes)

By itself this only gives (the not quite so useful) possibility to have
users change their mind in the middle of a formatting
string (%(trailers:only=true,only=false)). However, it gives users the
opportunity to override defaults from future options.

Signed-off-by: Anders Waldenborg <anders@0x63.nu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 10:03:09 -08:00
Stephen P. Smith
110a6a161d Add human date format tests.
When using `human` several fields are suppressed depending on the time
difference between the reference date and the local computer date. In
cases where the difference is less than a year, the year field is
supppressed. If the time is less than a day; the month and year is
suppressed.

Use TEST_DATE_NOW environment variable when using the test-tool to
hold the expected output strings constant.

Signed-off-by: Stephen P. Smith <ischis2@cox.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:40:08 -08:00
Stephen P. Smith
b841d4ff43 Add human format to test-tool
Add the human format support to the test tool so that
GIT_TEST_DATE_NOW can be used to specify the current time.

The get_time() helper function was created and and checks the
GIT_TEST_DATE_NOW environment variable.  If GIT_TEST_DATE_NOW is set,
then that date is used instead of the date returned by by
gettimeofday().

All calls to gettimeofday() were replaced by calls to get_time().

Renamed occurances of TEST_DATE_NOW to GIT_TEST_DATE_NOW since the
variable is now used in the get binary and not just in the test-tool.

Signed-off-by: Stephen P. Smith <ischis2@cox.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:40:08 -08:00
Johannes Schindelin
b819f1d2ce ci: parallelize testing on Windows
The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.

For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.

To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.

The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.

To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).

We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:47 -08:00
Johannes Schindelin
dd167a3001 tests: optionally skip bin-wrappers/
This speeds up the tests by a bit on Windows, where running Unix shell
scripts (and spawning processes) is not exactly a cheap operation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:47 -08:00
Johannes Schindelin
06718d4a1e t0061: workaround issues with --with-dashes and RUNTIME_PREFIX
When building Git with RUNTIME_PREFIX and starting a test helper from
t/helper/, it fails to detect a system prefix. The reason is that the
RUNTIME_PREFIX feature wants to use the location of the Git executable
to determine where the support files can be found, e.g. system-wide Git
config or the translations. This does not make any sense for the test
helpers, though, as they are distinctly not in a directory structure
resembling the final installation location of Git.

That is the reason why the test helpers rely on environment variables to
indicate the location of the needed support files, e.g.
GIT_TEXTDOMAINDIR. If this information is missing, the output will
contain warnings like this one:

	RUNTIME_PREFIX requested, but prefix computation failed. [...]

In t0061, we did not expect that to happen, and it actually does not
happen in the regular case, because bin-wrappers/test-tool specifically
sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool needs
to know anything about any runtime prefix).

However, with --with-dashes, bin-wrappers/test-tool is no longer called,
but t/helper/test-tool is called directly instead.

So let's just ignore the RUNTIME_PREFIX warning.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:47 -08:00
Johannes Schindelin
ca7312d3ef tests: add t/helper/ to the PATH with --with-dashes
We really need to be able to find the test helpers... Really. This
change was forgotten when we moved the test helpers into t/helper/

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:47 -08:00
Johannes Schindelin
046e90d1c9 mingw: try to work around issues with the test cleanup
It seems that every once in a while in the Git for Windows SDK, there
are some transient file locking issues preventing the test clean up to
delete the trash directory. Let's be gentle and try again five seconds
later, and only error out if it still fails the second time.

This change helps Windows, and does not hurt any other platform
(normally, it is highly unlikely that said deletion fails, and if it
does, normally it will fail again even 5 seconds later).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:47 -08:00
Johannes Schindelin
af9912efaf tests: include detailed trace logs with --write-junit-xml upon failure
The JUnit XML format lends itself to be presented in a powerful UI,
where you can drill down to the information you are interested in very
quickly.

For test failures, this usually means that you want to see the detailed
trace of the failing tests.

With Travis CI, we passed the `--verbose-log` option to get those
traces. However, that seems excessive, as we do not need/use the logs in
almost all of those cases: only when a test fails do we have a way to
include the trace.

So let's do something different when using Azure DevOps: let's run all
the tests with `--quiet` first, and only if a failure is encountered,
try to trace the commands as they are executed.

Of course, we cannot turn on `--verbose-log` after the fact. So let's
just re-run the test with all the same options, adding `--verbose-log`.
And then munging the output file into the JUnit XML on the fly.

Note: there is an off chance that re-running the test in verbose mode
"fixes" the failures (and this does happen from time to time!). That is
a possibility we should be able to live with. Ideally, we would label
this as "Passed upon rerun", and Azure Pipelines even know about that
outcome, but it is not available when using the JUnit XML format for
now:
https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:47 -08:00
Johannes Schindelin
5868bd8620 tests: avoid calling Perl just to determine file sizes
It is a bit ridiculous to spin up a full-blown Perl instance (especially
on Windows, where that means spinning up a full POSIX emulation layer,
AKA the MSYS2 runtime) just to tell how large a given file is.

So let's just use the test-tool to do that job instead.

This command will also be used over the next commits, to allow for
cutting out individual test cases' verbose log from the file generated
via --verbose-log.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:47 -08:00
Johannes Schindelin
2223190815 tests: optionally write results as JUnit-style .xml
This will come in handy when publishing the results of Git's test suite
during an automated Azure DevOps run.

Note: we need to make extra sure that invalid UTF-8 encoding is turned
into valid UTF-8 (using the Replacement Character, \uFFFD) because
t9902's trace contains such invalid byte sequences, and the task in the
Azure Pipeline that uploads the test results would refuse to do anything
if it was asked to parse an .xml file with invalid UTF-8 in it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29 09:26:46 -08:00
Torsten Bögershausen
e62e225ffb test-lint: only use only sed [-n] [-e command] [-f command_file]
From `man sed` (on a Mac OS X box):
The -E, -a and -i options are non-standard FreeBSD extensions and may not be available
on other operating systems.

From `man sed` on a Linux box:
REGULAR EXPRESSIONS
       POSIX.2 BREs should be supported, but they aren't completely because of
       performance problems.  The \n sequence in a regular expression matches the newline
       character,  and  similarly  for \a, \t, and other sequences.
       The -E option switches to using extended regular expressions instead; the -E option
       has been supported for years by GNU sed, and is now included in POSIX.

Well, there are still a lot of systems out there, which don't support it.
Beside that, IEEE Std 1003.1TM-2017, see
http://pubs.opengroup.org/onlinepubs/9699919799/
does not mention -E either.

To be on the safe side, don't allow -E (or -r, which is GNU).
Change check-non-portable-shell.pl to only accept the portable options:
sed [-n] [-e command] [-f command_file]

Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 11:25:16 -08:00
Johannes Schindelin
4419de9164 test-date: add a subcommand to measure times in shell scripts
In the next commit, we want to teach Git's test suite to optionally
output test results in JUnit-style .xml files. These files contain
information about the time spent. So we need a way to measure time.

While we could use `date +%s` for that, this will give us only seconds,
i.e. very coarse-grained timings.

GNU `date` supports `date +%s.%N` (i.e. nanosecond-precision output),
but there is no equivalent in BSD `date` (read: on macOS, we would not
be able to obtain precise timings).

So let's introduce `test-tool date getnanos`, with an optional start
time, that outputs preciser values. Note that this might not actually
give us nanosecond precision on some platforms, but it will give us as
precise information as possible, without the portability issues of shell
commands.

Granted, it is a bit pointless to try measuring times accurately in
shell scripts, certainly to nanosecond precision. But it is better than
second-granularity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 10:34:28 -08:00
Phillip Wood
891d4a0313 implicit interactive rebase: don't run sequence editor
If GIT_SEQUENCE_EDITOR is set then rebase runs it when executing
implicit interactive rebases which are supposed to appear
non-interactive to the user. Fix this by setting GIT_SEQUENCE_EDITOR=:
rather than GIT_EDITOR=:. A couple of tests relied on the old behavior
so they are updated to work with the new regime.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28 10:25:39 -08:00
Nguyễn Thái Ngọc Duy
baa4adc66a parse-options: disable option abbreviation with PARSE_OPT_KEEP_UNKNOWN
parse-options can unambiguously find an abbreviation only if it sees
all available options. This is usually the case when you use
parse_options(). But there are other callers like blame or shortlog
which uses parse_options_start() in combination with a custom option
parser, like rev-list. parse-options cannot see all options in this
case and will get abbrev detection wrong. Disable it.

t7800 needs update because --symlink no longer expands to --symlinks
and will be passed down to git-diff, which will not recognize it. I
still think this is the correct thing to do. But if --symlink has been
actually used in the wild, we would just add an option alias for it.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-27 16:28:17 -08:00
Jeff King
3d42034a18 show_date_relative(): drop unused "tz" parameter
The timestamp we receive is in epoch time, so there's no need for a
timezone parameter to interpret it. The matching show_date() uses "tz"
to show dates in author local time, but relative dates show only the
absolute time difference. The author's location is irrelevant, barring
relativistic effects from using Git close to the speed of light.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 12:35:44 -08:00
Jeff King
dac03b5518 combine-diff: treat --dirstat like --stat
Currently "--cc --dirstat" will show nothing for a merge.  Like
--shortstat and --summary in the previous two patches, it probably makes
sense to treat it like we do --stat, and show a stat against the
first-parent.

This case is less obviously correct than for --shortstat and --summary,
as those are basically variants of --stat themselves. It's possible we
could develop a multi-parent combined dirstat format, in which case we
might regret defining this first-parent behavior. But the same could be
said for --stat, and in the 12+ years of it showing first-parent stats,
nobody has complained.

So showing the first-parent dirstat is at least _useful_, and if we
later develop a clever multi-parent stat format, we'd probably have to
deal with --stat anyway.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 12:18:53 -08:00
Jeff King
04b19fcafd combine-diff: treat --summary like --stat
Currently "--cc --summary" on a merge shows nothing. Since we show "--cc
--stat" as a stat against the first parent, and because --summary is
typically used in combination with --stat, it makes sense to treat them
both the same way.

Note that we have to tweak t4013's setup a bit to test this case, as the
existing merges do not have any --summary results against their first
parent. But since the merge at the tip of 'master' does add and remove
files with respect to the second parent, we can just make a reversed
doppelganger merge where the parents are swapped.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 12:18:53 -08:00
Jeff King
8290faa077 combine-diff: treat --shortstat like --stat
The --stat of a combined diff is defined as the first-parent stat,
going all the way back to 965f803c32 (combine-diff: show diffstat with
the first parent., 2006-04-17).

Naturally, we gave --numstat the same treatment in 74e2abe5b7 (diff
--numstat, 2006-10-12).

But --shortstat, which is really just the final line of --stat, does
nothing, which produces confusing results:

  $ git show --oneline --stat eab7584e37
  eab7584e37 Merge branch 'en/show-ref-doc-fix'

   Documentation/git-show-ref.txt | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)

  $ git show --oneline --shortstat eab7584e37
  eab7584e37 Merge branch 'en/show-ref-doc-fix'

  [nothing! We'd expect to see the "1 file changed..." line]

This patch teaches combine-diff to treats the two formats identically.

Reported-by: David Turner <novalis@novalis.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 12:18:53 -08:00
Jeff King
48edf3a02a diff: clear emitted_symbols flag after use
There's an odd bug when "log --color-moved" is used with the combination
of "--cc --stat -p": the stat for merge commits is erroneously shown
with the diff of the _next_ commit.

The included test demonstrates the issue. Our history looks something
like this:

  A-B-M--D
   \ /
    C

When we run "git log --cc --stat -p --color-moved" starting at D, we get
this sequence of events:

  1. The diff for D is using -p, so diff_flush() calls into
     diff_flush_patch_all_file_pairs(). There we see that o->color_moved
     is in effect, so we point o->emitted_symbols to a static local
     struct, causing diff_flush_patch() to queue the symbols instead of
     actually writing them out.

     We then do our move detection, emit the symbols, and clear the
     struct. But we leave o->emitted_symbols pointing to our struct.

  2. Next we compute the diff for M. This is a merge, so we use the
     combined diff code. In find_paths_generic(), we compute the
     pairwise diff between each commit and its parent. Normally this is
     done with DIFF_FORMAT_NO_OUTPUT, since we're just looking for
     intersecting paths. But since "--stat --cc" shows the first-parent
     stat, and since we're computing that diff anyway, we enable
     DIFF_FORMAT_DIFFSTAT for the first parent. This outputs the stat
     information immediately, saving us from running a separate
     first-parent diff later.

     But where does that output go? Normally it goes directly to stdout,
     but because o->emitted_symbols is set, we queue it. As a result, we
     don't actually print the diffstat for the merge commit (yet), which
     is wrong.

  3. Next we compute the diff for C. We're actually showing a patch
     again, so we end up in diff_flush_patch_all_file_pairs(), but this
     time we have the queued stat from step 2 waiting in our struct.

     We add new elements to it for C's diff, and then flush the whole
     thing. And we see the diffstat from M as part of C's diff, which is
     wrong.

So triggering the bug really does require the combination of all of
those options.

To fix it, we can simply restore o->emitted_symbols to NULL after
flushing it, so that it does not affect anything outside of
diff_flush_patch_all_file_pairs(). This intuitively makes sense, since
nobody outside of that function is going to bother flushing it, so we
would not want them to write to it either.

In fact, we could take this a step further and turn the local "esm"
struct into a non-static variable that goes away after the function
ends. However, since it contains a dynamically sized array, we benefit
from amortizing the cost of allocations over many calls. So we'll leave
it as static to retain that benefit.

But let's push the zero-ing of esm.nr into the conditional for "if
(o->emitted_symbols)" to make it clear that we do not expect esm to hold
any values if we did not just try to use it. With the code as it is
written now, if we did encounter such a case (which I think would be a
bug), we'd silently leak those values without even bothering to display
them. With this change, we'd at least eventually show them, and somebody
would notice.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 11:59:07 -08:00
Jeff King
426fd36f4a t4006: resurrect commented-out tests
This set of tests was added by 4434e6ba6c (tests: check --[short]stat
output after chmod, 2012-05-01), and is primarily about the handling of
binary versus text files.

Later, 74faaa16f0 (Fix "git diff --stat" for interesting - but empty -
file changes, 2012-10-17) changed the stat output so that the empty text
file is mentioned rather than omitted. That commit just comments out
these tests. There's no discussion in the commit message, but the
original email[1] says:

  NOTE! This does break two of our tests, so we clearly did this on
  purpose, or at least tested for it. I just uncommented the subtests
  that this makes irrelevant, and changed the output of another one.

I don't think they're irrelevant, though. We should be testing this
"mode change only" case and making sure that it has the post-74faaa16f0
behavior. So this commit brings back those tests, with the current
expected output.

[1] https://public-inbox.org/git/CA+55aFz88GPJcfMSqiyY+u0Cdm48bEyrsTGxHVJbGsYsDg=Q5w@mail.gmail.com/

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 11:58:38 -08:00
Nguyễn Thái Ngọc Duy
f8adbec9fe cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch
By default, index compat macros are off from now on, because they
could hide the_index dependency.

Only those in builtin can use it.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-24 11:55:06 -08:00
Ben Peart
8424bfd45b checkout: fix regression in checkout -b on intitial checkout
When doing a 'checkout -b' do a full checkout including updating the working
tree when doing the initial checkout. As the new test involves an filesystem
access, do it later in the sequence to give chance to other cheaper tests to
leave early. This fixes the regression in behavior caused by fa655d8411
(checkout: optimize "git checkout -b <new_branch>", 2018-08-16).

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-23 13:22:48 -08:00
Ben Peart
91e3d7ca9b checkout: add test demonstrating regression with checkout -b on initial commit
Commit fa655d8411 (checkout: optimize "git checkout -b <new_branch>",
2018-08-16) introduced an unintentional change in behavior for 'checkout -b'
after doing 'clone --no-checkout'.  Add a test to demonstrate the changed
behavior to be used in a later patch to verify the fix.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-23 13:22:47 -08:00
Jeff King
7b95849be4 attr: do not mark queried macros as unset
Since 60a12722ac (attr: remove maybe-real, maybe-macro from git_attr,
2017-01-27), we will always mark an attribute macro (e.g., "binary")
that is specifically queried for as "unspecified", even though listing
_all_ attributes would display it at set. E.g.:

  $ echo "* binary" >.gitattributes

  $ git check-attr -a file
  file: binary: set
  file: diff: unset
  file: merge: unset
  file: text: unset

  $ git check-attr binary file
  file: binary: unspecified

The problem stems from an incorrect conversion of the optimization from
06a604e670 (attr: avoid heavy work when we know the specified attr is
not defined, 2014-12-28). There we tried in collect_some_attrs() to
avoid even looking at the attr_stack when the user has asked for "foo"
and we know that "foo" did not ever appear in any .gitattributes file.

It used a flag "maybe_real" in each attribute struct, where "real" meant
that the attribute appeared in an actual file (we have to make this
distinction because we also create an attribute struct for any names
that are being queried). But as explained in that commit message, the
meaning of "real" was tangled with some special cases around macros.

When 60a12722ac later refactored the macro code, it dropped maybe_real
entirely. This missed the fact that "maybe_real" could be unset for two
reasons: because of a macro, or because it was never found during
parsing. This had two results:

  - the optimization in collect_some_attrs() ceased doing anything
    meaningful, since it no longer kept track of "was it found during
    parsing"

  - worse, it actually kicked in when the caller _did_ ask about a macro
    by name, causing us to mark it as unspecified

It should be possible to salvage this optimization, but let's start with
just removing the remnants. It hasn't been doing anything (except
creating bugs) since 60a12722ac, and nobody seems to have noticed the
performance regression. It's more important to fix the correctness
problem clearly first.

I've added two tests here. The second one actually shows off the bug.
The test of "check-attr -a" is not strictly necessary, but we currently
do not test attribute macros much, and the builtin "binary" not at all.
So this increases our general test coverage, as well as making sure we
didn't mess up this related case.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-22 13:48:15 -08:00
Johannes Schindelin
d609615f48 tests: explicitly use test-tool.exe on Windows
In 8abfdf44c8 (tests: explicitly use `git.exe` on Windows,
2018-11-14), we made sure to use the `.exe` file extension when
using an absolute path to `git.exe`, to avoid getting confused with a
file or directory in the same place that lacks said file extension.

For the same reason, we need to handle test-tool.exe the same way.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-22 12:35:59 -08:00
SZEDER Gábor
5af7417bd8 commit-graph: rename "large edges" to "extra edges"
The optional 'Large Edge List' chunk of the commit graph file stores
parent information for commits with more than two parents, and the
names of most of the macros, variables, struct fields, and functions
related to this chunk contain the term "large edges", e.g.
write_graph_chunk_large_edges().  However, it's not a really great
term, as the edges to the second and subsequent parents stored in this
chunk are not any larger than the edges to the first and second
parents stored in the "main" 'Commit Data' chunk.  It's the number of
edges, IOW number of parents, that is larger compared to non-merge and
"regular" two-parent merge commits.  And indeed, two functions in
'commit-graph.c' have a local variable called 'num_extra_edges' that
refer to the same thing, and this "extra edges" term is much better at
describing these edges.

So let's rename all these references to "large edges" in macro,
variable, function, etc. names to "extra edges".  There is a
GRAPH_OCTOPUS_EDGES_NEEDED macro as well; for the sake of consistency
rename it to GRAPH_EXTRA_EDGES_NEEDED.

We can do so safely without causing any incompatibility issues,
because the term "large edges" doesn't come up in the file format
itself in any form (the chunk's magic is {'E', 'D', 'G', 'E'}, there
is no 'L' in there), but only in the specification text.  The string
"large edges", however, does come up in the output of 'git
commit-graph read' and in tests looking at its input, but that command
is explicitly documented as debugging aid, so we can change its output
and the affected tests safely.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-22 11:33:46 -08:00
Brandon Richardson
70ddbd7767 commit-tree: add missing --gpg-sign flag
Add --gpg-sign option in commit-tree, which was documented, but not
implemented, in 55ca3f99ae. Add tests for the --gpg-sign option.

Signed-off-by: Brandon Richardson <brandon1024.br@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-22 11:08:35 -08:00
Martin Ågren
41a74bd013 t7510: invoke git as part of &&-chain
If `git commit-tree HEAD^{tree}` fails on us and produces no output on
stdout, we will substitute that empty string and execute `git tag
ninth-unsigned`, i.e., we will tag HEAD rather than a newly created
object. But we are lucky: we have a signature on HEAD, so we should
eventually fail the next test, where we verify that "ninth-unsigned" is
indeed unsigned.

We have a similar problem a few lines later. If `git commit-tree -S`
fails with no output, we will happily tag HEAD as "tenth-signed". Here,
we are not so lucky. The tag ends up on the same commit as
"eighth-signed-alt", and that's a signed commit, so t7510-signed-commit
will pass, despite `git commit-tree -S` failing.

Make these `git commit-tree` invocations a direct part of the &&-chain,
so that we can rely less on luck and set a better example for future
tests modeled after this one. Fix a 9/10 copy/paste error while at it.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Brandon Richardson <brandon1024.br@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-22 11:08:33 -08:00