Commit Graph

10837 Commits

Author SHA1 Message Date
Michael Haggerty
433860f3d0 diff: improve positioning of add/delete blocks in diffs
Some groups of added/deleted lines in diffs can be slid up or down,
because lines at the edges of the group are not unique. Picking good
shifts for such groups is not a matter of correctness but definitely has
a big effect on aesthetics. For example, consider the following two
diffs. The first is what standard Git emits:

    --- a/9c572b21dd090a1e5c5bb397053bf8043ffe7fb4:git-send-email.perl
    +++ b/6dcfa306f2b67b733a7eb2d7ded1bc9987809edb:git-send-email.perl
    @@ -231,6 +231,9 @@ if (!defined $initial_reply_to && $prompting) {
     }

     if (!$smtp_server) {
    +       $smtp_server = $repo->config('sendemail.smtpserver');
    +}
    +if (!$smtp_server) {
            foreach (qw( /usr/sbin/sendmail /usr/lib/sendmail )) {
                    if (-x $_) {
                            $smtp_server = $_;

The following diff is equivalent, but is obviously preferable from an
aesthetic point of view:

    --- a/9c572b21dd090a1e5c5bb397053bf8043ffe7fb4:git-send-email.perl
    +++ b/6dcfa306f2b67b733a7eb2d7ded1bc9987809edb:git-send-email.perl
    @@ -230,6 +230,9 @@ if (!defined $initial_reply_to && $prompting) {
            $initial_reply_to =~ s/(^\s+|\s+$)//g;
     }

    +if (!$smtp_server) {
    +       $smtp_server = $repo->config('sendemail.smtpserver');
    +}
     if (!$smtp_server) {
            foreach (qw( /usr/sbin/sendmail /usr/lib/sendmail )) {
                    if (-x $_) {

This patch teaches Git to pick better positions for such "diff sliders"
using heuristics that take the positions of nearby blank lines and the
indentation of nearby lines into account.

The existing Git code basically always shifts such "sliders" as far down
in the file as possible. The only exception is when the slider can be
aligned with a group of changed lines in the other file, in which case
Git favors depicting the change as one add+delete block rather than one
add and a slightly offset delete block. This naive algorithm often
yields ugly diffs.

Commit d634d61ed6 improved the situation somewhat by preferring to
position add/delete groups to make their last line a blank line, when
that is possible. This heuristic does more good than harm, but (1) it
can only help if there are blank lines in the right places, and (2)
always picks the last blank line, even if there are others that might be
better. The end result is that it makes perhaps 1/3 as many errors as
the default Git algorithm, but that still leaves a lot of ugly diffs.

This commit implements a new and much better heuristic for picking
optimal "slider" positions using the following approach: First observe
that each hypothetical positioning of a diff slider introduces two
splits: one between the context lines preceding the group and the first
added/deleted line, and the other between the last added/deleted line
and the first line of context following it. It tries to find the
positioning that creates the least bad splits.

Splits are evaluated based only on the presence and locations of nearby
blank lines, and the indentation of lines near the split. Basically, it
prefers to introduce splits adjacent to blank lines, between lines that
are indented less, and between lines with the same level of indentation.
In more detail:

1. It measures the following characteristics of a proposed splitting
   position in a `struct split_measurement`:

   * the number of blank lines above the proposed split
   * whether the line directly after the split is blank
   * the number of blank lines following that line
   * the indentation of the nearest non-blank line above the split
   * the indentation of the line directly below the split
   * the indentation of the nearest non-blank line after that line

2. It combines the measured attributes using a bunch of
   empirically-optimized weighting factors to derive a `struct
   split_score` that measures the "badness" of splitting the text at
   that position.

3. It combines the `split_score` for the top and the bottom of the
   slider at each of its possible positions, and selects the position
   that has the best `split_score`.

I determined the initial set of weighting factors by collecting a corpus
of Git histories from 29 open-source software projects in various
programming languages. I generated many diffs from this corpus, and
determined the best positioning "by eye" for about 6600 diff sliders. I
used about half of the repositories in the corpus (corresponding to
about 2/3 of the sliders) as a training set, and optimized the weights
against this corpus using a crude automated search of the parameter
space to get the best agreement with the manually-determined values.
Then I tested the resulting heuristic against the full corpus. The
results are summarized in the following table, in column `indent-1`:

| repository            | count |      Git 2.9.0 |     compaction | compaction-fixed |       indent-1 |       indent-2 |
| --------------------- | ----- | -------------- | -------------- | ---------------- | -------------- | -------------- |
| afnetworking          |   109 |    89  (81.7%) |    37  (33.9%) |      37  (33.9%) |     2   (1.8%) |     2   (1.8%) |
| alamofire             |    30 |    18  (60.0%) |    14  (46.7%) |      15  (50.0%) |     0   (0.0%) |     0   (0.0%) |
| angular               |   184 |   127  (69.0%) |    39  (21.2%) |      23  (12.5%) |     5   (2.7%) |     5   (2.7%) |
| animate               |   313 |     2   (0.6%) |     2   (0.6%) |       2   (0.6%) |     2   (0.6%) |     2   (0.6%) |
| ant                   |   380 |   356  (93.7%) |   152  (40.0%) |     148  (38.9%) |    15   (3.9%) |    15   (3.9%) | *
| bugzilla              |   306 |   263  (85.9%) |   109  (35.6%) |      99  (32.4%) |    14   (4.6%) |    15   (4.9%) | *
| corefx                |   126 |    91  (72.2%) |    22  (17.5%) |      21  (16.7%) |     6   (4.8%) |     6   (4.8%) |
| couchdb               |    78 |    44  (56.4%) |    26  (33.3%) |      28  (35.9%) |     6   (7.7%) |     6   (7.7%) | *
| cpython               |   937 |   158  (16.9%) |    50   (5.3%) |      49   (5.2%) |     5   (0.5%) |     5   (0.5%) | *
| discourse             |   160 |    95  (59.4%) |    42  (26.2%) |      36  (22.5%) |    18  (11.2%) |    13   (8.1%) |
| docker                |   307 |   194  (63.2%) |   198  (64.5%) |     253  (82.4%) |     8   (2.6%) |     8   (2.6%) | *
| electron              |   163 |   132  (81.0%) |    38  (23.3%) |      39  (23.9%) |     6   (3.7%) |     6   (3.7%) |
| git                   |   536 |   470  (87.7%) |    73  (13.6%) |      78  (14.6%) |    16   (3.0%) |    16   (3.0%) | *
| gitflow               |   127 |     0   (0.0%) |     0   (0.0%) |       0   (0.0%) |     0   (0.0%) |     0   (0.0%) |
| ionic                 |   133 |    89  (66.9%) |    29  (21.8%) |      38  (28.6%) |     1   (0.8%) |     1   (0.8%) |
| ipython               |   482 |   362  (75.1%) |   167  (34.6%) |     169  (35.1%) |    11   (2.3%) |    11   (2.3%) | *
| junit                 |   161 |   147  (91.3%) |    67  (41.6%) |      66  (41.0%) |     1   (0.6%) |     1   (0.6%) | *
| lighttable            |    15 |     5  (33.3%) |     0   (0.0%) |       2  (13.3%) |     0   (0.0%) |     0   (0.0%) |
| magit                 |    88 |    75  (85.2%) |    11  (12.5%) |       9  (10.2%) |     1   (1.1%) |     0   (0.0%) |
| neural-style          |    28 |     0   (0.0%) |     0   (0.0%) |       0   (0.0%) |     0   (0.0%) |     0   (0.0%) |
| nodejs                |   781 |   649  (83.1%) |   118  (15.1%) |     111  (14.2%) |     4   (0.5%) |     5   (0.6%) | *
| phpmyadmin            |   491 |   481  (98.0%) |    75  (15.3%) |      48   (9.8%) |     2   (0.4%) |     2   (0.4%) | *
| react-native          |   168 |   130  (77.4%) |    79  (47.0%) |      81  (48.2%) |     0   (0.0%) |     0   (0.0%) |
| rust                  |   171 |   128  (74.9%) |    30  (17.5%) |      27  (15.8%) |    16   (9.4%) |    14   (8.2%) |
| spark                 |   186 |   149  (80.1%) |    52  (28.0%) |      52  (28.0%) |     2   (1.1%) |     2   (1.1%) |
| tensorflow            |   115 |    66  (57.4%) |    48  (41.7%) |      48  (41.7%) |     5   (4.3%) |     5   (4.3%) |
| test-more             |    19 |    15  (78.9%) |     2  (10.5%) |       2  (10.5%) |     1   (5.3%) |     1   (5.3%) | *
| test-unit             |    51 |    34  (66.7%) |    14  (27.5%) |       8  (15.7%) |     2   (3.9%) |     2   (3.9%) | *
| xmonad                |    23 |    22  (95.7%) |     2   (8.7%) |       2   (8.7%) |     1   (4.3%) |     1   (4.3%) | *
| --------------------- | ----- | -------------- | -------------- | ---------------- | -------------- | -------------- |
| totals                |  6668 |  4391  (65.9%) |  1496  (22.4%) |    1491  (22.4%) |   150   (2.2%) |   144   (2.2%) |
| totals (training set) |  4552 |  3195  (70.2%) |  1053  (23.1%) |    1061  (23.3%) |    86   (1.9%) |    88   (1.9%) |
| totals (test set)     |  2116 |  1196  (56.5%) |   443  (20.9%) |     430  (20.3%) |    64   (3.0%) |    56   (2.6%) |

In this table, the numbers are the count and percentage of human-rated
sliders that the corresponding algorithm got *wrong*. The columns are

* "repository" - the name of the repository used. I used the diffs
  between successive non-merge commits on the HEAD branch of the
  corresponding repository.

* "count" - the number of sliders that were human-rated. I chose most,
  but not all, sliders to rate from those among which the various
  algorithms gave different answers.

* "Git 2.9.0" - the default algorithm used by `git diff` in Git 2.9.0.

* "compaction" - the heuristic used by `git diff --compaction-heuristic`
  in Git 2.9.0.

* "compaction-fixed" - the heuristic used by `git diff
  --compaction-heuristic` after the fixes from earlier in this patch
  series. Note that the results are not dramatically different than
  those for "compaction". Both produce non-ideal diffs only about 1/3 as
  often as the default `git diff`.

* "indent-1" - the new `--indent-heuristic` algorithm, using the first
  set of weighting factors, determined as described above.

* "indent-2" - the new `--indent-heuristic` algorithm, using the final
  set of weighting factors, determined as described below.

* `*` - indicates that repo was part of training set used to determine
  the first set of weighting factors.

The fact that the heuristic performed nearly as well on the test set as
on the training set in column "indent-1" is a good indication that the
heuristic was not over-trained. Given that fact, I ran a second round of
optimization, using the entire corpus as the training set. The resulting
set of weights gave the results in column "indent-2". These are the
weights included in this patch.

The final result gives consistently and significantly better results
across the whole corpus than either `git diff` or `git diff
--compaction-heuristic`. It makes only about 1/30 as many errors as the
former and about 1/10 as many errors as the latter. (And a good fraction
of the remaining errors are for diffs that involve weirdly-formatted
code, sometimes apparently machine-generated.)

The tools that were used to do this optimization and analysis, along
with the human-generated data values, are recorded in a separate project
[1].

This patch adds a new command-line option `--indent-heuristic`, and a
new configuration setting `diff.indentHeuristic`, that activate this
heuristic. This interface is only meant for testing purposes, and should
be finalized before including this change in any release.

[1] https://github.com/mhagger/diff-slider-tools

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-19 10:25:11 -07:00
Junio C Hamano
34bf3bbb30 Merge branch 'nd/test-lib-httpd-show-error-log-in-verbose'
HTTPd tests learned to show the server error log to help diagnosing
a failing tests.

* nd/test-lib-httpd-show-error-log-in-verbose:
  lib-httpd.sh: print error.log on error
2016-07-06 13:38:08 -07:00
Junio C Hamano
b8b6365a8a Merge branch 'jk/string-list-static-init'
Instead of taking advantage of a struct string_list that is
allocated with all NULs happens to be STRING_LIST_INIT_NODUP kind,
initialize them explicitly as such, to document their behaviour
better.

* jk/string-list-static-init:
  use string_list initializer consistently
  blame,shortlog: don't make local option variables static
  interpret-trailers: don't duplicate option strings
  parse_opt_string_list: stop allocating new strings
2016-07-06 13:38:08 -07:00
Junio C Hamano
c49fd57bf4 Merge branch 'et/add-chmod-x'
"git update-index --add --chmod=+x file" may be usable as an escape
hatch, but not a friendly thing to force for people who do need to
use it regularly.  "git add --chmod=+x file" can be used instead.

* et/add-chmod-x:
  add: add --chmod=+x / --chmod=-x options
2016-06-27 09:56:49 -07:00
Junio C Hamano
3873075a12 Merge branch 'sg/reflog-past-root'
"git reflog" stopped upon seeing an entry that denotes a branch
creation event (aka "unborn"), which made it appear as if the
reflog was truncated.

* sg/reflog-past-root:
  reflog: continue walking the reflog past root commits
2016-06-27 09:56:46 -07:00
Junio C Hamano
d15c05a5d0 Merge branch 'rs/xdiff-hunk-with-func-line'
"git show -W" (extend hunks to cover the entire function, delimited
by lines that match the "funcname" pattern) used to show the entire
file when a change added an entire function at the end of the file,
which has been fixed.

* rs/xdiff-hunk-with-func-line:
  xdiff: fix merging of appended hunk with -W
  grep: -W: don't extend context to trailing empty lines
  t7810: add test for grep -W and trailing empty context lines
  xdiff: don't trim common tail with -W
  xdiff: -W: don't include common trailing empty lines in context
  xdiff: ignore empty lines before added functions with -W
  xdiff: handle appended chunks better with -W
  xdiff: factor out match_func_rec()
  t4051: rewrite, add more tests
2016-06-20 11:01:04 -07:00
Junio C Hamano
6d8c5454b6 Merge branch 'jk/rev-list-count-with-bitmap'
"git rev-list --count" whose walk-length is limited with "-n"
option did not work well with the counting optimized to look at the
bitmap index.

* jk/rev-list-count-with-bitmap:
  rev-list: disable bitmaps when "-n" is used with listing objects
  rev-list: "adjust" results of "--count --use-bitmap-index -n"
2016-06-20 11:01:03 -07:00
Junio C Hamano
8699b74ae1 Merge branch 'wd/userdiff-css'
Update the funcname definition to support css files.

* wd/userdiff-css:
  userdiff: add built-in pattern for CSS
2016-06-20 11:01:02 -07:00
Junio C Hamano
6d41eb685a Merge branch 'jg/dash-is-last-branch-in-worktree-add'
"git worktree add" learned that '-' can be used as a short-hand for
"@{-1}", the previous branch.

* jg/dash-is-last-branch-in-worktree-add:
  worktree: allow "-" short-hand for @{-1} in add command
2016-06-20 11:01:02 -07:00
Junio C Hamano
1b3d14c1c8 Merge branch 'et/pretty-format-c-auto'
The commands in `git log` family take %C(auto) in a custom format
string.  This unconditionally turned the color on, ignoring
--no-color or with --color=auto when the output is not connected to
a tty; this was corrected to make the format truly behave as
"auto".

* et/pretty-format-c-auto:
  format_commit_message: honor `color=auto` for `%C(auto)`
2016-06-20 11:01:01 -07:00
Junio C Hamano
3807098cd6 Merge branch 'sb/submodule-recommend-shallowness'
An upstream project can make a recommendation to shallowly clone
some submodules in the .gitmodules file it ships.

* sb/submodule-recommend-shallowness:
  submodule update: learn `--[no-]recommend-shallow` option
  submodule-config: keep shallow recommendation around
2016-06-20 11:01:01 -07:00
Junio C Hamano
73bc4b4928 Merge branch 'ah/no-verify-signature-with-pull-rebase'
"git pull --rebase --verify-signature" learned to warn the user
that "--verify-signature" is a no-op when rebasing.

* ah/no-verify-signature-with-pull-rebase:
  pull: warn on --verify-signatures with --rebase
2016-06-20 11:01:00 -07:00
Junio C Hamano
8d6a7e9a19 Merge branch 'ew/fast-import-unpack-limit'
"git fast-import" learned the same performance trick to avoid
creating too small a packfile as "git fetch" and "git push" have,
using *.unpackLimit configuration.

* ew/fast-import-unpack-limit:
  fast-import: invalidate pack_id references after loosening
  fast-import: implement unpack limit
2016-06-20 11:01:00 -07:00
Nguyễn Thái Ngọc Duy
44f243d356 lib-httpd.sh: print error.log on error
Failure to bring up httpd for testing is not considered an error, so the
trash directory, which contains this error.log file, is removed and we
don't know what made httpd fail to start. Improve the situation a bit,
print error.log but only in verbose mode.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 11:50:44 -07:00
Jeff King
2721ce21e4 use string_list initializer consistently
There are two types of string_lists: those that own the
string memory, and those that don't. You can tell the
difference by the strdup_strings flag, and one should use
either STRING_LIST_INIT_DUP, or STRING_LIST_INIT_NODUP as an
initializer.

Historically, the normal all-zeros initialization has
corresponded to the NODUP case. Many sites use no
initializer at all, and that works as a shorthand for that
case. But for a reader of the code, it can be hard to
remember which is which. Let's be more explicit and actually
have each site declare which type it means to use.

This is a fairly mechanical conversion; I assumed each site
was correct as-is, and just switched them all to NODUP.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 10:37:51 -07:00
Junio C Hamano
45c0c21eb9 Merge branch 'jk/shell-portability'
test fixes.

* jk/shell-portability:
  t5500 & t7403: lose bash-ism "local"
  test-lib: add in-shell "env" replacement
2016-06-10 15:26:05 -07:00
Junio C Hamano
8ffc9d26e4 Merge branch 'jc/t2300-setup'
A test fix.

* jc/t2300-setup:
  t2300: run git-sh-setup in an environment that better mimics the real life
2016-06-10 15:26:04 -07:00
René Scharfe
6f8d9bccb2 xdiff: fix merging of appended hunk with -W
When -W is given we search the lines between the end of the current
context and the next change for a function line.  If there is none then
we merge those two hunks as they must be part of the same function.

If the next change is an appended chunk we abort the search early in
get_func_line(), however, because its line number is out of range.  Fix
that by searching from the end of the pre-image in that case instead.

Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-09 15:27:26 -07:00
Edward Thomson
4e55ed32db add: add --chmod=+x / --chmod=-x options
The executable bit will not be detected (and therefore will not be
set) for paths in a repository with `core.filemode` set to false,
though the users may still wish to add files as executable for
compatibility with other users who _do_ have `core.filemode`
functionality.  For example, Windows users adding shell scripts may
wish to add them as executable for compatibility with users on
non-Windows.

Although this can be done with a plumbing command
(`git update-index --add --chmod=+x foo`), teaching the `git-add`
command allows users to set a file executable with a command that
they're already familiar with.

Signed-off-by: Edward Thomson <ethomson@edwardthomson.com>
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-07 17:43:39 -07:00
SZEDER Gábor
71abeb753f reflog: continue walking the reflog past root commits
If a repository contains more than one root commit, then its HEAD
reflog may contain multiple "creation events", i.e. entries whose
"from" value is the null sha1.  Listing such a reflog currently stops
prematurely at the first such entry, even when the reflog still
contains older entries.  This can scare users into thinking that their
reflog got truncated after 'git checkout --orphan'.

Continue walking the reflog past such creation events based on the
preceeding reflog entry's "new" value.

The test 'symbolic-ref writes reflog entry' in t1401-symbolic-ref
implicitly relies on the current behavior of the reflog walker to stop
at a root commit and thus to list only the reflog entries that are
relevant for that test.  Adjust the test to explicitly specify the
number of relevant reflog entries to be listed.

Reported-by: Patrik Gustafsson <pvn@textalk.se>
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-06 15:06:44 -07:00
William Duclot
0719f3eecd userdiff: add built-in pattern for CSS
CSS is widely used, motivating it being included as a built-in pattern.

It must be noted that the word_regex for CSS (i.e. the regex defining
what is a word in the language) does not consider '.' and '#' characters
(in CSS selectors) to be part of the word. This behavior is documented
by the test t/t4018/css-rule.
The logic behind this behavior is the following: identifiers in CSS
selectors are identifiers in a HTML/XML document. Therefore, the '.'/'#'
character are not part of the identifier, but an indicator of the nature
of the identifier in HTML/XML (class or id). Diffing ".class1" and
".class2" must show that the class name is changed, but we still are
selecting a class.

Logic behind the "pattern" regex is:
    1. reject lines ending with a colon/semicolon (properties)
    2. if a line begins with a name in column 1, pick the whole line

Credits to Johannes Sixt (j6t@kdbg.org) for the pattern regex and most
of the tests.

Signed-off-by: William Duclot <william.duclot@ensimag.grenoble-inp.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>
Reviewed-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-03 14:45:56 -07:00
Junio C Hamano
be3ac81f0c Merge branch 'js/perf-rebase-i'
The one in 'master' has a brown-paper-bag bug that breaks the perf
test when used inside a usual Git repository with a working tree.

* js/perf-rebase-i:
  perf: make the tests work without a worktree
2016-06-03 14:38:00 -07:00
Jeff King
5c9f9bf313 rev-list: "adjust" results of "--count --use-bitmap-index -n"
If you ask rev-list for:

    git rev-list --count --use-bitmap-index HEAD

we optimize out the actual traversal and just give you the
number of bits set in the commit bitmap. This is faster,
which is good.

But if you ask to limit the size of the traversal, like:

    git rev-list --count --use-bitmap-index -n 100 HEAD

we'll still output the full bitmapped number we found. On
the surface, that might even seem OK. You explicitly asked
to use the bitmap index, and it was cheap to compute the
real answer, so we gave it to you.

But there's something much more complicated going on under
the hood. If we don't have a bitmap directly for HEAD, then
we have to actually traverse backwards, looking for a
bitmapped commit. And _that_ traversal is bounded by our
`-n` count.

This is a good thing, because it bounds the work we have to
do, which is probably what the user wanted by asking for
`-n`. But now it makes the output quite confusing. You might
get many values:

  - your `-n` value, if we walked back and never found a
    bitmap (or fewer if there weren't that many commits)

  - the actual full count, if we found a bitmap root for
    every path of our traversal with in the `-n` limit

  - any number in between! We might have walked back and
    found _some_ bitmaps, but then cut off the traversal
    early with some commits not accounted for in the result.

So you cannot even see a value higher than your `-n` and say
"OK, bitmaps kicked in, this must be the real full count".
The only sane thing is for git to just clamp the value to a
maximum of the `-n` value, which means we should output the
exact same results whether bitmaps are in use or not.

The test in t5310 demonstrates this by using `-n 1`.
Without this patch we fail in the full-bitmap case (where we
do not have to traverse at all) but _not_ in the
partial-bitmap case (where we have to walk down to find an
actual bitmap). With this patch, both cases just work.

I didn't implement the crazy in-between case, just because
it's complicated to set up, and is really a subset of the
full-count case, which we do cover.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-03 09:00:59 -07:00
Junio C Hamano
fe17fc0006 t2300: run git-sh-setup in an environment that better mimics the real life
When we run scripted Porcelains, "git" potty has set up the $PATH by
prepending $GIT_EXEC_PATH, the path given by "git --exec-path=$there
$cmd", etc. already.  Because of this, scripted Porcelains can
dot-source shell script library like git-sh-setup with simple dot
without specifying any path.

t2300 however dot-sources git-sh-setup without adjusting $PATH like
the real "git" potty does.  This has not been a problem so far, but
once git-sh-setup wants to rely on the $PATH adjustment, just like
any scripted Porcelains already do, it would become one.  It cannot
for example dot-source another shell library without specifying the
full path to it by prefixing $(git --exec-path).

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 14:15:17 -07:00
Junio C Hamano
e256eec79d t5500 & t7403: lose bash-ism "local"
In t5500::check_prot_host_port_path(), diagport is not a variable
used elsewhere and the function is not recursively called so this
can simply lose the "local", which may not be supported by shell
(besides, the function liberally clobbers other variables without
making them "local").

t7403::reset_submodule_urls() overrides the "root" variable used
in the test framework for no good reason; its use is not about
temporarily relocating where the test repositories are created.
This assignment can be made not to clobber the variable by moving
them into the subshells it already uses.  Its value is always
$TRASH_DIRECTORY, so we could use it instead there, and this
function that is called only once and its two subshells may not be
necessary (instead, the caller can use "git -C $there config" and
set a value that is derived from $TRASH_DIRECTORY), but this is a
minimum fix that is needed to lose "local".

Helped-by: John Keeping <john@keeping.me.uk>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 14:00:33 -07:00
Jeff King
d2554c7207 test-lib: add in-shell "env" replacement
The one-shot environment variable syntax:

  FOO=BAR some-program

is unportable when some-program is actually a shell
function, like test_must_fail (on some shells FOO remains
set after the function returns, and on others it does not).

We sometimes get around this by using env, like:

  test_must_fail env FOO=BAR some-program

But that only works because test_must_fail's arguments are
themselves a command which can be run. You can't run:

  env FOO=BAR test_must_fail some-program

because env does not know about our shell functions. So
there is no equivalent for test_commit, for example, and one
must resort to:

  (
    FOO=BAR
    export FOO
    test_commit
  )

which is a bit verbose.  Let's add a version of "env" that
works _inside_ the shell, by creating a subshell, exporting
variables from its argument list, and running the command.

Its use is demonstrated on a currently-unportable case in
t4014.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 08:04:08 -07:00
Junio C Hamano
3296e1a93a Merge branch 'sb/submodule-deinit-all' into maint
Correct faulty recommendation to use "git submodule deinit ." when
de-initialising all submodules, which would result in a strange
error message in a pathological corner case.

* sb/submodule-deinit-all:
  submodule deinit: require '--all' instead of '.' for all submodules
2016-05-31 14:09:46 -07:00
Junio C Hamano
68a6e976a8 Merge branch 'jk/test-send-sh-x-trace-elsewhere' into maint
Running tests with '-x' option to trace the individual command
executions is a useful way to debug test scripts, but some tests
that capture the standard error stream and check what the command
said can be broken with the trace output mixed in.  When running
our tests under "bash", however, we can redirect the trace output
to another file descriptor to keep the standard error of programs
being tested intact.

* jk/test-send-sh-x-trace-elsewhere:
  test-lib: set BASH_XTRACEFD automatically
2016-05-31 14:08:27 -07:00
Junio C Hamano
9ee8f9409c Merge branch 'js/name-rev-use-oldest-ref' into maint
"git describe --contains" often made a hard-to-justify choice of
tag to give name to a given commit, because it tried to come up
with a name with smallest number of hops from a tag, causing an old
commit whose close descendant that is recently tagged were not
described with respect to an old tag but with a newer tag.  It did
not help that its computation of "hop" count was further tweaked to
penalize being on a side branch of a merge.  The logic has been
updated to favor using the tag with the oldest tagger date, which
is a lot easier to explain to the end users: "We describe a commit
in terms of the (chronologically) oldest tag that contains the
commit."

* js/name-rev-use-oldest-ref:
  name-rev: include taggerdate in considering the best name
2016-05-31 14:08:26 -07:00
René Scharfe
e2522f2aca perf: make the tests work without a worktree
In regular repositories $source_git and $objects_dir contain relative
paths based on $source.  Go there to allow cp to resolve them.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 13:44:59 -07:00
René Scharfe
4aa2c4753d grep: -W: don't extend context to trailing empty lines
Empty lines between functions are shown by grep -W, as it considers them
to be part of the function preceding them.  They are not interesting in
most languages.  The previous patches stopped showing them for diff -W.

Stop showing empty lines trailing a function with grep -W.  Grep scans
the lines of a buffer from top to bottom and prints matching lines
immediately.  Thus we need to peek ahead in order to determine if an
empty line is part of a function body and worth showing or not.

Remember how far ahead we peeked in order to avoid having to do so
repeatedly when handling multiple consecutive empty lines.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 13:08:56 -07:00
René Scharfe
799e09e5fb t7810: add test for grep -W and trailing empty context lines
Add a test demonstrating that git grep -W prints empty lines following
the function context we're actually interested in.  The modified test
file makes it necessary to adjust three unrelated test cases.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 13:08:56 -07:00
René Scharfe
e0876bca4d xdiff: don't trim common tail with -W
The function trim_common_tail() exits early if context lines are
requested.  If -U0 and -W are specified together then it can still trim
context lines that might belong to a changed function.  As a result
that function is shown incompletely.

Fix that by calling trim_common_tail() only if no function context or
fixed context is requested.  The parameter ctx is no longer needed now;
remove it.

While at it fix an outdated comment as well.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 13:08:56 -07:00
René Scharfe
9e6a4cfc38 xdiff: -W: don't include common trailing empty lines in context
Empty lines between functions are shown by diff -W, as it considers them
to be part of the function preceding them.  They are not interesting in
most languages.  The previous patch stopped showing them in the special
case of a function added at the end of a file.

Stop extending context to those empty lines by skipping back over them
from the start of the next function.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 13:08:56 -07:00
René Scharfe
392f6d3166 xdiff: ignore empty lines before added functions with -W
If a new function and a preceding empty line is appended, diff -W shows
the previous function in full in order to provide context for that empty
line.  In most languages empty lines between sections are not
interesting in and off themselves and showing a whole extra function for
them is not what we want.

Skip empty lines when checking of the appended chunk starts with a
function line, thereby avoiding to extend the context just for them.

Helped-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 13:08:56 -07:00
René Scharfe
6d5badb238 xdiff: handle appended chunks better with -W
If lines are added at the end of a file, diff -W shows the whole file.
That's because get_func_line() only considers the pre-image and gives up
if it sees a record index beyond its end.

Consider the post-image as well to see if the added lines already make
up a full function.  If it doesn't then search for the previous function
line by starting from the bottom of the pre-image, thereby avoiding to
confuse get_func_line().

Reuse the existing label called "again", as it's exactly where we need
to jump to when we're done handling the pre-context, but rename it to
"post_context_calculation" in order to document its new purpose better.

Reported-by: Junio C Hamano <gitster@pobox.com>
Initial-patch-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 13:08:56 -07:00
René Scharfe
d3621de789 t4051: rewrite, add more tests
Remove the tests that checked against a fixed result and replace them
with more focused checks of desired properties of the created diffs.
That way we get more detailed and meaningful diagnostics.

Store test file contents in files in a subdirectory in order to avoid
cluttering the test script with them.

Use tagged commits to store the changes to test diff -W against instead
of using changes to the worktree.  Use the worktree instead to try and
apply the generated patch in order to validate it.

Document unwanted features: trailing empty lines, too much context for
appended functions, insufficient context at the end with -U0.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 13:07:58 -07:00
Junio C Hamano
39fbe92248 Merge branch 'es/t1500-modernize'
test updates to make it more readable and maintainable.

* es/t1500-modernize:
  t1500: avoid setting environment variables outside of tests
  t1500: avoid setting configuration options outside of tests
  t1500: avoid changing working directory outside of tests
  t1500: test_rev_parse: facilitate future test enhancements
  t1500: be considerate to future potential tests
2016-05-31 12:40:55 -07:00
Junio C Hamano
bc4b9247df Merge branch 'fc/fast-import-broken-marks-file'
"git fast-import --export-marks" would overwrite the existing marks
file even when it makes a dump from its custom die routine.
Prevent it from doing so when we have an import-marks file but
haven't finished reading it.

* fc/fast-import-broken-marks-file:
  fast-import: do not truncate exported marks file
2016-05-31 12:40:53 -07:00
Jordan DE GEA
1a450e2fd1 worktree: allow "-" short-hand for @{-1} in add command
Since `git worktree add` uses `git checkout` when `[<branch>]` is used,
and `git checkout -` is already supported, it makes sense to allow the
same shortcut in `git worktree add`.

Signed-off-by: Jordan DE GEA <jordan.de-gea@grenoble-inp.org>
Signed-off-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 12:28:25 -07:00
Junio C Hamano
c6c655fdb1 Merge branch 'ak/t0008-ksh88-workaround'
Test portability workaround.

* ak/t0008-ksh88-workaround:
  t0008: 4 tests fail with ksh88
2016-05-29 18:06:44 -07:00
Junio C Hamano
10184b2718 Merge branch 'js/t6044-use-test-seq'
Test portability fix.

* js/t6044-use-test-seq:
  t6044: replace seq by test_seq
2016-05-29 18:06:43 -07:00
Eric Wong
d2986d0f29 fast-import: invalidate pack_id references after loosening
When loosening a pack, the current pack_id gets reused when
checkpointing and the import does not terminate.  This causes
problems after checkpointing as the object table, branch, and
tag lists still contains pre-checkpoint references to the
recycled pack_id.

Merely clearing the object_table as suggested by Jeff King in
http://mid.gmane.org/20160517121330.GA7346@sigill.intra.peff.net
is insufficient as the marks set still contains references
to object entries.

Wrong pack_id references branch and tags lists do not cause
errors, but can lead to misleading crash reports and core dumps,
so they are also invalidated.

Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-29 17:58:34 -07:00
Edward Thomson
b15a3e005a format_commit_message: honor color=auto for %C(auto)
git-log(1) documents that when specifying the `%C(auto)` format
placeholder will "turn on auto coloring on the next %placeholders
until the color is switched again."

However, when `%C(auto)` is used, the present implementation will turn
colors on unconditionally (even if the color configuration is turned off
for the current context - for example, `--no-color` was specified or the
color is `auto` and the output is not a tty).

Update `format_commit_one` to examine the current context when a format
string of `%C(auto)` is specified, which ensures that we will not
unconditionally write colors.  This brings that behavior in line with
the behavior of `%C(auto,<colorname>)`, and allows the user the ability
to specify that color should be displayed only when the output is a
tty.

Additionally, add a test for `%C(auto)` and update the existing tests
for `%C(auto,...)` as they were misidentified as being applicable to
`%C(auto)`.

Tests from Jeff King.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Edward Thomson <ethomson@edwardthomson.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-27 11:24:54 -07:00
Stefan Beller
abed000aca submodule update: learn --[no-]recommend-shallow option
Sometimes the history of a submodule is not considered important by
the projects upstream. To make it easier for downstream users, allow
a boolean field 'submodule.<name>.shallow' in .gitmodules, which can
be used to recommend whether upstream considers the history important.

This field is honored in the initial clone by default, it can be
ignored by giving the `--no-recommend-shallow` option.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-27 10:40:46 -07:00
Junio C Hamano
f14acabf3a Merge branch 'jc/fsck-nul-in-commit' into maint
"git fsck" learned to catch NUL byte in a commit object as
potential error and warn.

* jc/fsck-nul-in-commit:
  fsck: detect and warn a commit with embedded NUL
  fsck_commit_buffer(): do not special case the last validation
2016-05-26 13:17:24 -07:00
Junio C Hamano
e29300d69f Merge branch 'js/windows-dotgit' into maint
On Windows, .git and optionally any files whose name starts with a
dot are now marked as hidden, with a core.hideDotFiles knob to
customize this behaviour.

* js/windows-dotgit:
  mingw: remove unnecessary definition
  mingw: introduce the 'core.hideDotFiles' setting
2016-05-26 13:17:23 -07:00
Junio C Hamano
d07211b5fa Merge branch 'lp/typofixes' into maint
Typofixes.

* lp/typofixes:
  typofix: assorted typofixes in comments, documentation and messages
2016-05-26 13:17:21 -07:00
Junio C Hamano
1f62b9256d Merge branch 'sb/z-is-gnutar-ism' into maint
Test fix.

* sb/z-is-gnutar-ism:
  t6041: do not compress backup tar file
  t3513: do not compress backup tar file
2016-05-26 13:17:21 -07:00
Junio C Hamano
b262b8f889 Merge branch 'va/i18n-misc-updates' into maint
Mark several messages for translation.

* va/i18n-misc-updates:
  i18n: unpack-trees: avoid substituting only a verb in sentences
  i18n: builtin/pull.c: split strings marked for translation
  i18n: builtin/pull.c: mark placeholders for translation
  i18n: git-parse-remote.sh: mark strings for translation
  i18n: branch: move comment for translators
  i18n: branch: unmark string for translation
  i18n: builtin/rm.c: remove a comma ',' from string
  i18n: unpack-trees: mark strings for translation
  i18n: builtin/branch.c: mark option for translation
  i18n: index-pack: use plural string instead of normal one
2016-05-26 13:17:20 -07:00