Commit Graph

12511 Commits

Author SHA1 Message Date
Christian Couder
19cf57a92e perf/run: read GIT_PERF_REPO_NAME from perf.repoName
The GIT_PERF_REPO_NAME env variable is used in
the `aggregate.perl` script to set the 'environment'
field in the JSON Codespeed output.

Let's make it easy to set this variable by setting it
in a config file.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-05 12:31:08 -08:00
Christian Couder
fccec20f0b perf/run: learn to send output to codespeed server
Let's make it possible to set in a config file the URL of
a codespeed server. And then let's make the `run` script
send the perf test results to this URL at the end of the
tests.

This should make is possible to easily automate the process
of running perf tests and having their results available in
Codespeed.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-05 12:31:08 -08:00
Christian Couder
5d6bb93090 perf/run: learn about perf.codespeedOutput
Let's make it possible to set in a config file the output
format (regular or codespeed) of the perf tests.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-05 12:31:08 -08:00
Christian Couder
3ae7d2b0cd perf/run: add conf_opts argument to get_var_from_env_or_config()
Let's make it possible to use `git config` type specifiers like
`--int` or `--bool`, so that config values are converted to the
canonical form and easier to use.

This additional argument is now the fourth argument of
get_var_from_env_or_config() instead of the fifth because we
want the default value argument to be unset if it is not
passed, and this is simpler if it is the last argument.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-05 12:31:07 -08:00
Christian Couder
05eb1c37ed perf/aggregate: implement codespeed JSON output
Codespeed (https://github.com/tobami/codespeed/) is an open source
project that can be used to track how some software performs over
time. It stores performance test results in a database and can show
nice graphs and charts on a web interface.

As it can be interesting to use Codespeed to see how Git performance
evolves over time and releases, let's implement a Codespeed output
in "perf/aggregate.perl".

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-05 12:31:07 -08:00
Christian Couder
30ffff6ee2 perf/aggregate: refactor printing results
As we want to implement another kind of output than
the current output for the perf test results, let's
refactor the existing code that outputs the results
in its own print_default_results() function.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-05 12:31:07 -08:00
Christian Couder
6f5ecad6a5 perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION}
The way we check ENV{GIT_PERF_SUBSECTION} could trigger
comparison between undef and "" that may be flagged by
use of strict & warnings. Let's fix that.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-05 12:31:07 -08:00
Christian Couder
5d445f3416 perf: store subsection results in "test-results/$GIT_PERF_SUBSECTION/"
When tests are run for a subsection defined in a config file, it is
better if the results for the current subsection are not overwritting
the results of a previous subsection.

So let's store the results for a subsection in a subdirectory of
"test-results/" with the subsection name.

The aggregate.perl, when it is run for a subsection, should then
aggregate the results found in "test-results/$GIT_PERF_SUBSECTION/".

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Christian Couder
ffdd01076e perf/run: show name of rev being built
It is nice for the user to not just show the sha1 of the
current revision being built but also the actual name of
this revision.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Christian Couder
afda85c25d perf/run: add run_subsection()
Let's actually use the subsections we find in the config file
to run the perf tests separately for each subsection.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Christian Couder
9ba95ed23c perf/run: update get_var_from_env_or_config() for subsections
As we will set some config options in subsections, let's
teach get_var_from_env_or_config() to get the config options
from the subsections if they are set there.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Christian Couder
2638441e07 perf/run: add get_subsections()
This function makes it possible to find subsections, so that
we will be able to run different tests for different subsections
in a later commit.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Christian Couder
948e22e2bb perf/run: add calls to get_var_from_env_or_config()
These calls make it possible to have the make command or the
make options in a config file, instead of in environment
variables.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Christian Couder
91c4339e19 perf/run: add GIT_PERF_DIRS_OR_REVS
This environment variable can be set to some revisions or
directories whose Git versions should be tested, in addition
to the revisions or directories passed as arguments to the
'run' script.

This enables a "perf.dirsOrRevs" configuration variable to
be used to set revisions or directories whose Git versions
should be tested.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Christian Couder
e6b71539de perf/run: add get_var_from_env_or_config()
Add get_var_from_env_or_config() to easily set variables
from a config file if they are defined there and not already set.

This can also set them to a default value if one is provided.

As an example, use this function to set GIT_PERF_REPEAT_COUNT
from the perf.repeatCount config option or from the default
value.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Christian Couder
e3d5e1207e perf/run: add '--config' option to the 'run' script
It is error prone and tiring to use many long environment
variables to give parameters to the 'run' script.

Let's make it easy to store some parameters in a config
file and to pass them to the run script.

The GIT_PERF_CONFIG_FILE variable will be set to the
argument of the '--config' option. This variable is not
used yet. It will be used in a following commit.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 16:58:34 +09:00
Junio C Hamano
07f0542da3 Merge branch 'mh/packed-ref-transactions'
Implement transactional update to the packed-ref representation of
references.

* mh/packed-ref-transactions:
  files_transaction_finish(): delete reflogs before references
  packed-backend: rip out some now-unused code
  files_ref_store: use a transaction to update packed refs
  t1404: demonstrate two problems with reference transactions
  files_initial_transaction_commit(): use a transaction for packed refs
  prune_refs(): also free the linked list
  files_pack_refs(): use a reference transaction to write packed refs
  packed_delete_refs(): implement method
  packed_ref_store: implement reference transactions
  struct ref_transaction: add a place for backends to store data
  packed-backend: don't adjust the reference count on lock/unlock
2017-09-19 10:47:56 +09:00
Junio C Hamano
0543de438f Merge branch 'sb/merge-commit-msg-hook'
As "git commit" to conclude a conflicted "git merge" honors the
commit-msg hook, "git merge" that recoreds a merge commit that
cleanly auto-merges should, but it didn't.

* sb/merge-commit-msg-hook:
  builtin/merge: honor commit-msg hook for merges
2017-09-19 10:47:56 +09:00
Junio C Hamano
09595ab381 Merge branch 'jk/leak-checkers'
Many of our programs consider that it is OK to release dynamic
storage that is used throughout the life of the program by simply
exiting, but this makes it harder to leak detection tools to avoid
reporting false positives.  Plug many existing leaks and introduce
a mechanism for developers to mark that the region of memory
pointed by a pointer is not lost/leaking to help these tools.

* jk/leak-checkers:
  add UNLEAK annotation for reducing leak false positives
  set_git_dir: handle feeding gitdir to itself
  repository: free fields before overwriting them
  reset: free allocated tree buffers
  reset: make tree counting less confusing
  config: plug user_config leak
  update-index: fix cache entry leak in add_one_file()
  add: free leaked pathspec after add_files_to_cache()
  test-lib: set LSAN_OPTIONS to abort by default
  test-lib: --valgrind should not override --verbose-log
2017-09-19 10:47:55 +09:00
Junio C Hamano
df80c5760c Merge branch 'nm/pull-submodule-recurse-config'
"git -c submodule.recurse=yes pull" did not work as if the
"--recurse-submodules" option was given from the command line.
This has been corrected.

* nm/pull-submodule-recurse-config:
  pull: honor submodule.recurse config option
  pull: fix cli and config option parsing order
2017-09-19 10:47:55 +09:00
Junio C Hamano
daafb5062c Merge branch 'mh/packed-ref-store-prep'
Fix regression to "gitk --bisect" by a recent update.

* mh/packed-ref-store-prep:
  rev-parse: don't trim bisect refnames
2017-09-19 10:47:55 +09:00
Junio C Hamano
b86e112056 Merge branch 'jh/hashmap-disable-counting'
Our hashmap implementation in hashmap.[ch] is not thread-safe when
adding a new item needs to expand the hashtable by rehashing; add
an API to disable the automatic rehashing to work it around.

* jh/hashmap-disable-counting:
  hashmap: add API to disable item counting when threaded
2017-09-19 10:47:54 +09:00
Junio C Hamano
8a044c7f1d Merge branch 'nd/prune-in-worktree'
"git gc" and friends when multiple worktrees are used off of a
single repository did not consider the index and per-worktree refs
of other worktrees as the root for reachability traversal, making
objects that are in use only in other worktrees to be subject to
garbage collection.

* nd/prune-in-worktree:
  refs.c: reindent get_submodule_ref_store()
  refs.c: remove fallback-to-main-store code get_submodule_ref_store()
  rev-list: expose and document --single-worktree
  revision.c: --reflog add HEAD reflog from all worktrees
  files-backend: make reflog iterator go through per-worktree reflog
  revision.c: --all adds HEAD from all worktrees
  refs: remove dead for_each_*_submodule()
  refs.c: move for_each_remote_ref_submodule() to submodule.c
  revision.c: use refs_for_each*() instead of for_each_*_submodule()
  refs: add refs_head_ref()
  refs: move submodule slash stripping code to get_submodule_ref_store
  refs.c: refactor get_submodule_ref_store(), share common free block
  revision.c: --indexed-objects add objects from all worktrees
  revision.c: refactor add_index_objects_to_pending()
  refs.c: use is_dir_sep() in resolve_gitlink_ref()
  revision.h: new flag in struct rev_info wrt. worktree-related refs
2017-09-19 10:47:53 +09:00
Junio C Hamano
79553b94f9 Merge branch 'nd/worktree-kill-parse-ref'
"git branch -M a b" while on a branch that is completely unrelated
to either branch a or branch b misbehaved when multiple worktree
was in use.  This has been fixed.

* nd/worktree-kill-parse-ref:
  branch: fix branch renaming not updating HEADs correctly
2017-09-10 17:08:24 +09:00
Junio C Hamano
5064d66f5b Merge branch 'mm/send-email-cc-cruft'
In addition to "cc: <a@dd.re.ss> # cruft", "cc: a@dd.re.ss # cruft"
was taught to "git send-email" as a valid way to tell it that it
needs to also send a carbon copy to <a@dd.re.ss> in the trailer
section.

* mm/send-email-cc-cruft:
  send-email: don't use Mail::Address, even if available
  send-email: fix garbage removal after address
2017-09-10 17:08:23 +09:00
Junio C Hamano
8e36002add Merge branch 'ma/up-to-date'
Message and doc updates.

* ma/up-to-date:
  treewide: correct several "up-to-date" to "up to date"
  Documentation/user-manual: update outdated example output
2017-09-10 17:08:22 +09:00
Michael Haggerty
dc39e09942 files_ref_store: use a transaction to update packed refs
When processing a `files_ref_store` transaction, it is sometimes
necessary to delete some references from the "packed-refs" file. Do
that using a reference transaction conducted against the
`packed_ref_store`.

This change further decouples `files_ref_store` from
`packed_ref_store`. It also fixes multiple problems, including the two
revealed by test cases added in the previous commit.

First, the old code didn't obtain the `packed-refs` lock until
`files_transaction_finish()`. This means that a failure to acquire the
`packed-refs` lock (e.g., due to contention with another process)
wasn't detected until it was too late (problems like this are supposed
to be detected in the "prepare" phase). The new code acquires the
`packed-refs` lock in `files_transaction_prepare()`, the same stage of
the processing when the loose reference locks are being acquired,
removing another reason why the "prepare" phase might succeed and the
"finish" phase might nevertheless fail.

Second, the old code deleted the loose version of a reference before
deleting any packed version of the same reference. This left a moment
when another process might think that the packed version of the
reference is current, which is incorrect. (Even worse, the packed
version of the reference can be arbitrarily old, and might even point
at an object that has since been garbage-collected.)

Third, if a reference deletion fails to acquire the `packed-refs` lock
altogether, then the old code might leave the repository in the
incorrect state (possibly corrupt) described in the previous
paragraph.

Now we activate the new "packed-refs" file (sans any references that
are being deleted) *before* deleting the corresponding loose
references. But we hold the "packed-refs" lock until after the loose
references have been finalized, thus preventing a simultaneous
"pack-refs" process from packing the loose version of the reference in
the time gap, which would otherwise defeat our attempt to delete it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-09 03:18:04 +09:00
Michael Haggerty
6a2a7736d8 t1404: demonstrate two problems with reference transactions
Currently, a loose reference is deleted even before locking the
`packed-refs` file, let alone deleting any packed version of the
reference. This leads to two problems, demonstrated by two new tests:

* While a reference is being deleted, other processes might see the
  old, packed value of the reference for a moment before the packed
  version is deleted. Normally this would be hard to observe, but we
  can prolong the window by locking the `packed-refs` file externally
  before running `update-ref`, then unlocking it before `update-ref`'s
  attempt to acquire the lock times out.

* If the `packed-refs` file is locked so long that `update-ref` fails
  to lock it, then the reference can be left permanently in the
  incorrect state described in the previous point.

In a moment, both problems will be fixed.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-09 03:18:04 +09:00
Stefan Beller
f8b863598c builtin/merge: honor commit-msg hook for merges
Similar to 65969d43d1 (merge: honor prepare-commit-msg hook, 2011-02-14)
merge should also honor the commit-msg hook: When a merge is stopped due
to conflicts or --no-commit, the subsequent commit calls the commit-msg
hook.  However, it is not called after a clean merge. Fix this
inconsistency by invoking the hook after clean merges as well.

This change is motivated by Gerrit's commit-msg hook to install a ChangeId
trailer into the commit message. Without such a ChangeId, Gerrit refuses
to accept any commit by default, such that the inconsistency of (not)
running the commit-msg hook between commit and merge leads to confusion
and might block people from getting their work done.

As the githooks man page is very vocal about the possibility of skipping
the commit-msg hook via the --no-verify option, implement the option
in merge, too.

'git merge --continue' is currently implemented as calling cmd_commit
with no further arguments. This works for most other merge related options,
such as demonstrated via the --allow-unrelated-histories flag in the
test. The --no-verify option however is not remembered across invocations
of git-merge. Originally the author assumed an alternative in which the
'git merge --continue' command accepts the --no-verify flag, but that
opens up the discussion which flags are allows to the continued merge
command and which must be given in the first invocation.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-08 09:52:11 +09:00
Nicolas Morey-Chaisemartin
121e43fa53 pull: honor submodule.recurse config option
"git pull" supports a --recurse-submodules option but does not parse the
submodule.recurse configuration item to set the default for that option.
Meanwhile "git fetch" does support submodule.recurse, producing
confusing behavior: when submodule.recurse is enabled, "git pull"
recursively fetches submodules but does not update them after fetch.

Handle submodule.recurse in "git pull" to fix this.

Reported-by: Magnus Homann <magnus@homann.se>
Signed-off-by: Nicolas Morey-Chaisemartin <nicolas@morey-chaisemartin.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-07 09:52:18 +09:00
Jeff Hostetler
8b604d1951 hashmap: add API to disable item counting when threaded
This is to address concerns raised by ThreadSanitizer on the mailing list
about threaded unprotected R/W access to map.size with my previous "disallow
rehash" change (0607e10009).

See:
https://public-inbox.org/git/adb37b70139fd1e2bac18bfd22c8b96683ae18eb.1502780344.git.martin.agren@gmail.com/

Add API to hashmap to disable item counting and thus automatic rehashing.
Also include API to later re-enable them.

When item counting is disabled, the map.size field is invalid.  So to
prevent accidents, the field has been renamed and an accessor function
hashmap_get_size() has been added.  All direct references to this
field have been been updated.  And the name of the field changed
to map.private_size to communicate this.

Here is the relevant output from ThreadSanitizer showing the problem:

WARNING: ThreadSanitizer: data race (pid=10554)
  Read of size 4 at 0x00000082d488 by thread T2 (mutexes: write M16):
    #0 hashmap_add hashmap.c:209
    #1 hash_dir_entry_with_parent_and_prefix name-hash.c:302
    #2 handle_range_dir name-hash.c:347
    #3 handle_range_1 name-hash.c:415
    #4 lazy_dir_thread_proc name-hash.c:471
    #5 <null> <null>

  Previous write of size 4 at 0x00000082d488 by thread T1 (mutexes: write M31):
    #0 hashmap_add hashmap.c:209
    #1 hash_dir_entry_with_parent_and_prefix name-hash.c:302
    #2 handle_range_dir name-hash.c:347
    #3 handle_range_1 name-hash.c:415
    #4 handle_range_dir name-hash.c:380
    #5 handle_range_1 name-hash.c:415
    #6 lazy_dir_thread_proc name-hash.c:471
    #7 <null> <null>

Martin gives instructions for running TSan on test t3008 in this post:
https://public-inbox.org/git/CAN0heSoJDL9pWELD6ciLTmWf-a=oyxe4EXXOmCKvsG5MSuzxsA@mail.gmail.com/

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-07 09:42:02 +09:00
Jeff King
1d0538e486 rev-parse: don't trim bisect refnames
Using for_each_ref_in() with a full refname has always been
a questionable practice, but it became an error with
b9c8e7f2fb (prefix_ref_iterator: don't trim too much,
2017-05-22), making "git rev-parse --bisect" pretty reliably
show a BUG.

Commit 03df567fbf (for_each_bisect_ref(): don't trim
refnames, 2017-06-18) fixed this case for revision.c, but
rev-parse handles this option on its own. We can use the
same solution here (and piggy-back on its test).

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-07 08:46:53 +09:00
Jeff King
85b81b35ff test-lib: set LSAN_OPTIONS to abort by default
We already set ASAN_OPTIONS to abort if it finds any errors.
As we start to experiment with LSAN, the leak sanitizer,
it's convenient if we give it the same treatment.

Note that ASAN is actually a superset of LSAN and can do the
leak detection itself. So this only has an effect if you
specifically build with "make SANITIZE=leak" (leak detection
but not the rest of ASAN). Building with just LSAN results
in a build that runs much faster. That makes the
build-test-fix cycle more pleasant.

In the long run, once we've fixed or suppressed all the
leaks, it will probably be worth turning leak-detection on
for ASAN and just using that (to check both leaks _and_
memory errors in a single test run). But there's still a lot
of work before we get there.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-06 18:06:26 +09:00
Jeff King
88c6e9d31c test-lib: --valgrind should not override --verbose-log
The --verbose test option cannot be used with test harnesses
like "prove". Instead, you must use --verbose-log.

Since the --valgrind option implies --verbose, that means
that it cannot be used with prove. I.e., this does not work:

  prove t0000-basic.sh :: --valgrind

You'd think it could be fixed by doing:

  prove t0000-basic.sh :: --valgrind --verbose-log

but that doesn't work either, because the implied --verbose
takes precedence over --verbose-log. If the user has given
us a specific option, we should prefer that.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-06 18:06:26 +09:00
Junio C Hamano
a7d7f125e3 Merge branch 'rs/archive-excluded-directory'
"git archive" did not work well with pathspecs and the
export-ignore attribute.

* rs/archive-excluded-directory:
  archive: don't queue excluded directories
  archive: factor out helper functions for handling attributes
  t5001: add tests for export-ignore attributes and exclude pathspecs
2017-09-06 13:11:25 +09:00
Junio C Hamano
1fb77b3ee5 Merge branch 'ks/branch-set-upstream'
"branch --set-upstream" that has been deprecated in Git 1.8 has
finally been retired.

* ks/branch-set-upstream:
  branch: quote branch/ref names to improve readability
  builtin/branch: stop supporting the "--set-upstream" option
  t3200: cleanup cruft of a test
2017-09-06 13:11:24 +09:00
Junio C Hamano
6e6ba65a7c Merge branch 'mg/killed-merge'
Killing "git merge --edit" before the editor returns control left
the repository in a state with MERGE_MSG but without MERGE_HEAD,
which incorrectly tells the subsequent "git commit" that there was
a squash merge in progress.  This has been fixed.

* mg/killed-merge:
  merge: save merge state earlier
  merge: split write_merge_state in two
  merge: clarify call chain
  Documentation/git-merge: explain --continue
2017-08-26 22:55:10 -07:00
Junio C Hamano
96352ef9b4 Merge branch 'jc/cutoff-config'
"[gc] rerereResolved = 5.days" used to be invalid, as the variable
is defined to take an integer counting the number of days.  It now
is allowed.

* jc/cutoff-config:
  rerere: allow approxidate in gc.rerereResolved/gc.rerereUnresolved
  rerere: represent time duration in timestamp_t internally
  t4200: parameterize "rerere gc" custom expiry test
  t4200: gather "rerere gc" together
  t4200: make "rerere gc" test more robust
  t4200: give us a clean slate after "rerere gc" tests
2017-08-26 22:55:08 -07:00
Junio C Hamano
030faf2fa5 Merge branch 'kw/write-index-reduce-alloc'
We used to spend more than necessary cycles allocating and freeing
piece of memory while writing each index entry out.  This has been
optimized.

* kw/write-index-reduce-alloc:
  read-cache: avoid allocating every ondisk entry when writing
  read-cache: fix memory leak in do_write_index
  perf: add test for writing the index
2017-08-26 22:55:08 -07:00
Junio C Hamano
614ea03a71 Merge branch 'bw/submodule-config-cleanup'
Code clean-up to avoid mixing values read from the .gitmodules file
and values read from the .git/config file.

* bw/submodule-config-cleanup:
  submodule: remove gitmodules_config
  unpack-trees: improve loading of .gitmodules
  submodule-config: lazy-load a repository's .gitmodules file
  submodule-config: move submodule-config functions to submodule-config.c
  submodule-config: remove support for overlaying repository config
  diff: stop allowing diff to have submodules configured in .git/config
  submodule: remove submodule_config callback routine
  unpack-trees: don't respect submodule.update
  submodule: don't rely on overlayed config when setting diffopts
  fetch: don't overlay config with submodule-config
  submodule--helper: don't overlay config in update-clone
  submodule--helper: don't overlay config in remote_submodule_branch
  add, reset: ensure submodules can be added or reset
  submodule: don't use submodule_from_name
  t7411: check configuration parsing errors
2017-08-26 22:55:08 -07:00
Junio C Hamano
a17483fcfe Merge branch 'tb/apply-with-crlf'
"git apply" that is used as a better "patch -p1" failed to apply a
taken from a file with CRLF line endings to a file with CRLF line
endings.  The root cause was because it misused convert_to_git()
that tried to do "safe-crlf" processing by looking at the index
entry at the same path, which is a nonsense---in that mode, "apply"
is not working on the data in (or derived from) the index at all.
This has been fixed.

* tb/apply-with-crlf:
  apply: file commited with CRLF should roundtrip diff and apply
  convert: add SAFE_CRLF_KEEP_CRLF
2017-08-26 22:55:05 -07:00
Junio C Hamano
f6a47f9b7a Merge branch 'jt/stash-tests'
Test update to improve coverage for "git stash" operations.

* jt/stash-tests:
  stash: add a test for stashing in a detached state
  stash: add a test for when apply fails during stash branch
  stash: add a test for stash create with no files
2017-08-26 22:55:04 -07:00
Junio C Hamano
06cf4f2d87 Merge branch 'jk/trailers-parse'
"git interpret-trailers" has been taught a "--parse" and a few
other options to make it easier for scripts to grab existing
trailer lines from a commit log message.

* jk/trailers-parse:
  doc/interpret-trailers: fix "the this" typo
  pretty: support normalization options for %(trailers)
  t4205: refactor %(trailers) tests
  pretty: move trailer formatting to trailer.c
  interpret-trailers: add --parse convenience option
  interpret-trailers: add an option to unfold values
  interpret-trailers: add an option to show only existing trailers
  interpret-trailers: add an option to show only the trailers
  trailer: put process_trailers() options into a struct
2017-08-26 22:55:04 -07:00
Junio C Hamano
bfd91b4134 Merge branch 'pb/trailers-from-command-line'
"git interpret-trailers" learned to take the trailer specifications
from the command line that overrides the configured values.

* pb/trailers-from-command-line:
  interpret-trailers: fix documentation typo
  interpret-trailers: add options for actions
  trailers: introduce struct new_trailer_item
  trailers: export action enums and corresponding lookup functions
2017-08-26 22:55:04 -07:00
Junio C Hamano
0b96358479 Merge branch 'jt/diff-color-move-fix'
A handful of bugfixes and an improvement to "diff --color-moved".

* jt/diff-color-move-fix:
  diff: define block by number of alphanumeric chars
  diff: respect MIN_BLOCK_LENGTH for last block
  diff: avoid redundantly clearing a flag
2017-08-26 22:55:04 -07:00
Junio C Hamano
b6c4058f97 Merge branch 'sb/diff-color-move'
"git diff" has been taught to optionally paint new lines that are
the same as deleted lines elsewhere differently from genuinely new
lines.

* sb/diff-color-move: (25 commits)
  diff: document the new --color-moved setting
  diff.c: add dimming to moved line detection
  diff.c: color moved lines differently, plain mode
  diff.c: color moved lines differently
  diff.c: buffer all output if asked to
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_SUMMARY
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_STAT_SEP
  diff.c: convert word diffing to use emit_diff_symbol
  diff.c: convert show_stats to use emit_diff_symbol
  diff.c: convert emit_binary_diff_body to use emit_diff_symbol
  submodule.c: migrate diff output to use emit_diff_symbol
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_REWRITE_DIFF
  diff.c: emit_diff_symbol learns about DIFF_SYMBOL_BINARY_FILES
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_HEADER
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_FILEPAIR_{PLUS, MINUS}
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_INCOMPLETE
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_WORDS[_PORCELAIN]
  diff.c: migrate emit_line_checked to use emit_diff_symbol
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_NO_LF_EOF
  diff.c: emit_diff_symbol learns DIFF_SYMBOL_CONTEXT_FRAGINFO
  ...
2017-08-26 22:55:03 -07:00
Nguyễn Thái Ngọc Duy
acd9544a8f revision.c: --reflog add HEAD reflog from all worktrees
Note that add_other_reflogs_to_pending() is a bit inefficient, since
it scans reflog for all refs of each worktree, including shared refs,
so the shared ref's reflog is scanned over and over again.

We could update refs API to pass "per-worktree only" flag to avoid
that. But long term we should be able to obtain a "per-worktree only"
ref store and would need to revert the changes in reflog iteration
API. So let's just wait until then.

add_reflogs_to_pending() is called by reachable.c so by default "git
prune" will examine reflog from all worktrees.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:58:47 -07:00
Nguyễn Thái Ngọc Duy
944b4e3013 files-backend: make reflog iterator go through per-worktree reflog
refs/bisect is unfortunately per-worktree, so we need to look in
per-worktree logs/refs/bisect in addition to per-repo logs/refs. The
current iterator only goes through per-repo logs/refs.

Use merge iterator to walk two ref stores at the same time and pick
per-worktree refs from the right iterator.

PS. Note the unsorted order of for_each_reflog in the test. This is
supposed to be OK, for now. If we enforce order on for_each_reflog()
then some more work will be required.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:57:56 -07:00
Nguyễn Thái Ngọc Duy
d0c39a49cc revision.c: --all adds HEAD from all worktrees
Unless single_worktree is set, --all now adds HEAD from all worktrees.

Since reachable.c code does not use setup_revisions(), we need to call
other_head_refs_submodule() explicitly there to have the same effect on
"git prune", so that we won't accidentally delete objects needed by some
other HEADs.

A new FIXME is added because we would need something like

    int refs_other_head_refs(struct ref_store *, each_ref_fn, cb_data);

in addition to other_head_refs() to handle it, which might require

    int get_submodule_worktrees(const char *submodule, int flags);

It could be a separate topic to reduce the scope of this one.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:56:43 -07:00
Nguyễn Thái Ngọc Duy
be489d02d2 revision.c: --indexed-objects add objects from all worktrees
This is the result of single_worktree flag never being set (no way to up
until now). To get objects from current index only, set single_worktree.

The other add_index_objects_to_pending's caller is mark_reachable_objects()
(e.g. "git prune") which also mark objects from all indexes.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24 14:44:41 -07:00