Commit Graph

307 Commits

Author SHA1 Message Date
Stefan Beller
3ef2538032 recursive submodules: detach HEAD from new state
When a submodule is on a branch and in its superproject you run a
recursive checkout, the branch of the submodule is updated to what the
superproject checks out. This is very unexpected in the current model of
Git as e.g. 'submodule update' always detaches the submodule HEAD.

Despite having plans to have submodule HEADS not detached in the future,
the current behavior is really bad as it doesn't match user expectations
and it is not checking for loss of commits (only to be recovered via the
reflog).

Detach the HEAD unconditionally in the submodule when updating it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-25 11:05:41 -07:00
Brandon Williams
c7be7201a7 submodule--helper: teach push-check to handle HEAD
In 06bf4ad1d (push: propagate remote and refspec with
--recurse-submodules) push was taught how to propagate a refspec down to
submodules when the '--recurse-submodules' flag is given.  The only refspecs
that are allowed to be propagated are ones which name a ref which exists
in both the superproject and the submodule, with the caveat that 'HEAD'
was disallowed.

This patch teaches push-check (the submodule helper which determines if
a refspec can be propagated to a submodule) to permit propagating 'HEAD'
if and only if the superproject and the submodule both have the same
named branch checked out and the submodule is not in a detached head
state.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-20 14:58:26 -07:00
brian m. carlson
cd73de4714 submodule: convert submodule config lookup to use object_id
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-17 13:54:37 -07:00
Junio C Hamano
c9c63ee558 Merge branch 'sb/pull-rebase-submodule'
"git pull --rebase --recurse-submodules" learns to rebase the
branch in the submodules to an updated base.

* sb/pull-rebase-submodule:
  builtin/fetch cleanup: always set default value for submodule recursing
  pull: optionally rebase submodules (remote submodule changes only)
  builtin/fetch: parse recurse-submodules-default at default options parsing
  builtin/fetch: factor submodule recurse parsing out to submodule config
2017-07-13 16:14:54 -07:00
Junio C Hamano
85ce4a6828 Merge branch 'bw/repo-object'
Introduce a "repository" object to eventually make it easier to
work in multiple repositories (the primary focus is to work with
the superproject and its submodules) in a single process.

* bw/repo-object:
  ls-files: use repository object
  repository: enable initialization of submodules
  submodule: convert is_submodule_initialized to work on a repository
  submodule: add repo_read_gitmodules
  submodule-config: store the_submodule_cache in the_repository
  repository: add index_state to struct repo
  config: read config from a repository object
  path: add repo_worktree_path and strbuf_repo_worktree_path
  path: add repo_git_path and strbuf_repo_git_path
  path: worktree_git_path() should not use file relocation
  path: convert do_git_path to take a 'struct repository'
  path: convert strbuf_git_common_path to take a 'struct repository'
  path: always pass in commondir to update_common_dir
  path: create path.h
  environment: store worktree in the_repository
  environment: place key repository state in the_repository
  repository: introduce the repository object
  environment: remove namespace_len variable
  setup: add comment indicating a hack
  setup: don't perform lazy initialization of repository state
2017-07-05 13:32:56 -07:00
Stefan Beller
f3597138df submodule.c: migrate diff output to use emit_diff_symbol
As the submodule process is no longer attached to the same file pointer
'o->file' as the superprojects process, there is a different result in
color.c::check_auto_color. That is why we need to pass coloring explicitly,
such that the submodule coloring decision will be made by the child process
processing the submodule. Only DIFF_SYMBOL_SUBMODULE_PIPETHROUGH contains
color, the other symbols are for embedding the submodule output into the
superprojects output.

Remove the colors from the function signatures, as all the coloring
decisions will be made either inside the child process or the final
emit_diff_symbol, but not in the functions driving the submodule diff.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-30 13:13:02 -07:00
Ville Skyttä
6412757514 Spelling fixes
Signed-off-by: Ville Skyttä <ville.skytta@iki.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-27 10:35:49 -07:00
Junio C Hamano
f31d23a399 Merge branch 'bw/config-h'
Fix configuration codepath to pay proper attention to commondir
that is used in multi-worktree situation, and isolate config API
into its own header file.

* bw/config-h:
  config: don't implicitly use gitdir or commondir
  config: respect commondir
  setup: teach discover_git_directory to respect the commondir
  config: don't include config.h by default
  config: remove git_config_iter
  config: create config.h
2017-06-24 14:28:41 -07:00
Brandon Williams
627d9342fe submodule: convert is_submodule_initialized to work on a repository
Convert 'is_submodule_initialized()' to take a repository object and
while we're at it, lets rename the function to 'is_submodule_active()'
and remove the NEEDSWORK comment.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23 18:24:34 -07:00
Brandon Williams
69aba5329e submodule: add repo_read_gitmodules
Teach the repo object to be able to populate the submodule_cache by
reading the repository's gitmodules file.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23 18:24:34 -07:00
Junio C Hamano
25bf951381 Merge branches 'bw/ls-files-sans-the-index' and 'bw/config-h' into bw/repo-object
* bw/ls-files-sans-the-index:
  ls-files: factor out tag calculation
  ls-files: factor out debug info into a function
  ls-files: convert show_files to take an index
  ls-files: convert show_ce_entry to take an index
  ls-files: convert prune_cache to take an index
  ls-files: convert ce_excluded to take an index
  ls-files: convert show_ru_info to take an index
  ls-files: convert show_other_files to take an index
  ls-files: convert show_killed_files to take an index
  ls-files: convert write_eolinfo to take an index
  ls-files: convert overlay_tree_on_cache to take an index
  tree: convert read_tree to take an index parameter
  convert: convert renormalize_buffer to take an index
  convert: convert convert_to_git to take an index
  convert: convert convert_to_git_filter_fd to take an index
  convert: convert crlf_to_git to take an index
  convert: convert get_cached_convert_stats_ascii to take an index

* bw/config-h:
  config: don't implicitly use gitdir or commondir
  config: respect commondir
  setup: teach discover_git_directory to respect the commondir
  config: don't include config.h by default
  config: remove git_config_iter
  config: create config.h
  alias: use the early config machinery to expand aliases
  t7006: demonstrate a problem with aliases in subdirectories
  t1308: relax the test verifying that empty alias values are disallowed
  help: use early config when autocorrecting aliases
  config: report correct line number upon error
  discover_git_directory(): avoid setting invalid git_dir
2017-06-23 18:24:00 -07:00
Stefan Beller
a6d7eb2c7a pull: optionally rebase submodules (remote submodule changes only)
Teach pull to optionally update submodules when '--recurse-submodules'
is provided.  This will teach pull to run 'submodule update --rebase'
when the '--recurse-submodules' and '--rebase' flags are given under
specific circumstances.

On a rebase workflow:
=====================

1. Both sides change the submodule
 ------------------------------
Let's assume the following history in a submodule:

  H---I---J---K---L local branch
       \
        M---N---O---P remote branch

and the following in the superproject (recorded submodule in parens):

  A(H)---B(I)---F(K)---G(L)  local branch
          \
           C(N)---D(N)---E(P) remote branch

In an ideal world this would rebase the submodule and rewrite
the submodule pointers that the superproject points at such that
the superproject looks like

  A(H)---B(I)              F(K')---G(L')  rebased branch
           \                /
           C(N)---D(N)---E(P) remote branch

and the submodule as:

        J---K---L (old dangeling tip)
       /
  H---I               J'---K'---L' rebased branch
       \             /
        M---N---O---P remote branch

And if a conflict arises in the submodule the superproject rebase
would stop at that commit at which the submodule conflict occurs.

Currently a "pull --rebase" in the superproject produces
a merge conflict as the submodule pointer changes are
conflicting and cannot be resolved.

2. Local submodule changes only
 -----------------------
Assuming histories as above, except that the remote branch
would not contain submodule changes, then a result as

  A(H)---B(I)               F(K)---G(L)  rebased branch
           \                /
           C(I)---D(I)---E(I) remote branch

is desire-able. This is what currently happens in rebase.

If the recursive flag is given, the ideal git would
produce a superproject as:

  A(H)---B(I)              F(K')---G(L')  rebased branch (incl. sub rebase!)
           \                /
           C(I)---D(I)---E(I) remote branch

and the submodule as:

        J---K---L (old dangeling tip)
       /
  H---I               J'---K'---L' locally rebased branch
       \             /
        M---N---O---P advanced branch

This patch doesn't address this issue, however
a test is added that this fails up front.

3. Remote submodule changes only
 ----------------------
Assuming histories as in (1) except that the local superproject branch
would not have touched the submodule the rebase already works out in the
superproject with no conflicts:

  A(H)---B(I)               F(P)---G(P)  rebased branch (no sub changes)
           \                 /
           C(N)---D(N)---E(P) remote branch

The recurse flag as presented in this patch would additionally
update the submodule as:

  H---I              J'---K'---L' rebased branch
       \            /
        M---N---O---P remote branch

As neither J, K, L nor J', K', L' are referred to from the superproject,
no rewriting of the superproject commits is required.

Conclusion for 'pull --rebase --recursive'
 -----------------------------------------
If there are no local superproject changes it is sufficient to call
"submodule update --rebase" as this produces the desired results. In case
of conflicts, the behavior is the same as in 'submodule update --recursive'
which is assumed to be sane.

This patch implements (3) only.

On a merge workflow:
====================

We'll start off with the same underlying DAG as in (1) in the rebase
workflow. So in an ideal world a 'pull --merge --recursive' would
produce this:

  H---I---J---K---L----X
       \              /
        M---N---O---P

with X as the new merge-commit in the submodule and the superproject
as:

  A(H)---B(I)---F(K)---G(L)---Y(X)
          \                  /
           C(N)---D(N)---E(P)

However modifying the submodules on the fly is not supported in git-merge
such that Y(X) is not easy to produce in a single patch. In fact git-merge
doesn't know about submodules at all.

However when at least one side does not contain commits touching the
submodule at all, then we do not need to perform the merge for the
submodule but a fast-forward can be done via checking out either L or P
in the submodule.  This strategy is implemented in 68d03e4a6e (Implement
automatic fast-forward merge for submodules, 2010-07-07) already, so
to align with the rebase behavior we need to also update the worktree
of the submodule.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23 15:36:53 -07:00
Brandon Williams
b2141fc1d2 config: don't include config.h by default
Stop including config.h by default in cache.h.  Instead only include
config.h in those files which require use of the config system.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-15 12:56:22 -07:00
Junio C Hamano
3c548de378 Merge branch 'sb/submodule-blanket-recursive'
Many commands learned to pay attention to submodule.recurse
configuration.

* sb/submodule-blanket-recursive:
  builtin/fetch.c: respect 'submodule.recurse' option
  builtin/push.c: respect 'submodule.recurse' option
  builtin/grep.c: respect 'submodule.recurse' option
  Introduce 'submodule.recurse' option for worktree manipulators
  submodule loading: separate code path for .gitmodules and config overlay
  reset/checkout/read-tree: unify config callback for submodule recursion
  submodule test invocation: only pass additional arguments
  submodule recursing: do not write a config variable twice
2017-06-13 13:47:07 -07:00
Stefan Beller
046b48239e Introduce 'submodule.recurse' option for worktree manipulators
Any command that understands '--recurse-submodules' can have its
default changed to true, by setting the new 'submodule.recurse'
option.

This patch includes read-tree/checkout/reset for working tree
manipulating commands. Later patches will cover other commands.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-01 10:36:36 +09:00
Stefan Beller
1d789d0892 submodule loading: separate code path for .gitmodules and config overlay
The .gitmodules file is not supposed to have all the options available,
that are available in the configuration so separate it out.

A configuration option such as the hypothetical submodule.color.diff
that determines in which color a submodule change is printed,
is a very user specific thing, that the .gitmodules file should
not tamper with.

The .gitmodules file should only be used for settings that required
to setup the project in which the .gitmodules file is tracked. As the
minimum this would only include the name<->path mapping of the
submodule and its URL and branch.

Any further setting (such as 'fetch.recursesubmodules' or
'submodule.<name>.{update, ignore, shallow}') is not specific
to the project setup requirements, but rather is a distribution
of suggested developer configurations.  In other areas of Git
a suggested developer configuration is not transported in-tree
but via other means.  In an organisation this could be done
by deploying an opinionated system wide config (/etc/gitconfig)
or by putting the settings in the users home directory when
they start at the organisation. In open source projects this
is often accomplished via extensive READMEs (cf. our
SubmittingPatches/CodingGuidlines).

As a later patch in this series wants to introduce
a generic submodule recursion option, we want to make
sure that switch is not exposed via the gitmodules file.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-30 14:28:54 +09:00
Stefan Beller
d7a3803f9e reset/checkout/read-tree: unify config callback for submodule recursion
The callback function is essentially duplicated 3 times. Remove all
of them and offer a new callback function, that lives in submodule.c

By putting the callback function there, we no longer need the function
'set_config_update_recurse_submodules', nor duplicate the global variable
in each builtin as well as submodule.c

In the three builtins we have different 2 ways how to load the .gitmodules
and config file, which are slightly different. git-checkout has to load
the submodule config all the time due to 23b4c7bcc5 (checkout: Use
submodule.*.ignore settings from .git/config and .gitmodules, 2010-08-28)

git-reset and git-read-tree do not respect these diff settings, so loading
the submodule configuration is optional. Also put that into submodule.c
for code deduplication.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-30 14:28:53 +09:00
Junio C Hamano
3c5a78280f Merge branch 'bw/pathspec-sans-the-index'
Simplify parse_pathspec() codepath and stop it from looking at the
default in-core index.

* bw/pathspec-sans-the-index:
  pathspec: convert find_pathspecs_matching_against_index to take an index
  pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP
  ls-files: prevent prune_cache from overeagerly pruning submodules
  pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag
  submodule: add die_in_unpopulated_submodule function
  pathspec: provide a more descriptive die message
2017-05-30 11:16:40 +09:00
Junio C Hamano
6b526ced6f Merge branch 'bc/object-id'
Conversion from uchar[20] to struct object_id continues.

* bc/object-id: (53 commits)
  object: convert parse_object* to take struct object_id
  tree: convert parse_tree_indirect to struct object_id
  sequencer: convert do_recursive_merge to struct object_id
  diff-lib: convert do_diff_cache to struct object_id
  builtin/ls-tree: convert to struct object_id
  merge: convert checkout_fast_forward to struct object_id
  sequencer: convert fast_forward_to to struct object_id
  builtin/ls-files: convert overlay_tree_on_cache to object_id
  builtin/read-tree: convert to struct object_id
  sha1_name: convert internals of peel_onion to object_id
  upload-pack: convert remaining parse_object callers to object_id
  revision: convert remaining parse_object callers to object_id
  revision: rename add_pending_sha1 to add_pending_oid
  http-push: convert process_ls_object and descendants to object_id
  refs/files-backend: convert many internals to struct object_id
  refs: convert struct ref_update to use struct object_id
  ref-filter: convert some static functions to struct object_id
  Convert struct ref_array_item to struct object_id
  Convert the verify_pack callback to struct object_id
  Convert lookup_tag to struct object_id
  ...
2017-05-29 12:34:43 +09:00
Junio C Hamano
a531ecf399 Merge branch 'jk/diff-submodule-diff-inline'
"git diff --submodule=diff" now recurses into nested submodules.

* jk/diff-submodule-diff-inline:
  diff: recurse into nested submodules for inline diff
2017-05-29 12:34:42 +09:00
Junio C Hamano
f1101cefbd Merge branch 'sb/checkout-recurse-submodules'
"git checkout --recurse-submodules" did not quite work with a
submodule that itself has submodules.

* sb/checkout-recurse-submodules:
  submodule: properly recurse for read-tree and checkout
  submodule: avoid auto-discovery in new working tree manipulator code
  submodule_move_head: reuse child_process structure for futher commands
2017-05-29 12:34:41 +09:00
Junio C Hamano
5f074ca7e8 Merge branch 'sb/reset-recurse-submodules'
"git reset" learned "--recurse-submodules" option.

* sb/reset-recurse-submodules:
  builtin/reset: add --recurse-submodules switch
  submodule.c: submodule_move_head works with broken submodules
  submodule.c: uninitialized submodules are ignored in recursive commands
  entry.c: submodule recursing: respect force flag correctly
2017-05-29 12:34:40 +09:00
Brandon Williams
c08397e3aa pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag
Since (ae8d08242 pathspec: pass directory indicator to
match_pathspec_item()) the path matching logic has been able to cope
with submodules without needing to strip off a trailing slash if a path
refers to a submodule.

Since the stripping the trailing slash is no longer necessary, remove
the PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag.  In addition, factor
out the logic which dies if a path decends into a submodule so that it
can still be used as a check after a pathspec struct has been
initialized.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-12 14:23:46 +09:00
Brandon Williams
bdab972153 submodule: add die_in_unpopulated_submodule function
Currently 'git add' is the only command which dies when launched from an
unpopulated submodule (the place-holder directory for a submodule which
hasn't been checked out).  This is triggered implicitly by passing the
PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag to 'parse_pathspec()'.

Instead make this desire more explicit by creating a function
'die_in_unpopulated_submodule()' which dies if the provided 'prefix' has
a leading path component which matches a submodule in the the index.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-10 14:47:39 +09:00
brian m. carlson
bc83266abe Convert lookup_commit* to struct object_id
Convert lookup_commit, lookup_commit_or_die,
lookup_commit_reference, and lookup_commit_reference_gently to take
struct object_id arguments.

Introduce a temporary in parse_object buffer in order to convert this
function.  This is required since in order to convert parse_object and
parse_object_buffer, lookup_commit_reference_gently and
lookup_commit_or_die would need to be converted.  Not introducing a
temporary would therefore require that lookup_commit_or_die take a
struct object_id *, but lookup_commit would take unsigned char *,
leaving a confusing and hard-to-use interface.

parse_object_buffer will lose this temporary in a later patch.

This commit was created with manual changes to commit.c, commit.h, and
object.c, plus the following semantic patch:

@@
expression E1, E2;
@@
- lookup_commit_reference_gently(E1.hash, E2)
+ lookup_commit_reference_gently(&E1, E2)

@@
expression E1, E2;
@@
- lookup_commit_reference_gently(E1->hash, E2)
+ lookup_commit_reference_gently(E1, E2)

@@
expression E1;
@@
- lookup_commit_reference(E1.hash)
+ lookup_commit_reference(&E1)

@@
expression E1;
@@
- lookup_commit_reference(E1->hash)
+ lookup_commit_reference(E1)

@@
expression E1;
@@
- lookup_commit(E1.hash)
+ lookup_commit(&E1)

@@
expression E1;
@@
- lookup_commit(E1->hash)
+ lookup_commit(E1)

@@
expression E1, E2;
@@
- lookup_commit_or_die(E1.hash, E2)
+ lookup_commit_or_die(&E1, E2)

@@
expression E1, E2;
@@
- lookup_commit_or_die(E1->hash, E2)
+ lookup_commit_or_die(E1, E2)

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-08 15:12:57 +09:00
brian m. carlson
1e43ed9867 Convert remaining callers of lookup_commit_reference* to object_id
There are a small number of remaining callers of lookup_commit_reference
and lookup_commit_reference_gently that still need to be converted to
struct object_id.  Convert these.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-08 15:12:57 +09:00
brian m. carlson
71f35d5cbc submodule: convert merge_submodule to use struct object_id
This is a caller of lookup_commit_reference, which we will convert
later.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-08 15:12:56 +09:00
Stefan Beller
5a5221427c diff: recurse into nested submodules for inline diff
When fd47ae6a5b (diff: teach diff to display submodule difference with an
inline diff, 2016-08-31) was introduced, we did not think of recursing
into nested submodules.

When showing the inline diff for submodules, automatically recurse
into nested submodules as well with inline submodule diffs.

Signed-off-by: Stefan Beller <sbeller@google.com>
Acked-by: Jacob Keller <jacob.keller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-08 09:44:54 +09:00
Stefan Beller
218c883783 submodule: properly recurse for read-tree and checkout
We forgot to prepare the submodule env, which is only a problem for
nested submodules. See 2e5d6503bd (ls-files: fix recurse-submodules
with nested submodules, 2017-04-13) for further explanation.

To come up with a proper test for this, we'd need to look at nested
submodules just as in that given commit. It turns out we're lucky
and these tests already exist, but are marked as failing. We need
to pass `--recurse-submodules` to read-tree additionally to make
these tests pass. Passing that flag alone would not make the tests
pass, such that this covers testing for the bug fix of the submodule
env as well.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-04 13:42:05 +09:00
Stefan Beller
da27bc81f0 submodule: avoid auto-discovery in new working tree manipulator code
All commands that are run in a submodule, are run in a correct setup,
there is no need to prepare the environment without setting the GIT_DIR
variable. By setting the GIT_DIR variable we fix issues as discussed in
10f5c52656 (submodule: avoid auto-discovery in
prepare_submodule_repo_env(), 2016-09-01)

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-04 13:42:04 +09:00
Stefan Beller
a17062cf42 submodule_move_head: reuse child_process structure for futher commands
We do not need to declare another struct child_process, but we can just
reuse the existing `cp` struct.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-04 13:42:04 +09:00
Brandon Williams
aacc5c1a81 submodule: refactor logic to determine changed submodules
There are currently two instances (fetch and push) where we want to
determine if submodules have changed given some revision specification.
These two instances don't use the same logic to generate a list of
changed submodules and as a result there is a fair amount of code
duplication.

This patch refactors these two code paths such that they both use the
same logic to generate a list of changed submodules.  This also makes it
easier for future callers to be able to reuse this logic as they only
need to create an argv_array with the revision specification to be using
during the revision walk.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-02 10:45:11 +09:00
Brandon Williams
7c8d2b00f2 submodule: improve submodule_has_commits()
Teach 'submodule_has_commits()' to ensure that if a commit exists in a
submodule, that it is also reachable from a ref.

This is a preparatory step prior to merging the logic which checks for
changed submodules when fetching or pushing.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-02 10:45:04 +09:00
Brandon Williams
a6bb78c3b1 submodule: change string_list changed_submodule_paths
Eliminate a call to 'xstrdup()' by changing the string_list
'changed_submodule_paths' to duplicated strings added to it.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-01 12:19:15 +09:00
Brandon Williams
d1a8460caa submodule: remove add_oid_to_argv()
The function 'add_oid_to_argv()' provides the same functionality as
'append_oid_to_argv()'.  Remove this duplicate function and instead use
'append_oid_to_argv()' where 'add_oid_to_argv()' was previously used.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-01 12:19:06 +09:00
Brandon Williams
610b233704 submodule: rename free_submodules_sha1s()
Rename 'free_submodules_sha1s()' to 'free_submodules_oids()' since the
function frees a 'struct string_list' which has a 'struct oid_array'
stored in the 'util' field.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-01 12:18:38 +09:00
Brandon Williams
419fd7866c submodule: rename add_sha1_to_array()
Rename 'add_sha1_to_array()' to 'append_oid_to_array()' to more
accurately describe what the function does, since it handles
'struct object_id' and not sha1 character arrays.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-01 12:17:10 +09:00
Junio C Hamano
5000cd45d8 Merge branch 'sb/checkout-recurse-submodules'
Code cleanup.

* sb/checkout-recurse-submodules:
  submodule: remove a superfluous second check for the "new" variable
2017-04-23 22:07:54 -07:00
Junio C Hamano
5ab8f2261f Merge branch 'nd/files-backend-git-dir'
The "submodule" specific field in the ref_store structure is
replaced with a more generic "gitdir" that can later be used also
when dealing with ref_store that represents the set of refs visible
from the other worktrees.

* nd/files-backend-git-dir: (28 commits)
  refs.h: add a note about sorting order of for_each_ref_*
  t1406: new tests for submodule ref store
  t1405: some basic tests on main ref store
  t/helper: add test-ref-store to test ref-store functions
  refs: delete pack_refs() in favor of refs_pack_refs()
  files-backend: avoid ref api targeting main ref store
  refs: new transaction related ref-store api
  refs: add new ref-store api
  refs: rename get_ref_store() to get_submodule_ref_store() and make it public
  files-backend: replace submodule_allowed check in files_downcast()
  refs: move submodule code out of files-backend.c
  path.c: move some code out of strbuf_git_path_submodule()
  refs.c: make get_main_ref_store() public and use it
  refs.c: kill register_ref_store(), add register_submodule_ref_store()
  refs.c: flatten get_ref_store() a bit
  refs: rename lookup_ref_store() to lookup_submodule_ref_store()
  refs.c: introduce get_main_ref_store()
  files-backend: remove the use of git_path()
  files-backend: add and use files_ref_path()
  files-backend: add and use files_reflog_path()
  ...
2017-04-19 21:37:19 -07:00
Junio C Hamano
a06380be54 Merge branch 'sb/submodule-rm-absorb'
Error message fix.

* sb/submodule-rm-absorb:
  submodule.c: add missing ' in error messages
2017-04-19 21:37:17 -07:00
Junio C Hamano
872e2cf00a Merge branch 'bw/push-options-recursively-to-submodules'
"git push --recurse-submodules --push-option=<string>" learned to
propagate the push option recursively down to pushes in submodules.

* bw/push-options-recursively-to-submodules:
  push: propagate remote and refspec with --recurse-submodules
  submodule--helper: add push-check subcommand
  remote: expose parse_push_refspec function
  push: propagate push-options with --recurse-submodules
  push: unmark a local variable as static
2017-04-19 21:37:14 -07:00
Junio C Hamano
b1081e4004 Merge branch 'bc/object-id'
Conversion from unsigned char [40] to struct object_id continues.

* bc/object-id:
  Documentation: update and rename api-sha1-array.txt
  Rename sha1_array to oid_array
  Convert sha1_array_for_each_unique and for_each_abbrev to object_id
  Convert sha1_array_lookup to take struct object_id
  Convert remaining callers of sha1_array_lookup to object_id
  Make sha1_array_append take a struct object_id *
  sha1-array: convert internal storage for struct sha1_array to object_id
  builtin/pull: convert to struct object_id
  submodule: convert check_for_new_submodule_commits to object_id
  sha1_name: convert disambiguate_hint_fn to take object_id
  sha1_name: convert struct disambiguate_state to object_id
  test-sha1-array: convert most code to struct object_id
  parse-options-cb: convert sha1_array_append caller to struct object_id
  fsck: convert init_skiplist to struct object_id
  builtin/receive-pack: convert portions to struct object_id
  builtin/pull: convert portions to struct object_id
  builtin/diff: convert to struct object_id
  Convert GIT_SHA1_RAWSZ used for allocation to GIT_MAX_RAWSZ
  Convert GIT_SHA1_HEXSZ used for allocation to GIT_MAX_HEXSZ
  Define new hash-size constants for allocating memory
2017-04-19 21:37:13 -07:00
Junio C Hamano
c703555cc8 Merge branch 'sb/submodule-short-status'
The output from "git status --short" has been extended to show
various kinds of dirtyness in submodules differently; instead of to
"M" for modified, 'm' and '?' can be shown to signal changes only
to the working tree of the submodule but not the commit that is
checked out.

* sb/submodule-short-status:
  submodule.c: correctly handle nested submodules in is_submodule_modified
  short status: improve reporting for submodule changes
  submodule.c: stricter checking for submodules in is_submodule_modified
  submodule.c: port is_submodule_modified to use porcelain 2
  submodule.c: convert is_submodule_modified to use strbuf_getwholeline
  submodule.c: factor out early loop termination in is_submodule_modified
  submodule.c: use argv_array in is_submodule_modified
2017-04-19 21:37:12 -07:00
Stefan Beller
f2d48994dc submodule.c: submodule_move_head works with broken submodules
Early on in submodule_move_head just after the check if the submodule is
initialized, we need to check if the submodule is populated correctly.

If the submodule is initialized but doesn't look like it is populated,
this is a red flag and can indicate multiple sorts of failures:
(1) The submodule may be recorded at an object name, that is missing.
(2) The submodule '.git' file link may be broken and it is not pointing
    at a repository.

In both cases we want to complain to the user in the non-forced mode,
and in the forced mode ignoring the old state and just moving the
submodule into its new state with a fixed '.git' file link.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-18 21:18:29 -07:00
Stefan Beller
823bab09c6 submodule.c: uninitialized submodules are ignored in recursive commands
This was an oversight when working on the working tree modifying commands
recursing into submodules.

To test for uninitialized submodules, introduce another submodule
"uninitialized_sub". Adding it via `submodule add` will activate the
submodule in the preparation area (in create_lib_submodule_repo we
setup all the things in submodule_update_repo), but the later tests
will use a new testing repo that clones the preparation repo
in which the new submodule is not initialized.

By adding it to the branch "add_sub1", which is the starting point of
all other branches, we have wide coverage.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-18 21:18:29 -07:00
Sebastian Schuberth
731eb176a3 submodule: remove a superfluous second check for the "new" variable
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-17 17:41:13 -07:00
Junio C Hamano
1776a710d6 Merge branch 'sb/show-diff-for-submodule-in-diff-fix'
"git diff --submodule=diff" learned to work better in a project
with a submodule that in turn has its own submodules.

* sb/show-diff-for-submodule-in-diff-fix:
  diff: submodule inline diff to initialize env array.
2017-04-16 23:29:32 -07:00
Junio C Hamano
cb054eb264 Merge branch 'jk/snprintf-cleanups'
Code clean-up.

* jk/snprintf-cleanups:
  daemon: use an argv_array to exec children
  gc: replace local buffer with git_path
  transport-helper: replace checked snprintf with xsnprintf
  convert unchecked snprintf into xsnprintf
  combine-diff: replace malloc/snprintf with xstrfmt
  replace unchecked snprintf calls with heap buffers
  receive-pack: print --pack-header directly into argv array
  name-rev: replace static buffer with strbuf
  create_branch: use xstrfmt for reflog message
  create_branch: move msg setup closer to point of use
  avoid using mksnpath for refs
  avoid using fixed PATH_MAX buffers for refs
  fetch: use heap buffer to format reflog
  tag: use strbuf to format tag header
  diff: avoid fixed-size buffer for patch-ids
  odb_mkstemp: use git_path_buf
  odb_mkstemp: write filename into strbuf
  do not check odb_mkstemp return value for errors
2017-04-16 23:29:26 -07:00
Ralf Thielow
35ad44cbd8 submodule.c: add missing ' in error messages
Signed-off-by: Ralf Thielow <ralf.thielow@gmail.com>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-13 17:59:21 -07:00
Brandon Williams
06bf4ad1db push: propagate remote and refspec with --recurse-submodules
Teach "push --recurse-submodules" to propagate, if given a name as remote, the
provided remote and refspec recursively to the pushes performed in the
submodules. The push will therefore only succeed if all submodules have a
remote with such a name configured.

Note that "push --recurse-submodules" with a path or URL as remote will not
propagate the remote or refspec and instead use the default remote and refspec
configured in the submodule, preserving the current behavior.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-11 00:45:26 -07:00
Brandon Williams
2a90556dde push: propagate push-options with --recurse-submodules
Teach push --recurse-submodules to propagate push-options recursively to
the pushes performed in the submodules.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-11 00:45:03 -07:00
Stefan Beller
17b254cda6 diff: submodule inline diff to initialize env array.
David reported:
> When I try to run `git diff --submodule=diff` in a submodule which has
> it's own submodules that have changes I get the error: fatal: bad
> object.

This happens, because we do not properly initialize the environment
in which the diff is run in the submodule. That means we inherit the
environment from the main process, which sets environment variables.
(Apparently we do set environment variables which we do not set
when not in a submodules, i.e. the .git directory is linked)

This commit, just like fd47ae6a5b (diff: teach diff to display
submodule difference with an inline diff, 2016-08-31) introduces bad
test code (i.e. hard coded hash values), which will be cleanup up in
a later patch.

Reported-by: David Parrish <daveparrish@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-02 09:51:03 -07:00
brian m. carlson
910650d2f8 Rename sha1_array to oid_array
Since this structure handles an array of object IDs, rename it to struct
oid_array.  Also rename the accessor functions and the initialization
constant.

This commit was produced mechanically by providing non-Documentation
files to the following Perl one-liners:

    perl -pi -E 's/struct sha1_array/struct oid_array/g'
    perl -pi -E 's/\bsha1_array_/oid_array_/g'
    perl -pi -E 's/SHA1_ARRAY_INIT/OID_ARRAY_INIT/g'

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-31 08:33:56 -07:00
brian m. carlson
1b7ba794d2 Convert sha1_array_for_each_unique and for_each_abbrev to object_id
Make sha1_array_for_each_unique take a callback using struct object_id.
Since one of these callbacks is an argument to for_each_abbrev, convert
those as well.  Rename various functions, replacing "sha1" with "oid".

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-31 08:33:55 -07:00
brian m. carlson
98a72ddc12 Make sha1_array_append take a struct object_id *
Convert the callers to pass struct object_id by changing the function
declaration and definition and applying the following semantic patch:

@@
expression E1, E2;
@@
- sha1_array_append(E1, E2.hash)
+ sha1_array_append(E1, &E2)

@@
expression E1, E2;
@@
- sha1_array_append(E1, E2->hash)
+ sha1_array_append(E1, E2)

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-31 08:33:55 -07:00
Jeff King
1a168e5c86 convert unchecked snprintf into xsnprintf
These calls to snprintf should always succeed, because their
input is small and fixed. Let's use xsnprintf to make sure
this is the case (and to make auditing for actual truncation
easier).

These could be candidates for turning into heap buffers, but
they fall into a few broad categories that make it not worth
doing:

  - formatting single numbers is simple enough that we can
    see the result should fit

  - the size of a sha1 is likewise well-known, and I didn't
    want to cause unnecessary conflicts with the ongoing
    process to convert these constants to GIT_MAX_HEXSZ

  - the interface for curl_errorstr is dictated by curl

Signed-off-by: Jeff King <peff@peff.net>
2017-03-30 14:59:50 -07:00
Junio C Hamano
a93dcb0a56 Merge branch 'bw/submodule-is-active'
"what URL do we want to update this submodule?" and "are we
interested in this submodule?" are split into two distinct
concepts, and then the way used to express the latter got extended,
paving a way to make it easier to manage a project with many
submodules and make it possible to later extend use of multiple
worktrees for a project with submodules.

* bw/submodule-is-active:
  submodule add: respect submodule.active and submodule.<name>.active
  submodule--helper init: set submodule.<name>.active
  clone: teach --recurse-submodules to optionally take a pathspec
  submodule init: initialize active submodules
  submodule: decouple url and submodule interest
  submodule--helper clone: check for configured submodules using helper
  submodule sync: use submodule--helper is-active
  submodule sync: skip work for inactive submodules
  submodule status: use submodule--helper is-active
  submodule--helper: add is-active subcommand
2017-03-30 14:07:14 -07:00
Stefan Beller
40069d6e3a submodule.c: correctly handle nested submodules in is_submodule_modified
Suppose I have a superproject 'super', with two submodules 'super/sub'
and 'super/sub1'. 'super/sub' itself contains a submodule
'super/sub/subsub'. Now suppose I run, from within 'super':

    echo hi >sub/subsub/stray-file
    echo hi >sub1/stray-file

Currently we get would see the following output in git-status:

    git status --short
     m sub
     ? sub1

With this patch applied, the untracked file in the nested submodule is
displayed as an untracked file on the 'super' level as well.

    git status --short
     ? sub
     ? sub1

This doesn't change the output of 'git status --porcelain=1' for nested
submodules, because its output is always ' M' for either untracked files
or local modifications no matter the nesting level of the submodule.

'git status --porcelain=2' is affected by this change in a nested
submodule, though. Without this patch it would report the direct submodule
as modified and having no untracked files. With this patch it would report
untracked files. Chalk this up as a bug fix.

This bug fix also affects the default output (non-short, non-porcelain)
of git-status, which is not tested here.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-29 17:37:21 -07:00
Junio C Hamano
e394fa01d6 Merge branch 'sb/checkout-recurse-submodules'
"git checkout" is taught the "--recurse-submodules" option.

* sb/checkout-recurse-submodules:
  builtin/read-tree: add --recurse-submodules switch
  builtin/checkout: add --recurse-submodules switch
  entry.c: create submodules when interesting
  unpack-trees: check if we can perform the operation for submodules
  unpack-trees: pass old oid to verify_clean_submodule
  update submodules: add submodule_move_head
  submodule.c: get_super_prefix_or_empty
  update submodules: move up prepare_submodule_repo_env
  submodules: introduce check to see whether to touch a submodule
  update submodules: add a config option to determine if submodules are updated
  update submodules: add submodule config parsing
  make is_submodule_populated gently
  lib-submodule-update.sh: define tests for recursing into submodules
  lib-submodule-update.sh: replace sha1 by hash
  lib-submodule-update: teach test_submodule_content the -C <dir> flag
  lib-submodule-update.sh: do not use ./. as submodule remote
  lib-submodule-update.sh: reorder create_lib_submodule_repo
  submodule--helper.c: remove duplicate code
  connect_work_tree_and_git_dir: safely create leading directories
2017-03-28 14:05:58 -07:00
brian m. carlson
2eb80bcdcc submodule: convert check_for_new_submodule_commits to object_id
All of the callers of this function have been converted, so convert this
function and update the callers.  This function also calls
sha1_array_append, which we'll convert shortly.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-28 09:59:34 -07:00
Nguyễn Thái Ngọc Duy
bbbb7de7ac path.c: move some code out of strbuf_git_path_submodule()
refs is learning to avoid path rewriting that is done by
strbuf_git_path_submodule(). Factor out this code so it could be reused
by refs_* functions.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 10:23:40 -07:00
Stefan Beller
5c896f7c3e submodule.c: stricter checking for submodules in is_submodule_modified
By having a stricter check in the superproject we catch errors earlier,
instead of spawning a child process to tell us.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 09:48:19 -07:00
Stefan Beller
fcecf0b968 submodule.c: port is_submodule_modified to use porcelain 2
Migrate 'is_submodule_modified' to the new porcelain format of
git-status. This conversion attempts to convert faithfully, i.e.
the behavior ought to be exactly the same.

As the output in the parsing only distinguishes between untracked files
and the rest, this is easy to port to the new format, as we only
need to identify untracked files and the rest is handled in the "else"
case.

untracked files are indicated by only a single question mark instead of
two question marks, so the conversion is easy.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 09:48:19 -07:00
Stefan Beller
af6865a7f1 submodule.c: convert is_submodule_modified to use strbuf_getwholeline
Instead of implementing line reading yet again, make use of our beautiful
library function to read one line.  By using strbuf_getwholeline instead
of strbuf_read, we avoid having to allocate memory for the entire child
process output at once.  That is, we limit maximum memory usage.
Also we can start processing the output as it comes in, no need to
wait for all of it.

Once we know all information that we care about, we can terminate
the child early. In that case we do not care about its exit code as well.
By just closing our side of the pipe the child process will get a SIGPIPE
signal, which it will not report nor do we report it in finish_command,
ac78663b0d (run-command: don't warn on SIGPIPE deaths, 2015-12-29).

Helped-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 09:48:19 -07:00
Stefan Beller
64f9a946f0 submodule.c: factor out early loop termination in is_submodule_modified
This makes it easier for a follow up patch.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 09:48:19 -07:00
Stefan Beller
d0d7fed1a2 submodule.c: use argv_array in is_submodule_modified
struct argv_array is easier to use and maintain.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-27 09:48:18 -07:00
Brandon Williams
a086f921a7 submodule: decouple url and submodule interest
Currently the submodule.<name>.url config option is used to determine if
a given submodule is of interest to the user.  This ends up being
cumbersome in a world where we want to have different submodules checked
out in different worktrees or a more generalized mechanism to select
which submodules are of interest.

In a future with worktree support for submodules, there will be multiple
working trees, each of which may only need a subset of the submodules
checked out.  The URL (which is where the submodule repository can be
obtained) should not differ between different working trees.

It may also be convenient for users to more easily specify groups of
submodules they are interested in as opposed to running "git submodule
init <path>" on each submodule they want checked out in their working
tree.

To this end two config options are introduced, submodule.active and
submodule.<name>.active.  The submodule.active config holds a pathspec
that specifies which submodules should exist in the working tree.  The
submodule.<name>.active config is a boolean flag used to indicate if
that particular submodule should exist in the working tree.

Its important to note that submodule.active functions differently than
the other configuration options since it takes a pathspec.  This allows
users to adopt at least two new workflows:

  1. Submodules can be grouped with a leading directory, such that a
     pathspec e.g. 'lib/' would cover all library-ish modules to allow
     those who are interested in library-ish modules to set
     "submodule.active = lib/" just once to say any and all modules in
     'lib/' are interesting.

  2. Once the pathspec-attribute feature is invented, users can label
     submodules with attributes to group them, so that a broad pathspec
     with attribute requirements, e.g. ':(attr:lib)', can be used to say
     any and all modules with the 'lib' attribute are interesting.
     Since the .gitattributes file, just like the .gitmodules file, is
     tracked by the superproject, when a submodule moves in the
     superproject tree, the project can adjust which path gets the
     attribute in .gitattributes, just like it can adjust which path has
     the submodule in .gitmodules.

Neither of these two additional configuration options solve the problem
of wanting different submodules checked out in different worktrees
because multiple worktrees share .git/config.  Only once per-worktree
configurations become a reality can this be solved, but this is a
necessary preparatory step for that future.

Given these multiple ways to check if a submodule is of interest, the
more fine-grained submodule.<name>.active option has the highest order
of precedence followed by the pathspec check against submodule.active.
To ensure backwards compatibility, if neither of these options are set,
git falls back to checking the submodule.<name>.url option to determine
if a submodule is interesting.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18 09:51:23 -07:00
Junio C Hamano
3edcc04862 Merge branch 'sb/rev-parse-show-superproject-root'
From a working tree of a repository, a new option of "rev-parse"
lets you ask if the repository is used as a submodule of another
project, and where the root level of the working tree of that
project (i.e. your superproject) is.

* sb/rev-parse-show-superproject-root:
  rev-parse: add --show-superproject-working-tree
2017-03-17 13:50:25 -07:00
Stefan Beller
6e3c1595c6 update submodules: add submodule_move_head
In later patches we introduce the options and flag for commands
that modify the working directory, e.g. git-checkout.

This piece of code will be used universally for
all these working tree modifications as it
* supports dry run to answer the question:
  "Is it safe to change the submodule to this new state?"
  e.g. is it overwriting untracked files or are there local
  changes that would be overwritten?
* supports a force flag that can be used for resetting
  the tree.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-16 14:07:16 -07:00
Stefan Beller
202275b96b submodule.c: get_super_prefix_or_empty
In a later patch we need to use the super_prefix, and
in case it is NULL we can just assume it is empty.
Create a helper function for this.

We already have some use cases for this helper function,
convert them, too.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-16 14:07:16 -07:00
Stefan Beller
6cd5757c89 update submodules: move up prepare_submodule_repo_env
In a later patch we need to prepare the submodule environment with
another git directory, so split up the function.

Also move it up in the file such that we do not need to declare the
function later before using it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-15 18:15:54 -07:00
Stefan Beller
84f8925eeb submodules: introduce check to see whether to touch a submodule
In later patches we introduce the --recurse-submodule flag for commands
that modify the working directory, e.g. git-checkout.

It is potentially expensive to check if a submodule needs an update,
because a common theme to interact with submodules is to spawn a child
process for each interaction.

So let's introduce a function that checks if a submodule needs
to be checked for an update before attempting the update.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-15 18:15:54 -07:00
Stefan Beller
cf055e465f update submodules: add a config option to determine if submodules are updated
In later patches we introduce the options and flag for commands
that modify the working directory, e.g. git-checkout.

Have a central place to store such settings whether we want to update
a submodule.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-15 18:15:54 -07:00
Stefan Beller
15cdc64776 make is_submodule_populated gently
We need the gentle version in a later patch. As we have just one caller,
migrate the caller.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-15 18:15:54 -07:00
Stefan Beller
365444a6a5 connect_work_tree_and_git_dir: safely create leading directories
In a later patch we'll use connect_work_tree_and_git_dir when the
directory for the gitlink file doesn't exist yet. This patch makes
connect_work_tree_and_git_dir safe to use for both cases of
either the git dir or the working dir missing.

To do so, we need to call safe_create_leading_directories[_const]
on both directories. However this has to happen before we construct
the absolute paths as real_pathdup assumes the directories to
be there already.

So for both the config file in the git dir as well as the .git link
file we need to
a) construct the name
b) call SCLD
c) get the absolute path
d) once a-c is done for both we can consume the absolute path
   to compute the relative path to each other and store those
   relative paths.

The implementation provided here puts a) and b) for both cases first,
and then performs c and d after.

One of the two users of 'connect_work_tree_and_git_dir' already checked
for the directory being there, so we can loose that check as
connect_work_tree_and_git_dir handles this functionality now.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-15 18:15:53 -07:00
Stefan Beller
bf0231c661 rev-parse: add --show-superproject-working-tree
In some situations it is useful to know if the given repository
is a submodule of another repository.

Add the flag --show-superproject-working-tree to git-rev-parse
to make it easy to find out if there is a superproject. When no
superproject exists, the output will be empty.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-08 15:52:03 -08:00
Johannes Schindelin
ce83eadd9a real_pathdup(): fix callsites that wanted it to die on error
In 4ac9006f83 (real_path: have callers use real_pathdup and
strbuf_realpath, 2016-12-12), we changed the xstrdup(real_path())
pattern to use real_pathdup() directly.

The problem with this change is that real_path() calls
strbuf_realpath() with die_on_error = 1 while real_pathdup() calls
it with die_on_error = 0. Meaning that in cases where real_path()
causes Git to die() with an error message, real_pathdup() is silent
and returns NULL instead.

The callers, however, are ill-prepared for that change, as they expect
the return value to be non-NULL (and otherwise the function died
with an appropriate error message).

Fix this by extending real_pathdup()'s signature to accept the
die_on_error flag and simply pass it through to strbuf_realpath(),
and then adjust all callers after a careful audit whether they would
handle NULLs well.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-08 14:38:41 -08:00
Stefan Beller
ec9629b3b9 submodule absorbing: fix worktree/gitdir pointers recursively for non-moves
Consider having a submodule 'sub' and a nested submodule at 'sub/nested'.
When nested is already absorbed into sub, but sub is not absorbed into
its superproject, then we need to fixup the gitfile and core.worktree
setting for 'nested' when absorbing 'sub', but we do not need to move
its git dir around.

Previously 'nested's gitfile contained "gitdir: ../.git/modules/nested";
it has to be corrected to "gitdir: ../../.git/modules/sub1/modules/nested".

An alternative I considered to do this work lazily, i.e. when resolving
"../.git/modules/nested", we would notice the ".git" being a gitfile
linking to another path.  That seemed to be robuster by design, but harder
to get the implementation right.  Maybe we have to do that anyway once we
try to have submodules and worktrees working nicely together, but for now
just produce 'correct' (i.e. direct) pointers.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-26 11:01:04 -08:00
Junio C Hamano
8b5b0e8db5 Merge branch 'sb/submodule-cleanup-export-git-dir-env'
Code cleanup.

* sb/submodule-cleanup-export-git-dir-env:
  submodule.c: use GIT_DIR_ENVIRONMENT consistently
2017-01-18 15:12:14 -08:00
Junio C Hamano
3ccd681c2a Merge branch 'sb/submodule-rm-absorb'
"git rm" used to refuse to remove a submodule when it has its own
git repository embedded in its working tree.  It learned to move
the repository away to $GIT_DIR/modules/ of the superproject
instead, and allow the submodule to be deleted (as long as there
will be no loss of local modifications, that is).

* sb/submodule-rm-absorb:
  rm: absorb a submodules git dir before deletion
  submodule: rename and add flags to ok_to_remove_submodule
  submodule: modernize ok_to_remove_submodule to use argv_array
  submodule.h: add extern keyword to functions
2017-01-18 15:12:11 -08:00
Junio C Hamano
55d128ae06 Merge branch 'bw/grep-recurse-submodules'
"git grep" has been taught to optionally recurse into submodules.

* bw/grep-recurse-submodules:
  grep: search history of moved submodules
  grep: enable recurse-submodules to work on <tree> objects
  grep: optionally recurse into submodules
  grep: add submodules as a grep source type
  submodules: load gitmodules file from commit sha1
  submodules: add helper to determine if a submodule is initialized
  submodules: add helper to determine if a submodule is populated
  real_path: canonicalize directory separators in root parts
  real_path: have callers use real_pathdup and strbuf_realpath
  real_path: create real_pathdup
  real_path: convert real_path_internal to strbuf_realpath
  real_path: resolve symlinks by hand
2017-01-18 15:12:11 -08:00
Junio C Hamano
3075e40c75 Merge branch 'bw/push-dry-run' into maint
"git push --dry-run --recurse-submodule=on-demand" wasn't
"--dry-run" in the submodules.

* bw/push-dry-run:
  push: fix --dry-run to not push submodules
  push: --dry-run updates submodules when --recurse-submodules=on-demand
2017-01-17 14:49:27 -08:00
Junio C Hamano
9da9965ba6 Merge branch 'hv/submodule-not-yet-pushed-fix' into maint
The code in "git push" to compute if any commit being pushed in the
superproject binds a commit in a submodule that hasn't been pushed
out was overly inefficient, making it unusable even for a small
project that does not have any submodule but have a reasonable
number of refs.

* hv/submodule-not-yet-pushed-fix:
  submodule_needs_pushing(): explain the behaviour when we cannot answer
  batch check whether submodule needs pushing into one call
  serialize collection of refs that contain submodule changes
  serialize collection of changed submodules
2017-01-17 14:49:26 -08:00
Junio C Hamano
da2b74eeec Merge branch 'sb/submodule-embed-gitdir'
A new submodule helper "git submodule embedgitdirs" to make it
easier to move embedded .git/ directory for submodules in a
superproject to .git/modules/ (and point the latter with the former
that is turned into a "gitdir:" file) has been added.

* sb/submodule-embed-gitdir:
  worktree: initialize return value for submodule_uses_worktrees
  submodule: add absorb-git-dir function
  move connect_work_tree_and_git_dir to dir.h
  worktree: check if a submodule uses worktrees
  test-lib-functions.sh: teach test_commit -C <dir>
  submodule helper: support super prefix
  submodule: use absolute path for computing relative path connecting
2017-01-10 15:24:28 -08:00
Stefan Beller
c5f3cba126 submodule.c: use GIT_DIR_ENVIRONMENT consistently
In C code we have the luxury of having constants for all the important
things that are hard coded. This is the only place in C that hard codes
the git directory environment variable, so fix it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-07 13:10:02 -08:00
Stefan Beller
83b7696605 submodule: rename and add flags to ok_to_remove_submodule
In different contexts the question "Is it ok to delete a submodule?"
may be answered differently.

In 293ab15eea (submodule: teach rm to remove submodules unless they
contain a git directory, 2012-09-26) a case was made that we can safely
ignore ignored untracked files for removal as we explicitely ask for the
removal of the submodule.

In a later patch we want to remove submodules even when the user doesn't
explicitly ask for it (e.g. checking out a tree-ish in which the submodule
doesn't exist).  In that case we want to be more careful when it comes
to deletion of untracked files. As of this patch it is unclear how this
will be implemented exactly, so we'll offer flags in which the caller
can specify how the different untracked files ought to be handled.

As the flags allow the function to not die on an error when spawning
a child process, we need to find an appropriate return code for the
case when the child process could not be started. As in that case we
cannot tell if the submodule is ok to remove, we'd want to return 'false'.

As only 0 is understood as false, rename the function to invert the
meaning, i.e. the return code of 0 signals the removal of the submodule
is fine, and other values can be used to return a more precise answer
what went wrong.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-27 14:19:35 -08:00
Stefan Beller
5a1c824f70 submodule: modernize ok_to_remove_submodule to use argv_array
Instead of constructing the NULL terminated array ourselves, we
should make use of the argv_array infrastructure.

While at it, adapt the error messages to reflect the actual invocation.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-27 14:19:35 -08:00
Brandon Williams
9ebf689aad submodules: load gitmodules file from commit sha1
teach submodules to load a '.gitmodules' file from a commit sha1.  This
enables the population of the submodule_cache to be based on the state
of the '.gitmodules' file from a particular commit.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-22 11:47:33 -08:00
Brandon Williams
f9f42560e2 submodules: add helper to determine if a submodule is initialized
Add the `is_submodule_initialized()` helper function to submodules.c.
`is_submodule_initialized()` performs a check to determine if the
submodule at the given path has been initialized.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-22 11:47:33 -08:00
Brandon Williams
5688c28d81 submodules: add helper to determine if a submodule is populated
Add the `is_submodule_populated()` helper function to submodules.c.
`is_submodule_populated()` performes a check to see if a submodule has
been checkout out (and has a valid .git directory/file) at the given path.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-22 11:47:33 -08:00
Junio C Hamano
12cf1135df Merge branch 'bw/push-dry-run'
"git push --dry-run --recurse-submodule=on-demand" wasn't
"--dry-run" in the submodules.

* bw/push-dry-run:
  push: fix --dry-run to not push submodules
  push: --dry-run updates submodules when --recurse-submodules=on-demand
2016-12-16 15:27:48 -08:00
Junio C Hamano
af952dac7a Merge branch 'hv/submodule-not-yet-pushed-fix'
The code in "git push" to compute if any commit being pushed in the
superproject binds a commit in a submodule that hasn't been pushed
out was overly inefficient, making it unusable even for a small
project that does not have any submodule but have a reasonable
number of refs.

* hv/submodule-not-yet-pushed-fix:
  submodule_needs_pushing(): explain the behaviour when we cannot answer
  batch check whether submodule needs pushing into one call
  serialize collection of refs that contain submodule changes
  serialize collection of changed submodules
2016-12-16 15:27:47 -08:00
Brandon Williams
4ac9006f83 real_path: have callers use real_pathdup and strbuf_realpath
Migrate callers of real_path() who duplicate the retern value to use
real_pathdup or strbuf_realpath.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12 15:22:32 -08:00
Stefan Beller
f6f8586140 submodule: add absorb-git-dir function
When a submodule has its git dir inside the working dir, the submodule
support for checkout that we plan to add in a later patch will fail.

Add functionality to migrate the git directory to be absorbed
into the superprojects git directory.

The newly added code in this patch is structured such that other areas of
Git can also make use of it. The code in the submodule--helper is a mere
wrapper and option parser for the function
`absorb_git_dir_into_superproject`, that takes care of embedding the
submodules git directory into the superprojects git dir. That function
makes use of the more abstract function for this use case
`relocate_gitdir`, which can be used by e.g. the worktree code eventually
to move around a git directory.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12 15:15:07 -08:00
Stefan Beller
47e83eb3b7 move connect_work_tree_and_git_dir to dir.h
That function was primarily used by submodule code, but the function
itself is not inherently about submodules. In the next patch we'll
introduce relocate_git_dir, which can be used by worktrees as well,
so find a neutral middle ground in dir.h.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12 15:15:07 -08:00
Stefan Beller
90c0011619 submodule: use absolute path for computing relative path connecting
The current caller of connect_work_tree_and_git_dir passes
an absolute path for the `git_dir` parameter. In the future patch
we will also pass in relative path for `git_dir`. Extend the functionality
of connect_work_tree_and_git_dir to take relative paths for parameters.

We could work around this in the future patch by computing the absolute
path for the git_dir in the calling site, however accepting relative
paths for either parameter makes the API for this function much harder
to misuse.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09 14:52:57 -08:00
Brandon Williams
0301c821c5 push: fix --dry-run to not push submodules
Teach push to respect the --dry-run option when configured to
recursively push submodules 'on-demand'.  This is done by passing the
--dry-run flag to the child process which performs a push for a
submodules when performing a dry-run.

In order to preserve good user experience, the additional check for
unpushed submodules is skipped during a dry-run when
--recurse-submodules=on-demand.  The check is skipped because the submodule
pushes were performed as dry-runs and this check would always fail as the
submodules would still need to be pushed.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-23 08:39:14 -08:00
Heiko Voigt
250ab24ab3 submodule_needs_pushing(): explain the behaviour when we cannot answer
When we do not have commits that are involved in the update of the
superproject in our copy of submodule, we cannot tell if the remote
end needs to acquire these commits to be able to check out the
superproject tree.  Explain why we answer "no there is no need/point
in pushing from our submodule repository" in this case.

Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-16 11:13:58 -08:00
Heiko Voigt
5b6607d23f batch check whether submodule needs pushing into one call
We run a command for each sha1 change in a submodule. This is
unnecessary since we can simply batch all sha1's we want to check into
one command. Lets do it so we can speedup the check when many submodule
changes are in need of checking.

Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-16 11:12:50 -08:00
Heiko Voigt
9cfa1c260f serialize collection of refs that contain submodule changes
We are iterating over each pushed ref and want to check whether it
contains changes to submodules. Instead of immediately checking each ref
lets first collect them and then do the check for all of them in one
revision walk.

Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-16 11:12:50 -08:00