Commit Graph

177 Commits

Author SHA1 Message Date
Brandon Williams
1c41c82bc4 grep: convert to struct object_id
Convert the remaining parts of grep to use struct object_id.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-02 09:36:06 +09:00
Junio C Hamano
6b526ced6f Merge branch 'bc/object-id'
Conversion from uchar[20] to struct object_id continues.

* bc/object-id: (53 commits)
  object: convert parse_object* to take struct object_id
  tree: convert parse_tree_indirect to struct object_id
  sequencer: convert do_recursive_merge to struct object_id
  diff-lib: convert do_diff_cache to struct object_id
  builtin/ls-tree: convert to struct object_id
  merge: convert checkout_fast_forward to struct object_id
  sequencer: convert fast_forward_to to struct object_id
  builtin/ls-files: convert overlay_tree_on_cache to object_id
  builtin/read-tree: convert to struct object_id
  sha1_name: convert internals of peel_onion to object_id
  upload-pack: convert remaining parse_object callers to object_id
  revision: convert remaining parse_object callers to object_id
  revision: rename add_pending_sha1 to add_pending_oid
  http-push: convert process_ls_object and descendants to object_id
  refs/files-backend: convert many internals to struct object_id
  refs: convert struct ref_update to use struct object_id
  ref-filter: convert some static functions to struct object_id
  Convert struct ref_array_item to struct object_id
  Convert the verify_pack callback to struct object_id
  Convert lookup_tag to struct object_id
  ...
2017-05-29 12:34:43 +09:00
brian m. carlson
c251c83df2 object: convert parse_object* to take struct object_id
Make parse_object, parse_object_or_die, and parse_object_buffer take a
pointer to struct object_id.  Remove the temporary variables inserted
earlier, since they are no longer necessary.  Transform all of the
callers using the following semantic patch:

@@
expression E1;
@@
- parse_object(E1.hash)
+ parse_object(&E1)

@@
expression E1;
@@
- parse_object(E1->hash)
+ parse_object(E1)

@@
expression E1, E2;
@@
- parse_object_or_die(E1.hash, E2)
+ parse_object_or_die(&E1, E2)

@@
expression E1, E2;
@@
- parse_object_or_die(E1->hash, E2)
+ parse_object_or_die(E1, E2)

@@
expression E1, E2, E3, E4, E5;
@@
- parse_object_buffer(E1.hash, E2, E3, E4, E5)
+ parse_object_buffer(&E1, E2, E3, E4, E5)

@@
expression E1, E2, E3, E4, E5;
@@
- parse_object_buffer(E1->hash, E2, E3, E4, E5)
+ parse_object_buffer(E1, E2, E3, E4, E5)

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-08 15:12:58 +09:00
Brandon Williams
0d32c183b6 dir: convert fill_directory to take an index
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-06 19:15:39 +09:00
Junio C Hamano
df3b119265 Merge branch 'ab/grep-plug-pathspec-leak'
Call clear_pathspec() to release resources immediately before the
cmd_grep() function returns.

* ab/grep-plug-pathspec-leak:
  grep: plug a trivial memory leak
2017-04-19 21:37:21 -07:00
Ævar Arnfjörð Bjarmason
7861fa07c3 grep: plug a trivial memory leak
Change the cleanup phase for the grep command to free the pathspec
struct that's allocated earlier in the same block, and used just a few
lines earlier.

With "grep hi README.md" valgrind reports a loss of 239 bytes now,
down from 351.

The relevant --num-callers=40 --leak-check=full --show-leak-kinds=all
backtrace is:

    [...] 187 (112 direct, 75 indirect) bytes in 1 blocks are definitely lost in loss record 70 of 110
    [...]    at 0x4C2BBAF: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
    [...]    by 0x60B339: do_xmalloc (wrapper.c:59)
    [...]    by 0x60B2F6: xmalloc (wrapper.c:86)
    [...]    by 0x576B37: parse_pathspec (pathspec.c:652)
    [...]    by 0x4519F0: cmd_grep (grep.c:1215)
    [...]    by 0x4062EF: run_builtin (git.c:371)
    [...]    by 0x40544D: handle_builtin (git.c:572)
    [...]    by 0x4060A2: run_argv (git.c:624)
    [...]    by 0x4051C6: cmd_main (git.c:701)
    [...]    by 0x4C5901: main (common-main.c:43)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-16 17:56:47 -07:00
Junio C Hamano
3736c92558 Merge branch 'bw/recurse-submodules-relative-fix'
A few commands that recently learned the "--recurse-submodule"
option misbehaved when started from a subdirectory of the
superproject.

* bw/recurse-submodules-relative-fix:
  ls-files: fix bug when recursing with relative pathspec
  ls-files: fix typo in variable name
  grep: fix bug when recursing with relative pathspec
  setup: allow for prefix to be passed to git commands
  grep: fix help text typo
2017-03-30 14:07:15 -07:00
Junio C Hamano
e394fa01d6 Merge branch 'sb/checkout-recurse-submodules'
"git checkout" is taught the "--recurse-submodules" option.

* sb/checkout-recurse-submodules:
  builtin/read-tree: add --recurse-submodules switch
  builtin/checkout: add --recurse-submodules switch
  entry.c: create submodules when interesting
  unpack-trees: check if we can perform the operation for submodules
  unpack-trees: pass old oid to verify_clean_submodule
  update submodules: add submodule_move_head
  submodule.c: get_super_prefix_or_empty
  update submodules: move up prepare_submodule_repo_env
  submodules: introduce check to see whether to touch a submodule
  update submodules: add a config option to determine if submodules are updated
  update submodules: add submodule config parsing
  make is_submodule_populated gently
  lib-submodule-update.sh: define tests for recursing into submodules
  lib-submodule-update.sh: replace sha1 by hash
  lib-submodule-update: teach test_submodule_content the -C <dir> flag
  lib-submodule-update.sh: do not use ./. as submodule remote
  lib-submodule-update.sh: reorder create_lib_submodule_repo
  submodule--helper.c: remove duplicate code
  connect_work_tree_and_git_dir: safely create leading directories
2017-03-28 14:05:58 -07:00
Junio C Hamano
ff8b7e63de Merge branch 'bw/grep-recurse-submodules'
Build fix for NO_PTHREADS build.

* bw/grep-recurse-submodules:
  grep: fix builds with with no thread support
  grep: set default output method
2017-03-28 14:05:57 -07:00
Brandon Williams
2225e1ea20 grep: fix builds with with no thread support
Commit 0281e487fd ("grep: optionally recurse into submodules")
added functions grep_submodule() and grep_submodule_launch() which
use "struct work_item" which is defined only when thread support
is available.

The original implementation of grep_submodule() used the "struct
work_item" in order to gain access to a strbuf to store its output which
was to be printed at a later point in time.  This differs from how both
grep_file() and grep_sha1() handle their output.  This patch eliminates
the reliance on the "struct work_item" and instead opts to use the
output function stored in the output field of the "struct grep_opt"
object directly, making it behave similarly to both grep_file() and
grep_sha1().

Reported-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Brandon Williams <bmwill@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18 10:02:01 -07:00
Junio C Hamano
e1fae93019 Merge branch 'bc/object-id'
"uchar [40]" to "struct object_id" conversion continues.

* bc/object-id:
  wt-status: convert to struct object_id
  builtin/merge-base: convert to struct object_id
  Convert object iteration callbacks to struct object_id
  sha1_file: introduce an nth_packed_object_oid function
  refs: simplify parsing of reflog entries
  refs: convert each_reflog_ent_fn to struct object_id
  reflog-walk: convert struct reflog_info to struct object_id
  builtin/replace: convert to struct object_id
  Convert remaining callers of resolve_refdup to object_id
  builtin/merge: convert to struct object_id
  builtin/clone: convert to struct object_id
  builtin/branch: convert to struct object_id
  builtin/grep: convert to struct object_id
  builtin/fmt-merge-message: convert to struct object_id
  builtin/fast-export: convert to struct object_id
  builtin/describe: convert to struct object_id
  builtin/diff-tree: convert to struct object_id
  builtin/commit: convert to struct object_id
  hex: introduce parse_oid_hex
2017-03-17 13:50:25 -07:00
Brandon Williams
be80a2392f grep: fix bug when recursing with relative pathspec
When using the --recurse-submodules flag with a relative pathspec which
includes "..", an error is produced inside the child process spawned for
a submodule.  When creating the pathspec struct in the child, the ".."
is interpreted to mean "go up a directory" which causes an error stating
that the path ".." is outside of the repository.

While it is true that ".." is outside the scope of the submodule, it is
confusing to a user who originally invoked the command where ".." was
indeed still inside the scope of the superproject.  Since the child
process launched for the submodule has some context that it is operating
underneath a superproject, this error could be avoided.

This patch fixes the bug by passing the 'prefix' to the child process.
Now each child process that works on a submodule has two points of
reference to the superproject: (1) the 'super_prefix' which is the path
from the root of the superproject down to root of the submodule and (2)
the 'prefix' which is the path from the root of the superproject down to
the directory where the user invoked the git command.

With these two pieces of information a child process can correctly
interpret the pathspecs provided by the user as well as being able to
properly format its output relative to the directory the user invoked
the original command from.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-17 11:54:50 -07:00
Brandon Williams
4fb1c6aad8 grep: fix help text typo
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-17 11:54:49 -07:00
Stefan Beller
15cdc64776 make is_submodule_populated gently
We need the gentle version in a later patch. As we have just one caller,
migrate the caller.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-15 18:15:54 -07:00
brian m. carlson
1db1108654 builtin/grep: convert to struct object_id
Convert several functions to use struct object_id, and rename them so
that they no longer refer to SHA-1.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-22 10:12:15 -08:00
Jeff King
131f3c96d2 grep: treat revs the same for --untracked as for --no-index
git-grep has always disallowed grepping in a tree (as
opposed to the working directory) with both --untracked
and --no-index. But we traditionally did so by first
collecting the revs, and then complaining when any were
provided.

The --no-index option recently learned to detect revs
much earlier. This has two user-visible effects:

  - we don't bother to resolve revision names at all. So
    when there's a rev/path ambiguity, we always choose to
    treat it as a path.

  - likewise, when you do specify a revision without "--",
    the error you get is "no such path" and not "--untracked
    cannot be used with revs".

The rationale for doing this with --no-index is that it is
meant to be used outside a repository, and so parsing revs
at all does not make sense.

This patch gives --untracked the same treatment. While it
_is_ meant to be used in a repository, it is explicitly
about grepping the non-repository contents. Telling the user
"we found a rev, but you are not allowed to use revs" is
not really helpful compared to "we treated your argument as
a path, and could not find it".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-14 13:59:25 -08:00
Jeff King
73fc7b6b9b grep: do not diagnose misspelt revs with --no-index
If we are using --no-index, then our arguments cannot be
revs in the first place. Not only is it pointless to
diagnose them, but if we are not in a repository, we should
not be trying to resolve any names.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-14 11:26:37 -08:00
Jeff King
d0ffc06933 grep: avoid resolving revision names in --no-index case
We disallow the use of revisions with --no-index, but we
don't actually check and complain until well after we've
parsed the revisions.

This is the cause of a few problems:

 1. We shouldn't be calling get_sha1() at all when we aren't
    in a repository, as it might access the ref or object
    databases. For now, this should generally just return
    failure, but eventually it will become a BUG().

 2. When there's a "--" disambiguator and you're outside a
    repository, we'll complain early with "unable to resolve
    revision". But we can give a much more specific error.

 3. When there isn't a "--" disambiguator, we still do the
    normal rev/path checks. This is silly, as we know we
    cannot have any revs with --no-index. Everything we see
    must be a path.

    Outside of a repository this doesn't matter (since we
    know it won't resolve), but inside one, we may complain
    unnecessarily if a filename happens to also match a
    refname.

This patch skips the get_sha1() call entirely in the
no-index case, and behaves as if it failed (with the
exception of giving a better error message).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-14 11:26:37 -08:00
Jeff King
b5b81136da grep: fix "--" rev/pathspec disambiguation
If we see "git grep pattern rev -- file" then we apply the
usual rev/pathspec disambiguation rules: any "rev" before
the "--" must be a revision, and we do not need to apply the
verify_non_filename() check.

But there are two bugs here:

  1. We keep a seen_dashdash flag to handle this case, but
     we set it in the same left-to-right pass over the
     arguments in which we parse "rev".

     So when we see "rev", we do not yet know that there is
     a "--", and we mistakenly complain if there is a
     matching file.

     We can fix this by making a preliminary pass over the
     arguments to find the "--", and only then checking the rev
     arguments.

  2. If we can't resolve "rev" but there isn't a dashdash,
     that's OK. We treat it like a path, and complain later
     if it doesn't exist.

     But if there _is_ a dashdash, then we know it must be a
     rev, and should treat it as such, complaining if it
     does not resolve. The current code instead ignores it
     and tries to treat it like a path.

This patch fixes both bugs, and tries to comment the parsing
flow a bit better.

It adds tests that cover the two bugs, but also some related
situations (which already worked, but this confirms that our
fixes did not break anything).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-14 11:26:37 -08:00
Jeff King
20d6421cae grep: re-order rev-parsing loop
We loop over the arguments, but every branch of the loop
hits either a "continue" or a "break". Surely we can make
this simpler.

The final conditional is:

  if (arg is a rev) {
	  ... handle rev ...
	  continue;
  }
  break;

We can rewrite this as:

  if (arg is not a rev)
	  break;

  ... handle rev ...

That makes the flow a little bit simpler, and will make
things much easier to follow when we add more logic in
future patches.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-14 11:26:37 -08:00
Jonathan Tan
dca3b5f5ce grep: do not unnecessarily query repo for "--"
When running a command of the form

  git grep --no-index pattern -- path

in the absence of a Git repository, an error message will be printed:

  fatal: BUG: setup_git_env called without repository

This is because "git grep" tries to interpret "--" as a rev. "git grep"
has always tried to first interpret "--" as a rev for at least a few
years, but this issue was upgraded from a pessimization to a bug in
commit 59332d1 ("Resurrect "git grep --no-index"", 2010-02-06), which
calls get_sha1 regardless of whether --no-index was specified. This bug
appeared to be benign until commit b1ef400 ("setup_git_env: avoid blind
fall-back to ".git"", 2016-10-20) when Git was taught to die in this
situation.  (This "git grep" bug appears to be one of the bugs that
commit b1ef400 is meant to flush out.)

Therefore, always interpret "--" as signaling the end of options,
instead of trying to interpret it as a rev first.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-14 11:26:37 -08:00
Jeff King
a0fe2b0d23 grep: move thread initialization a little lower
Originally, we set up the threads for grep before parsing
the non-option arguments. In 53b8d931b (grep: disable
threading in non-worktree case, 2011-12-12), the thread code
got bumped lower in the function because it now needed to
know whether we got any revision arguments.

That put a big block of code in between the parsing of revs
and the parsing of pathspecs, both of which share some loop
variables. That makes it harder to read the code than the
original, where the shared loops were right next to each
other.

Let's bump the thread initialization until after all of the
parsing is done.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-14 11:13:25 -08:00
Brandon Williams
e6fac7f3d3 grep: search history of moved submodules
If a submodule was renamed at any point since it's inception then if you
were to try and grep on a commit prior to the submodule being moved, you
wouldn't be able to find a working directory for the submodule since the
path in the past is different from the current path.

This patch teaches grep to find the .git directory for a submodule in
the parents .git/modules/ directory in the event the path to the
submodule in the commit that is being searched differs from the state of
the currently checked out commit.  If found, the child process that is
spawned to grep the submodule will chdir into its gitdir instead of a
working directory.

In order to override the explicit setting of submodule child process's
gitdir environment variable (which was introduced in '10f5c526')
`GIT_DIR_ENVIORMENT` needs to be pushed onto child process's env_array.
This allows the searching of history from a submodule's gitdir, rather
than from a working directory.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-22 11:47:33 -08:00
Brandon Williams
74ed43711f grep: enable recurse-submodules to work on <tree> objects
Teach grep to recursively search in submodules when provided with a
<tree> object. This allows grep to search a submodule based on the state
of the submodule that is present in a commit of the super project.

When grep is provided with a <tree> object, the name of the object is
prefixed to all output.  In order to provide uniformity of output
between the parent and child processes the option `--parent-basename`
has been added so that the child can preface all of it's output with the
name of the parent's object instead of the name of the commit SHA1 of
the submodule. This changes output from the command
`git grep -e. -l --recurse-submodules HEAD` from:

      HEAD:file
      <commit sha1 of submodule>:sub/file

to:

      HEAD:file
      HEAD:sub/file

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-22 11:47:33 -08:00
Brandon Williams
0281e487fd grep: optionally recurse into submodules
Allow grep to recognize submodules and recursively search for patterns in
each submodule.  This is done by forking off a process to recursively
call grep on each submodule.  The top level --super-prefix option is
used to pass a path to the submodule which can in turn be used to
prepend to output or in pathspec matching logic.

Recursion only occurs for submodules which have been initialized and
checked out by the parent project.  If a submodule hasn't been
initialized and checked out it is simply skipped.

In order to support the existing multi-threading infrastructure in grep,
output from each child process is captured in a strbuf so that it can be
later printed to the console in an ordered fashion.

To limit the number of theads that are created, each child process has
half the number of threads as its parents (minimum of 1), otherwise we
potentailly have a fork-bomb.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-22 11:47:33 -08:00
brian m. carlson
99d1a9861a cache: convert struct cache_entry to use struct object_id
Convert struct cache_entry to use struct object_id by applying the
following semantic patch and the object_id transforms from contrib, plus
the actual change to the struct:

@@
struct cache_entry E1;
@@
- E1.sha1
+ E1.oid.hash

@@
struct cache_entry *E1;
@@
- E1->sha1
+ E1->oid.hash

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-07 12:59:42 -07:00
Junio C Hamano
42bd66816b Merge branch 'nd/ita-cleanup'
Git does not know what the contents in the index should be for a
path added with "git add -N" yet, so "git grep --cached" should not
show hits (or show lack of hits, with -L) in such a path, but that
logic does not apply to "git grep", i.e. searching in the working
tree files.  But we did so by mistake, which has been corrected.

* nd/ita-cleanup:
  grep: fix grepping for "intent to add" files
  t7810-grep.sh: fix a whitespace inconsistency
  t7810-grep.sh: fix duplicated test name
2016-07-13 11:24:18 -07:00
Charles Bailey
b8e47d1acf grep: fix grepping for "intent to add" files
This reverts commit 4d5520053 (grep: make it clear i-t-a entries are
ignored, 2015-12-27) and adds an alternative fix to maintain the -L
--cached behavior.

4d5520053 caused 'git grep' to no longer find matches in new files in
the working tree where the corresponding index entry had the "intent to
add" bit set, despite the fact that these files are tracked.

The content in the index of a file for which the "intent to add" bit is
set is considered indeterminate and not empty. For most grep queries we
want these to behave the same, however for -L --cached (files without a
match) we don't want to respond positively for "intent to add" files as
their contents are indeterminate. This is in contrast to files with
empty contents in the index (no lines implies no matches for any grep
query expression) which should be reported in the output of a grep -L
--cached invocation.

Add tests to cover this case and a few related cases which previously
lacked coverage.

Helped-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Charles Bailey <cbailey32@bloomberg.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 13:27:41 -07:00
Junio C Hamano
8429f2b42d Merge branch 'bc/object-id'
Move from unsigned char[20] to struct object_id continues.

* bc/object-id:
  match-trees: convert several leaf functions to use struct object_id
  tree-walk: convert tree_entry_extract() to use struct object_id
  struct name_entry: use struct object_id instead of unsigned char sha1[20]
  match-trees: convert shift_tree() and shift_tree_by() to use object_id
  test-match-trees: convert to use struct object_id
  sha1-name: introduce a get_oid() function
2016-05-06 14:45:44 -07:00
brian m. carlson
7d924c9139 struct name_entry: use struct object_id instead of unsigned char sha1[20]
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-25 14:23:42 -07:00
Jeff King
85975c0c7f grep: turn off gitlink detection for --no-index
If we are running "git grep --no-index" outside of a git
repository, we behave roughly like "grep -r", examining all
files in the current directory and its subdirectories.
However, because we use fill_directory() to do the
recursion, it will skip over any directories which look like
sub-repositories.

For a normal git operation (like "git grep" in a repository)
this makes sense; we do not want to cross the boundary out
of our current repository into a submodule. But for
"--no-index" without a repository, we should look at all
files, including embedded repositories.

There is one exception, though: we probably should _not_
descend into ".git" directories. Doing so is inefficient and
unlikely to turn up useful hits.

This patch drops our use of dir.c's gitlink-detection, but
we do still avoid ".git". That makes us more like tools such
as "ack" or "ag", which also know to avoid cruft in .git.

As a bonus, this also drops our usage of the ref code
when we are outside of a repository, making the transition
to pluggable ref backends cleaner.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-07 12:27:28 -08:00
Junio C Hamano
11529ecec9 Merge branch 'jk/tighten-alloc'
Update various codepaths to avoid manually-counted malloc().

* jk/tighten-alloc: (22 commits)
  ewah: convert to REALLOC_ARRAY, etc
  convert ewah/bitmap code to use xmalloc
  diff_populate_gitlink: use a strbuf
  transport_anonymize_url: use xstrfmt
  git-compat-util: drop mempcpy compat code
  sequencer: simplify memory allocation of get_message
  test-path-utils: fix normalize_path_copy output buffer size
  fetch-pack: simplify add_sought_entry
  fast-import: simplify allocation in start_packfile
  write_untracked_extension: use FLEX_ALLOC helper
  prepare_{git,shell}_cmd: use argv_array
  use st_add and st_mult for allocation size computation
  convert trivial cases to FLEX_ARRAY macros
  use xmallocz to avoid size arithmetic
  convert trivial cases to ALLOC_ARRAY
  convert manual allocations to argv_array
  argv-array: add detach function
  add helpers for allocating flex-array structs
  harden REALLOC_ARRAY and xcalloc against size_t overflow
  tree-diff: catch integer overflow in combine_diff_path allocation
  ...
2016-02-26 13:37:16 -08:00
Jeff King
850d2fec53 convert manual allocations to argv_array
There are many manual argv allocations that predate the
argv_array API. Switching to that API brings a few
advantages:

  1. We no longer have to manually compute the correct final
     array size (so it's one less thing we can screw up).

  2. In many cases we had to make a separate pass to count,
     then allocate, then fill in the array. Now we can do it
     in one pass, making the code shorter and easier to
     follow.

  3. argv_array handles memory ownership for us, making it
     more obvious when things should be free()d and and when
     not.

Most of these cases are pretty straightforward. In some, we
switch from "run_command_v" to "run_command" which lets us
directly use the argv_array embedded in "struct
child_process".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-22 14:50:32 -08:00
Junio C Hamano
017565525f Merge branch 'jc/peace-with-crlf'
Many commands that read files that are expected to contain text
that is generated (or can be edited) by the end user to control
their behaviour (e.g. "git grep -f <filename>") have been updated
to be more tolerant to lines that are terminated with CRLF (they
used to treat such a line to contain payload that ends with CR,
which is usually not what the users expect).

* jc/peace-with-crlf:
  test-sha1-array: read command stream with strbuf_getline()
  grep: read -f file with strbuf_getline()
  send-pack: read list of refs with strbuf_getline()
  column: read lines with strbuf_getline()
  cat-file: read batch stream with strbuf_getline()
  transport-helper: read helper response with strbuf_getline()
  clone/sha1_file: read info/alternates with strbuf_getline()
  remote.c: read $GIT_DIR/remotes/* with strbuf_getline()
  ident.c: read /etc/mailname with strbuf_getline()
  rev-parse: read parseopt spec with strbuf_getline()
  revision: read --stdin with strbuf_getline()
  hash-object: read --stdin-paths with strbuf_getline()
2016-02-03 14:15:58 -08:00
Junio C Hamano
b62624b51a Merge branch 'jc/strbuf-getline'
The preliminary clean-up for jc/peace-with-crlf topic.

* jc/strbuf-getline:
  strbuf: give strbuf_getline() to the "most text friendly" variant
  checkout-index: there are only two possible line terminations
  update-index: there are only two possible line terminations
  check-ignore: there are only two possible line terminations
  check-attr: there are only two possible line terminations
  mktree: there are only two possible line terminations
  strbuf: introduce strbuf_getline_{lf,nul}()
  strbuf: make strbuf_getline_crlf() global
  strbuf: miniscule style fix
2016-01-28 16:10:14 -08:00
Junio C Hamano
52bae62f78 Merge branch 'tg/grep-no-index-fallback'
"git grep" by default does not fall back to its "--no-index"
behaviour outside a directory under Git's control (otherwise the
user may by mistake end up running a huge recursive search); with a
new configuration (set in $HOME/.gitconfig--by definition this
cannot be set in the config file per project), this safety can be
disabled.

* tg/grep-no-index-fallback:
  builtin/grep: add grep.fallbackToNoIndex config
  t7810: correct --no-index test
2016-01-20 11:43:39 -08:00
Junio C Hamano
cc14ea8cf4 Merge branch 'nd/ita-cleanup'
Paths that have been told the index about with "add -N" are not
quite yet in the index, but a few commands behaved as if they
already are in a harmful way.

* nd/ita-cleanup:
  grep: make it clear i-t-a entries are ignored
  add and use a convenience macro ce_intent_to_add()
  blame: remove obsolete comment
2016-01-20 11:43:25 -08:00
Junio C Hamano
a551843129 grep: read -f file with strbuf_getline()
List of patterns file could come from a DOS editor.

This is iffy; you may actually be trying to find a line with ^M in
it on a system whose line ending is LF.  You can of course work it
around by having a line that has "^M^M^J", let the strbuf_getline()
eat the last "^M^J", leaving just the single "^M" as the pattern.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-15 10:35:07 -08:00
Junio C Hamano
8f309aeb82 strbuf: introduce strbuf_getline_{lf,nul}()
The strbuf_getline() interface allows a byte other than LF or NUL as
the line terminator, but this is only because I wrote these
codepaths anticipating that there might be a value other than NUL
and LF that could be useful when I introduced line_termination long
time ago.  No useful caller that uses other value has emerged.

By now, it is clear that the interface is overly broad without a
good reason.  Many codepaths have hardcoded preference to read
either LF terminated or NUL terminated records from their input, and
then call strbuf_getline() with LF or NUL as the third parameter.

This step introduces two thin wrappers around strbuf_getline(),
namely, strbuf_getline_lf() and strbuf_getline_nul(), and
mechanically rewrites these call sites to call either one of
them.  The changes contained in this patch are:

 * introduction of these two functions in strbuf.[ch]

 * mechanical conversion of all callers to strbuf_getline() with
   either '\n' or '\0' as the third parameter to instead call the
   respective thin wrapper.

After this step, output from "git grep 'strbuf_getline('" would
become a lot smaller.  An interim goal of this series is to make
this an empty set, so that we can have strbuf_getline_crlf() take
over the shorter name strbuf_getline().

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-15 10:12:51 -08:00
Junio C Hamano
bdd1cc2092 Merge branch 'vl/grep-configurable-threads'
"git grep" can now be configured (or told from the command line)
how many threads to use when searching in the working tree files.

* vl/grep-configurable-threads:
  grep: add --threads=<num> option and grep.threads configuration
  grep: slight refactoring to the code that disables threading
  grep: allow threading even on a single-core machine
2016-01-12 15:16:55 -08:00
Thomas Gummerer
ecd9ba6177 builtin/grep: add grep.fallbackToNoIndex config
Currently when git grep is used outside of a git repository without the
--no-index option git simply dies.  For convenience, add a
grep.fallbackToNoIndex configuration variable.  If set to true, git grep
behaves like git grep --no-index if it is run outside of a git
repository.  It defaults to false, preserving the current behavior.

Helped-by: Jeff King <peff@peff.net>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-12 10:54:31 -08:00
Nguyễn Thái Ngọc Duy
4d55200532 grep: make it clear i-t-a entries are ignored
The expression "!S_ISREG(ce)" covers i-t-a entries as well because
ce->ce_mode would be zero then. I could make a comment saying that, but
it's probably better just to comment with code, in case i-t-a entry
content changes in future.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-28 12:42:35 -08:00
Victor Leschuk
89f09dd34e grep: add --threads=<num> option and grep.threads configuration
"git grep" can now be configured (or told from the command line) how
many threads to use when searching in the working tree files.

Signed-off-by: Victor Leschuk <vleschuk@accesssoftek.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-16 12:03:23 -08:00
Victor Leschuk
044b1f3cb4 grep: slight refactoring to the code that disables threading
When show-in-pager option is used, threading is unconditionally
disabled, but this happened much earlier than the code that
determines the use of threading based on the operand (i.e. we do not
thread search in the object database).  Consolidate the code to
disable threading to just one place.

Signed-off-by: Victor Leschuk <vleschuk@accesssoftek.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-15 10:49:57 -08:00
Victor Leschuk
b6b468b2bf grep: allow threading even on a single-core machine
Earlier we disabled threading when online_cpus() said "1", but on a
filesystem with long latency (or in a cold cache situation), using
multiple threads to drive I/O in parallel would improve performance
even on a single-core machines.

Signed-off-by: Victor Leschuk <vleschuk@accesssoftek.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-15 10:43:30 -08:00
Junio C Hamano
844a9ce472 Merge branch 'bc/object-id'
More transition from "unsigned char[40]" to "struct object_id".

This needed a few merge fixups, but is mostly disentangled from other
topics.

* bc/object-id:
  remote: convert functions to struct object_id
  Remove get_object_hash.
  Convert struct object to object_id
  Add several uses of get_object_hash.
  object: introduce get_object_hash macro.
  ref_newer: convert to use struct object_id
  push_refs_with_export: convert to struct object_id
  get_remote_heads: convert to struct object_id
  parse_fetch: convert to use struct object_id
  add_sought_entry_mem: convert to struct object_id
  Convert struct ref to use object_id.
  sha1_file: introduce has_object_file helper.
2015-12-10 12:36:13 -08:00
René Scharfe
4441549995 grep: stop using PARSE_OPT_NO_INTERNAL_HELP
The flag PARSE_OPT_NO_INTERNAL_HELP is set to allow overriding the
option -h, except when it's the only one given.  This is the default
behavior now, so remove the flag and the hand-rolled --help-all
handling.  The internal --help-all handler now actually shows hidden
options, i.e. --debug in this case.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 08:02:07 -05:00
brian m. carlson
ed1c9977cb Remove get_object_hash.
Convert all instances of get_object_hash to use an appropriate reference
to the hash member of the oid member of struct object.  This provides no
functional change, as it is essentially a macro substitution.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 08:02:05 -05:00
brian m. carlson
f2fd0760f6 Convert struct object to object_id
struct object is one of the major data structures dealing with object
IDs.  Convert it to use struct object_id instead of an unsigned char
array.  Convert get_object_hash to refer to the new member as well.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 08:02:05 -05:00
brian m. carlson
7999b2cf77 Add several uses of get_object_hash.
Convert most instances where the sha1 member of struct object is
dereferenced to use get_object_hash.  Most instances that are passed to
functions that have versions taking struct object_id, such as
get_sha1_hex/get_oid_hex, or instances that can be trivially converted
to use struct object_id instead, are not converted.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 08:02:05 -05:00