The convention for a command line is to follow "git cmdname
--options" with revisions followed by an optional "--"
disambiguator and then finally pathspecs. When "--" is not there,
we make sure early ones are all interpretable as revs (and do not
look like paths) and later ones are the other way around. A
pathspec with "magic" (e.g. ":/p/a/t/h" that matches p/a/t/h from
the top-level of the working tree, no matter what subdirectory you
are working from) are conservatively judged as "not a path", which
required disambiguation more often. The command line parser
learned to say "it's a pathspec" a bit more often when the syntax
looks like so.
* jk/pathspec-magic-disambiguation:
verify_filename(): flip order of checks
verify_filename(): treat ":(magic)" as a pathspec
check_filename(): handle ":^" path magic
check_filename(): use skip_prefix
check_filename(): refactor ":/" handling
t4208: add check for ":/" without matching file
Check the console output when using --autostash and the stash does not
apply is what we expect. The test is quite strict but should catch any
changes to the console output from the various rebase flavors.
Thanks-to: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Check the console output when using --autostash and the stash applies
cleanly is what we expect. The test is quite strict but should catch
any changes to the console output from the various rebase flavors.
Thanks-to: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Check that the reflog message written to the branch reflog when the
rebase is completed is correct
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce '--show-stash' and its configuration option 'status.showStash'
to allow git-status to show information about currently stashed entries.
Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`for_each_bisect_ref()` is called by `for_each_bad_bisect_ref()` with
a term "bad". This used to make it call `for_each_ref_in_submodule()`
with a prefix "refs/bisect/bad". But the latter is the name of the
reference that is being sought, so the empty string was being passed
to the callback as the trimmed refname. Moreover, this questionable
practice was turned into an error by
b9c8e7f2fb prefix_ref_iterator: don't trim too much, 2017-05-22
It makes more sense (and agrees better with the documentation of
`--bisect`) for the callers to receive the full reference names. So
* Add a new function, `for_each_fullref_in_submodule()`, to the refs
API. This plugs a gap in the existing functionality, analogous to
`for_each_fullref_in()` but accepting a `submodule` argument.
* Change `for_each_bad_bisect_ref()` to call the new function rather
than `for_each_ref_in_submodule()`.
* Add a test.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A negated character class that does not include '/', e.g. [^a-z]:
- Should match '/' when doing "wildmatch"
- Should not match '/' when doing "pathmatch"
Add two tests to cover these cases.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we convert seconds-since-epochs timestamps into a
broken-down "struct tm", we do so by adjusting the timestamp
according to the known offset and then using gmtime() to
break down the result. This means that the resulting struct
"knows" that it's in GMT, even though the time it represents
is adjusted for a different zone. The fields where it stores
this data are not portably accessible, so we have no way to
override them to tell them the real zone info.
For the most part, this works. Our date-formatting routines
don't pay attention to these inaccessible fields, and use
the same tz info we provided for adjustment. The one
exception is when we call strftime(), whose %Z format
reveals this hidden timezone data.
We solved that by always showing the empty string for %Z.
This is allowed by POSIX, but not very helpful to the user.
We can't make this work in the general case, as there's no
portable function for setting an arbitrary timezone (and
anyway, we don't have the zone name for the author zones,
only their offsets).
But for the special case of the "-local" formats, we can
just skip the adjustment and use localtime() instead of
gmtime(). This makes --date=format-local:%Z work correctly,
showing the local timezone instead of an empty string.
The new test checks the result for "UTC", our default
test-lib value for $TZ. Using something like EST5 might be
more interesting, but the actual zone string is
system-dependent (for instance, on my system it expands to
just EST). Hopefully "UTC" is vanilla enough that every
system treats it the same.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We already test that "%z" and "%Z" show the right thing, but
we don't actually check that the time we display is the
correct one. Let's add two new tests:
1. Test that "format:" shows the time in the author's
timezone, just like the other time formats.
2. Test that "format-local:" shows time in the local
timezone. We don't want to use our normal UTC for this,
because its offset is zero (so the result would be
"correct" even if the code forgot to apply the offset
or applied it in the wrong direction).
We'll use the EST5 zone, which is already used
elsewhere in the script (and so is assumed to be
available everywhere).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There is no portable way to pass timezone information to strftime. Add
parameters for timezone offset and name to strbuf_addftime and let it
handle the timezone-related format specifiers %z and %Z internally.
Callers can opt out for %Z by passing NULL as timezone name. %z is
always handled internally -- this helps on Windows, where strftime would
expand it to a timezone name (same as %Z), in violation of POSIX.
Modifiers are not handled, e.g. %Ez is still passed to strftime.
Use an empty string as timezone name in show_date (the only current
caller) for now because we only have the timezone offset in non-local
mode. POSIX allows %Z to resolve to an empty string in case of missing
information.
Helped-by: Ulrich Mueller <ulm@gentoo.org>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Stop including config.h by default in cache.h. Instead only include
config.h in those files which require use of the config system.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of discovering the .git/ directory, reading the config and then
trying to painstakingly reset all the global state if we did not find a
matching alias, let's use the early config machinery instead.
It may look like unnecessary work to discover the .git/ directory in the
early config machinery and then call setup_git_directory_gently() in the
case of a shell alias, repeating the very same discovery *again*.
However, we have to do this as the early config machinery takes pains
*not* to touch any global state, while shell aliases expect a possibly
changed working directory and at least the GIT_PREFIX and GIT_DIR
variables to be set.
This change also fixes a known issue where Git tried to read the pager
config from an incorrect path in a subdirectory of a Git worktree if an
alias expanded to a shell command.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When expanding aliases, the git_dir is set during the alias expansion
(by virtue of running setup_git_directory_gently()).
This git_dir may be relative to the current working directory, and
indeed often is simply ".git/".
When the alias expands to a shell command, we restore the original
working directory, though, yet we do not reset git_dir.
As a consequence, subsequent read_early_config() runs will mistake the
git_dir to be populated properly and not find the correct config.
Demonstrate this problem by adding a test case.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We are about to change the way aliases are expanded, to use the early
config machinery.
This machinery reports errors in a slightly different manner than the
cached config machinery.
Let's not get hung up by the precise wording of the message mentioning
the line number. It is really sufficient to verify that all the relevant
information is given to the user.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When get_value() parses a key/value pair, it is possible that the line
number is decreased (because the \n has been consumed already) before the
key/value pair is passed to the callback function, to allow for the
correct line to be attributed in case of an error.
However, when git_parse_source() asks get_value() to parse the key/value
pair, the error reporting is performed *after* get_value() returns.
Which means that we have to be careful not to increase the line number
in get_value() after the callback function returned an error.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some submodule tests do some setup outside of a test_expect
block. This is bad because we won't actually check the
outcome of those commands. But it's doubly so because "git
add submodule" now produces a warning to stderr, which is
not suppressed by the test scripts in non-verbose mode.
This patch does the minimal to fix the annoying warnings.
All three of these scripts could use more cleanup of related
setup.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's an easy mistake to add a repository inside another
repository, like:
git clone $url
git add .
The resulting entry is a gitlink, but there's no matching
.gitmodules entry. Trying to use "submodule init" (or clone
with --recursive) doesn't do anything useful. Prior to
v2.13, such an entry caused git-submodule to barf entirely.
In v2.13, the entry is considered "inactive" and quietly
ignored. Either way, no clone of your repository can do
anything useful with the gitlink without the user manually
adding the submodule config.
In most cases, the user probably meant to either add a real
submodule, or they forgot to put the embedded repository in
their .gitignore file.
Let's issue a warning when we see this case. There are a few
things to note:
- the warning will go in the git-add porcelain; anybody
wanting to do low-level manipulation of the index is
welcome to create whatever funny states they want.
- we detect the case by looking for a newly added gitlink;
updates via "git add submodule" are perfectly reasonable,
and this avoids us having to investigate .gitmodules
entirely
- there's a command-line option to suppress the warning.
This is needed for git-submodule itself (which adds the
entry before adding any submodule config), but also
provides a mechanism for other scripts doing
submodule-like things.
We could make this a hard error instead of a warning.
However, we do add lots of sub-repos in our test suite. It's
not _wrong_ to do so. It just creates a state where users
may be surprised. Pointing them in the right direction with
a gentle hint is probably the best option.
There is a config knob that can disable the (long) hint. But
I intentionally omitted a config knob to disable the warning
entirely. Whether the warning is sensible or not is
generally about context, not about the user's preferences.
If there's a tool or workflow that adds gitlinks without
matching .gitmodules, it should probably be taught about the
new command-line option, rather than blanket-disabling the
warning.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We often try to open a file for reading whose existence is
optional, and silently ignore errors from open/fopen; report such
errors if they are not due to missing files.
* nd/fopen-errors:
mingw_fopen: report ENOENT for invalid file names
mingw: verify that paths are not mistaken for remote nicknames
log: fix memory leak in open_next_file()
rerere.c: move error_errno() closer to the source system call
print errno when reporting a system call error
wrapper.c: make warn_on_inaccessible() static
wrapper.c: add and use fopen_or_warn()
wrapper.c: add and use warn_on_fopen_errors()
config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too
config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD
clone: use xfopen() instead of fopen()
use xfopen() in more places
git_fopen: fix a sparse 'not declared' warning
Many commands learned to pay attention to submodule.recurse
configuration.
* sb/submodule-blanket-recursive:
builtin/fetch.c: respect 'submodule.recurse' option
builtin/push.c: respect 'submodule.recurse' option
builtin/grep.c: respect 'submodule.recurse' option
Introduce 'submodule.recurse' option for worktree manipulators
submodule loading: separate code path for .gitmodules and config overlay
reset/checkout/read-tree: unify config callback for submodule recursion
submodule test invocation: only pass additional arguments
submodule recursing: do not write a config variable twice
perf-test update.
* jh/memihash-opt:
p0004: don't error out if test repo is too small
p0004: don't abort if multi-threaded is too slow
p0004: use test_perf
p0004: avoid using pipes
p0004: simplify calls of test-lazy-init-name-hash
"git pull --rebase --autostash" didn't auto-stash when the local history
fast-forwards to the upstream.
* tb/pull-ff-rebase-autostash:
pull: ff --rebase --autostash works in dirty repo
"git clean -d" used to clean directories that has ignored files,
even though the command should not lose ignored ones without "-x".
"git status --ignored" did not list ignored and untracked files
without "-uall". These have been corrected.
* sl/clean-d-ignored-fix:
clean: teach clean -d to preserve ignored paths
dir: expose cmp_name() and check_contains()
dir: hide untracked contents of untracked dirs
dir: recurse into untracked dirs for ignored files
t7061: status --ignored should search untracked dirs
t7300: clean -d should skip dirs with ignored files
A recent update to t5545-push-options.sh started skipping all the
tests in the script when a web server testing is disabled or
unavailable, not just the ones that require a web server. Non HTTP
tests have been salvaged to always run in this script.
* jc/skip-test-in-the-middle:
t5545: enhance test coverage when no http server is installed
test: allow skipping the remainder
The "run-command" API implementation has been made more robust
against dead-locking in a threaded environment.
* bw/forking-and-threading:
usage.c: drop set_error_handle()
run-command: restrict PATH search to executable files
run-command: expose is_executable function
run-command: block signals between fork and execve
run-command: add note about forking and threading
run-command: handle dup2 and close errors in child
run-command: eliminate calls to error handling functions in child
run-command: don't die in child when duping /dev/null
run-command: prepare child environment before forking
string-list: add string_list_remove function
run-command: use the async-signal-safe execv instead of execvp
run-command: prepare command before forking
t0061: run_command executes scripts without a #! line
t5550: use write_script to generate post-update hook
Introduce the BUG() macro to improve die("BUG: ...").
* jk/bug-to-abort:
usage: add NORETURN to BUG() function definitions
config: complain about --local outside of a git repo
setup_git_env: convert die("BUG") to BUG()
usage.c: add BUG() function
"git checkout --recurse-submodules" did not quite work with a
submodule that itself has submodules.
* sb/checkout-recurse-submodules:
submodule: properly recurse for read-tree and checkout
submodule: avoid auto-discovery in new working tree manipulator code
submodule_move_head: reuse child_process structure for futher commands
The `git stash push` command recently gained the ability to get a
pathspec as its argument to only stash matching files. Calling this
command from a subdirectory does not work, though, as one of the first
things we do is changing to the top level directory without keeping
track of the prefix from which the command is being run.
Fix the shortcoming by storing the prefix previous to the call to
`cd_to_toplevel` and then subsequently using `git rev-parse --prefix` to
correctly resolve the pathspec. Add a test to catch future breakage of
this usecase.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a test for the case when only one parameter is passed to '-m'
(move/rename) option.
For example - if 'git branch -m bbb' is run while checked out on aaa
branch, it should rename the currently checked out branch to bbb.
There was no test for this particular case with only one parameter
for -m option. However, there's one similar test case for -M option.
Add test for making sure HEAD points to the bbb (new branch name). Also
add a test for making sure the reflog that is moved to 'bbb' retains
entries created for the currently checked out branch. Note that since
the topmost entry on reflog for bbb will be about branch creation, we
compare bbb@{1} (instead of bbb@{0}) with aaa@{0} to make sure the
reflog for bbb retains entries from aaa.
Signed-off-by: Sahil Dua <sahildua2305@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add some more simple mailinfo tests including a few that
produce:
fatal: `pos + len' is too far after the end of the buffer
Mark those as 'test_expect_failure'.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use modern style in the test t4005. Remove hard coded sha1 values.
Combine test prep work and the actual test. Rename the first
test to contain the word "setup".
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit a1283866b (t5313: test bounds-checks of
corrupted/malicious pack/idx files, 2016-02-25) added a test
that requires our corrupted pack index to have two objects.
The entry for the first one remains untouched, but we
corrupt the entry for second one. Since the index stores the
entries in sha1-sorted order, this means that the test must
make sure that the sha1 of the object we expect to be
corrupted ("$object") sorts after the other placeholder
object.
That commit used the HEAD commit as the placeholder, but the
script never calls test_tick. That means that the commit
object (and thus its sha1) depends on the timestamp when the
test script is run. This usually works in practice, because
the sha1 of $object starts with "fff". The commit object
will sort after that only 1 in 4096 times, but when it does
the test will fail.
One obvious solution is to add the test_tick call to get a
deterministic commit sha1. But since we're relying on the
sort order for the test to function, let's make that very
explicit by just generating a second blob with a known sha1.
Reported-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since commit 99caeed05 (Let 'git <command> -h' show usage
without a git dir, 2009-11-09), the git wrapper handles "-h"
specially, skipping any repository setup but still calling
the builtin's cmd_foo() function. This means that every
cmd_foo() must be ready to handle this case, but we don't
have any systematic tests. This led to "git am -h" being
broken for some time without anybody noticing.
This patch just tests that "git foo -h" works for every
builtin, where we see a 129 exit code (the normal code for
our usage() helper), and that the word "usage" appears in
the output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the tested repo has an index.lock file it should be removed. This
file may be present if e.g. git-status previously crashed in that
repo, and it will make a lot of git commands fail. Let's try harder
and remove the lock.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git pull --rebase --autostash" didn't auto-stash when the local history
fast-forwards to the upstream.
* tb/pull-ff-rebase-autostash:
pull: ff --rebase --autostash works in dirty repo
The implementation of "ref" API around the "packed refs" have been
cleaned up, in preparation for further changes.
* mh/packed-ref-store-prep: (25 commits)
cache_ref_iterator_begin(): avoid priming unneeded directories
ref-filter: limit traversal to prefix
create_ref_entry(): remove `check_name` option
refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`
read_packed_refs(): report unexpected fopen() failures
read_packed_refs(): do more of the work of reading packed refs
get_packed_ref_cache(): assume "packed-refs" won't change while locked
should_pack_ref(): new function, extracted from `files_pack_refs()`
ref_update_reject_duplicates(): add a sanity check
ref_update_reject_duplicates(): use `size_t` rather than `int`
ref_update_reject_duplicates(): expose function to whole refs module
ref_transaction_prepare(): new optional step for reference updates
ref_transaction_commit(): check for valid `transaction->state`
files_transaction_cleanup(): new helper function
files_ref_store: put the packed files lock directly in this struct
files-backend: move `lock` member to `files_ref_store`
lockfile: add a new method, is_lock_file_locked()
ref_store: take a `msg` parameter when deleting references
refs: use `size_t` indexes when iterating over ref transaction updates
refs_ref_iterator_begin(): don't check prefixes redundantly
...
Make the "indent" heuristics the default in "diff" and diff.indentHeuristics
configuration variable an escape hatch for those who do no want it.
* mb/diff-default-to-indent-heuristics:
add--interactive: drop diff.indentHeuristic handling
diff: enable indent heuristic by default
diff: have the diff-* builtins configure diff before initializing revisions
diff: make the indent heuristic part of diff's basic configuration
A test allowed both "git push" and "git receive-pack" on the other
end write their traces into the same file. This is OK on platforms
that allows atomically appending to a file opened with O_APPEND,
but on other platforms led to a mangled output, causing
intermittent test failures. This has been fixed by disabling
traces from "receive-pack" in the test.
* jk/alternate-ref-optim:
t5400: avoid concurrent writes into a trace file
"git interpret-trailers", when used as GIT_EDITOR for "git commit
-v", looked for and appended to a trailer block at the very end,
i.e. at the end of the "diff" output. The command has been
corrected to pay attention to the cut-mark line "commit -v" adds to
the buffer---the real trailer block should appear just before it.
* bm/interpret-trailers-cut-line-is-eom:
interpret-trailers: honor the cut line
"git clone --config var=val" is a way to populate the
per-repository configuration file of the new repository, but it did
not work well when val is an empty string. This has been fixed.
* jn/clone-add-empty-config-from-command-line:
clone: handle empty config values in -c
"git cherry-pick" and other uses of the sequencer machinery
mishandled a trailer block whose last line is an incomplete line.
This has been fixed so that an additional sign-off etc. are added
after completing the existing incomplete line.
* jt/use-trailer-api-in-commands:
sequencer: add newline before adding footers
The receive-pack program now makes sure that the push certificate
records the same set of push options used for pushing.
* jt/push-options-doc:
receive-pack: verify push options in cert
docs: correct receive.advertisePushOptions default
Make sure our tests would pass when the sources are checked out
with "platform native" line ending convention by default on
Windows. Some "text" files out tests use and the test scripts
themselves that are meant to be run with /bin/sh, ought to be
checked out with eol=LF even on Windows.
* js/eol-on-ourselves:
t4051: mark supporting files as requiring LF-only line endings
Fix the remaining tests that failed with core.autocrlf=true
t3901: move supporting files into t/t3901/
completion: mark bash script as LF-only
git-new-workdir: mark script as LF-only
Fix build with core.autocrlf=true
Tag objects, which are not reachable from any ref, that point at
missing objects were mishandled by "git gc" and friends (they
should silently be ignored instead)
* jk/ignore-broken-tags-when-ignoring-missing-links:
revision.c: ignore broken tags with ignore_missing_links
"pack-objects" can stream a slice of an existing packfile out when
the pack bitmap can tell that the reachable objects are all needed
in the output, without inspecting individual objects. This
strategy however would not work well when "--local" and other
options are in use, and need to be disabled.
* jk/disable-pack-reuse-when-broken:
t5310: fix "; do" style
pack-objects: disable pack reuse for object-selection options
Setting "log.decorate=false" in the configuration file did not take
effect in v2.13, which has been corrected.
* ah/log-decorate-default-to-auto:
builtin/log: honor log.decorate
Update tests to pass under GETTEXT_POISON (a mechanism to ensure
that output strings that should not be translated are not
translated by mistake), and tell TravisCI to run them.
* ab/fix-poison-tests:
travis-ci: add job to run tests with GETTEXT_POISON
travis-ci: setup "prove cache" in "script" step
tests: fix tests broken under GETTEXT_POISON=YesPlease
The Web interface to gmane news archive is long gone, even though
the articles are still accessible via NTTP. Replace the links with
ones to public-inbox.org. Because their message identification is
based on the actual message-id, it is likely that it will be easier
to migrate away from it if/when necessary.
* ab/doc-replace-gmane-links:
doc: replace more gmane links
doc: replace a couple of broken gmane links
The internal implementation of "git grep" has seen some clean-up.
* ab/grep-preparatory-cleanup: (31 commits)
grep: assert that threading is enabled when calling grep_{lock,unlock}
grep: given --threads with NO_PTHREADS=YesPlease, warn
pack-objects: fix buggy warning about threads
pack-objects & index-pack: add test for --threads warning
test-lib: add a PTHREADS prerequisite
grep: move is_fixed() earlier to avoid forward declaration
grep: change internal *pcre* variable & function names to be *pcre1*
grep: change the internal PCRE macro names to be PCRE1
grep: factor test for \0 in grep patterns into a function
grep: remove redundant regflags assignments
grep: catch a missing enum in switch statement
perf: add a comparison test of log --grep regex engines with -F
perf: add a comparison test of log --grep regex engines
perf: add a comparison test of grep regex engines with -F
perf: add a comparison test of grep regex engines
perf: emit progress output when unpacking & building
perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do
grep: add tests to fix blind spots with \0 patterns
grep: prepare for testing binary regexes containing rx metacharacters
grep: add a test helper function for less verbose -f \0 tests
...
The result from "git diff" that compares two blobs, e.g. "git diff
$commit1:$path $commit2:$path", used to be shown with the full
object name as given on the command line, but it is more natural to
use the $path in the output and use it to look up .gitattributes.
* jk/diff-blob:
diff: use blob path for blob/file diffs
diff: use pending "path" if it is available
diff: use the word "path" instead of "name" for blobs
diff: pass whole pending entry in blobinfo
handle_revision_arg: record paths for pending objects
handle_revision_arg: record modes for "a..b" endpoints
t4063: add tests of direct blob diffs
get_sha1_with_context: dynamically allocate oc->path
get_sha1_with_context: always initialize oc->symlink_path
sha1_name: consistently refer to object_context as "oc"
handle_revision_arg: add handle_dotdot() helper
handle_revision_arg: hoist ".." check out of range parsing
handle_revision_arg: stop using "dotdot" as a generic pointer
handle_revision_arg: simplify commit reference lookups
handle_revision_arg: reset "dotdot" consistently
"git clean -d" used to clean directories that has ignored files,
even though the command should not lose ignored ones without "-x".
"git status --ignored" did not list ignored and untracked files
without "-uall". These have been corrected.
* sl/clean-d-ignored-fix:
clean: teach clean -d to preserve ignored paths
dir: expose cmp_name() and check_contains()
dir: hide untracked contents of untracked dirs
dir: recurse into untracked dirs for ignored files
t7061: status --ignored should search untracked dirs
t7300: clean -d should skip dirs with ignored files
Unless --no-validate is passed, send-email will invoke
$repo->repo_path() in its search for a validate hook regardless of
whether a Git repo is actually present. Teach send-email to first check
for repo existence.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Windows, certain characters are prohibited in file names, most
prominently the colon. When fopen() is called with such an invalid file
name, the underlying Windows API actually reports a particular error,
but since there is no suitable errno value, this error is translated
to EINVAL. Detect the case and report ENOENT instead.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This added test case simply verifies that users will not be bothered
with bogus complaints à la
warning: unable to access '.git/remotes/D:\repo': Invalid argument
when fetching from a Windows path (in this case, D:\repo).
[j6t: mark the new test as test_expect_failure]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add support for v2 of the PCRE API. This is a new major version of
PCRE that came out in early 2015[1].
The regular expression syntax is the same, but while the API is
similar, pretty much every function is either renamed or takes
different arguments. Thus using it via entirely new functions makes
sense, as opposed to trying to e.g. have one compile_pcre_pattern()
that would call either PCRE v1 or v2 functions.
Git can now be compiled with either USE_LIBPCRE1=YesPlease or
USE_LIBPCRE2=YesPlease, with USE_LIBPCRE=YesPlease currently being a
synonym for the former. Providing both is a compile-time error.
With earlier patches to enable JIT for PCRE v1 the performance of the
release versions of both libraries is almost exactly the same, with
PCRE v2 being around 1% slower.
However after I reported this to the pcre-dev mailing list[2] I got a
lot of help with the API use from Zoltán Herczeg, he subsequently
optimized some of the JIT functionality in v2 of the library.
Running the p7820-grep-engines.sh performance test against the latest
Subversion trunk of both, with both them and git compiled as -O3, and
the test run against linux.git, gives the following results. Just the
/perl/ tests shown:
$ GIT_PERF_REPEAT_COUNT=30 GIT_PERF_LARGE_REPO=~/g/linux GIT_PERF_MAKE_COMMAND='grep -q LIBPCRE2 Makefile && make -j8 USE_LIBPCRE2=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre2/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre2/inst/lib || make -j8 USE_LIBPCRE=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre/inst/lib' ./run HEAD~5 HEAD~ HEAD p7820-grep-engines.sh
[...]
Test HEAD~5 HEAD~ HEAD
-----------------------------------------------------------------------------------------------------------------
7820.3: perl grep 'how.to' 0.31(1.10+0.48) 0.21(0.35+0.56) -32.3% 0.21(0.34+0.55) -32.3%
7820.7: perl grep '^how to' 0.56(2.70+0.40) 0.24(0.64+0.52) -57.1% 0.20(0.28+0.60) -64.3%
7820.11: perl grep '[how] to' 0.56(2.66+0.38) 0.29(0.95+0.45) -48.2% 0.23(0.45+0.54) -58.9%
7820.15: perl grep '(e.t[^ ]*|v.ry) rare' 1.02(5.77+0.42) 0.31(1.02+0.54) -69.6% 0.23(0.50+0.54) -77.5%
7820.19: perl grep 'm(ú|u)lt.b(æ|y)te' 0.38(1.57+0.42) 0.27(0.85+0.46) -28.9% 0.21(0.33+0.57) -44.7%
See commit ("perf: add a comparison test of grep regex engines",
2017-04-19) for details on the machine the above test run was executed
on.
Here HEAD~2 is git with PCRE v1 without JIT, HEAD~ is PCRE v1 with
JIT, and HEAD is PCRE v2 (also with JIT). See previous commits of mine
mentioning p7820-grep-engines.sh for more details on the test setup.
For ease of readability, a different run just of HEAD~ (PCRE v1 with
JIT v.s. PCRE v2), again with just the /perl/ tests shown:
[...]
Test HEAD~ HEAD
----------------------------------------------------------------------------------------
7820.3: perl grep 'how.to' 0.21(0.42+0.52) 0.21(0.31+0.58) +0.0%
7820.7: perl grep '^how to' 0.25(0.65+0.50) 0.20(0.31+0.57) -20.0%
7820.11: perl grep '[how] to' 0.30(0.90+0.50) 0.23(0.46+0.53) -23.3%
7820.15: perl grep '(e.t[^ ]*|v.ry) rare' 0.30(1.19+0.38) 0.23(0.51+0.51) -23.3%
7820.19: perl grep 'm(ú|u)lt.b(æ|y)te' 0.27(0.84+0.48) 0.21(0.34+0.57) -22.2%
I.e. the two are either neck-to-neck, but PCRE v2 usually pulls ahead,
when it does it's around 20% faster.
A brief note on thread safety: As noted in pcre2api(3) & pcre2jit(3)
the compiled pattern can be shared between threads, but not some of
the JIT context, however the grep threading support does all pattern &
JIT compilation in separate threads, so this code doesn't need to
concern itself with thread safety.
See commit 63e7e9d8b6 ("git-grep: Learn PCRE", 2011-05-09) for the
initial addition of PCRE v1. This change follows some of the same
patterns it did (and which were discussed on list at the time),
e.g. mocking up types with typedef instead of ifdef-ing them out when
USE_LIBPCRE2 isn't defined. This adds some trivial memory use to the
program, but makes the code look nicer.
1. https://lists.exim.org/lurker/message/20150105.162835.0666407a.en.html
2. https://lists.exim.org/lurker/thread/20170419.172322.833ee099.en.html
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When `git pull --rebase --autostash` in a dirty repository resulted in a
fast-forward, nothing was being autostashed and the pull failed. This
was due to a shortcut to avoid running rebase when we can fast-forward,
but autostash is ignored on that codepath.
Now we will only take the shortcut if autostash is not in effect.
Based on a few tests against the git.git repo, the shortcut does not
seem to give us significant performance benefits, on Linux at least.
Regardless, it is more important to be correct than to be fast.
Signed-off-by: Tyler Brazier <tyler@tylerbrazier.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The closest mapping from the boolean 'submodule.recurse' set to "yes"
to the variety of submodule push modes is "on-demand", so implement that.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In builtin/grep.c we parse the config before evaluating the command line
options. This makes the task of teaching grep to respect the new config
option 'submodule.recurse' very easy by just parsing that option.
As an alternative I had implemented a similar structure to treat
submodules as the fetch/push command have, including
* aligning the meaning of the 'recurse_submodules' to possible submodule
values RECURSE_SUBMODULES_* as defined in submodule.h.
* having a callback to parse the value and
* reacting to the RECURSE_SUBMODULES_DEFAULT state that was the initial
state.
However all this is not needed for a true boolean value, so let's keep
it simple. However this adds another place where "submodule.recurse" is
parsed.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Any command that understands '--recurse-submodules' can have its
default changed to true, by setting the new 'submodule.recurse'
option.
This patch includes read-tree/checkout/reset for working tree
manipulating commands. Later patches will cover other commands.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a later patch we want to introduce a config option to trigger the
submodule recursing by default. As this option should be available and
uniform across all commands that deal with submodules we'd want to test
for this option in the submodule update library.
So instead of calling the whole test set again for
"git -c submodule.recurse foo" instead of "git foo --recurse-submodules",
we'd only want to introduce one basic test that tests if the option is
recognized and respected to not overload the test suite.
Change the test functions by taking only the argument and assemble the
command inside the test function by embedding the arguments into the
command that is "git $arguments --recurse-submodules".
It would be nice to do this for all functions in lib-submodule-update,
but we cannot do that for the non-recursing tests, as there we do not
just pass in a git command but whole functions. (See t3426 for example)
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ab/grep-preparatory-cleanup: (31 commits)
grep: assert that threading is enabled when calling grep_{lock,unlock}
grep: given --threads with NO_PTHREADS=YesPlease, warn
pack-objects: fix buggy warning about threads
pack-objects & index-pack: add test for --threads warning
test-lib: add a PTHREADS prerequisite
grep: move is_fixed() earlier to avoid forward declaration
grep: change internal *pcre* variable & function names to be *pcre1*
grep: change the internal PCRE macro names to be PCRE1
grep: factor test for \0 in grep patterns into a function
grep: remove redundant regflags assignments
grep: catch a missing enum in switch statement
perf: add a comparison test of log --grep regex engines with -F
perf: add a comparison test of log --grep regex engines
perf: add a comparison test of grep regex engines with -F
perf: add a comparison test of grep regex engines
perf: emit progress output when unpacking & building
perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do
grep: add tests to fix blind spots with \0 patterns
grep: prepare for testing binary regexes containing rx metacharacters
grep: add a test helper function for less verbose -f \0 tests
...
When "git checkout", "git merge", etc. manipulates the in-core
index, various pieces of information in the index extensions are
discarded from the original state, as it is usually not the case
that they are kept up-to-date and in-sync with the operation on the
main index. The untracked cache extension is copied across these
operations now, which would speed up "git status" (as long as the
cache is properly invalidated).
* dt/unpack-save-untracked-cache-extension:
unpack-trees: preserve index extensions
A recent update to t5545-push-options.sh started skipping all the
tests in the script when a web server testing is disabled or
unavailable, not just the ones that require a web server. Non HTTP
tests have been salvaged to always run in this script.
* jc/skip-test-in-the-middle:
t5545: enhance test coverage when no http server is installed
test: allow skipping the remainder
The recently introduced "[includeIf "gitdir:$dir"] path=..."
mechansim has further been taught to take symlinks into account.
The directory "$dir" specified in "gitdir:$dir" may be a symlink to
a real location, not something that $(getcwd) may return. In such
a case, a realpath of "$dir" is compared with the real path of the
current repository to determine if the contents from the named path
should be included.
* ab/conditional-config-with-symlinks:
config: match both symlink & realpath versions in IncludeIf.gitdir:*
There is no good reason why "git fetch $there $sha1" should fail
when the $sha1 names an object at the tip of an advertised ref,
even when the other side hasn't enabled allowTipSHA1InWant.
* jt/fetch-allow-tip-sha1-implicitly:
fetch-pack: always allow fetching of literal SHA1s
"git send-email" learned to run sendemail-validate hook to inspect
and reject a message before sending it out.
* jt/send-email-validate-hook:
send-email: support validate hook
perf-test update.
* jh/memihash-opt:
p0004: don't error out if test repo is too small
p0004: don't abort if multi-threaded is too slow
p0004: use test_perf
p0004: avoid using pipes
p0004: simplify calls of test-lazy-init-name-hash
The "run-command" API implementation has been made more robust
against dead-locking in a threaded environment.
* bw/forking-and-threading:
usage.c: drop set_error_handle()
run-command: restrict PATH search to executable files
run-command: expose is_executable function
run-command: block signals between fork and execve
run-command: add note about forking and threading
run-command: handle dup2 and close errors in child
run-command: eliminate calls to error handling functions in child
run-command: don't die in child when duping /dev/null
run-command: prepare child environment before forking
string-list: add string_list_remove function
run-command: use the async-signal-safe execv instead of execvp
run-command: prepare command before forking
t0061: run_command executes scripts without a #! line
t5550: use write_script to generate post-update hook
Add perf-test for wildmatch.
* ab/perf-wildmatch:
perf: add test showing exponential growth in path globbing
perf: add function to setup a fresh test repo
Simplify parse_pathspec() codepath and stop it from looking at the
default in-core index.
* bw/pathspec-sans-the-index:
pathspec: convert find_pathspecs_matching_against_index to take an index
pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP
ls-files: prevent prune_cache from overeagerly pruning submodules
pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag
submodule: add die_in_unpopulated_submodule function
pathspec: provide a more descriptive die message
"git describe --contains" penalized light-weight tags so much that
they were almost never considered. Instead, give them about the
same chance to be considered as an annotated tag that is the same
age as the underlying commit would.
* jc/name-rev-lw-tag:
name-rev: favor describing with tags and use committer date to tiebreak
name-rev: refactor logic to see if a new candidate is a better name
Fixes the test by changing "branch.s/s/dummy" to "branch.s/s.dummy" which is
the right way of accessing config key "branch.s/s.dummy". Purpose of
this test is to confirm that this key doesn't exist after the branch
"s/s" has been renamed to "s".
Earlier it was trying to access invalid config key and hence was getting
an error. However, this wasn't caught because we were expecting the
command to fail for other reason as mentioned above.
Signed-off-by: Sahil Dua <sahildua2305@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Tag objects, which are not reachable from any ref, that point at
missing objects were mishandled by "git gc" and friends (they
should silently be ignored instead)
* jk/ignore-broken-tags-when-ignoring-missing-links:
revision.c: ignore broken tags with ignore_missing_links
A test allowed both "git push" and "git receive-pack" on the other
end write their traces into the same file. This is OK on platforms
that allows atomically appending to a file opened with O_APPEND,
but on other platforms led to a mangled output, causing
intermittent test failures. This has been fixed by disabling
traces from "receive-pack" in the test.
* jk/alternate-ref-optim:
t5400: avoid concurrent writes into a trace file
"git interpret-trailers", when used as GIT_EDITOR for "git commit
-v", looked for and appended to a trailer block at the very end,
i.e. at the end of the "diff" output. The command has been
corrected to pay attention to the cut-mark line "commit -v" adds to
the buffer---the real trailer block should appear just before it.
* bm/interpret-trailers-cut-line-is-eom:
interpret-trailers: honor the cut line
Introduce the BUG() macro to improve die("BUG: ...").
* jk/bug-to-abort:
usage: add NORETURN to BUG() function definitions
config: complain about --local outside of a git repo
setup_git_env: convert die("BUG") to BUG()
usage.c: add BUG() function
Make sure our tests would pass when the sources are checked out
with "platform native" line ending convention by default on
Windows. Some "text" files out tests use and the test scripts
themselves that are meant to be run with /bin/sh, ought to be
checked out with eol=LF even on Windows.
* js/eol-on-ourselves:
t4051: mark supporting files as requiring LF-only line endings
Fix the remaining tests that failed with core.autocrlf=true
t3901: move supporting files into t/t3901/
completion: mark bash script as LF-only
git-new-workdir: mark script as LF-only
Fix build with core.autocrlf=true
"pack-objects" can stream a slice of an existing packfile out when
the pack bitmap can tell that the reachable objects are all needed
in the output, without inspecting individual objects. This
strategy however would not work well when "--local" and other
options are in use, and need to be disabled.
* jk/disable-pack-reuse-when-broken:
t5310: fix "; do" style
pack-objects: disable pack reuse for object-selection options
Conversion from uchar[20] to struct object_id continues.
* bc/object-id: (53 commits)
object: convert parse_object* to take struct object_id
tree: convert parse_tree_indirect to struct object_id
sequencer: convert do_recursive_merge to struct object_id
diff-lib: convert do_diff_cache to struct object_id
builtin/ls-tree: convert to struct object_id
merge: convert checkout_fast_forward to struct object_id
sequencer: convert fast_forward_to to struct object_id
builtin/ls-files: convert overlay_tree_on_cache to object_id
builtin/read-tree: convert to struct object_id
sha1_name: convert internals of peel_onion to object_id
upload-pack: convert remaining parse_object callers to object_id
revision: convert remaining parse_object callers to object_id
revision: rename add_pending_sha1 to add_pending_oid
http-push: convert process_ls_object and descendants to object_id
refs/files-backend: convert many internals to struct object_id
refs: convert struct ref_update to use struct object_id
ref-filter: convert some static functions to struct object_id
Convert struct ref_array_item to struct object_id
Convert the verify_pack callback to struct object_id
Convert lookup_tag to struct object_id
...
Plug some leaks and updates internal API used to implement the
split index feature to make it easier to avoid such a leak in the
future.
* nd/split-index-unshare:
p3400: add perf tests for rebasing many changes
split-index: add and use unshare_split_index()
"git checkout --recurse-submodules" did not quite work with a
submodule that itself has submodules.
* sb/checkout-recurse-submodules:
submodule: properly recurse for read-tree and checkout
submodule: avoid auto-discovery in new working tree manipulator code
submodule_move_head: reuse child_process structure for futher commands
For commands that take revisions and pathspecs, magic
pathspecs like ":(exclude)foo" require the user to specify
a disambiguating "--", since they do not match a file in the
filesystem, like:
git grep foo -- :(exclude)bar
This makes them more annoying to use than they need to be.
We loosened the rules for wildcards in 28fcc0b71 (pathspec:
avoid the need of "--" when wildcard is used, 2015-05-02).
Let's do the same for pathspecs with long-form magic.
We already handle the short-forms ":/" and ":^" specially in
check_filename(), so we don't need to handle them here. And
in fact, we could do the same with long-form magic, parsing
out the actual filename and making sure it exists. But there
are a few reasons not to do it that way:
- the parsing gets much more complicated, and we'd want to
hand it off to the pathspec code. But that code isn't
ready to do this kind of speculative parsing (it's happy
to die() when it sees a syntactically invalid pathspec).
- not all pathspec magic maps to a filesystem path. E.g.,
:(attr) should be treated as a pathspec regardless of
what is in the filesystem
- we can be a bit looser with ":(" than with the
short-form ":/", because it is much less likely to have
a false positive. Whereas ":/" also means "search for a
commit with this regex".
Note that because the change is in verify_filename() and not
in its helper check_filename(), this doesn't affect the
verify_non_filename() case. I.e., if an item that matches
our new rule doesn't resolve as an object, we may fallback
to treating it as a pathspec (rather than complaining it
doesn't exist). But if it does resolve (e.g., as a file in
the index that starts with an open-paren), we won't then
complain that it's also a valid pathspec. This matches the
wildcard-exception behavior.
And of course in either case, one can always insert the "--"
to get more precise results.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We special-case "git log :/foo" to work when "foo" exists in
the working tree. But :^ (and its alias :!) do not get the
same treatment, requiring the user to supply a
disambiguating "--". Let's make them work without requiring
the user to type the "--".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The DWIM magic in check_filename() doesn't just recognize
":/". It actually makes sure that the file it points to
exists. t4208 checks only the case where the path is
present, not the opposite. Since the next patches will be
touching this area, let's add a test to make sure it
continues working.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove a test line that's exactly the same as the preceding
line.
This was brought in in commit feabcc173b ("Integrate wildmatch to
git", 2012-10-15), these tests are originally copied from rsync.git,
but the duplicate line was never present there, so must have just
snuck in during integration with git by accident.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a short -P option as a synonym for the longer --perl-regexp, for
consistency with the options the corresponding grep invocations
accept.
This was intentionally omitted in commit 727b6fc3ed ("log --grep:
accept --basic-regexp and --perl-regexp", 2012-10-03) for unspecified
future use.
Make it consistent with "grep" rather than to keep it open for future
use, and to avoid the confusion of -P meaning different things for
grep & log, as is the case with the -G option.
As noted in the aforementioned commit the --basic-regexp option can't
have a corresponding -G argument, as the log command already uses that
for -G<regex>.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a warning about missing thread support when grep.threads or
--threads is set to a non 0 (default) or 1 (no parallelism) value
under NO_PTHREADS=YesPlease.
This is for consistency with the index-pack & pack-objects commands,
which also take a --threads option & are configurable via
pack.threads, and have long warned about the same under
NO_PTHREADS=YesPlease.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix a buggy warning about threads under NO_PTHREADS=YesPlease. Due to
re-using the delta_search_threads variable for both the state of the
"pack.threads" config & the --threads option, setting "pack.threads"
but not supplying --threads would trigger the warning for both
"pack.threads" & --threads.
Solve this bug by resetting the delta_search_threads variable in
git_pack_config(), it might then be set by --threads again and be
subsequently warned about, as the test I'm changing here asserts.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
