Commit Graph

2412 Commits

Author SHA1 Message Date
Junio C Hamano
33e4ae9c50 Merge branch 'bc/sha-256'
Add sha-256 hash and plug it through the code to allow building Git
with the "NewHash".

* bc/sha-256:
  hash: add an SHA-256 implementation using OpenSSL
  sha256: add an SHA-256 implementation using libgcrypt
  Add a base implementation of SHA-256 support
  commit-graph: convert to using the_hash_algo
  t/helper: add a test helper to compute hash speed
  sha1-file: add a constant for hash block size
  t: make the sha1 test-tool helper generic
  t: add basic tests for our SHA-1 implementation
  cache: make hashcmp and hasheq work with larger hashes
  hex: introduce functions to print arbitrary hashes
  sha1-file: provide functions to look up hash algorithms
  sha1-file: rename algorithm to "sha1"
2019-01-29 12:47:55 -08:00
Junio C Hamano
0722553177 Merge branch 'sb/cocci-pending'
A coding convention around the Coccinelle semantic patches to have
two classes to ease code migration process has been proposed and
its support has been added to the Makefile.

* sb/cocci-pending:
  coccicheck: introduce 'pending' semantic patches
2018-11-19 16:24:41 +09:00
Junio C Hamano
2488849c7e Merge branch 'js/test-git-installed'
Update the "test installed Git" mode of our test suite to work better.

* js/test-git-installed:
  tests: explicitly use `git.exe` on Windows
  tests: do not require Git to be built when testing an installed Git
  t/lib-gettext: test installed git-sh-i18n if GIT_TEST_INSTALLED is set
  tests: respect GIT_TEST_INSTALLED when initializing repositories
  tests: fix GIT_TEST_INSTALLED's PATH to include t/helper/
2018-11-19 16:24:41 +09:00
Junio C Hamano
1c6e646235 Merge branch 'dd/poll-dot-h'
A build update.

* dd/poll-dot-h:
  git-compat-util: prefer poll.h to sys/poll.h
2018-11-19 16:24:41 +09:00
Junio C Hamano
954932667d Merge branch 'ab/dynamic-gettext-poison'
Our testing framework uses a special i18n "poisoned localization"
feature to find messages that ought to stay constant but are
incorrectly marked to be translated.  This feature has been made
into a runtime option (it used to be a compile-time option).

* ab/dynamic-gettext-poison:
  Makefile: ease dynamic-gettext-poison transition
  i18n: make GETTEXT_POISON a runtime option
2018-11-19 16:24:39 +09:00
Junio C Hamano
7b9bb3876d Merge branch 'js/fuzz-cxxflags'
The build procedure to link for fuzzing test has been made
customizable with a new Makefile variable.

* js/fuzz-cxxflags:
  Makefile: use FUZZ_CXXFLAGS for linking fuzzers
2018-11-18 18:23:58 +09:00
Junio C Hamano
39847644ad Merge branch 'js/mingw-res-rebuild'
Windows build update.

* js/mingw-res-rebuild:
  Windows: force-recompile git.res for differing architectures
2018-11-18 18:23:53 +09:00
Junio C Hamano
fa2f2f085e Merge branch 'jk/curl-ldflags'
The way -lcurl library gets linked has been simplified by taking
advantage of the fact that we can just ask curl-config command how.

* jk/curl-ldflags:
  build: link with curl-defined linker flags
2018-11-18 18:23:53 +09:00
Junio C Hamano
26b80a841a Merge branch 'nd/pthreads'
The codebase has been cleaned up to reduce "#ifndef NO_PTHREADS".

* nd/pthreads:
  Clean up pthread_create() error handling
  read-cache.c: initialize copy_len to shut up gcc 8
  read-cache.c: reduce branching based on HAVE_THREADS
  read-cache.c: remove #ifdef NO_PTHREADS
  pack-objects: remove #ifdef NO_PTHREADS
  preload-index.c: remove #ifdef NO_PTHREADS
  grep: clean up num_threads handling
  grep: remove #ifdef NO_PTHREADS
  attr.c: remove #ifdef NO_PTHREADS
  name-hash.c: remove #ifdef NO_PTHREADS
  index-pack: remove #ifdef NO_PTHREADS
  send-pack.c: move async's #ifdef NO_PTHREADS back to run-command.c
  run-command.h: include thread-utils.h instead of pthread.h
  thread-utils: macros to unconditionally compile pthreads API
2018-11-18 18:23:52 +09:00
Josh Steadmon
927c77e7d4 Makefile: use FUZZ_CXXFLAGS for linking fuzzers
OSS-Fuzz requires C++-specific flags to link fuzzers. Passing these in
CFLAGS causes lots of build warnings. Using separate FUZZ_CXXFLAGS
avoids this.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-16 14:25:06 +09:00
Johannes Schindelin
8abfdf44c8 tests: explicitly use git.exe on Windows
On Windows, when we refer to `/an/absolute/path/to/git`, it magically
resolves `git.exe` at that location. Except if something of the name
`git` exists next to that `git.exe`. So if we call `$BUILD_DIR/git`, it
will find `$BUILD_DIR/git.exe` *only* if there is not, say, a directory
called `$BUILD_DIR/git`.

Such a directory, however, exists in Git for Windows when building with
Visual Studio (our Visual Studio project generator defaults to putting
the build files into a directory whose name is the base name of the
corresponding `.exe`).

In the bin-wrappers/* scripts, we already take pains to use `git.exe`
rather than `git`, as this could pick up the wrong thing on Windows
(i.e. if there exists a `git` file or directory in the build directory).

Now we do the same in the tests' start-up code.

This also helps when testing an installed Git, as there might be even
more likely some stray file or directory in the way.

Note: the only way we can record whether the `.exe` suffix is by writing
it to the `GIT-BUILD-OPTIONS` file and sourcing it at the beginning of
`t/test-lib.sh`. This is not a requirement introduced by this patch, but
we move the call to be able to use the `$X` variable that holds the file
extension, if any.

Note also: the many, many calls to `git this` and `git that` are
unaffected, as the regular PATH search will find the `.exe` files on
Windows (and not be confused by a directory of the name `git` that is
in one of the directories listed in the `PATH` variable), while
`/path/to/git` would not, per se, know that it is looking for an
executable and happily prefer such a directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-16 14:18:00 +09:00
brian m. carlson
4b4e291809 hash: add an SHA-256 implementation using OpenSSL
We already have OpenSSL routines available for SHA-1, so add routines
for SHA-256 as well.

On a Core i7-6600U, this SHA-256 implementation compares favorably to
the SHA1DC SHA-1 implementation:

SHA-1: 157 MiB/s (64 byte chunks); 337 MiB/s (16 KiB chunks)
SHA-256: 165 MiB/s (64 byte chunks); 408 MiB/s (16 KiB chunks)

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 16:54:53 +09:00
brian m. carlson
27dc04c545 sha256: add an SHA-256 implementation using libgcrypt
Generally, one gets better performance out of cryptographic routines
written in assembly than C, and this is also true for SHA-256.  In
addition, most Linux distributions cannot distribute Git linked against
OpenSSL for licensing reasons.

Most systems with GnuPG will also have libgcrypt, since it is a
dependency of GnuPG.  libgcrypt is also faster than the SHA1DC
implementation for messages of a few KiB and larger.

For comparison, on a Core i7-6600U, this implementation processes 16 KiB
chunks at 355 MiB/s while SHA1DC processes equivalent chunks at 337
MiB/s.

In addition, libgcrypt is licensed under the LGPL 2.1, which is
compatible with the GPL.  Add an implementation of SHA-256 that uses
libgcrypt.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 16:54:53 +09:00
brian m. carlson
13eeedb5d1 Add a base implementation of SHA-256 support
SHA-1 is weak and we need to transition to a new hash function.  For
some time, we have referred to this new function as NewHash.  Recently,
we decided to pick SHA-256 as NewHash.  The reasons behind the choice of
SHA-256 are outlined in the thread starting at [1] and in the commit
history for the hash function transition document.

Add a basic implementation of SHA-256 based off libtomcrypt, which is in
the public domain.  Optimize it and restructure it to meet our coding
standards.  Pull in the update and final functions from the SHA-1 block
implementation, as we know these function correctly with all compilers.
This implementation is slower than SHA-1, but more performant
implementations will be introduced in future commits.

Wire up SHA-256 in the list of hash algorithms, and add a test that the
algorithm works correctly.

Note that with this patch, it is still not possible to switch to using
SHA-256 in Git.  Additional patches are needed to prepare the code to
handle a larger hash algorithm and further test fixes are needed.

[1] https://public-inbox.org/git/20180609224913.GC38834@genre.crustytoothpaste.net/

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 16:54:53 +09:00
brian m. carlson
37649b7f80 t/helper: add a test helper to compute hash speed
Add a utility (which is less for the testsuite and more for developers)
that can compute hash speeds for whatever hash algorithms are
implemented.  This allows developers to test their personal systems to
determine the performance characteristics of various algorithms.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 16:54:52 +09:00
brian m. carlson
50c817e0c0 t: make the sha1 test-tool helper generic
Since we're going to have multiple hash algorithms to test, it makes
sense to share as much of the test code as possible.  Convert the sha1
helper for the test-tool to be generic and move it out into its own
module.  This will allow us to share most of this code with our NewHash
implementation.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 16:54:52 +09:00
Đoàn Trần Công Danh
2648ccc231 git-compat-util: prefer poll.h to sys/poll.h
POSIX specifies that <poll.h> is the correct header for poll(2)
whereas <sys/poll.h> is only needed for some old libc.

Let's follow the POSIX way by default.

This effectively eliminates musl's warning:

    warning redirecting incorrect #include <sys/poll.h> to <poll.h>

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 16:32:24 +09:00
SZEDER Gábor
dd5d052c39 coccicheck: introduce 'pending' semantic patches
Teach `make coccicheck` to avoid patches named "*.pending.cocci" and
handle them separately in a new `make coccicheck-pending` instead.
This means that we can separate "critical" patches from "FYI" patches.
The former target can continue causing Travis to fail its static
analysis job, while the latter can let us keep an eye on ongoing
(pending) transitions without them causing too much fallout.

Document the intended use-cases around these two targets.
As the process around the pending patches is not yet fully explored,
leave that out.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Based-on-work-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-14 11:22:36 +09:00
Junio C Hamano
abb4824d13 Merge branch 'ao/submodule-wo-gitmodules-checked-out'
The submodule support has been updated to read from the blob at
HEAD:.gitmodules when the .gitmodules file is missing from the
working tree.

* ao/submodule-wo-gitmodules-checked-out:
  t/helper: add test-submodule-nested-repo-config
  submodule: support reading .gitmodules when it's not in the working tree
  submodule: add a helper to check if it is safe to write to .gitmodules
  t7506: clean up .gitmodules properly before setting up new scenario
  submodule: use the 'submodule--helper config' command
  submodule--helper: add a new 'config' subcommand
  t7411: be nicer to future tests and really clean things up
  t7411: merge tests 5 and 6
  submodule: factor out a config_set_in_gitmodules_file_gently function
  submodule: add a print_config_from_gitmodules() helper
2018-11-13 22:37:22 +09:00
Junio C Hamano
fd7761a1cd Merge branch 'nd/config-split'
Split the overly large Documentation/config.txt file into million
little pieces.  This potentially allows each individual piece
included into the manual page of the command it affects more easily.

* nd/config-split: (81 commits)
  config.txt: remove config/dummy.txt
  config.txt: move worktree.* to a separate file
  config.txt: move web.* to a separate file
  config.txt: move versionsort.* to a separate file
  config.txt: move user.* to a separate file
  config.txt: move url.* to a separate file
  config.txt: move uploadpack.* to a separate file
  config.txt: move uploadarchive.* to a separate file
  config.txt: move transfer.* to a separate file
  config.txt: move tag.* to a separate file
  config.txt: move submodule.* to a separate file
  config.txt: move stash.* to a separate file
  config.txt: move status.* to a separate file
  config.txt: move splitIndex.* to a separate file
  config.txt: move showBranch.* to a separate file
  config.txt: move sequencer.* to a separate file
  config.txt: move sendemail-config.txt to config/
  config.txt: move reset.* to a separate file
  config.txt: move rerere.* to a separate file
  config.txt: move repack.* to a separate file
  ...
2018-11-13 22:37:16 +09:00
Junio C Hamano
c6a9a30e6a Makefile: ease dynamic-gettext-poison transition
Earlier we made the entire build to fail when GETTEXT_POISON=Yes is
given to make, to notify those who did not notice that text poisoning
is now a runtime behaviour.

It turns out that this is too irritating for those who need to build
and test different versions of Git that cross the boundary between
history with and without this topic to switch between two
environment variables.  Demote the error to a warning, so that you
can say something like

    make GETTEXT_POISON=Yes GIT_TEST_GETTEXT_POISON=Yes test

during the transition period, without having to worry about whether
exact version you are testing has or does not have this topic.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-09 11:25:19 +09:00
Ævar Arnfjörð Bjarmason
6cdccfce1e i18n: make GETTEXT_POISON a runtime option
Change the GETTEXT_POISON compile-time + runtime GIT_GETTEXT_POISON
test parameter to only be a GIT_TEST_GETTEXT_POISON=<non-empty?>
runtime parameter, to be consistent with other parameters documented
in "Running tests with special setups" in t/README.

When I added GETTEXT_POISON in bb946bba76 ("i18n: add GETTEXT_POISON
to simulate unfriendly translator", 2011-02-22) I was concerned with
ensuring that the _() function would get constant folded if NO_GETTEXT
was defined, and likewise that GETTEXT_POISON would be compiled out
unless it was defined.

But as the benchmark in my [1] shows doing a one-off runtime
getenv("GIT_TEST_[...]") is trivial, and since GETTEXT_POISON was
originally added the GIT_TEST_* env variables have become the common
idiom for turning on special test setups.

So change GETTEXT_POISON to work the same way. Now the
GETTEXT_POISON=YesPlease compile-time option is gone, and running the
tests with GIT_TEST_GETTEXT_POISON=[YesPlease|] can be toggled on/off
without recompiling.

This allows for conditionally amending tests to test with/without
poison, similar to what 859fdc0c3c ("commit-graph: define
GIT_TEST_COMMIT_GRAPH", 2018-08-29) did for GIT_TEST_COMMIT_GRAPH. Do
some of that, now we e.g. always run the t0205-gettext-poison.sh test.

I did enough there to remove the GETTEXT_POISON prerequisite, but its
inverse C_LOCALE_OUTPUT is still around, and surely some tests using
it can be converted to e.g. always set GIT_TEST_GETTEXT_POISON=.

Notes on the implementation:

 * We still compile a dedicated GETTEXT_POISON build in Travis
   CI. Perhaps this should be revisited and integrated into the
   "linux-gcc" build, see ae59a4e44f ("travis: run tests with
   GIT_TEST_SPLIT_INDEX", 2018-01-07) for prior art in that area. Then
   again maybe not, see [2].

 * We now skip a test in t0000-basic.sh under
   GIT_TEST_GETTEXT_POISON=YesPlease that wasn't skipped before. This
   test relies on C locale output, but due to an edge case in how the
   previous implementation of GETTEXT_POISON worked (reading it from
   GIT-BUILD-OPTIONS) wasn't enabling poison correctly. Now it does,
   and needs to be skipped.

 * The getenv() function is not reentrant, so out of paranoia about
   code of the form:

       printf(_("%s"), getenv("some-env"));

   call use_gettext_poison() in our early setup in git_setup_gettext()
   so we populate the "poison_requested" variable in a codepath that's
   won't suffer from that race condition.

 * We error out in the Makefile if you're still saying
   GETTEXT_POISON=YesPlease to prompt users to change their
   invocation.

 * We should not print out poisoned messages during the test
   initialization itself to keep it more readable, so the test library
   hides the variable if set in $GIT_TEST_GETTEXT_POISON_ORIG during
   setup. See [3].

See also [4] for more on the motivation behind this patch, and the
history of the GETTEXT_POISON facility.

1. https://public-inbox.org/git/871s8gd32p.fsf@evledraar.gmail.com/
2. https://public-inbox.org/git/20181102163725.GY30222@szeder.dev/
3. https://public-inbox.org/git/20181022202241.18629-2-szeder.dev@gmail.com/
4. https://public-inbox.org/git/878t2pd6yu.fsf@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-09 11:25:19 +09:00
Johannes Schindelin
adb59def10 Windows: force-recompile git.res for differing architectures
When git.rc is compiled into git.res, the result is actually dependent
on the architecture. That is, you cannot simply link a 32-bit git.res
into a 64-bit git.exe.

Therefore, to allow 32-bit and 64-bit builds in the same directory, we
let git.res depend on GIT-PREFIX so that it gets recompiled when
compiling for a different architecture (this works because the exec path
changes based on the architecture: /mingw32/libexec/git-core for 32-bit
and /mingw64/libexec/git-core for 64-bit).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-07 11:58:24 +09:00
James Knight
23c4bbe28e build: link with curl-defined linker flags
Adjusting the build process to rely more on curl-config to populate
linker flags instead of manually populating flags based off detected
features.

Originally, a configure-invoked build would check for SSL-support in the
target curl library. If enabled, NEEDS_SSL_WITH_CURL would be set and
used in the Makefile to append additional libraries to link against. As
for systems building solely with make, the defines NEEDS_IDN_WITH_CURL
and NEEDS_SSL_WITH_CURL could be set to indirectly enable respective
linker flags. Since both configure.ac and Makefile already rely on
curl-config utility to provide curl-related build information, adjusting
the respective assets to populate required linker flags using the
utility (unless explicitly configured).

Signed-off-by: James Knight <james.d.knight@live.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-05 10:19:25 +09:00
Junio C Hamano
b49ef560ed Merge branch 'ag/rebase-i-in-c'
Rewrite of the remaining "rebase -i" machinery in C.

* ag/rebase-i-in-c:
  rebase -i: move rebase--helper modes to rebase--interactive
  rebase -i: remove git-rebase--interactive.sh
  rebase--interactive2: rewrite the submodes of interactive rebase in C
  rebase -i: implement the main part of interactive rebase as a builtin
  rebase -i: rewrite init_basic_state() in C
  rebase -i: rewrite write_basic_state() in C
  rebase -i: rewrite the rest of init_revisions_and_shortrevisions() in C
  rebase -i: implement the logic to initialize $revisions in C
  rebase -i: remove unused modes and functions
  rebase -i: rewrite complete_action() in C
  t3404: todo list with commented-out commands only aborts
  sequencer: change the way skip_unnecessary_picks() returns its result
  sequencer: refactor append_todo_help() to write its message to a buffer
  rebase -i: rewrite checkout_onto() in C
  rebase -i: rewrite setup_reflog_action() in C
  sequencer: add a new function to silence a command, except if it fails
  rebase -i: rewrite the edit-todo functionality in C
  editor: add a function to launch the sequence editor
  rebase -i: rewrite append_todo_help() in C
  sequencer: make three functions and an enum from sequencer.c public
2018-11-02 11:04:53 +09:00
Junio C Hamano
5ae50845d8 Merge branch 'pk/rebase-in-c'
Rewrite of the "rebase" machinery in C.

* pk/rebase-in-c:
  builtin/rebase: support running "git rebase <upstream>"
  rebase: refactor common shell functions into their own file
  rebase: start implementing it as a builtin
2018-11-02 11:04:52 +09:00
Antonio Ospite
2b1257e463 t/helper: add test-submodule-nested-repo-config
Add a test tool to exercise config_from_gitmodules(), in particular for
the case of nested submodules.

Add also a test to document that reading the submoudles config of nested
submodules does not work yet when the .gitmodules file is not in the
working tree but it still in the index.

This is because the git API does not always make it possible access the
object store  of an arbitrary repository (see get_oid() usage in
config_from_gitmodules()).

When this git limitation gets fixed the aforementioned use case will be
supported too.

Signed-off-by: Antonio Ospite <ao2@ao2.it>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-31 15:01:30 +09:00
Nguyễn Thái Ngọc Duy
0ec79358d0 thread-utils: macros to unconditionally compile pthreads API
When built with NO_PTHREADS, the macros are used make the code build
even though pthreads header and library may be missing. The code can
still have different code paths for no threads support with
HAVE_THREADS variable.

There are of course impacts on no-pthreads builds:

- data structure may get slightly bigger because all the mutexes and
  pthread_t are present (as an int)

- code execution is not impacted much. Locking (in hot path) is
  no-op. Other wrapper function calls really should not matter much.

- the binary size grows bigger because of threaded code. But at least
  on Linux this does not matter, if some code is not executed, it's
  not mapped in memory.

This is a preparation step to remove "#ifdef NO_PTHREADS" in the code
mostly because of maintainability. As Jeff put it

> it's probably OK to stop thinking of it as "non-threaded platforms
> are the default and must pay zero cost" and more as "threaded
> platforms are the default, and non-threaded ones are OK to pay a
> small cost as long as they still work".

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-29 11:22:48 +09:00
Nguyễn Thái Ngọc Duy
76b993afd6 Update makefile in preparation for Documentation/config/*.txt
config.txt is going to be broken down in smaller pieces and put under
Documentation/config directory. Update build rules to take these files
into account.

A dummy file is added to make sure wildcard expansion is predictable
(depending on shell setting it could expand to nothing or becomes a
path if config directory is empty). The file will be deleted once the
move is over.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-29 10:16:59 +09:00
Junio C Hamano
0c41b3b1a7 Merge branch 'js/fuzzer'
An experiment to fuzz test a few areas, hopefully we can gain more
coverage to various areas.

* js/fuzzer:
  fuzz: add fuzz testing for packfile indices.
  fuzz: add basic fuzz testing target.
2018-10-26 14:22:14 +09:00
Junio C Hamano
340fde61be Merge branch 'bp/rename-test-env-var'
Some environment variables that control the runtime options of Git
used during tests are getting renamed for consistency.

* bp/rename-test-env-var:
  t0000: do not get self-test disrupted by environment warnings
  preload-index: update GIT_FORCE_PRELOAD_TEST support
  read-cache: update TEST_GIT_INDEX_VERSION support
  fsmonitor: update GIT_TEST_FSMONITOR support
  preload-index: use git_env_bool() not getenv() for customization
  t/README: correct spelling of "uncommon"
2018-10-19 13:34:03 +09:00
Junio C Hamano
ff6bbce6e3 Merge branch 'rj/header-check'
Header files clean-up.

* rj/header-check:
  delta-islands.h: add missing forward declarations (hdr-check)
  midx.h: add missing forward declarations (hdr-check)
  refs/refs-internal.h: add missing declarations (hdr-check)
  refs/packed-backend.h: add missing declaration (hdr-check)
  refs/ref-cache.h: add missing declarations (hdr-check)
  ewah/ewok_rlw.h: add missing include (hdr-check)
  json-writer.h: add missing include (hdr-check)
  Makefile: add a hdr-check target
2018-10-16 16:16:00 +09:00
Josh Steadmon
1127a98cce fuzz: add fuzz testing for packfile indices.
Breaks the majority of check_packed_git_idx() into a separate function,
load_idx(). The latter function operates on arbitrary buffers, which
makes it suitable as a fuzzing test target.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-15 14:29:03 +09:00
Josh Steadmon
5e47215080 fuzz: add basic fuzz testing target.
fuzz-pack-headers.c provides a fuzzing entry point compatible with
libFuzzer (and possibly other fuzzing engines).

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-15 14:28:59 +09:00
Junio C Hamano
77b5046ae3 Merge branch 'nd/test-tool'
Test helper binaries clean-up.

* nd/test-tool:
  Makefile: add a hint about TEST_BUILTINS_OBJS
  t/helper: merge test-dump-fsmonitor into test-tool
  t/helper: merge test-parse-options into test-tool
  t/helper: merge test-pkt-line into test-tool
  t/helper: merge test-dump-untracked-cache into test-tool
  t/helper: keep test-tool command list sorted
2018-10-10 12:37:16 +09:00
Alban Gruin
34b47315d9 rebase -i: move rebase--helper modes to rebase--interactive
This moves the rebase--helper modes still used by
git-rebase--preserve-merges.sh (`--shorten-ids`, `--expand-ids`,
`--check-todo-list`, `--rearrange-squash` and `--add-exec-commands`) to
rebase--interactive.c.

git-rebase--preserve-merges.sh is modified accordingly, and
rebase--helper.c is removed as it is useless now.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-09 10:44:10 +09:00
Alban Gruin
cf808208ec rebase -i: remove git-rebase--interactive.sh
This removes git-rebase--interactive.sh, as its functionnality has been
replaced by git-rebase--interactive2.

git-rebase--interactive2.c is then renamed to git-rebase--interactive.c.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-09 10:44:10 +09:00
Alban Gruin
53bbcfbde7 rebase -i: implement the main part of interactive rebase as a builtin
This rewrites the part of interactive rebase which initializes the
basic state, make the script and complete the action, as a buitin, named
git-rebase--interactive2 for now.  Others modes (`--continue`,
`--edit-todo`, etc.) will be rewritten in the next commit.

git-rebase--interactive.sh is modified to call git-rebase--interactive2
instead of git-rebase--helper.

Signed-off-by: Alban Gruin <alban.gruin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-09 10:44:10 +09:00
Ben Peart
1f357b045b read-cache: update TEST_GIT_INDEX_VERSION support
Rename TEST_GIT_INDEX_VERSION to GIT_TEST_INDEX_VERSION for consistency with
the other GIT_TEST_ special setups and properly document its use.

Add logic in t/test-lib.sh to give a warning when the old variable is set to
let people know they need to update their environment to use the new
variable.

Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-28 11:41:01 -07:00
Junio C Hamano
e3d4ff037d Merge branch 'js/mingw-o-append'
Further fix for O_APPEND emulation on Windows

* js/mingw-o-append:
  mingw: fix mingw_open_append to work with named pipes
  t0051: test GIT_TRACE to a windows named pipe
2018-09-24 10:30:47 -07:00
Ramsay Jones
ebb7baf02f Makefile: add a hdr-check target
Commit ef3ca95475 ("Add missing includes and forward declarations",
2018-08-15) resulted from the author employing a manual method to
create a C file consisting of a pair of pre-processor #include
lines (for 'git-compat-util.h' and a given toplevel header), and
fixing any resulting compiler errors or warnings.

Add a Makefile target to automate this process. This implementation
relies on the '-include' and '-xc' arguments to the 'gcc' and 'clang'
compilers, which allows us to effectively create the required C
compilation unit on-the-fly. This limits the portability of this
solution to those systems which have such a compiler.

The new 'hdr-check' target can be used to check most header files in
the project (for various reasons, the 'compat' and 'xdiff' directories
are not included). Also, note that individual header files can be
checked directly using the '.hco' extension (read: Hdr-Check Object)
like so:

    $ make config.hco
        HDR config.h
    $

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-20 11:47:38 -07:00
Junio C Hamano
688cb1c989 Merge branch 'es/format-patch-interdiff'
"git format-patch" learned a new "--interdiff" option to explain
the difference between this version and the previous atttempt in
the cover letter (or after the tree-dashes as a comment).

* es/format-patch-interdiff:
  format-patch: allow --interdiff to apply to a lone-patch
  log-tree: show_log: make commentary block delimiting reusable
  interdiff: teach show_interdiff() to indent interdiff
  format-patch: teach --interdiff to respect -v/--reroll-count
  format-patch: add --interdiff option to embed diff in cover letter
  format-patch: allow additional generated content in make_cover_letter()
2018-09-17 13:53:55 -07:00
Junio C Hamano
f3504ea3dd Merge branch 'cc/delta-islands'
Lift code from GitHub to restrict delta computation so that an
object that exists in one fork is not made into a delta against
another object that does not appear in the same forked repository.

* cc/delta-islands:
  pack-objects: move 'layer' into 'struct packing_data'
  pack-objects: move tree_depth into 'struct packing_data'
  t5320: tests for delta islands
  repack: add delta-islands support
  pack-objects: add delta-islands support
  pack-objects: refactor code into compute_layer_order()
  Add delta-islands.{c,h}
2018-09-17 13:53:55 -07:00
Junio C Hamano
1b7a91da71 Merge branch 'ds/reachable'
The code for computing history reachability has been shuffled,
obtained a bunch of new tests to cover them, and then being
improved.

* ds/reachable:
  commit-reach: correct accidental #include of C file
  commit-reach: use can_all_from_reach
  commit-reach: make can_all_from_reach... linear
  commit-reach: replace ref_newer logic
  test-reach: test commit_contains
  test-reach: test can_all_from_reach_with_flags
  test-reach: test reduce_heads
  test-reach: test get_merge_bases_many
  test-reach: test is_descendant_of
  test-reach: test in_merge_bases
  test-reach: create new test tool for ref_newer
  commit-reach: move can_all_from_reach_with_flags
  upload-pack: generalize commit date cutoff
  upload-pack: refactor ok_to_give_up()
  upload-pack: make reachable() more generic
  commit-reach: move commit_contains from ref-filter
  commit-reach: move ref_newer from remote.c
  commit.h: remove method declarations
  commit-reach: move walk methods from commit.c
2018-09-17 13:53:52 -07:00
Junio C Hamano
49f210fd52 Merge branch 'ds/multi-pack-index'
When there are too many packfiles in a repository (which is not
recommended), looking up an object in these would require
consulting many pack .idx files; a new mechanism to have a single
file that consolidates all of these .idx files is introduced.

* ds/multi-pack-index: (32 commits)
  pack-objects: consider packs in multi-pack-index
  midx: test a few commands that use get_all_packs
  treewide: use get_all_packs
  packfile: add all_packs list
  midx: fix bug that skips midx with alternates
  midx: stop reporting garbage
  midx: mark bad packed objects
  multi-pack-index: store local property
  multi-pack-index: provide more helpful usage info
  midx: clear midx on repack
  packfile: skip loading index if in multi-pack-index
  midx: prevent duplicate packfile loads
  midx: use midx in approximate_object_count
  midx: use existing midx when writing new one
  midx: use midx in abbreviation calculations
  midx: read objects from multi-pack-index
  config: create core.multiPackIndex setting
  midx: write object offsets
  midx: write object id fanout chunk
  midx: write object ids in a chunk
  ...
2018-09-17 13:53:50 -07:00
Jeff Hostetler
06ba9d03e3 t0051: test GIT_TRACE to a windows named pipe
Create a test-tool helper to create the server side of
a windows named pipe, wait for a client connection, and
copy data written to the pipe to stdout.

Create t0051 test to route GIT_TRACE output of a command
to a named pipe using the above test-tool helper.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 13:54:25 -07:00
Nguyễn Thái Ngọc Duy
c9a1f4161f Makefile: add a hint about TEST_BUILTINS_OBJS
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 10:54:19 -07:00
Nguyễn Thái Ngọc Duy
f1ef0b024c t/helper: merge test-dump-fsmonitor into test-tool
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 10:54:19 -07:00
Nguyễn Thái Ngọc Duy
2f17c78ceb t/helper: merge test-parse-options into test-tool
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 10:54:19 -07:00
Nguyễn Thái Ngọc Duy
8ea40cc55d t/helper: merge test-pkt-line into test-tool
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11 10:54:19 -07:00