Commit Graph

2492 Commits

Author SHA1 Message Date
Junio C Hamano
c62bc49139 Merge branch 'js/visual-studio'
Support building Git with Visual Studio

The bits about .git/branches/* have been dropped from the series.
We may want to drop the support for it, but until that happens, the
tests should rely on the existence of the support to pass.

* js/visual-studio: (23 commits)
  git: avoid calling aliased builtins via their dashed form
  bin-wrappers: append `.exe` to target paths if necessary
  .gitignore: ignore Visual Studio's temporary/generated files
  .gitignore: touch up the entries regarding Visual Studio
  vcxproj: also link-or-copy builtins
  msvc: add a Makefile target to pre-generate the Visual Studio solution
  contrib/buildsystems: add a backend for modern Visual Studio versions
  contrib/buildsystems: handle options starting with a slash
  contrib/buildsystems: also handle -lexpat
  contrib/buildsystems: handle libiconv, too
  contrib/buildsystems: handle the curl library option
  contrib/buildsystems: error out on unknown option
  contrib/buildsystems: optionally capture the dry-run in a file
  contrib/buildsystems: redirect errors of the dry run into a log file
  contrib/buildsystems: ignore gettext stuff
  contrib/buildsystems: handle quoted spaces in filenames
  contrib/buildsystems: fix misleading error message
  contrib/buildsystems: ignore irrelevant files in Generators/
  contrib/buildsystems: ignore invalidcontinue.obj
  Vcproj.pm: urlencode '<' and '>' when generating VC projects
  ...
2019-08-02 13:12:02 -07:00
Johannes Schindelin
3a94cb31d5 bin-wrappers: append .exe to target paths if necessary
When compiling with Visual Studio, the projects' names are identical to
the executables modulo the extensions. Read: there will exist both a
directory called `git` as well as an executable called `git.exe` in the
end. Which means that the bin-wrappers *need* to target the `.exe` files
lest they try to execute directories.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-29 14:51:43 -07:00
Junio C Hamano
080af915a3 Merge branch 'mt/dir-iterator-updates'
Adjust the dir-iterator API and apply it to the local clone
optimization codepath.

* mt/dir-iterator-updates:
  clone: replace strcmp by fspathcmp
  clone: use dir-iterator to avoid explicit dir traversal
  clone: extract function from copy_or_link_directory
  clone: copy hidden paths at local clone
  dir-iterator: add flags parameter to dir_iterator_begin
  dir-iterator: refactor state machine model
  dir-iterator: use warning_errno when possible
  dir-iterator: add tests for dir-iterator API
  clone: better handle symlinked files at .git/objects/
  clone: test for our behavior on odd objects/* content
2019-07-25 13:59:22 -07:00
Junio C Hamano
023ff4cdf5 Merge branch 'ab/test-env'
Many GIT_TEST_* environment variables control various aspects of
how our tests are run, but a few followed "non-empty is true, empty
or unset is false" while others followed the usual "there are a few
ways to spell true, like yes, on, etc., and also ways to spell
false, like no, off, etc." convention.

* ab/test-env:
  env--helper: mark a file-local symbol as static
  tests: make GIT_TEST_FAIL_PREREQS a boolean
  tests: replace test_tristate with "git env--helper"
  tests README: re-flow a previously changed paragraph
  tests: make GIT_TEST_GETTEXT_POISON a boolean
  t6040 test: stop using global "script" variable
  config.c: refactor die_bad_number() to not call gettext() early
  env--helper: new undocumented builtin wrapping git_env_*()
  config tests: simplify include cycle test
2019-07-25 13:59:20 -07:00
Junio C Hamano
c62bff2ced Merge branch 'cc/test-oidmap'
Extend the test coverage a bit.

* cc/test-oidmap:
  t0016: add 'remove' subcommand test
  test-oidmap: remove 'add' subcommand
  test-hashmap: remove 'hash' command
  oidmap: use sha1hash() instead of static hash() function
  t: add t0016-oidmap.sh
  t/helper: add test-oidmap.c
2019-07-19 11:30:19 -07:00
Daniel Ferreira
150791adbf dir-iterator: add tests for dir-iterator API
Create t/helper/test-dir-iterator.c, which prints relevant information
about a directory tree iterated over with dir-iterator.

Create t/t0066-dir-iterator.sh, which tests that dir-iterator does
iterate through a whole directory tree as expected.

Signed-off-by: Daniel Ferreira <bnmvco@gmail.com>
[matheus.bernardino: update to use test-tool and some minor aesthetics]
Helped-by: Matheus Tavares <matheus.bernardino@usp.br>
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-11 13:52:15 -07:00
Junio C Hamano
88b1075759 Merge branch 'jh/msvc'
Support to build with MSVC has been updated.

* jh/msvc:
  msvc: ignore .dll and incremental compile output
  msvc: avoid debug assertion windows in Debug Mode
  msvc: do not pretend to support all signals
  msvc: add pragmas for common warnings
  msvc: add a compile-time flag to allow detailed heap debugging
  msvc: support building Git using MS Visual C++
  msvc: update Makefile to allow for spaces in the compiler path
  msvc: fix detect_msys_tty()
  msvc: define ftello()
  msvc: do not re-declare the timespec struct
  msvc: mark a variable as non-const
  msvc: define O_ACCMODE
  msvc: include sigset_t definition
  msvc: fix dependencies of compat/msvc.c
  mingw: replace mingw_startup() hack
  obstack: fix compiler warning
  cache-tree/blame: avoid reusing the DEBUG constant
  t0001 (mingw): do not expect a specific order of stdout/stderr
  Mark .bat files as requiring CR/LF endings
  mingw: fix a typo in the msysGit-specific section
2019-07-09 15:25:45 -07:00
Junio C Hamano
f496b064fc Merge branch 'nd/switch-and-restore'
Two new commands "git switch" and "git restore" are introduced to
split "checking out a branch to work on advancing its history" and
"checking out paths out of the index and/or a tree-ish to work on
advancing the current history" out of the single "git checkout"
command.

* nd/switch-and-restore: (46 commits)
  completion: disable dwim on "git switch -d"
  switch: allow to switch in the middle of bisect
  t2027: use test_must_be_empty
  Declare both git-switch and git-restore experimental
  help: move git-diff and git-reset to different groups
  doc: promote "git restore"
  user-manual.txt: prefer 'merge --abort' over 'reset --hard'
  completion: support restore
  t: add tests for restore
  restore: support --patch
  restore: replace --force with --ignore-unmerged
  restore: default to --source=HEAD when only --staged is specified
  restore: reject invalid combinations with --staged
  restore: add --worktree and --staged
  checkout: factor out worktree checkout code
  restore: disable overlay mode by default
  restore: make pathspec mandatory
  restore: take tree-ish from --source option instead
  checkout: split part of it to new command 'restore'
  doc: promote "git switch"
  ...
2019-07-09 15:25:44 -07:00
Jeff Hostetler
dce7d29551 msvc: support building Git using MS Visual C++
With this patch, Git can be built using the Microsoft toolchain, via:

	make MSVC=1 [DEBUG=1]

Third party libraries are built from source using the open source
"vcpkg" tool set. See https://github.com/Microsoft/vcpkg

On a first build, the vcpkg tools and the third party libraries are
automatically downloaded and built. DLLs for the third party libraries
are copied to the top-level (and t/helper) directory to facilitate
debugging. See compat/vcbuild/README.

A series of .bat files are invoked by the Makefile to find the location
of the installed version of Visual Studio and the associated compiler
tools (essentially replicating the environment setup performed by a
"Developer Command Prompt"). This should find the most recent VS2015 or
VS2017 installation. Output from these scripts are used by the Makefile
to define compiler and linker pathnames and -I and -L arguments.

The build produces .pdb files for both debug and release builds.

Note: This commit was squashed from an organic series of commits
developed between 2016 and 2018 in Git for Windows' `master` branch.
This combined commit eliminates the obsolete commits related to fetching
NuGet packages for third party libraries. It is difficult to use NuGet
packages for C/C++ sources because they may be built by earlier versions
of the MSVC compiler and have CRT version and linking issues.

Additionally, the C/C++ NuGet packages that we were using tended to not
be updated concurrently with the sources.  And in the case of cURL and
OpenSSL, this could expose us to security issues.

Helped-by: Yue Lin Ho <b8732003@student.nsysu.edu.tw>
Helped-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-25 10:46:57 -07:00
Ævar Arnfjörð Bjarmason
b4f207f339 env--helper: new undocumented builtin wrapping git_env_*()
We have many GIT_TEST_* variables that accept a <boolean> because
they're implemented in C, and then some that take <non-empty?> because
they're implemented at least partially in shellscript.

Add a helper that wraps git_env_bool() and git_env_ulong() as the
first step in fixing this. This isn't being added as a test-tool mode
because some of these are used outside the test suite.

Part of what this tool does can be done via a trick with "git config"
added in 83d842dc8c ("tests: turn on network daemon tests by default",
2014-02-10) for test_tristate(), i.e.:

    git -c magic.variable="$1" config --bool magic.variable 2>/dev/null

But as subsequent changes will show being able to pass along the
default value makes all the difference, and we'll be able to replace
test_tristate() itself with that.

The --type=bool option will be used by subsequent patches, but not
--type=ulong. I figured it was easy enough to add it & test for it so
I left it in so we'd have wrappers for both git_env_*() functions, and
to have a template to make it obvious how we'd add --type=int etc. if
it's needed in the future.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-21 09:42:49 -07:00
Jeff Hostetler
c444bf8cb6 msvc: update Makefile to allow for spaces in the compiler path
It is quite common that MS Visual C++ is installed into a location whose
path contains spaces, therefore we need to quote it.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-20 14:03:06 -07:00
Christian Couder
11510decd0 t/helper: add test-oidmap.c
This new helper is very similar to "test-hashmap.c" and will help
test how `struct oidmap` from oidmap.{c,h} can be used.

Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-17 18:11:41 -07:00
Junio C Hamano
51d6c0f015 Merge branch 'ab/deprecate-R-for-dynpath'
The way of specifying the path to find dynamic libraries at runtime
has been simplified.  The old default to pass -R/path/to/dir has been
replaced with the new default to pass -Wl,-rpath,/path/to/dir,
which is the more recent GCC uses.  Those who need to build with an
old GCC can still use "CC_LD_DYNPATH=-R"

* ab/deprecate-R-for-dynpath:
  Makefile: remove the NO_R_TO_GCC_LINKER flag
2019-06-13 13:19:43 -07:00
Junio C Hamano
ed7f8acbaa Merge branch 'js/rebase-cleanup'
Update supporting parts of "git rebase" to remove code that should
no longer be used.

* js/rebase-cleanup:
  rebase: fold git-rebase--common into the -p backend
  sequencer: the `am` and `rebase--interactive` scripts are gone
  .gitignore: there is no longer a built-in `git-rebase--interactive`
  t3400: stop referring to the scripted rebase
  Drop unused git-rebase--am.sh
2019-06-13 13:19:40 -07:00
Junio C Hamano
6795fc8afd Merge branch 'jk/cocci-batch'
Optionally "make coccicheck" can feed multiple source files to
spatch, gaining performance while spending more memory.

* jk/cocci-batch:
  coccicheck: make batch size of 0 mean "unlimited"
  coccicheck: optionally batch spatch invocations
2019-05-19 16:45:28 +09:00
Ævar Arnfjörð Bjarmason
0f50c8e32c Makefile: remove the NO_R_TO_GCC_LINKER flag
Change our default CC_LD_DYNPATH invocation to something GCC likes
these days. Since the GCC 4.6 release unknown flags haven't been
passed through to ld(1). Thus our previous default of CC_LD_DYNPATH=-R
would cause an error on modern GCC unless NO_R_TO_GCC_LINKER was set.

This CC_LD_DYNPATH flag is really obscure, and I don't expect anyone
except those working on git development ever use this.

It's not needed to simply link to libraries like say libpcre,
but *only* for those cases where we're linking to such a library not
present in the OS's library directories. See e.g. ldconfig(8) on Linux
for more details.

I use this to compile my git with a LIBPCREDIR=$HOME/g/pcre2/inst as
I'm building that from source, but someone maintaining an OS package
is almost certainly not going to use this. They're just going to set
USE_LIBPCRE=YesPlease after installing the libpcre dependency,
which'll point to OS libraries which ld(1) will find without the help
of CC_LD_DYNPATH.

Another thing that helps mitigate any potential breakage is that we
detect the right type of invocation in configure.ac, which e.g. HP/UX
uses[1], as does IBM's AIX package[2]. From what I can tell both AIX
and Solaris packagers are building git with GCC, so I'm not adding a
corresponding config.mak.uname default to cater to their OS-native
linkers.

Now for an overview of past development in this area:

Our use of "-R" dates back to 455a7f3275 ("More portability.",
2005-09-30). Soon after that in bbfc63dd78 ("gcc does not necessarily
pass runtime libpath with -R", 2006-12-27) the NO_R_TO_GCC flag was
added, allowing optional use of "-Wl,-rpath=".

Then in f5b904db6b ("Makefile: Allow CC_LD_DYNPATH to be overriden",
2008-08-16) the ability to override this flag to something else
entirely was added, as some linkers use neither "-Wl,-rpath," nor
"-R".

From what I can tell we should, with the benefit of hindsight, have
made this change back in 2006. GCC & ld supported this type of
invocation back then, or since at least binutils-gdb.git's[3]
a1ad915dc4 ("[...]Add support for -rpath[...]", 1994-07-20).

Further reading and prior art can be found at [4][5][6][7]. Making a
plain "-R" an error seems from reading those reports to have been
introduced in GCC 4.6 released on March 25, 2011[8], but I couldn't
confirm this with absolute certainty, its release notes are ambiguous
on the subject, and I couldn't be bothered to try to build & bisect it
against GCC 4.5.

1. https://public-inbox.org/git/20190516093412.14795-1-avarab@gmail.com/
2. https://www.ibm.com/developerworks/aix/library/aix-toolbox/alpha.html
3. git://sourceware.org/git/binutils-gdb.git
4. https://github.com/tsuna/boost.m4/issues/15
5. https://bugzilla.gnome.org/show_bug.cgi?id=641416
6. https://stackoverflow.com/questions/12629042/g-4-6-real-error-unrecognized-option-r
7. https://curl.haxx.se/mail/archive-2014-11/0005.html
8. https://gcc.gnu.org/gcc-4.6/changes.html

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-19 09:46:24 +09:00
Johannes Schindelin
082ef75b7b rebase: fold git-rebase--common into the -p backend
The only remaining scripted part of `git rebase` is the
`--preserve-merges` backend. Meaning: there is little reason to keep the
"library of common rebase functions" as a separate file.

While moving the functions to `git-rebase--preserve-merges.sh`, we also
drop the `move_to_original_branch` function that is no longer used.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-15 10:57:32 +09:00
Johannes Schindelin
311c00aae8 Drop unused git-rebase--am.sh
Since 21853626ea (built-in rebase: call `git am` directly, 2019-01-18),
the built-in rebase already uses the built-in `git am` directly.

Now that d03ebd411c (rebase: remove the rebase.useBuiltin setting,
2019-03-18) even removed the scripted rebase, there is no longer any
user of `git-rebase--am.sh`, so let's just remove it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-15 10:57:31 +09:00
Junio C Hamano
40bef4992e Merge branch 'cc/access-on-aix-workaround'
Workaround for standard-compliant but less-than-useful behaviour of
access(2) for the root user.

* cc/access-on-aix-workaround:
  git-compat-util: work around for access(X_OK) under root
2019-05-13 23:50:35 +09:00
Junio C Hamano
7ba06bc3d0 Merge branch 'pw/rebase-i-internal'
The internal implementation of "git rebase -i" has been updated to
avoid forking a separate "rebase--interactive" process.

* pw/rebase-i-internal:
  rebase -i: run without forking rebase--interactive
  rebase: use a common action enum
  rebase -i: use struct rebase_options in do_interactive_rebase()
  rebase -i: use struct rebase_options to parse args
  rebase -i: use struct object_id for squash_onto
  rebase -i: use struct commit when parsing options
  rebase -i: remove duplication
  rebase -i: combine rebase--interactive.c with rebase.c
  rebase: use OPT_RERERE_AUTOUPDATE()
  rebase: rename write_basic_state()
  rebase: don't translate trace strings
  sequencer: always discard index after checkout
2019-05-13 23:50:34 +09:00
Junio C Hamano
a505f62f6f Merge branch 'jc/make-dedup-ls-files-output'
A "ls-files" that emulates "find" to enumerate files in the working
tree resulted in duplicated Makefile rules that caused the build to
issue an unnecessary warning during a trial build after merge
conflicts are resolved in working tree *.h files but before the
resolved results are added to the index.  This has been corrected.

* jc/make-dedup-ls-files-output:
  Makefile: dedup list of files obtained from ls-files
2019-05-13 23:50:33 +09:00
Junio C Hamano
5b2d1c0c6e Merge branch 'jh/trace2-sid-fix'
Polishing of the new trace2 facility continues.  The system-level
configuration can specify site-wide trace2 settings, which can be
overridden with per-user configuration and environment variables.

* jh/trace2-sid-fix:
  trace2: fixup access problem on /etc/gitconfig in read_very_early_config
  trace2: update docs to describe system/global config settings
  trace2: make SIDs more unique
  trace2: clarify UTC datetime formatting
  trace2: report peak memory usage of the process
  trace2: use system/global config for default trace2 settings
  config: add read_very_early_config()
  trace2: find exec-dir before trace2 initialization
  trace2: add absolute elapsed time to start event
  trace2: refactor setting process starting time
  config: initialize opts structure in repo_read_config()
2019-05-13 23:50:31 +09:00
Junio C Hamano
caa227ff45 Merge branch 'js/misc-doc-fixes'
"make check-docs", "git help -a", etc. did not account for cases
where a particular build may deliberately omit some subcommands,
which has been corrected.

* js/misc-doc-fixes:
  Turn `git serve` into a test helper
  test-tool: handle the `-C <directory>` option just like `git`
  check-docs: do not bother checking for legacy scripts' documentation
  docs: exclude documentation for commands that have been excluded
  check-docs: allow command-list.txt to contain excluded commands
  help -a: do not list commands that are excluded from the build
  Makefile: drop the NO_INSTALL variable
  remote-testgit: move it into the support directory for t5801
2019-05-09 00:37:27 +09:00
Jeff King
bcb4edf7af coccicheck: make batch size of 0 mean "unlimited"
If you have the memory to handle it, the ideal case is to run a single
spatch invocation with all of the source files. But the only way to do
so now is to pick an arbitrarily large batch size. Let's make "0" do
this, which is a little friendlier (and doesn't otherwise have a useful
meaning).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-08 19:25:14 +09:00
Nguyễn Thái Ngọc Duy
46e91b663b checkout: split part of it to new command 'restore'
Previously the switching branch business of 'git checkout' becomes a
new command 'switch'. This adds the restore command for the checking
out paths path.

Similar to git-switch, a new man page is added to describe what the
command will become. The implementation will be updated shortly to
match the man page.

A couple main differences from 'git checkout <paths>':

- 'restore' by default will only update worktree. This matters more
  when --source is specified ('checkout <tree> <paths>' updates both
  worktree and index).

- 'restore --staged' can be used to restore the index. This command
  overlaps with 'git reset <paths>'.

- both worktree and index could also be restored at the same time
  (from a tree) when both --staged and --worktree are specified. This
  overlaps with 'git checkout <tree> <paths>'

- default source for restoring worktree and index is the index and
  HEAD respectively. A different (tree) source could be specified as
  with --source (*).

- when both index and worktree are restored, --source must be
  specified since the default source for these two individual targets
  are different (**)

- --no-overlay is enabled by default, if an entry is missing in the
  source, restoring means deleting the entry

(*) I originally went with --from instead of --source. I still think
  --from is a better name. The short option -f however is already
  taken by force. And I do think short option is good to have, e.g. to
  write -s@ or -s@^ instead of --source=HEAD.

(**) If you sit down and think about it, moving worktree's source from
  the index to HEAD makes sense, but nobody is really thinking it
  through when they type the commands.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-07 13:04:47 +09:00
Jeff King
960154b9c1 coccicheck: optionally batch spatch invocations
In our "make coccicheck" rule, we currently feed each source file to its
own individual invocation of spatch. This has a few downsides:

  - it repeats any overhead spatch has for starting up and reading the
    patch file

  - any included header files may get processed from multiple
    invocations. This is slow (we see the same header files multiple
    times) and may produce a resulting patch with repeated hunks (which
    cannot be applied without further cleanup)

Ideally we'd just invoke a single instance of spatch per rule-file and
feed it all source files. But spatch can be rather memory hungry when
run in this way. I measured the peak RSS going from ~90MB for a single
file to ~1900MB for all files. Multiplied by multiple rule files being
processed at the same time (for "make -j"), this can make things slower
or even cause them to fail (e.g., this is reported to happen on our
Travis builds).

Instead, let's provide a tunable knob. We'll leave the default at "1",
but it can be cranked up to "999" for maximum CPU/memory tradeoff, or
people can find points in between that serve their particular machines.

Here are a few numbers running a single rule via:

  SIZES='1 4 16 999'
  RULE=contrib/coccinelle/object_id.cocci
  for i in $SIZES; do
    make clean
    /usr/bin/time -o $i.out --format='%e | %U | %S | %M' \
      make $RULE.patch SPATCH_BATCH_SIZE=$i
  done
  for i in $SIZES; do
    printf '%4d | %s\n' $i "$(cat $i.out)"
  done

which yields:

     1 | 97.73 | 93.38 | 4.33 | 100128
     4 | 52.80 | 51.14 | 1.69 | 135204
    16 | 35.82 | 35.09 | 0.76 | 284124
   999 | 23.30 | 23.13 | 0.20 | 1903852

The implementation is done with xargs, which should be widely available;
it's in POSIX, we rely on it already in the test suite. And "coccicheck"
is really a developer-only tool anyway, so it's not a big deal if
obscure systems can't run it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-07 11:37:17 +09:00
Clément Chigot
400caafb2b git-compat-util: work around for access(X_OK) under root
On AIX, access(X_OK) may succeed when run as root even if the
execution isn't possible. This behavior is allowed by POSIX
which says:

  ... for a process with appropriate privileges, an implementation
  may indicate success for X_OK even if execute permission is not
  granted to any user.

It can lead hook programs to have their execution refused:

   git commit -m content
   fatal: cannot exec '.git/hooks/pre-commit': Permission denied

Add NEED_ACCESS_ROOT_HANDLER in order to use an access helper function.
It checks with stat if any executable flags is set when the current user
is root.

Signed-off-by: Clément Chigot <clement.chigot@atos.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25 17:49:44 +09:00
Junio C Hamano
604a64641d Makefile: dedup list of files obtained from ls-files
Since 33533975 ("Makefile: ask "ls-files" to list source files if
available", 2011-10-18), we optionally asked "ls-files" to list the
source files that ought to exist, as a faster approximation for
"find" on working tree files.

This works reasonably well, except that it ends up listing the same
path multiple times if the index is unmerged.  Because the original
use of this construct was to name files to run etags over, and the
etags command happily takes the same filename multiple times without
causing any harm, there was no problem (other than perhaps spending
slightly more cycles, but who cares how fast the TAGS file gets
updated).

We however recently added a similar call to "ls-files" to list *.h
files, instead of using "find", in 92b88eba ("Makefile: use `git
ls-files` to list header files, if possible", 2019-03-04).  In this
new use of "ls-files", the resulting list $(LIB_H) is used for,
among other things, generating the header files to run hdr-check
target, and the duplicate unfortunately becomes a true problem.  It
causes $(MAKE) to notice that there are multiple %.hco targets and
complain.

Let the resulting list consumed by $(sort), which deduplicates,
to fix this.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-22 15:14:22 +09:00
Junio C Hamano
27ff787809 Merge branch 'js/check-docs-exe'
Dev support update.

* js/check-docs-exe:
  check-docs: fix for setups where executables have an extension
  check-docs: do not expect guide pages to correspond to commands
  check-docs: really look at the documented commands again
  docs: do not document the `git remote-testgit` command
  docs: move gitremote-helpers into section 7
2019-04-22 11:14:46 +09:00
Junio C Hamano
e36adf7122 Merge branch 'ps/stash-in-c'
"git stash" rewritten in C.

* ps/stash-in-c: (28 commits)
  tests: add a special setup where stash.useBuiltin is off
  stash: optionally use the scripted version again
  stash: add back the original, scripted `git stash`
  stash: convert `stash--helper.c` into `stash.c`
  stash: replace all `write-tree` child processes with API calls
  stash: optimize `get_untracked_files()` and `check_changes()`
  stash: convert save to builtin
  stash: make push -q quiet
  stash: convert push to builtin
  stash: convert create to builtin
  stash: convert store to builtin
  stash: convert show to builtin
  stash: convert list to builtin
  stash: convert pop to builtin
  stash: convert branch to builtin
  stash: convert drop and clear to builtin
  stash: convert apply to builtin
  stash: mention options in `show` synopsis
  stash: add tests for `git stash show` config
  stash: rename test cases to be more descriptive
  ...
2019-04-22 11:14:43 +09:00
Phillip Wood
0609b741a4 rebase -i: combine rebase--interactive.c with rebase.c
In order to run `rebase -i` without forking `rebase--interactive` it
will be convenient to have all the code from rebase--interactive.c in
rebase.c. This is a straight forward copy of the code from
rebase--interactive.c, it will be simplified slightly in the next
commit.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 17:32:10 +09:00
Johannes Schindelin
b7ce24d095 Turn git serve into a test helper
The `git serve` built-in was introduced in ed10cb952d (serve:
introduce git-serve, 2018-03-15) as a backend to serve Git protocol v2,
probably originally intended to be spawned by `git upload-pack`.

However, in the version that the protocol v2 patches made it into core
Git, `git upload-pack` calls the `serve()` function directly instead of
spawning `git serve`; The only reason in life for `git serve` to survive
as a built-in command is to provide a way to test the protocol v2
functionality.

Meaning that it does not even have to be a built-in that is installed
with end-user facing Git installations, but it can be a test helper
instead.

Let's make it so.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:03:24 +09:00
Johannes Schindelin
31cf4a6ba9 check-docs: do not bother checking for legacy scripts' documentation
In the recent years, there has been a big push to convert more and more
of Git's commands that are implemented as scripts to built-ins written
in pure, portable C, for robustness, speed and portability.

One strategy that served us well is to convert those scripts
incrementally, starting by renaming the scripts to
`git-legacy-<command>`, then introducing a built-in that does nothing
else at first than checking the config setting `<command>.useBuiltin`
(which defaults to `false` at the outset) and handing off to the legacy
script if so asked.

Obviously, those `git-legacy-<command>` commands share the documentation
with the built-in `git-<command>`, and are not intended to be called
directly anyway. So let's not try to ensure that they are documented
separately from their built-in versions.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:03:24 +09:00
Johannes Schindelin
faa7a096d8 docs: exclude documentation for commands that have been excluded
When building with certain build options, some commands are excluded
from the build. For example, `git-credential-cache` is skipped when
building with `NO_UNIX_SOCKETS`.

Let's not build or package documentation for those excluded commands.

This issue was pointed out rightfully when running `make check-docs` on
Windows, where we do not yet have Unix sockets, and therefore the
`credential-cache` command is excluded (yet its documentation was built
and shipped).

Note: building the documentation via `make -C Documentation` leaves the
build system with no way to determine which commands have been
excluded. If called thusly, we gracefully fail to exclude their
documentation. Only when building the documentation via the top-level
Makefile will it get excluded properly, or after building
`Documentation/GIT-EXCLUDED-PROGRAMS` manually.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:03:24 +09:00
Johannes Schindelin
7c3bd713b1 check-docs: allow command-list.txt to contain excluded commands
Among other things, the `check-docs` target ensures that
`command-list.txt` no longer contains commands that were dropped (or
that were never added in the first place).

To do so, it compares the list of commands from that file to the
commands listed in `$(ALL_COMMANDS)`.

However, some build options exclude commands from the latter. Fix the
target to handle this situation correctly by taking the just-introduced
`$(EXCLUDED_PROGRAMS)` into account.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:03:24 +09:00
Johannes Schindelin
724d63569f help -a: do not list commands that are excluded from the build
When built with NO_CURL or with NO_UNIX_SOCKETS, some commands are
skipped from the build. It does not make sense to list them in the
output of `git help -a`, so let's just not.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:03:24 +09:00
Johannes Schindelin
0b64e21cc2 Makefile: drop the NO_INSTALL variable
The last user was just removed; There is no longer any need to carry it
around. Should we ever run into a need for it again, it is easy enough
to revert this commit.

It is unlikely, though, that we need `NO_INSTALL` again: as we saw with
the just-removed item, `git-remote-testgit`, we have better locations
to put executables and scripts that we do not want to install, e.g.
a subdirectory in `t/`, or `contrib/`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-19 14:03:24 +09:00
Junio C Hamano
4f3036cfa1 Merge branch 'ab/drop-scripted-rebase'
Retire scripted "git rebase" implementation.

* ab/drop-scripted-rebase:
  rebase: remove the rebase.useBuiltin setting
2019-04-16 19:28:09 +09:00
Jeff Hostetler
bce9db6de9 trace2: use system/global config for default trace2 settings
Teach git to read the system and global config files for
default Trace2 settings.  This allows system-wide Trace2 settings to
be installed and inherited to make it easier to manage a collection of
systems.

The original GIT_TR2* environment variables are loaded afterwards and
can be used to override the system settings.

Only the system and global config files are used.  Repo and worktree
local config files are ignored.  Likewise, the "-c" command line
arguments are also ignored.  These limits are for performance reasons.

(1) For users not using Trace2, there should be minimal overhead to
detect that Trace2 is not enabled.  In particular, Trace2 should not
allocate lots of otherwise unused data strucutres.

(2) For accurate performance measurements, Trace2 should be initialized
as early in the git process as possible, and before most of the normal
git process initialization (which involves discovering the .git directory
and reading a hierarchy of config files).

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-16 13:37:06 +09:00
Johannes Schindelin
5afb2ce4cd remote-testgit: move it into the support directory for t5801
The `git-remote-testgit` script is really only used in
`t5801-remote-helpers.sh`. It does not even contain any `@@<MAGIC>@@`
placeholders that would need to be interpolated via `make
git-remote-testgit`.

Let's just move it to a new home, decluttering the top-level directory
and clarifying that this is just a test helper, not an official Git
command that we would want to ever support.

Suggested by Ævar Arnfjörð Bjarmason.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15 14:30:04 +09:00
Junio C Hamano
c063a537be Merge branch 'jk/sha1dc'
Build update for SHA-1 with collision detection.

* jk/sha1dc:
  Makefile: fix unaligned loads in sha1dc with UBSan
2019-04-10 02:14:26 +09:00
Nguyễn Thái Ngọc Duy
d787d311db checkout: split part of it to new command 'switch'
"git checkout" doing too many things is a source of confusion for many
users (and it even bites old timers sometimes). To remedy that, the
command will be split into two new ones: switch and restore. The good
old "git checkout" command is still here and will be until all (or most
of users) are sick of it.

See the new man page for the final design of switch. The actual
implementation though is still pretty much the same as "git checkout"
and not completely aligned with the man page. Following patches will
adjust their behavior to match the man page.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-02 13:56:59 +09:00
Johannes Schindelin
5ee4246339 check-docs: fix for setups where executables have an extension
On Windows, for example, executables (must) have the extension `.exe`.
Our `check-docs` target was not prepared for that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01 14:01:11 +09:00
Johannes Schindelin
8e6d69591a check-docs: do not expect guide pages to correspond to commands
When we want to see what commands are listed in `command-list.txt` but
not installed, we currently include lines that refer to guides, e.g.
`gitattributes` or `gitcli`.

Let's not include those lines, as they are not referring to commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01 14:01:09 +09:00
Johannes Schindelin
057ccba593 check-docs: really look at the documented commands again
As part of the `check-docs` target, we verify that commands that are
documented are actually in the current list of commands to be built.

However, this logic broke in 5fafce0b78 (check-docs: get documented
command list from Makefile, 2012-08-08), when we tried to make the logic
safer by not looking at the files in the worktree, but at the list of
files to be generated in `Documentation/Makefile`. While this was the
right thing to do, it failed to accommodate for the fact that `make -C
Documentation/ print-man1`, unlike `ls Documentation/*.txt`, would *not*
print lines starting with the prefix `Documentation/`.

At long last, let's fix this.

Note: This went undetected due to a funny side effect of the
`ALL_PROGRAMS` variable starting with a space. That space, together with
the extra space we inserted before `$(ALL_PROGRAMS)` in the

	case " $(ALL_PROGRAMS)" in
	*" $$cmd ")
		[...]

construct, is responsible that this case arm is used when `cmd` is empty
(which was clearly not intended to be the case).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01 14:01:06 +09:00
Junio C Hamano
3cef67694c Merge branch 'ab/makefile-help-devs-more'
CFLAGS now can be tweaked when invoking Make while using
DEVELOPER=YesPlease; this did not work well before.

* ab/makefile-help-devs-more:
  Makefile: allow for combining DEVELOPER=1 and CFLAGS="..."
  Makefile: move the setting of *FLAGS closer to "include"
  Makefile: Move *_LIBS assignment into its own section
  Makefile: add/remove comments at top and tweak whitespace
  Makefile: move "strip" assignment down from flags
  Makefile: remove an out-of-date comment
2019-03-20 15:16:04 +09:00
Ævar Arnfjörð Bjarmason
d03ebd411c rebase: remove the rebase.useBuiltin setting
Remove the rebase.useBuiltin setting, which was added as an escape
hatch to disable the builtin version of rebase first released with Git
2.20.

See [1] for the initial implementation of rebase.useBuiltin, and [2]
and [3] for the documentation and corresponding
GIT_TEST_REBASE_USE_BUILTIN option.

Carrying the legacy version is a maintenance burden as seen in
7e097e27d3 ("legacy-rebase: backport -C<n> and --whitespace=<option>
checks", 2018-11-20) and 9aea5e9286 ("rebase: fix regression in
rebase.useBuiltin=false test mode", 2019-02-13). Since the built-in
version has been shown to be stable enough let's remove the legacy
version.

As noted in [3] having use_builtin_rebase() shell out to get its
config doesn't make any sense anymore, that was done for the purposes
of spawning the legacy rebase without having modified any global
state. Let's instead handle this case in rebase_config().

There's still a bunch of references to git-legacy-rebase in po/*.po,
but those will be dealt with in time by the i18n effort.

Even though this configuration variable only existed two releases
let's not entirely delete the entry from the docs, but note its
absence. Individual versions of git tend to be around for a while due
to distro packaging timelines, so e.g. if we're "lucky" a given
version like 2.21 might be installed on say OSX for half a decade.

That'll mean some people probably setting this in config, and then
when they later wonder if it's needed they can Google search the
config option name or check it in git-config. It also allows us to
refer to the docs from the warning for details.

1. 55071ea248 ("rebase: start implementing it as a builtin",
   2018-08-07)
2. d8d0a546f0 ("rebase doc: document rebase.useBuiltin", 2018-11-14)
3. 62c23938fa ("tests: add a special setup where rebase.useBuiltin is
   off", 2018-11-14)
3. https://public-inbox.org/git/nycvar.QRO.7.76.6.1903141544110.41@tvgsbejvaqbjf.bet/

Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-20 09:25:10 +09:00
Jeff King
07a20f569b Makefile: fix unaligned loads in sha1dc with UBSan
The sha1dc library uses unaligned loads on platforms that support them.
This is normally what you'd want for performance, but it does cause
UBSan to complain when we compile with SANITIZE=undefined. Just like we
set -DNO_UNALIGNED_LOADS for our own code in that case, we should set
-DSHA1DC_FORCE_ALIGNED_ACCESS.

Of course that does nothing without pulling in the patches from sha1dc
to respect that define. So let's do that, too, updating both the
submodule link and our in-tree copy (from the same commit).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-13 13:45:52 +09:00
Junio C Hamano
c0d97d299e Merge branch 'js/find-lib-h-with-ls-files-when-possible'
The Makefile uses 'find' utility to enumerate all the *.h header
files, which is expensive on platforms with slow filesystems; it
now optionally uses "ls-files" if working within a repository,
which is a trick similar to how all sources are enumerated to run
ETAGS on.

* js/find-lib-h-with-ls-files-when-possible:
  Makefile: use `git ls-files` to list header files, if possible
2019-03-11 16:16:25 +09:00
Junio C Hamano
1de413bc1d Merge branch 'rj/hdr-check-gcrypt-fix'
The set of header files used by "make hdr-check" unconditionally
included sha256/gcrypt.h, even when it is not used, causing the
make target to fail.  We now skip it when GCRYPT_SHA256 is not in
use.

* rj/hdr-check-gcrypt-fix:
  Makefile: fix 'hdr-check' when GCRYPT not installed
2019-03-11 16:16:25 +09:00