Commit Graph

93 Commits

Author SHA1 Message Date
Junio C Hamano
ae7e4d4fed Merge branch 'ab/pcre-v2'
Update "perl-compatible regular expression" support to enable JIT
and also allow linking with the newer PCRE v2 library.

* ab/pcre-v2:
  grep: add support for PCRE v2
  grep: un-break building with PCRE >= 8.32 without --enable-jit
  grep: un-break building with PCRE < 8.20
  grep: un-break building with PCRE < 8.32
  grep: add support for the PCRE v1 JIT API
  log: add -P as a synonym for --perl-regexp
  grep: skip pthreads overhead when using one thread
  grep: don't redundantly compile throwaway patterns under threading
2017-06-19 12:38:43 -07:00
Junio C Hamano
b9a7d55d93 Merge branch 'nd/fopen-errors'
We often try to open a file for reading whose existence is
optional, and silently ignore errors from open/fopen; report such
errors if they are not due to missing files.

* nd/fopen-errors:
  mingw_fopen: report ENOENT for invalid file names
  mingw: verify that paths are not mistaken for remote nicknames
  log: fix memory leak in open_next_file()
  rerere.c: move error_errno() closer to the source system call
  print errno when reporting a system call error
  wrapper.c: make warn_on_inaccessible() static
  wrapper.c: add and use fopen_or_warn()
  wrapper.c: add and use warn_on_fopen_errors()
  config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too
  config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD
  clone: use xfopen() instead of fopen()
  use xfopen() in more places
  git_fopen: fix a sparse 'not declared' warning
2017-06-13 13:47:09 -07:00
Ævar Arnfjörð Bjarmason
fb95e2e38d grep: un-break building with PCRE >= 8.32 without --enable-jit
Amend my change earlier in this series ("grep: add support for the
PCRE v1 JIT API", 2017-04-11) to un-break the build on PCRE v1
versions later than 8.31 compiled without --enable-jit.

As explained in that change and a later compatibility change in this
series ("grep: un-break building with PCRE < 8.32", 2017-05-10) the
pcre_jit_exec() function is a faster path to execute the JIT.

Unfortunately there's no compatibility stub for that function compiled
into the library if pcre_config(PCRE_CONFIG_JIT, &ret) would return 0,
and no macro that can be used to check for it, so the only portable
option to support builds without --enable-jit is via a new
NO_LIBPCRE1_JIT=UnfortunatelyYes Makefile option[1].

Another option would be to make the JIT opt-in via
USE_LIBPCRE1_JIT=YesPlease, after all it's not a default option of
PCRE v1.

I think it makes more sense to make it opt-out since even though it's
not a default option, most packagers of PCRE seem to turn it on by
default, with the notable exception of the MinGW package.

Make the MinGW platform work by default by changing the build defaults
to turn on NO_LIBPCRE1_JIT=UnfortunatelyYes. It is the only platform
that turns on USE_LIBPCRE=YesPlease by default, see commit
df5218b4c3 ("config.mak.uname: support MSys2", 2016-01-13) for that
change.

1. "How do I support pcre1 JIT on all
   versions?"  (https://lists.exim.org/lurker/thread/20170601.103148.10253788.en.html)

2. https://github.com/Alexpux/MINGW-packages/blob/master/mingw-w64-pcre/PKGBUILD
   (referenced from "Re: PCRE v2 compile error, was Re: What's cooking
   in git.git (May 2017, #01; Mon, 1)";
   <alpine.DEB.2.20.1705021756530.3480@virtualbox>)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-02 08:29:05 +09:00
Junio C Hamano
8e178ec4d0 config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26 12:33:55 +09:00
Nguyễn Thái Ngọc Duy
e2d90fd1c3 config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD
This variable is added [1] with the assumption that on a sane system,
fopen(<dir>, "r") should return NULL. Linux and FreeBSD do not meet this
expectation while at least Windows and AIX do. Let's make sure they
behave the same way.

I only tested one version on Linux (4.7.0 with glibc 2.22) and
FreeBSD (11.0) but since GNU/kFreeBSD is fbsd kernel with gnu userspace,
I'm pretty sure it shares the same problem.

[1] cba22528fa (Add compat/fopen.c which returns NULL on attempt to open
    directory - 2008-02-08)

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26 12:33:55 +09:00
Ævar Arnfjörð Bjarmason
0e6d899fee config.mak.uname: set NO_REGEX=NeedsStartEnd on AIX
Set the NO_REGEX=NeedsStartEnd Makefile flag by default on AIX.

Since commit 2f8952250a ("regex: add regexec_buf() that can work on a
non NUL-terminated string", 2016-09-21) git has errored out at
compile-time if the regular expression library doesn't support
REG_STARTEND.

While looking through Google search results for the use of NO_REGEX I
found a Chef recipe that set this on AIX[1], looking through the
documentation for the latest version of AIX (7.2, released October
2015) shows that its regexec() doesn't have REG_STARTEND.

1. https://github.com/chef/omnibus-software/commit/e247e36761#diff-3df898345d670979b74acc0bf71d8c47
2. https://www.ibm.com/support/knowledgecenter/ssw_aix_72/com.ibm.aix.basetrf2/regexec.htm

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-04 16:10:51 +09:00
Junio C Hamano
f56a4390ee Merge branch 'rj/remove-unused-mktemp' into maint
Code cleanup.

* rj/remove-unused-mktemp:
  wrapper.c: remove unused gitmkstemps() function
  wrapper.c: remove unused git_mkstemp() function
2017-03-21 15:03:24 -07:00
Junio C Hamano
890d76504c Merge branch 'jh/mingw-openssl-sha1' into maint
Windows port wants to use OpenSSL's implementation of SHA-1
routines, so let them.

* jh/mingw-openssl-sha1:
  mingw: use OpenSSL's SHA-1 routines
2017-03-16 13:56:41 -07:00
Junio C Hamano
033328a5dc Merge branch 'jh/mingw-openssl-sha1'
Windows port wants to use OpenSSL's implementation of SHA-1
routines, so let them.

* jh/mingw-openssl-sha1:
  mingw: use OpenSSL's SHA-1 routines
2017-03-12 23:21:33 -07:00
Ramsay Jones
b2d593a779 wrapper.c: remove unused gitmkstemps() function
The last call to the mkstemps() function was removed in commit 659488326
("wrapper.c: delete dead function git_mkstemps()", 22-04-2016). In order
to support platforms without mkstemps(), this functionality was provided,
along with a Makefile build variable (NO_MKSTEMPS), by the gitmkstemps()
function. Remove the dead code, along with the defunct build machinery.

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-28 11:54:21 -08:00
Jeff Hostetler
2cfc70f0de mingw: use OpenSSL's SHA-1 routines
Use OpenSSL's SHA-1 routines rather than builtin block-sha1 routines.
This improves performance on SHA1 operations on Intel processors.

OpenSSL 1.0.2 has made considerable performance improvements and
support the Intel hardware acceleration features.  See:
https://software.intel.com/en-us/articles/improving-openssl-performance
https://software.intel.com/en-us/articles/intel-sha-extensions

To test this I added/staged a single file in a gigantic
repository having a 450MB index file.  The code in read-cache.c
verifies the header SHA as it reads the index and computes a new
header SHA as it writes out the new index.  Therefore, in this test
the SHA code must process 900MB of data.  Testing was done on an
Intel I7-4770 CPU @ 3.40GHz (Intel64, Family 6, Model 60) CPU.

The block-sha1 version averaged 5.27 seconds.
The OpenSSL    version averaged 4.50 seconds.

================================================================

$ echo xxx >> project.mk
$ time /e/blk_sha/bin/git.exe add project.mk

real    0m5.207s
user    0m0.000s
sys     0m0.250s

$ echo xxx >> project.mk
$ time /e/blk_sha/bin/git.exe add project.mk

real    0m5.362s
user    0m0.015s
sys     0m0.234s

$ echo xxx >> project.mk
$ time /e/blk_sha/bin/git.exe add project.mk

real    0m5.300s
user    0m0.016s
sys     0m0.250s

$ echo xxx >> project.mk
$ time /e/blk_sha/bin/git.exe add project.mk

real    0m5.216s
user    0m0.000s
sys     0m0.250s

================================================================
$ echo xxx >> project.mk
$ time /e/openssl/bin/git.exe add project.mk

real    0m4.431s
user    0m0.000s
sys     0m0.250s

$ echo xxx >> project.mk
$ time /e/openssl/bin/git.exe add project.mk

real    0m4.478s
user    0m0.000s
sys     0m0.265s

$ echo xxx >> project.mk
$ time /e/openssl/bin/git.exe add project.mk

real    0m4.690s
user    0m0.000s
sys     0m0.250s

$ echo xxx >> project.mk
$ time /e/openssl/bin/git.exe add project.mk

real    0m4.420s
user    0m0.000s
sys     0m0.234s

================================================================

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-09 15:40:06 -08:00
Jeff King
1f7c926132 xdiff: drop XDL_FAST_HASH
The xdiff code hashes every line of both sides of a diff,
and then compares those hashes to find duplicates. The
overall performance depends both on how fast we can compute
the hashes, but also on how many hash collisions we see.

The idea of XDL_FAST_HASH is to speed up the hash
computation. But the generated hashes have worse collision
behavior. This means that in some cases it speeds diffs up
(running "git log -p" on git.git improves by ~8% with it),
but in others it can slow things down. One pathological case
saw over a 100x slowdown[1].

There may be a better hash function that covers both
properties, but in the meantime we are better off with the
original hash. It's slightly slower in the common case, but
it has fewer surprising pathological cases.

[1] http://public-inbox.org/git/20141222041944.GA441@peff.net/

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06 13:27:11 -08:00
Junio C Hamano
43a42aa403 Merge branch 'ew/build-time-pager-tweaks'
The build procedure learned PAGER_ENV knob that lists what default
environment variable settings to export for popular pagers.  This
mechanism is used to tweak the default settings to MORE on FreeBSD.

* ew/build-time-pager-tweaks:
  pager: move pager-specific setup into the build
2016-08-08 14:48:44 -07:00
Eric Wong
995bc22d7f pager: move pager-specific setup into the build
Allowing PAGER_ENV to be set at build-time allows us to move
pager-specific knowledge out of our build.  This allows us to
set a better default for FreeBSD more(1), which pretends not to
understand ANSI color escapes if the MORE environment variable
is left empty, but accepts the same variables as less(1)

Originally-from:
 https://public-inbox.org/git/xmqq61piw4yf.fsf@gitster.dls.corp.google.com/

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-04 13:51:02 -07:00
Junio C Hamano
36f99a40a8 Merge branch 'ew/find-perl-on-freebsd-in-local'
Recent FreeBSD stopped making perl available at /usr/bin/perl;
switch the default the built-in path to /usr/local/bin/perl on not
too ancient FreeBSD releases.

* ew/find-perl-on-freebsd-in-local:
  config.mak.uname: correct perl path on FreeBSD
2016-07-28 10:34:44 -07:00
Nguyễn Thái Ngọc Duy
259f22af90 config.mak.uname: correct perl path on FreeBSD
It looks the the symlink /usr/bin/perl (to /usr/local/bin/perl) has
been removed at least on FreeBSD 10.3. See [1] for more information.

[1] https://svnweb.freebsd.org/ports/head/UPDATING?r1=386270&r2=386269&pathrev=386270&diff_format=c

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Helped-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-26 09:43:06 -07:00
Eric Wong
52fcec75ce config.mak.uname: define NEEDS_LIBRT under Linux, for now
My Debian wheezy LTS system is still on glibc 2.13; and LTS
distros may use older glibc, still, so lets not unnecessarily
break things out-of-the-box.

We seem to assume Linux is using glibc in our Makefiles anyways,
so I don't think this will introduce new breakage for users of
alternative libc implementations.

Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-11 11:44:18 -07:00
Karsten Blees
f7f90e0f4f mingw: make isatty() recognize MSYS2's pseudo terminals (/dev/pty*)
MSYS2 emulates pseudo terminals via named pipes, and isatty() returns 0
for such file descriptors. Therefore, some interactive functionality
(such as launching a pager, asking if a failed unlink should be repeated
etc.) doesn't work when run in a terminal emulator that uses MSYS2's
ptys (such as mintty).

However, MSYS2 uses special names for its pty pipes ('msys-*-pty*'),
which allows us to distinguish them from normal piped input / output.

On startup, check if stdin / stdout / stderr are connected to such pipes
using the NtQueryObject API from NTDll.dll. If the names match, adjust
the flags in MSVCRT's ioinfo structure accordingly.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-26 13:12:02 -07:00
Junio C Hamano
1c07e3eaaf Merge branch 'ad/cygwin-wants-rename' into maint
On Cygwin, object creation uses the "create a temporary and then
rename it to the final name" pattern, not "create a temporary,
hardlink it to the final name and then unlink the temporary"
pattern.

This is necessary to use Git on Windows shared directories, and is
already enabled for the MinGW and plain Windows builds.  It also
has been used in Cygwin packaged versions of Git for quite a while.
See http://thread.gmane.org/gmane.comp.version-control.git/291853
($gmane/275680, $gmane/291853).

* ad/cygwin-wants-rename:
  config.mak.uname: Cygwin needs OBJECT_CREATION_USES_RENAMES
2016-05-02 14:24:12 -07:00
Junio C Hamano
074677315c Merge branch 'tb/avoid-gcc-on-darwin-10-6'
* tb/avoid-gcc-on-darwin-10-6:
  Revert "config.mak.uname: use clang for Mac OS X 10.6"
2016-03-21 09:20:13 -07:00
Eric Sunshine
c0ed7590ce Revert "config.mak.uname: use clang for Mac OS X 10.6"
This reverts commit 7b6daf8d2f.

Now that st_add4() has been patched to work around the gcc 4.2.x
compiler crash, revert the sledge-hammer approach of forcing Mac OS X
10.6 to unconditionally use 'clang' rather than the default compiler
(gcc).

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-21 09:20:02 -07:00
Junio C Hamano
2df13639e7 Merge branch 'jc/sane-grep'
Recent versions of GNU grep is pickier than before to decide if a
file is "binary" and refuse to give line-oriented hits when we
expect it to, unless explicitly told with "-a" option.  As our
scripted Porcelains use sane_grep wrapper for line-oriented data,
even when the line may contain non-ASCII payload we took from
end-user data, use "grep -a" to implement sane_grep wrapper when
using an implementation of "grep" that takes the "-a" option.

* jc/sane-grep:
  rebase-i: clarify "is this commit relevant?" test
  sane_grep: pass "-a" if grep accepts it
2016-03-16 13:16:54 -07:00
Junio C Hamano
71b401032b sane_grep: pass "-a" if grep accepts it
Newer versions of GNU grep is reported to be pickier when we feed a
non-ASCII input and break some Porcelain scripts.  As we know we do
not feed random binary file to our own sane_grep wrapper, allow us
to always pass "-a" by setting SANE_TEXT_GREP=-a Makefile variable
to work it around.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-10 15:35:43 -08:00
Torsten Bögershausen
7b6daf8d2f config.mak.uname: use clang for Mac OS X 10.6
Gcc under Mac OX 10.6 throws an internal compiler error:

CC combine-diff.o
    combine-diff.c: In function ‘diff_tree_combined’:
    combine-diff.c:1391: internal compiler error: Segmentation fault

while attempting to build Git at 5b442c4f (tree-diff: catch integer
overflow in combine_diff_path allocation, 2016-02-19).

As clang that ships with the version does not have the same bug,
make Git compile under Mac OS X 10.6 by using clang instead of gcc
to work this around, as it is unlikely that we will see fixed GCC
on that platform.

Later versions of Mac OSX/Xcode only provide clang, and gcc is a
wrapper to it.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-28 16:34:23 -08:00
Johannes Schindelin
b640b77fea mingw: do not trust MSYS2's MinGW gettext.sh
It does not quite work because it produces DOS line endings which the
shell does not like at all.

This lets t0200-gettext-basic.sh, t0204-gettext-reencode-sanity.sh,
t3406-rebase-message.sh, t3903-stash.sh, t7400-submodule-basic.sh,
t7401-submodule-summary.sh, t7406-submodule-update.sh and
t7407-submodule-foreach.sh pass in Git for Windows' SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-26 13:42:59 -08:00
Johannes Schindelin
f9206ce268 mingw: let's use gettext with MSYS2
This solves two problems:

- we now have proper localisation even on Windows

- we sidestep the infamous "BUG: your vsnprintf is broken (returned -1)"
  message when running "git init" (which otherwise prevents the entire
  test suite from running) because libintl.h overrides vsnprintf() with
  libintl_vsnprintf() [*1*]

The latter issue is rather crucial, as *no* test passes in Git for
Windows without this fix.

Footnote *1*: gettext_git=http://git.savannah.gnu.org/cgit/gettext.git
$gettext_git/tree/gettext-runtime/intl/libgnuintl.in.h#n380

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-26 13:42:59 -08:00
Johannes Schindelin
7b40ae86a3 config.mak.uname: supporting 64-bit MSys2
This just makes things compile, the test suite needs extra tender loving
care in addition to this change. We will address these issues in later
commits.

While at it, also allow building MSys2 Git (i.e. a Git that uses MSys2's
POSIX emulation layer).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-13 11:26:35 -08:00
Johannes Schindelin
df5218b4c3 config.mak.uname: support MSys2
For a long time, Git for Windows lagged behind Git's 2.x releases because
the Git for Windows developers wanted to let that big jump coincide with
a well-needed jump away from MSys to MSys2.

To understand why this is such a big issue, it needs to be noted that
many parts of Git are not written in portable C, but instead Git relies
on a POSIX shell and Perl to be available.

To support the scripts, Git for Windows has to ship a minimal POSIX
emulation layer with Bash and Perl thrown in, and when the Git for
Windows effort started in August 2007, this developer settled on using
MSys, a stripped down version of Cygwin. Consequently, the original name
of the project was "msysGit" (which, sadly, caused a *lot* of confusion
because few Windows users know about MSys, and even less care).

To compile the C code of Git for Windows, MSys was used, too: it sports
two versions of the GNU C Compiler: one that links implicitly to the
POSIX emulation layer, and another one that targets the plain Win32 API
(with a few convenience functions thrown in).  Git for Windows'
executables are built using the latter, and therefore they are really
just Win32 programs. To discern executables requiring the POSIX
emulation layer from the ones that do not, the latter are called MinGW
(Minimal GNU for Windows) when the former are called MSys executables.

This reliance on MSys incurred challenges, too, though: some of our
changes to the MSys runtime -- necessary to support Git for Windows
better -- were not accepted upstream, so we had to maintain our own
fork. Also, the MSys runtime was not developed further to support e.g.
UTF-8 or 64-bit, and apart from lacking a package management system
until much later (when mingw-get was introduced), many packages provided
by the MSys/MinGW project lag behind the respective source code
versions, in particular Bash and OpenSSL. For a while, the Git for
Windows project tried to remedy the situation by trying to build newer
versions of those packages, but the situation quickly became untenable,
especially with problems like the Heartbleed bug requiring swift action
that has nothing to do with developing Git for Windows further.

Happily, in the meantime the MSys2 project (https://msys2.github.io/)
emerged, and was chosen to be the base of the Git for Windows 2.x. Just
like MSys, MSys2 is a stripped down version of Cygwin, but it is
actively kept up-to-date with Cygwin's source code.  Thereby, it already
supports Unicode internally, and it also offers the 64-bit support that
we yearned for since the beginning of the Git for Windows project.

MSys2 also ported the Pacman package management system from Arch Linux
and uses it heavily. This brings the same convenience to which Linux
users are used to from `yum` or `apt-get`, and to which MacOSX users are
used to from Homebrew or MacPorts, or BSD users from the Ports system,
to MSys2: a simple `pacman -Syu` will update all installed packages to
the newest versions currently available.

MSys2 is also *very* active, typically providing package updates
multiple times per week.

It still required a two-month effort to bring everything to a state
where Git's test suite passes, many more months until the first official
Git for Windows 2.x was released, and a couple of patches still await
their submission to the respective upstream projects. Yet without MSys2,
the modernization of Git for Windows would simply not have happened.

This commit lays the ground work to supporting MSys2-based Git builds.

Assisted-by: Waldek Maleska <weakcamel@users.github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-13 11:26:24 -08:00
Jeff King
e23a91b047 Makefile: drop D_INO_IN_DIRENT build knob
Now that fsck has dropped its inode-sorting, there are no
longer any users of this knob, and it can go away.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-05 11:08:06 -07:00
Adam Dinwoodie
e53a64b982 config.mak.uname: Cygwin needs OBJECT_CREATION_USES_RENAMES
This is necessary to use Git on Windows shared directories, and is
already enabled for the MinGW and plain Windows builds.

This problem was reported on the Cygwin mailing list at
https://cygwin.com/ml/cygwin/2015-08/msg00102.html (amongst others)
and is being applied as a manual patch to the Cygwin builds until
the patch is taken here.

Reported-by: Peter Rosin <peda@lysator.liu.se>
Signed-off-by: Adam Dinwoodie <adam@dinwoodie.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-07 14:36:45 -07:00
Eric Sunshine
4e3687858e config.mak.uname: Darwin: define HAVE_GETDELIM for modern OS X releases
On Mac OS X, getdelim() first became available with Xcode 4.1[1], which
was released the same day as OS X 10.7 "Lion", so assume getdelim()
availability from 10.7 onward. (As of this writing, OS X is at 10.10
"Yosemite".)

According to Wikipedia[2], 4.1 was also available for download by paying
developers on OS X 10.6 "Snow Leopard", so it's possible that some 10.6
machines may have getdelim(). However, as strbuf's use of getdelim() is
purely an optimization, let's be conservative and assume 10.6 and
earlier lack getdelim().

[1]: Or, possibly with Xcode 4.0, but that version is no longer
     available for download, or not available to non-paying developers,
     so testing is not possible.

[2]: http://en.wikipedia.org/wiki/Xcode

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-03 09:38:04 -07:00
Jeff King
0cc30e0e84 strbuf_getwholeline: use getdelim if it is available
We spend a lot of time in strbuf_getwholeline in a tight
loop reading characters from a stdio handle into a buffer.
The libc getdelim() function can do this for us with less
overhead. It's in POSIX.1-2008, and was a GNU extension
before that. Therefore we can't rely on it, but can fall
back to the existing getc loop when it is not available.

The HAVE_GETDELIM knob is turned on automatically for Linux,
where we have glibc. We don't need to set any new
feature-test macros, because we already define _GNU_SOURCE.
Other systems that implement getdelim may need to other
macros (probably _POSIX_C_SOURCE >= 200809L), but we can
address that along with setting the Makefile knob after
testing the feature on those systems.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):

  real    0m8.601s
  user    0m8.084s
  sys     0m0.524s

to:

  real    0m6.768s
  user    0m6.340s
  sys     0m0.432s

for a wall-clock speedup of 21%.

Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:05 -07:00
Kyle J. McKay
9529080de2 configure: support HAVE_BSD_SYSCTL option
On BSD-compatible systems some information such as the number
of available CPUs may only be available via the sysctl function.

Add support for a HAVE_BSD_SYSCTL option complete with autoconf
support and include the sys/syctl.h header when the option is
enabled to make the sysctl function available.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-10 15:13:25 -07:00
Reuben Hawkins
a6c3c638ac configure.ac: check for clock_gettime and CLOCK_MONOTONIC
Set or clear Makefile variables HAVE_CLOCK_GETTIME and
HAVE_CLOCK_MONOTONIC based upon results of the checks (overriding
default values from config.mak.uname).

CLOCK_MONOTONIC isn't available on RHEL3, but there are still RHEL3
systems being used in production.

Signed-off-by: Reuben Hawkins <reubenhwk@gmail.com>
Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-09 15:33:39 -08:00
Junio C Hamano
77933f4449 Sync with v2.1.4
* maint-2.1:
  Git 2.1.4
  Git 2.0.5
  Git 1.9.5
  Git 1.8.5.6
  fsck: complain about NTFS ".git" aliases in trees
  read-cache: optionally disallow NTFS .git variants
  path: add is_ntfs_dotgit() helper
  fsck: complain about HFS+ ".git" aliases in trees
  read-cache: optionally disallow HFS+ .git variants
  utf8: add is_hfs_dotgit() helper
  fsck: notice .git case-insensitively
  t1450: refactor ".", "..", and ".git" fsck tests
  verify_dotfile(): reject .git case-insensitively
  read-tree: add tests for confusing paths like ".." and ".git"
  unpack-trees: propagate errors adding entries to the index
2014-12-17 11:46:57 -08:00
Junio C Hamano
58f1d950e3 Sync with v2.0.5
* maint-2.0:
  Git 2.0.5
  Git 1.9.5
  Git 1.8.5.6
  fsck: complain about NTFS ".git" aliases in trees
  read-cache: optionally disallow NTFS .git variants
  path: add is_ntfs_dotgit() helper
  fsck: complain about HFS+ ".git" aliases in trees
  read-cache: optionally disallow HFS+ .git variants
  utf8: add is_hfs_dotgit() helper
  fsck: notice .git case-insensitively
  t1450: refactor ".", "..", and ".git" fsck tests
  verify_dotfile(): reject .git case-insensitively
  read-tree: add tests for confusing paths like ".." and ".git"
  unpack-trees: propagate errors adding entries to the index
2014-12-17 11:42:28 -08:00
Junio C Hamano
5e519fb8b0 Sync with v1.9.5
* maint-1.9:
  Git 1.9.5
  Git 1.8.5.6
  fsck: complain about NTFS ".git" aliases in trees
  read-cache: optionally disallow NTFS .git variants
  path: add is_ntfs_dotgit() helper
  fsck: complain about HFS+ ".git" aliases in trees
  read-cache: optionally disallow HFS+ .git variants
  utf8: add is_hfs_dotgit() helper
  fsck: notice .git case-insensitively
  t1450: refactor ".", "..", and ".git" fsck tests
  verify_dotfile(): reject .git case-insensitively
  read-tree: add tests for confusing paths like ".." and ".git"
  unpack-trees: propagate errors adding entries to the index
2014-12-17 11:28:54 -08:00
Junio C Hamano
6898b79721 Sync with v1.8.5.6
* maint-1.8.5:
  Git 1.8.5.6
  fsck: complain about NTFS ".git" aliases in trees
  read-cache: optionally disallow NTFS .git variants
  path: add is_ntfs_dotgit() helper
  fsck: complain about HFS+ ".git" aliases in trees
  read-cache: optionally disallow HFS+ .git variants
  utf8: add is_hfs_dotgit() helper
  fsck: notice .git case-insensitively
  t1450: refactor ".", "..", and ".git" fsck tests
  verify_dotfile(): reject .git case-insensitively
  read-tree: add tests for confusing paths like ".." and ".git"
  unpack-trees: propagate errors adding entries to the index
2014-12-17 11:20:31 -08:00
Johannes Schindelin
2b4c6efc82 read-cache: optionally disallow NTFS .git variants
The point of disallowing ".git" in the index is that we
would never want to accidentally overwrite files in the
repository directory. But this means we need to respect the
filesystem's idea of when two paths are equal. The prior
commit added a helper to make such a comparison for NTFS
and FAT32; let's use it in verify_path().

We make this check optional for two reasons:

  1. It restricts the set of allowable filenames, which is
     unnecessary for people who are not on NTFS nor FAT32.
     In practice this probably doesn't matter, though, as
     the restricted names are rather obscure and almost
     certainly would never come up in practice.

  2. It has a minor performance penalty for every path we
     insert into the index.

This patch ties the check to the core.protectNTFS config
option. Though this is expected to be most useful on Windows,
we allow it to be set everywhere, as NTFS may be mounted on
other platforms. The variable does default to on for Windows,
though.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-17 11:04:45 -08:00
Jeff King
a42643aa8d read-cache: optionally disallow HFS+ .git variants
The point of disallowing ".git" in the index is that we
would never want to accidentally overwrite files in the
repository directory. But this means we need to respect the
filesystem's idea of when two paths are equal. The prior
commit added a helper to make such a comparison for HFS+;
let's use it in verify_path.

We make this check optional for two reasons:

  1. It restricts the set of allowable filenames, which is
     unnecessary for people who are not on HFS+. In practice
     this probably doesn't matter, though, as the restricted
     names are rather obscure and almost certainly would
     never come up in practice.

  2. It has a minor performance penalty for every path we
     insert into the index.

This patch ties the check to the core.protectHFS config
option. Though this is expected to be most useful on OS X,
we allow it to be set everywhere, as HFS+ may be mounted on
other platforms. The variable does default to on for OS X,
though.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-17 11:04:44 -08:00
Junio C Hamano
55b6dffd13 Merge branch 'jc/config-mak-document-darwin-vs-macosx'
* jc/config-mak-document-darwin-vs-macosx:
  config.mak.uname: add hint on uname_R for MacOS X
  config.mak.uname: set NO_APPLE_COMMON_CRYPTO on older systems
2014-09-09 12:54:05 -07:00
Junio C Hamano
9eeff2f681 config.mak.uname: add hint on uname_R for MacOS X
I always have to scratch my head every time I see this cryptic
pattern "[15678]\."; leave a short note to remind the maintainer
and the reviewers.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-15 11:04:59 -07:00
Kyle J. McKay
9c7a0beee0 config.mak.uname: set NO_APPLE_COMMON_CRYPTO on older systems
Older MacOS systems prior to 10.5 do not have the CommonCrypto
support Git uses so set NO_APPLE_COMMON_CRYPTO on those systems.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-15 09:50:18 -07:00
Junio C Hamano
385e171a5b Merge branch 'sk/mingw-uni-fix-more'
Most of these are battle-tested in msysgit and are needed to
complete what has been merged to 'master' already.

* sk/mingw-uni-fix-more:
  Win32: enable color output in Windows cmd.exe
  Win32: patch Windows environment on startup
  Win32: keep the environment sorted
  Win32: use low-level memory allocation during initialization
  Win32: reduce environment array reallocations
  Win32: don't copy the environment twice when spawning child processes
  Win32: factor out environment block creation
  Win32: unify environment function names
  Win32: unify environment case-sensitivity
  Win32: fix environment memory leaks
  Win32: Unicode environment (incoming)
  Win32: Unicode environment (outgoing)
  Revert "Windows: teach getenv to do a case-sensitive search"
  tests: do not pass iso8859-1 encoded parameter
2014-07-30 14:21:09 -07:00
Junio C Hamano
9f2de9c121 Merge branch 'kb/perf-trace'
* kb/perf-trace:
  api-trace.txt: add trace API documentation
  progress: simplify performance measurement by using getnanotime()
  wt-status: simplify performance measurement by using getnanotime()
  git: add performance tracing for git's main() function to debug scripts
  trace: add trace_performance facility to debug performance issues
  trace: add high resolution timer function to debug performance issues
  trace: add 'file:line' to all trace output
  trace: move code around, in preparation to file:line output
  trace: add current timestamp to all trace output
  trace: disable additional trace output for unit tests
  trace: add infrastructure to augment trace output with additional info
  sha1_file: change GIT_TRACE_PACK_ACCESS logging to use trace API
  Documentation/git.txt: improve documentation of 'GIT_TRACE*' variables
  trace: improve trace performance
  trace: remove redundant printf format attribute
  trace: consistently name the format parameter
  trace: move trace declarations from cache.h to new trace.h
2014-07-22 10:59:19 -07:00
Karsten Blees
e96942e821 Win32: fix environment memory leaks
All functions that modify the environment have memory leaks.

Disable gitunsetenv in the Makefile and use env_setenv (via mingw_putenv)
instead (this frees removed environment entries).

Move xstrdup from env_setenv to make_augmented_environ, so that
mingw_putenv no longer copies the environment entries (according to POSIX
[1], "the string [...] shall become part of the environment"). This also
fixes the memory leak in gitsetenv, which expects a POSIX compliant putenv.

[1] http://pubs.opengroup.org/onlinepubs/009695399/functions/putenv.html

Note: This patch depends on taking control of char **environ and having
our own mingw_putenv (both introduced in "Win32: Unicode environment
(incoming)").

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-21 09:32:49 -07:00
Karsten Blees
148d6771bf trace: add high resolution timer function to debug performance issues
Add a getnanotime() function that returns nanoseconds since 01/01/1970 as
unsigned 64-bit integer (i.e. overflows in july 2554). This is easier to
work with than e.g. struct timeval or struct timespec. Basing the timer on
the epoch allows using the results with other time-related APIs.

To simplify adaption to different platforms, split the implementation into
a common getnanotime() and a platform-specific highres_nanos() function.

The common getnanotime() function handles errors, falling back to
gettimeofday() if highres_nanos() isn't implemented or doesn't work.

getnanotime() is also responsible for normalizing to the epoch. The offset
to the system clock is calculated only once on initialization, i.e.
manually setting the system clock has no impact on the timer (except if
the fallback gettimeofday() is in use). Git processes are typically short
lived, so we don't need to handle clock drift.

The highres_nanos() function returns monotonically increasing nanoseconds
relative to some arbitrary point in time (e.g. system boot), or 0 on
failure. Providing platform-specific implementations should be relatively
easy, e.g. adapting to clock_gettime() as defined by the POSIX realtime
extensions is seven lines of code.

This version includes highres_nanos() implementations for:
 * Linux: using clock_gettime(CLOCK_MONOTONIC)
 * Windows: using QueryPerformanceCounter()

Todo:
 * enable clock_gettime() on more platforms
 * add Mac OSX version, e.g. using mach_absolute_time + mach_timebase_info

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-13 21:25:20 -07:00
Junio C Hamano
b0bae7f0e4 Merge branch 'sk/mingw-dirent'
* sk/mingw-dirent:
  Win32 dirent: improve dirent implementation
  Win32 dirent: clarify #include directives
  Win32 dirent: change FILENAME_MAX to MAX_PATH
  Win32 dirent: remove unused dirent.d_reclen member
  Win32 dirent: remove unused dirent.d_ino member
2014-07-09 11:34:27 -07:00
Junio C Hamano
a9041df7ab Merge branch 'nd/index-pack-one-fd-per-thread' into maint
We used to disable threaded "git index-pack" on platforms without
thread-safe pread(); use a different workaround for such
platforms to allow threaded "git index-pack".

* nd/index-pack-one-fd-per-thread:
  index-pack: work around thread-unsafe pread()
2014-06-25 11:47:58 -07:00
Karsten Blees
1d94c403fd Win32 dirent: remove unused dirent.d_ino member
There are no proper inodes on Windows, so remove dirent.d_ino and #define
NO_D_INO_IN_DIRENT in the Makefile (this skips e.g. an ineffective qsort in
fsck.c).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-09 15:10:52 -07:00