We spend a lot of time in strbuf_getwholeline in a tight
loop reading characters from a stdio handle into a buffer.
The libc getdelim() function can do this for us with less
overhead. It's in POSIX.1-2008, and was a GNU extension
before that. Therefore we can't rely on it, but can fall
back to the existing getc loop when it is not available.
The HAVE_GETDELIM knob is turned on automatically for Linux,
where we have glibc. We don't need to set any new
feature-test macros, because we already define _GNU_SOURCE.
Other systems that implement getdelim may need to other
macros (probably _POSIX_C_SOURCE >= 200809L), but we can
address that along with setting the Makefile knob after
testing the feature on those systems.
Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):
real 0m8.601s
user 0m8.084s
sys 0m0.524s
to:
real 0m6.768s
user 0m6.340s
sys 0m0.432s
for a wall-clock speedup of 21%.
Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We now detect number of CPUs on older BSD-derived systems.
* km/bsd-sysctl:
thread-utils.c: detect CPU count on older BSD-like systems
configure: support HAVE_BSD_SYSCTL option
On BSD-compatible systems some information such as the number
of available CPUs may only be available via the sysctl function.
Add support for a HAVE_BSD_SYSCTL option complete with autoconf
support and include the sys/syctl.h header when the option is
enabled to make the sysctl function available.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Certain older vintages of cURL give irregular output from
"curl-config --vernum", which confused our build system.
* tc/curl-vernum-output-broken-in-7.11:
Makefile: handle broken curl version number in version check
curl 7.11.0 through 7.12.2 when built from their official release
archives will present a 5 digit version number instead of the documented
6 digits which breaks the version check in the Makefile.
Correct these broken version numbers on the fly when extracting them to
ensure the comparison works correctly.
[jc: shortened the new sed scripts a bit]
Signed-off-by: Tom G. Christensen <tgc@statsbiblioteket.dk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The gettext N_ macro is used to mark strings for translation
without actually translating them. At runtime the string is
expected to be passed to the gettext API for translation.
If two N_ macro invocations appear next to each other with only
whitespace (or nothing at all) between them, the two separate
strings will be marked for translation, but the preprocessor
will then silently combine the strings into one and at runtime
the string passed to gettext will not match the strings that
were translated so no translation will actually occur.
Avoid this by adding parentheses around the expansion of the
N_ macro so that instead of ending up with two adjacent strings
that are then combined by the preprocessor, two adjacent strings
surrounded by parentheses result instead which causes a compile
error so the mistake can be quickly found and corrected.
However, since these string literals are typically assigned to
static variables and not all compilers support parenthesized
string literal assignments, allow this to be controlled by the
Makefile with the default only enabled when the compiler is
known to support the syntax.
For now only __GNUC__ enables this by default which covers both
gcc and clang which should result in early detection of any
adjacent N_ macros.
Although the necessary tests make the affected files a bit less
elegant, the benefit of avoiding propagation of a translation-
marking error to all the translation teams thus creating extra
work for them when the error is eventually detected and fixed
would seem to outweigh the minor inelegance the additional
configuration tests introduce.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
OpenSSL version 0.9.6b and before defined the function HMAC_cleanup.
Newer versions define HMAC_CTX_cleanup. Check for HMAC_CTX_cleanup and
fall back to HMAC_cleanup when the newer function is missing.
Signed-off-by: Reuben Hawkins <reubenhwk@gmail.com>
Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Set or clear Makefile variables HAVE_CLOCK_GETTIME and
HAVE_CLOCK_MONOTONIC based upon results of the checks (overriding
default values from config.mak.uname).
CLOCK_MONOTONIC isn't available on RHEL3, but there are still RHEL3
systems being used in production.
Signed-off-by: Reuben Hawkins <reubenhwk@gmail.com>
Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Newer libCurl knows how to talk IMAP; "git imap-send" has been
updated to use this instead of a hand-rolled OpenSSL calls.
* br/imap-send-via-libcurl:
git-imap-send: use libcurl for implementation
Long overdue departure from the assumption that S_IFMT is shared by
everybody made in 2005.
* dm/compat-s-ifmt-for-zos:
compat: convert modes to use portable file type values
The build procedure did not bother fixing perl and python scripts
when NO_PERL and NO_PYTHON build-time configuration changed.
* jk/rebuild-perl-scripts-with-no-perl-seting-change:
Makefile: have python scripts depend on NO_PYTHON setting
Makefile: simplify by using SCRIPT_{PERL,SH}_GEN macros
Makefile: have perl scripts depend on NO_PERL setting
The build procedure did not bother fixing perl and python scripts
when NO_PERL and NO_PYTHON build-time configuration changed.
* jk/rebuild-perl-scripts-with-no-perl-seting-change:
Makefile: have python scripts depend on NO_PYTHON setting
Makefile: simplify by using SCRIPT_{PERL,SH}_GEN macros
Makefile: have perl scripts depend on NO_PERL setting
This adds simple wrapper functions around calls to stat(), fstat(),
and lstat() that translate the operating system's native file type
bits to those used by most operating systems. It also rewrites the
S_IF* macros to the common values, so all file type processing is
performed using the translated modes. This makes projects portable
across operating systems that use different file type definitions.
Only the file type bits may be affected by these compatibility
functions; the file permission bits are assumed to be 07777 and are
passed through unchanged.
Signed-off-by: David Michael <fedora.dm0@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Like the perl scripts, python scripts need a dependency to ensure they
are rebuilt when switching between the "dummy" versions that run
without Python and the real thing.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
SCRIPT_PERL_GEN is defined as $(patsubst %.perl,%,$(SCRIPT_PERL))
for use in targets like build-perl-script used by makefiles in
subdirectories that override SCRIPT_PERL (see v1.8.2-rc0~17^2,
"git-remote-mediawiki: use toplevel's Makefile", 2013-02-08).
The same expression is used in the rules that actually write the
generated perl scripts, and since these rules were introduced before
SCRIPT_PERL_GEN, they use the longhand instead of that macro. Use the
macro to make reading easier.
Likewise for SCRIPT_SH_GEN. The Python rules already got the same
simplification in v1.8.4-rc0~162^2~8 (2013-05-24).
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If NO_PERL is not set, our perl scripts are built as
usual. If it is set, then we build "dummy" versions that
tell you git was built without perl support and exit
gracefully.
However, if you switch to NO_PERL in a directory with
existing build artifacts, we do not notice that the files
need rebuilt. We see only that they are newer than the
"unimplemented.sh" wrapper and assume they are done. So
doing:
make
make NO_PERL=Nope
would result in a git-add--interactive script that uses perl
(and running the test suite would make use of it).
Instead, we should trigger a rebuild of the perl scripts
anytime NO_PERL changes.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use libcurl's high-level API functions to implement git-imap-send
instead of the previous low-level OpenSSL-based functions.
Since version 7.30.0, libcurl's API has been able to communicate with
IMAP servers. Using those high-level functions instead of the current
ones would reduce imap-send.c by some 1200 lines of code. For now,
the old ones are wrapped in #ifdefs, and the new functions are enabled
by make if curl's version is >= 7.34.0, from which version on curl's
CURLOPT_LOGIN_OPTIONS (enabling IMAP authentication) parameter has been
available. The low-level functions will still be used for tunneling
into the server for now.
As I don't have access to that many IMAP servers, I haven't been able to
test the new code with a wide variety of parameter combinations. I did
test both secure and insecure (imaps:// and imap://) connections and
values of "PLAIN" and "LOGIN" for the authMethod.
In order to suppress a sparse warning about "using sizeof on a
function", we use the same solution used in commit 9371322a6
("sparse: suppress some "using sizeof on a function" warnings",
06-10-2013) which solved exactly this problem for the other commands
using libcurl.
Helped-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Bernhard Reiter <ockham@raz.or.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
z/OS port
* dm/port2zos:
compat/bswap.h: detect endianness from XL C compiler macros
Makefile: reorder linker flags in the git executable rule
git-compat-util.h: support variadic macros with the XL C compiler
The XL C compiler can fail due to mixing library path and object
file arguments, for example when linking git while building with
"gmake LDFLAGS=-L$prefix/lib".
Move the ALL_LDFLAGS variable expansion in the git executable rule
to be consistent with all the other linking rules, namely to have
LDFLAGS such as -L$where before the object files *.o being linked
together.
Signed-off-by: David Michael <fedora.dm0@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A new filter to programatically edit the tail end of the commit log
messages.
* cc/interpret-trailers:
Documentation: add documentation for 'git interpret-trailers'
trailer: add tests for commands in config file
trailer: execute command from 'trailer.<name>.command'
trailer: add tests for "git interpret-trailers"
trailer: add interpret-trailers command
trailer: put all the processing together and print
trailer: parse trailers from file or stdin
trailer: process command line trailer arguments
trailer: read and process config information
trailer: process trailers from input message and arguments
trailer: add data structures and basic functions
This patch adds the "git interpret-trailers" command.
This command uses the previously added process_trailers()
function in trailer.c.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We will use a doubly linked list to store all information
about trailers and their configuration.
This way we can easily remove or add trailers to or from
trailer lists while traversing the lists in either direction.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Jeff King <peff@peff.net>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* tb/crlf-tests:
MinGW: update tests to handle a native eol of crlf
Makefile: propagate NATIVE_CRLF to C
t0027: Tests for core.eol=native, eol=lf, eol=crlf
Admit that keeping LIB_H up-to-date, only for those that do not use
the automatically generated dependencies, is a losing battle, and
make it conservative by making everything depend on anything.
* jk/make-simplify-dependencies:
Makefile: drop CHECK_HEADER_DEPENDENCIES code
Makefile: use `find` to determine static header dependencies
i18n: treat "make pot" as an explicitly-invoked target
Add in-core caching layer to let us avoid reading the same
configuration files number of times.
* ta/config-set:
test-config: add tests for the config_set API
add `config_set` API for caching config-like files
Commit 95f31e9a (convert: The native line-ending is \r\n on MinGW,
2010-09-04) correctly points out that the NATIVE_CRLF setting is
incorrectly set on Mingw git. However, the Makefile variable is not
propagated to the C preprocessor and results in no change. This patch
pushes the definition to the C code and adds a test to validate that
when core.eol as native is crlf, we actually normalize text files to
this line ending convention when core.autocrlf is false.
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This code was useful when we kept a static list of header
files, and it was easy to forget to update it. Since the last
commit, we generate the list dynamically.
Technically this could still be used to find a dependency
that our dynamic check misses (e.g., a header file without a
".h" extension). But that is reasonably unlikely to be
added, and even less likely to be noticed by this tool
(because it has to be run manually)., It is not worth
carrying around the cruft in the Makefile.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most modern platforms will use automatically computed header
dependencies to figure out when a C file needs rebuilt due
to a header changing. With old compilers, however, we
fallback to a static list of header files. If any of them
changes, we recompile everything. This is overly
conservative, but the best we can do on older platforms.
It is unfortunately easy for our static header list to grow
stale, as none of the regular developers make use of it.
Instead of trying to keep it up to date, let's invoke "find"
to generate the list dynamically.
Since we do not use the value $(LIB_H) unless either
COMPUTE_HEADER_DEPENDENCIES is turned on or the user is
building "po/git.pot" (where it comes in via $(LOCALIZED_C),
make is smart enough to not even run this "find" in most
cases. However, we do need to stop using the "immediate"
variable assignment ":=" for $(LOCALIZED_C). That's OK,
because it was not otherwise useful here.
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
po/git.pot is normally used as-is and not regenerated by people
building git, so it is okay if an explicit "make po/git.pot" always
automatically regenerates it. Depend on the magic FORCE target
instead of explicitly keeping track of dependencies.
This simplifies the makefile, in particular preparing for a moment
when $(LIB_H), which is part of $(LOCALIZED_C), can be computed on the
fly. It also fixes a slight breakage in which changes to perl and shell
scripts did not trigger a rebuild of po/git.pot.
We still need a dependency on GENERATED_H, to force those files to be
built when regenerating git.pot.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The perf tests need a repository to operate on; if none is
defined, we fall back to the repository containing our build
directory. That fails, though, for an exported tarball of
git.git, which has no repository.
Since 5d7fd6d we run the perf tests as part of "make
profile". Therefore "make profile" fails out of the box on
released tarballs of v2.1.0.
We can fix this by making the perf tests optional; if they
are skipped, we still run the regular test suite, which
should give a lot of profile data (and is what we used to do
prior to 5d7fd6d anyway).
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Expose the `config_set` C API as a set of simple commands in order to
facilitate testing. Add tests for the `config_set` API as well as for
`git_config_get_*()` family for the usual config files.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* kb/perf-trace:
api-trace.txt: add trace API documentation
progress: simplify performance measurement by using getnanotime()
wt-status: simplify performance measurement by using getnanotime()
git: add performance tracing for git's main() function to debug scripts
trace: add trace_performance facility to debug performance issues
trace: add high resolution timer function to debug performance issues
trace: add 'file:line' to all trace output
trace: move code around, in preparation to file:line output
trace: add current timestamp to all trace output
trace: disable additional trace output for unit tests
trace: add infrastructure to augment trace output with additional info
sha1_file: change GIT_TRACE_PACK_ACCESS logging to use trace API
Documentation/git.txt: improve documentation of 'GIT_TRACE*' variables
trace: improve trace performance
trace: remove redundant printf format attribute
trace: consistently name the format parameter
trace: move trace declarations from cache.h to new trace.h
* ak/profile-feedback-build:
Fix profile feedback with -jN and add profile-fast
Run the perf test suite for profile feedback too
Don't define away __attribute__ on gcc
Use BASIC_FLAGS for profile feedback
An experiment to use two files (the base file and incremental
changes relative to it) to represent the index to reduce I/O cost
of rewriting a large index when only small part of the working tree
changes.
* nd/split-index: (32 commits)
t1700: new tests for split-index mode
t2104: make sure split index mode is off for the version test
read-cache: force split index mode with GIT_TEST_SPLIT_INDEX
read-tree: note about dropping split-index mode or index version
read-tree: force split-index mode off on --index-output
rev-parse: add --shared-index-path to get shared index path
update-index --split-index: do not split if $GIT_DIR is read only
update-index: new options to enable/disable split index mode
split-index: strip pathname of on-disk replaced entries
split-index: do not invalidate cache-tree at read time
split-index: the reading part
split-index: the writing part
read-cache: mark updated entries for split index
read-cache: save deleted entries in split index
read-cache: mark new entries for split index
read-cache: split-index mode
read-cache: save index SHA-1 after reading
entry.c: update cache_changed if refresh_cache is set in checkout_entry()
cache-tree: mark istate->cache_changed on prime_cache_tree()
cache-tree: mark istate->cache_changed on cache tree update
...
Add a getnanotime() function that returns nanoseconds since 01/01/1970 as
unsigned 64-bit integer (i.e. overflows in july 2554). This is easier to
work with than e.g. struct timeval or struct timespec. Basing the timer on
the epoch allows using the results with other time-related APIs.
To simplify adaption to different platforms, split the implementation into
a common getnanotime() and a platform-specific highres_nanos() function.
The common getnanotime() function handles errors, falling back to
gettimeofday() if highres_nanos() isn't implemented or doesn't work.
getnanotime() is also responsible for normalizing to the epoch. The offset
to the system clock is calculated only once on initialization, i.e.
manually setting the system clock has no impact on the timer (except if
the fallback gettimeofday() is in use). Git processes are typically short
lived, so we don't need to handle clock drift.
The highres_nanos() function returns monotonically increasing nanoseconds
relative to some arbitrary point in time (e.g. system boot), or 0 on
failure. Providing platform-specific implementations should be relatively
easy, e.g. adapting to clock_gettime() as defined by the POSIX realtime
extensions is seven lines of code.
This version includes highres_nanos() implementations for:
* Linux: using clock_gettime(CLOCK_MONOTONIC)
* Windows: using QueryPerformanceCounter()
Todo:
* enable clock_gettime() on more platforms
* add Mac OSX version, e.g. using mach_absolute_time + mach_timebase_info
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Profile feedback always failed for me with -jN. The problem
was that there was no implicit ordering between the profile generate
stage and the profile use stage. So some objects in the later stage
would be linked with profile generate objects, and fail due
to the missing -lgcov.
This adds a new profile target that implicitely enforces the
correct ordering by using submakes. Plus a profile-install target
to also install. This is also nicer to type that PROFILE=...
Plus I always run the performance test suite now for the full
profile run.
In addition I also added a profile-fast / profile-fast-install
target the only runs the performance test suite instead of the
whole test suite. This significantly speeds up the profile build,
which was totally dominated by test suite run time. However
it may have less coverage of course.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use BASIC_CFLAGS instead of CFLAGS to set up the profile feedback
option in the Makefile.
This allows still overriding CFLAGS on the make command line
without disabling profile feedback.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit signatures can be verified using "git show -s --show-signature"
or the "%G?" pretty format and parsing the output, which is well suited
for user inspection, but not for scripting.
Provide a command "verify-commit" which is analogous to "verify-tag": It
returns 0 for good signatures and non-zero otherwise, has the gpg output
on stderr and (optionally) the commit object on stdout, sans the
signature, just like "verify-tag" does.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This split-index mode is designed to keep write cost proportional to
the number of changes the user has made, not the size of the work
tree. (Read cost is another matter, to be dealt separately.)
This mode stores index info in a pair of $GIT_DIR/index and
$GIT_DIR/sharedindex.<SHA-1>. sharedindex is large and unchanged over
time while "index" is smaller and updated often. Format details are in
index-format.txt, although not everything is implemented in this
patch.
Shared indexes are not automatically removed, because it's unclear if
the shared index is needed by any (even temporary) indexes by just
looking at it. After a while you'll collect stale shared indexes. The
good news is one shared index is useable for long, until
$GIT_DIR/index becomes too big and sluggish that the new shared index
must be created.
The safest way to clean shared indexes is to turn off split index
mode, so shared files are all garbage, delete them all, then turn on
split index mode again.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update the logic to compute the display width needed for utf8
strings and allow us to more easily maintain the tables used in
that logic.
We may want to let the users choose if codepoints with ambiguous
widths are treated as a double or single width in a follow-up patch.
* tb/unicode-6.3-zero-width:
utf8: make it easier to auto-update git_wcwidth()
utf8.c: use a table for double_width
Enable threaded index-pack on platforms without thread-unsafe
pread() emulation.
* nd/index-pack-one-fd-per-thread:
index-pack: work around thread-unsafe pread()
Instead of running N pair-wise diff-trees when inspecting a
N-parent merge, find the set of paths that were touched by walking
N+1 trees in parallel. These set of paths can then be turned into
N pair-wise diff-tree results to be processed through rename
detections and such. And N=2 case nicely degenerates to the usual
2-way diff-tree, which is very nice.
* ks/tree-diff-nway:
mingw: activate alloca
combine-diff: speed it up, by using multiparent diff tree-walker directly
tree-diff: rework diff_tree() to generate diffs for multiparent cases as well
Portable alloca for Git
tree-diff: reuse base str(buf) memory on sub-tree recursion
tree-diff: no need to call "full" diff_tree_sha1 from show_path()
tree-diff: rework diff_tree interface to be sha1 based
tree-diff: diff_tree() should now be static
tree-diff: remove special-case diff-emitting code for empty-tree cases
tree-diff: simplify tree_entry_pathcmp
tree-diff: show_path prototype is not needed anymore
tree-diff: rename compare_tree_entry -> tree_entry_pathcmp
tree-diff: move all action-taking code out of compare_tree_entry()
tree-diff: don't assume compare_tree_entry() returns -1,0,1
tree-diff: consolidate code for emitting diffs and recursion in one place
tree-diff: show_tree() is not needed
tree-diff: no need to pass match to skip_uninteresting()
tree-diff: no need to manually verify that there is no mode change for a path
combine-diff: move changed-paths scanning logic into its own function
combine-diff: move show_log_first logic/action out of paths scanning
