Commit Graph

2141 Commits

Author SHA1 Message Date
Jeff King
e951ebca91 Makefile: reformat FIND_SOURCE_FILES
As we add to this in future commits, the formatting is going
to make it harder and harder to read. Let's write it more as
we would in a shell script, putting each logical block on
its own line.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-14 09:54:49 -08:00
Jeff King
1f7c926132 xdiff: drop XDL_FAST_HASH
The xdiff code hashes every line of both sides of a diff,
and then compares those hashes to find duplicates. The
overall performance depends both on how fast we can compute
the hashes, but also on how many hash collisions we see.

The idea of XDL_FAST_HASH is to speed up the hash
computation. But the generated hashes have worse collision
behavior. This means that in some cases it speeds diffs up
(running "git log -p" on git.git improves by ~8% with it),
but in others it can slow things down. One pathological case
saw over a 100x slowdown[1].

There may be a better hash function that covers both
properties, but in the meantime we are better off with the
original hash. It's slightly slower in the common case, but
it has fewer surprising pathological cases.

[1] http://public-inbox.org/git/20141222041944.GA441@peff.net/

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06 13:27:11 -08:00
Junio C Hamano
332fd5655a Merge branch 'ls/macos-update'
Portability update and workaround for builds on recent Mac OS X.

* ls/macos-update:
  travis-ci: disable GIT_TEST_HTTPD for macOS
  Makefile: set NO_OPENSSL on macOS by default
2016-11-11 13:56:30 -08:00
Lars Schneider
f01fe92b82 Makefile: set NO_OPENSSL on macOS by default
Apple removed the OpenSSL header files in macOS 10.11 and above. OpenSSL
was deprecated since macOS 10.7.

Set `NO_OPENSSL` and `APPLE_COMMON_CRYPTO` to `YesPlease` as default for
macOS. It is possible to override this and use OpenSSL by defining
`NO_APPLE_COMMON_CRYPTO`.

Original-patch-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-10 11:10:36 -08:00
Junio C Hamano
25ab004c53 Merge branch 'jk/quarantine-received-objects'
In order for the receiving end of "git push" to inspect the
received history and decide to reject the push, the objects sent
from the sending end need to be made available to the hook and
the mechanism for the connectivity check, and this was done
traditionally by storing the objects in the receiving repository
and letting "git gc" to expire it.  Instead, store the newly
received objects in a temporary area, and make them available by
reusing the alternate object store mechanism to them only while we
decide if we accept the check, and once we decide, either migrate
them to the repository or purge them immediately.

* jk/quarantine-received-objects:
  tmp-objdir: do not migrate files starting with '.'
  tmp-objdir: put quarantine information in the environment
  receive-pack: quarantine objects until pre-receive accepts
  tmp-objdir: introduce API for temporary object directories
  check_connected: accept an env argument
2016-10-17 13:25:20 -07:00
Jeff King
2564d994c9 tmp-objdir: introduce API for temporary object directories
Once objects are added to the object database by a process,
they cannot easily be deleted, as we don't know what other
processes may have started referencing them. We have to
clean them up with git-gc, which will apply the usual
reachability and grace-period checks.

This patch provides an alternative: it helps callers create
a temporary directory inside the object directory, and a
temporary environment which can be passed to sub-programs to
ask them to write there (the original object directory
remains accessible as an alternate of the temporary one).

See tmp-objdir.h for details on the API.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10 13:54:02 -07:00
René Scharfe
a9a884aea5 coccicheck: use --all-includes by default
Add a make variable, SPATCH_FLAGS, for specifying flags for spatch, and
set it to --all-includes by default.  This option lets it consider
header files which would otherwise be ignored.  That's important for
some rules that rely on type information.  It doubles the duration of
coccicheck, however.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-29 20:40:18 -07:00
Junio C Hamano
6a67695268 Merge branch 'js/regexec-buf'
Some codepaths in "git diff" used regexec(3) on a buffer that was
mmap(2)ed, which may not have a terminating NUL, leading to a read
beyond the end of the mapped region.  This was fixed by introducing
a regexec_buf() helper that takes a <ptr,len> pair with REG_STARTEND
extension.

* js/regexec-buf:
  regex: use regexec_buf()
  regex: add regexec_buf() that can work on a non NUL-terminated string
  regex: -G<pattern> feeds a non NUL-terminated string to regexec() and fails
2016-09-26 16:09:19 -07:00
Junio C Hamano
85f34a929d Merge branch 'rs/cocci'
Code cleanup.

* rs/cocci:
  use strbuf_addstr() for adding constant strings to a strbuf, part 2
  add coccicheck make target
  contrib/coccinelle: fix semantic patch for oid_to_hex_r()
2016-09-26 16:09:14 -07:00
Johannes Schindelin
2f8952250a regex: add regexec_buf() that can work on a non NUL-terminated string
We just introduced a test that demonstrates that our sloppy use of
regexec() on a mmap()ed area can result in incorrect results or even
hard crashes.

So what we need to fix this is a function that calls regexec() on a
length-delimited, rather than a NUL-terminated, string.

Happily, there is an extension to regexec() introduced by the NetBSD
project and present in all major regex implementation including
Linux', MacOSX' and the one Git includes in compat/regex/: by using
the (non-POSIX) REG_STARTEND flag, it is possible to tell the
regexec() function that it should only look at the offsets between
pmatch[0].rm_so and pmatch[0].rm_eo.

That is exactly what we need.

Since support for REG_STARTEND is so widespread by now, let's just
introduce a helper function that always uses it, and tell people
on a platform whose regex library does not support it to use the
one from our compat/regex/ directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-21 13:56:15 -07:00
Junio C Hamano
81358dc238 Merge branch 'cc/apply-am'
"git am" has been taught to make an internal call to "git apply"'s
innards without spawning the latter as a separate process.

* cc/apply-am: (41 commits)
  builtin/am: use apply API in run_apply()
  apply: learn to use a different index file
  apply: pass apply state to build_fake_ancestor()
  apply: refactor `git apply` option parsing
  apply: change error_routine when silent
  usage: add get_error_routine() and get_warn_routine()
  usage: add set_warn_routine()
  apply: don't print on stdout in verbosity_silent mode
  apply: make it possible to silently apply
  apply: use error_errno() where possible
  apply: make some parsing functions static again
  apply: move libified code from builtin/apply.c to apply.{c,h}
  apply: rename and move opt constants to apply.h
  builtin/apply: rename option parsing functions
  builtin/apply: make create_one_file() return -1 on error
  builtin/apply: make try_create_file() return -1 on error
  builtin/apply: make write_out_results() return -1 on error
  builtin/apply: make write_out_one_result() return -1 on error
  builtin/apply: make create_file() return -1 on error
  builtin/apply: make add_index_file() return -1 on error
  ...
2016-09-19 13:47:18 -07:00
René Scharfe
63f0a758a0 add coccicheck make target
Provide a simple way to run Coccinelle against all source files, in the
form of a Makefile target.  Running "make coccicheck" applies each
.cocci file in contrib/coccinelle/ on all source files.  It generates
a .patch file for each .cocci file, containing the actual changes for
effecting the transformations described by the semantic patches.

Non-empty .patch files are reported.  They can be applied to the work
tree using "patch -p0", but should be checked to e.g. make sure they
don't screw up formatting or create circular references.

Coccinelle's diagnostic output (stderr) is piped into .log files.

Linux has a much more elaborate make target of the same name; let's
start nice and easy.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-15 12:23:37 -07:00
Junio C Hamano
27853a85ed Merge branch 'rs/compat-strdup'
* rs/compat-strdup:
  compat: move strdup(3) replacement to its own file
2016-09-12 15:34:36 -07:00
Junio C Hamano
faacc8efe5 Merge branch 'jk/common-main' into maint
There are certain house-keeping tasks that need to be performed at
the very beginning of any Git program, and programs that are not
built-in commands had to do them exactly the same way as "git"
potty does.  It was easy to make mistakes in one-off standalone
programs (like test helpers).  A common "main()" function that
calls cmd_main() of individual program has been introduced to
make it harder to make mistakes.

* jk/common-main:
  mingw: declare main()'s argv as const
  common-main: call git_setup_gettext()
  common-main: call restore_sigpipe_to_default()
  common-main: call sanitize_stdfds()
  common-main: call git_extract_argv0_path()
  add an extra level of indirection to main()
2016-09-08 21:35:51 -07:00
René Scharfe
ca2baa3f75 compat: move strdup(3) replacement to its own file
Move our implementation of strdup(3) out of compat/nedmalloc/ and
allow it to be used independently from USE_NED_ALLOCATOR.  The
original nedmalloc doesn't come with strdup() and doesn't need it.
Only _users_ of nedmalloc need it, which was added when we imported
it to our compat/ hierarchy.

This reduces the difference of our copy of nedmalloc from the
original, making it easier to update, and allows for easier testing
and reusing of our version of strdup().

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-07 10:41:45 -07:00
Christian Couder
bb493a5c14 builtin/apply: move init_apply_state() to apply.c
To libify `git apply` functionality we must make init_apply_state()
usable outside "builtin/apply.c".

Let's do that by moving it into a new "apply.c".

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-11 12:41:47 -07:00
Junio C Hamano
43a42aa403 Merge branch 'ew/build-time-pager-tweaks'
The build procedure learned PAGER_ENV knob that lists what default
environment variable settings to export for popular pagers.  This
mechanism is used to tweak the default settings to MORE on FreeBSD.

* ew/build-time-pager-tweaks:
  pager: move pager-specific setup into the build
2016-08-08 14:48:44 -07:00
Junio C Hamano
78849622ec Merge branch 'jk/pack-objects-optim'
"git pack-objects" has a few options that tell it not to pack
objects found in certain packfiles, which require it to scan .idx
files of all available packs.  The codepaths involved in these
operations have been optimized for a common case of not having any
non-local pack and/or any .kept pack.

* jk/pack-objects-optim:
  pack-objects: compute local/ignore_pack_keep early
  pack-objects: break out of want_object loop early
  find_pack_entry: replace last_found_pack with MRU cache
  add generic most-recently-used list
  sha1_file: drop free_pack_by_name
  t/perf: add tests for many-pack scenarios
2016-08-08 14:48:39 -07:00
Junio C Hamano
970994deb1 Merge branch 'nd/test-helpers' into maint
Build clean-up.

* nd/test-helpers:
  t/test-lib.sh: fix running tests with --valgrind
  Makefile: use VCSSVN_LIB to refer to svn library
  Makefile: drop extra dependencies for test helpers
2016-08-08 14:21:43 -07:00
Eric Wong
995bc22d7f pager: move pager-specific setup into the build
Allowing PAGER_ENV to be set at build-time allows us to move
pager-specific knowledge out of our build.  This allows us to
set a better default for FreeBSD more(1), which pretends not to
understand ANSI color escapes if the MORE environment variable
is left empty, but accepts the same variables as less(1)

Originally-from:
 https://public-inbox.org/git/xmqq61piw4yf.fsf@gitster.dls.corp.google.com/

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-04 13:51:02 -07:00
Jeff King
002f206faf add generic most-recently-used list
There are a few places in Git that would benefit from a fast
most-recently-used cache (e.g., the list of packs, which we
search linearly but would like to order based on locality).
This patch introduces a generic list that can be used to
store arbitrary pointers in most-recently-used order.

The implementation is just a doubly-linked list, where
"marking" an item as used moves it to the front of the list.
Insertion and marking are O(1), and iteration is O(n).

There's no lookup support provided; if you need fast
lookups, you are better off with a different data structure
in the first place.

There is also no deletion support. This would not be hard to
do, but it's not necessary for handling pack structs, which
are created and never removed.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-29 11:05:07 -07:00
Junio C Hamano
ae9ca20c85 Merge branch 'nd/test-helpers'
Build clean-up.

* nd/test-helpers:
  t/test-lib.sh: fix running tests with --valgrind
  Makefile: use VCSSVN_LIB to refer to svn library
  Makefile: drop extra dependencies for test helpers
2016-07-25 14:13:42 -07:00
Junio C Hamano
937be62993 Merge branch 'rw/make-needs-librt'
Makefile assumed that -lrt is always available on platforms that
want to use clock_gettime() and CLOCK_MONOTONIC, which is not a
case for recent Mac OS X.  The necessary symbols are often found in
libc on many modern systems and having -lrt on the command line, as
long as the library exists, had no effect, but when the platform
removes librt.a that is a different matter--having -lrt will break
the linkage.

This change could be seen as a regression for those who do need to
specify -lrt, as they now specifically ask for NEEDS_LIBRT when
building. Hopefully they are in the minority these days.

* rw/make-needs-librt:
  config.mak.uname: define NEEDS_LIBRT under Linux, for now
  Makefile: add NEEDS_LIBRT to optionally link with librt
2016-07-25 14:13:36 -07:00
Junio C Hamano
87492cb24d Merge branch 'mh/ref-iterators'
The API to iterate over all the refs (i.e. for_each_ref(), etc.)
has been revamped.

* mh/ref-iterators:
  for_each_reflog(): reimplement using iterators
  dir_iterator: new API for iterating over a directory tree
  for_each_reflog(): don't abort for bad references
  do_for_each_ref(): reimplement using reference iteration
  refs: introduce an iterator interface
  ref_resolves_to_object(): new function
  entry_resolves_to_object(): rename function from ref_resolves_to_object()
  get_ref_cache(): only create an instance if there is a submodule
  remote rm: handle symbolic refs correctly
  delete_refs(): add a flags argument
  refs: use name "prefix" consistently
  do_for_each_ref(): move docstring to the header file
  refs: remove unnecessary "extern" keywords
2016-07-25 14:13:33 -07:00
Junio C Hamano
d4c6375fd8 Merge branch 'jk/common-main'
There are certain house-keeping tasks that need to be performed at
the very beginning of any Git program, and programs that are not
built-in commands had to do them exactly the same way as "git"
potty does.  It was easy to make mistakes in one-off standalone
programs (like test helpers).  A common "main()" function that
calls cmd_main() of individual program has been introduced to
make it harder to make mistakes.

* jk/common-main:
  mingw: declare main()'s argv as const
  common-main: call git_setup_gettext()
  common-main: call restore_sigpipe_to_default()
  common-main: call sanitize_stdfds()
  common-main: call git_extract_argv0_path()
  add an extra level of indirection to main()
2016-07-19 13:22:19 -07:00
Junio C Hamano
2703572b3a Merge branch 'va/i18n-even-more'
More markings of messages for i18n, with updates to various tests
to pass GETTEXT_POISON tests.

One patch from the original submission dropped due to conflicts
with jk/upload-pack-hook, which is still in flux.

* va/i18n-even-more: (38 commits)
  t5541: become resilient to GETTEXT_POISON
  i18n: branch: mark comment when editing branch description for translation
  i18n: unmark die messages for translation
  i18n: submodule: escape shell variables inside eval_gettext
  i18n: submodule: join strings marked for translation
  i18n: init-db: join message pieces
  i18n: remote: allow translations to reorder message
  i18n: remote: mark URL fallback text for translation
  i18n: standardise messages
  i18n: sequencer: add period to error message
  i18n: merge: change command option help to lowercase
  i18n: merge: mark messages for translation
  i18n: notes: mark options for translation
  i18n: notes: mark strings for translation
  i18n: transport-helper.c: change N_() call to _()
  i18n: bisect: mark strings for translation
  t5523: use test_i18ngrep for negation
  t4153: fix negated test_i18ngrep call
  t9003: become resilient to GETTEXT_POISON
  tests: unpack-trees: update to use test_i18n* functions
  ...
2016-07-13 11:24:10 -07:00
Ronald Wampler
d19e3a5b21 Makefile: add NEEDS_LIBRT to optionally link with librt
We unconditionally link with librt, when HAVE_CLOCK_GETTIME is defined.
But clock_gettime() has been available in most libc implementations for
some time now (e.g., for glibc since version 2.17) and no longer
requires linking with librt. Furthermore, commit a6c3c63 (configure.ac:
check for clock_gettime() and CLOCK_MONOTONIC) will automatically
determined which library (libc or librt) is required for linking when
checking for clock_gettime().

The assumption to unconditionally link with librt was OK, since either
almost every Unix-like system provides a version of librt for backwards
compatibility or other systems, namely Windows or OS X, never provided
clock_gettime(). However, in the latest release of OS X (macOS Sierra),
this function has been added to OS X libc version. As a result, when
running the configuration script, HAVE_CLOCK_GETTIME is set and since
librt is not present, it causes a linker error.

This patches requires those not building via the configuration scripts
to define NEEDS_LIBRT in addition to HAVE_CLOCK_GETTIME, if needed.

Signed-off-by: Ronald Wampler <rdwampler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-07 14:15:08 -07:00
Jeff King
4df7c8a037 Makefile: use VCSSVN_LIB to refer to svn library
We have an abstracted variable; let's use it consistently.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 11:51:03 -07:00
Jeff King
1be36b60f1 Makefile: drop extra dependencies for test helpers
A few test-helpers have Makefile dependencies on specific
object files. But since these files are part of libgit.a
(which all of the helpers link against), the inclusion is
simply redundant.

These were once necessary, but became redundant due to
5c5ba73 (Makefile: Use generic rule to build test programs,
2007-05-31), which added the $(GITLIBS) dependency (but
didn't prune the extra dependency lines). Later commits then
cargo-culted the practice (e.g., b4285c7).

Note that we _do_ need to leave the dependencies on the svn
library, as that is not part of the usual link command.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06 11:50:48 -07:00
Junio C Hamano
de61cebde7 Merge branch 'jk/common-main-2.8' into jk/common-main
* jk/common-main-2.8:
  mingw: declare main()'s argv as const
  common-main: call git_setup_gettext()
  common-main: call restore_sigpipe_to_default()
  common-main: call sanitize_stdfds()
  common-main: call git_extract_argv0_path()
  add an extra level of indirection to main()
2016-07-06 10:02:57 -07:00
Jeff King
3f2e2297b9 add an extra level of indirection to main()
There are certain startup tasks that we expect every git
process to do. In some cases this is just to improve the
quality of the program (e.g., setting up gettext()). In
others it is a requirement for using certain functions in
libgit.a (e.g., system_path() expects that you have called
git_extract_argv0_path()).

Most commands are builtins and are covered by the git.c
version of main(). However, there are still a few external
commands that use their own main(). Each of these has to
remember to include the correct startup sequence, and we are
not always consistent.

Rather than just fix the inconsistencies, let's make this
harder to get wrong by providing a common main() that can
run this standard startup.

We basically have two options to do this:

 - the compat/mingw.h file already does something like this by
   adding a #define that replaces the definition of main with a
   wrapper that calls mingw_startup().

   The upside is that the code in each program doesn't need
   to be changed at all; it's rewritten on the fly by the
   preprocessor.

   The downside is that it may make debugging of the startup
   sequence a bit more confusing, as the preprocessor is
   quietly inserting new code.

 - the builtin functions are all of the form cmd_foo(),
   and git.c's main() calls them.

   This is much more explicit, which may make things more
   obvious to somebody reading the code. It's also more
   flexible (because of course we have to figure out _which_
   cmd_foo() to call).

   The downside is that each of the builtins must define
   cmd_foo(), instead of just main().

This patch chooses the latter option, preferring the more
explicit approach, even though it is more invasive. We
introduce a new file common-main.c, with the "real" main. It
expects to call cmd_main() from whatever other objects it is
linked against.

We link common-main.o against anything that links against
libgit.a, since we know that such programs will need to do
this setup. Note that common-main.o can't actually go inside
libgit.a, as the linker would not pick up its main()
function automatically (it has no callers).

The rest of the patch is just adjusting all of the various
external programs (mostly in t/helper) to use cmd_main().
I've provided a global declaration for cmd_main(), which
means that all of the programs also need to match its
signature. In particular, many functions need to switch to
"const char **" instead of "char **" for argv. This effect
ripples out to a few other variables and functions, as well.

This makes the patch even more invasive, but the end result
is much better. We should be treating argv strings as const
anyway, and now all programs conform to the same signature
(which also matches the way builtins are defined).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01 15:09:10 -07:00
Michael Haggerty
0fe5043dad dir_iterator: new API for iterating over a directory tree
The iterator interface is modeled on that for references, though no
vtable is necessary because there is (so far?) only one type of
dir_iterator.

There are obviously a lot of features that could easily be added to this
class:

* Skip/include directory paths in the iteration
* Shallow/deep iteration
* Letting the caller decide which subdirectories to recurse into (e.g.,
  via a dir_iterator_advance_into() function)
* Option to iterate in sorted order
* Option to iterate over directory paths before vs. after their contents

But these are not needed for the current patch series, so I refrain.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-20 11:38:21 -07:00
Michael Haggerty
3bc581b940 refs: introduce an iterator interface
Currently, the API for iterating over references is via a family of
for_each_ref()-type functions that invoke a callback function for each
selected reference. All of these eventually call do_for_each_ref(),
which knows how to do one thing: iterate in parallel through two
ref_caches, one for loose and one for packed refs, giving loose
references precedence over packed refs. This is rather complicated code,
and is quite specialized to the files backend. It also requires callers
to encapsulate their work into a callback function, which often means
that they have to define and use a "cb_data" struct to manage their
context.

The current design is already bursting at the seams, and will become
even more awkward in the upcoming world of multiple reference storage
backends:

* Per-worktree vs. shared references are currently handled via a kludge
  in git_path() rather than iterating over each part of the reference
  namespace separately and merging the results. This kludge will cease
  to work when we have multiple reference storage backends.

* The current scheme is inflexible. What if we sometimes want to bypass
  the ref_cache, or use it only for packed or only for loose refs? What
  if we want to store symbolic refs in one type of storage backend and
  non-symbolic ones in another?

In the future, each reference backend will need to define its own way of
iterating over references. The crux of the problem with the current
design is that it is impossible to compose for_each_ref()-style
iterations, because the flow of control is owned by the for_each_ref()
function. There is nothing that a caller can do but iterate through all
references in a single burst, so there is no way for it to interleave
references from multiple backends and present the result to the rest of
the world as a single compound backend.

This commit introduces a new iteration primitive for references: a
ref_iterator. A ref_iterator is a polymorphic object that a reference
storage backend can be asked to instantiate. There are three functions
that can be applied to a ref_iterator:

* ref_iterator_advance(): move to the next reference in the iteration
* ref_iterator_abort(): end the iteration before it is exhausted
* ref_iterator_peel(): peel the reference currently being looked at

Iterating using a ref_iterator leaves the flow of control in the hands
of the caller, which means that ref_iterators from multiple
sources (e.g., loose and packed refs) can be composed and presented to
the world as a single compound ref_iterator.

It also means that the backend code for implementing reference iteration
will sometimes be more complicated. For example, the
cache_ref_iterator (which iterates over a ref_cache) can't use the C
stack to recurse; instead, it must manage its own stack internally as
explicit data structures. There is also a lot of boilerplate connected
with object-oriented programming in C.

Eventually, end-user callers will be able to be written in a more
natural way—managing their own flow of control rather than having to
work via callbacks. Since there will only be a few reference backends
but there are many consumers of this API, this is a good tradeoff.

More importantly, we gain composability, and especially the possibility
of writing interchangeable parts that can work with any ref_iterator.

For example, merge_ref_iterator implements a generic way of merging the
contents of any two ref_iterators. It is used to merge loose + packed
refs as part of the implementation of the files_ref_iterator. But it
will also be possible to use it to merge other pairs of reference
sources (e.g., per-worktree vs. shared refs).

Another example is prefix_ref_iterator, which can be used to trim a
prefix off the front of reference names before presenting them to the
caller (e.g., "refs/heads/master" -> "master").

In this patch, we introduce the iterator abstraction and many utilities,
and implement a reference iterator for the files ref storage backend.
(I've written several other obvious utilities, for example a generic way
to filter references being iterated over. These will probably be useful
in the future. But they are not needed for this patch series, so I am
not including them at this time.)

In a moment we will rewrite do_for_each_ref() to work via reference
iterators (allowing some special-purpose code to be discarded), and do
something similar for reflogs. In future patch series, we will expose
the ref_iterator abstraction in the public refs API so that callers can
use it directly.

Implementation note: I tried abstracting this a layer further to allow
generic iterators (over arbitrary types of objects) and generic
utilities like a generic merge_iterator. But the implementation in C was
very cumbersome, involving (in my opinion) too much boilerplate and too
much unsafe casting, some of which would have had to be done on the
caller side. However, I did put a few iterator-related constants in a
top-level header file, iterator.h, as they will be useful in a moment to
implement iteration over directory trees and possibly other types of
iterators in the future.

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-20 11:38:20 -07:00
Vasco Almeida
9588c52b75 i18n: rebase-interactive: mark strings for translation
Mark strings in git-rebase--interactive.sh for translation. There is no
need to source git-sh-i18n since git-rebase.sh already does so.

Add git-rebase--interactive.sh to LOCALIZED_SH in Makefile in order to
enable extracting strings marked for translation by xgettext.

Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-17 15:45:48 -07:00
Vasco Almeida
d323c6b641 i18n: git-sh-setup.sh: mark strings for translation
Positional arguments, such as $0, $1, etc, need to be stored on shell
variables for use in translatable strings, according to gettext manual
[1].

Add git-sh-setup.sh to LOCALIZED_SH variable in Makefile to enable
extraction of string marked for translation by xgettext.

Source git-sh-i18n in git-sh-setup.sh for gettext support.
git-sh-setup.sh is a shell library to be sourced by other shell scripts.
In order to avoid other scripts from sourcing git-sh-i18n twice, remove
line that sources it from them.  Not sourcing git-sh-i18n in any script
that uses gettext would lead to failure due to, for instance, gettextln
not being found.

[1] http://www.gnu.org/software/gettext/manual/html_node/Preparing-Shell-Scripts.html

Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-17 15:45:48 -07:00
Junio C Hamano
ec5ad66ee2 Merge branch 'mm/makefile-developer-can-be-in-config-mak'
"make DEVELOPER=1" worked as expected; setting DEVELOPER=1 in
config.mak didn't.

* mm/makefile-developer-can-be-in-config-mak:
  Makefile: add $(DEVELOPER_CFLAGS) variable
  Makefile: move 'ifdef DEVELOPER' after config.mak* inclusion
2016-06-03 14:38:02 -07:00
Matthieu Moy
51dd3e81d4 Makefile: add $(DEVELOPER_CFLAGS) variable
This does not change the behavior, but allows the user to tweak
DEVELOPER_CFLAGS on the command-line or in a config.mak* file if
needed.

This also makes the code somewhat cleaner as it follows the pattern

<initialisation of variables>
<include statements>
<actual build logic>

by specifying which flags to activate in the first part, and actually
activating them in the last one.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 08:17:15 -07:00
Matthieu Moy
d615628c35 Makefile: move 'ifdef DEVELOPER' after config.mak* inclusion
The DEVELOPER knob was introduced in 658df95 (add DEVELOPER makefile
knob to check for acknowledged warnings, 2016-02-25), and works well
when used as "make DEVELOPER=1", and when the configure script was not
used.

However, the advice given in CodingGuidelines to add DEVELOPER=1 to
config.mak does not: config.mak is included after testing for
DEVELOPER in the Makefile, and at least GNU Make's manual specifies
"Conditional directives are parsed immediately", hence the config.mak
declaration is not visible at the time the conditional is evaluated.

Also, when using the configure script to generate a
config.mak.autogen, the later file contained a "CFLAGS = <flags>"
initialization, which overrode the "CFLAGS += -W..." triggered by
DEVELOPER.

This patch fixes both issues.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31 10:01:51 -07:00
Junio C Hamano
b262b8f889 Merge branch 'va/i18n-misc-updates' into maint
Mark several messages for translation.

* va/i18n-misc-updates:
  i18n: unpack-trees: avoid substituting only a verb in sentences
  i18n: builtin/pull.c: split strings marked for translation
  i18n: builtin/pull.c: mark placeholders for translation
  i18n: git-parse-remote.sh: mark strings for translation
  i18n: branch: move comment for translators
  i18n: branch: unmark string for translation
  i18n: builtin/rm.c: remove a comma ',' from string
  i18n: unpack-trees: mark strings for translation
  i18n: builtin/branch.c: mark option for translation
  i18n: index-pack: use plural string instead of normal one
2016-05-26 13:17:20 -07:00
Junio C Hamano
fa4f29b8a8 Merge branch 'jc/doc-lint'
Find common mistakes when writing gitlink: in our documentation and
drive the check from "make check-docs".

I am not entirely happy with the way the script chooses what input
file to validate, but it is not worse than not having anything, so
let's move it forward and have the logic improved later when people
care about it deeply.

* jc/doc-lint:
  ci: validate "linkgit:" in documentation
2016-05-23 14:54:34 -07:00
Junio C Hamano
e5e7a9115d Merge branch 'va/i18n-misc-updates'
Mark several messages for translation.

* va/i18n-misc-updates:
  i18n: unpack-trees: avoid substituting only a verb in sentences
  i18n: builtin/pull.c: split strings marked for translation
  i18n: builtin/pull.c: mark placeholders for translation
  i18n: git-parse-remote.sh: mark strings for translation
  i18n: branch: move comment for translators
  i18n: branch: unmark string for translation
  i18n: builtin/rm.c: remove a comma ',' from string
  i18n: unpack-trees: mark strings for translation
  i18n: builtin/branch.c: mark option for translation
  i18n: index-pack: use plural string instead of normal one
2016-05-17 14:38:23 -07:00
Junio C Hamano
ab81411ced ci: validate "linkgit:" in documentation
It is easy to add incorrect "linkgit:<page>[<section>]" references
to our documentation suite.  Catch these common classes of errors:

 * Referring to Documentation/<page>.txt that does not exist.

 * Referring to a <page> outside the Git suite.  In general, <page>
   must begin with "git".

 * Listing the manual <section> incorrectly.  The first line of the
   Documentation/<page>.txt must end with "(<section>)".

with a new script "ci/lint-gitlink", and drive it from "make check-docs".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-10 11:15:04 -07:00
Junio C Hamano
a0c9cf51c0 Merge branch 'ky/imap-send-openssl-1.1.0' into maint
Upcoming OpenSSL 1.1.0 will break compilation b updating a few APIs
we use in imap-send, which has been adjusted for the change.

* ky/imap-send-openssl-1.1.0:
  configure: remove checking for HMAC_CTX_cleanup
  imap-send: avoid deprecated TLSv1_method()
  imap-send: check NULL return of SSL_CTX_new()
  imap-send: use HMAC() function provided by OpenSSL
2016-05-06 14:53:24 -07:00
Junio C Hamano
54b0ac57ab Merge branch 'jc/drop-git-spec-in'
As nobody maintains our in-tree git.spec.in and distros use their
own spec file, we stopped pretending that we support "make rpm".

* jc/drop-git-spec-in:
  Makefile: remove dependency on git.spec
  Makefile: stop pretending to support rpmbuild
2016-05-06 14:45:44 -07:00
Junio C Hamano
98eef48257 Merge branch 'jc/makefile-redirection-stderr' into maint
A minor fix in the Makefile.

* jc/makefile-redirection-stderr:
  Makefile: fix misdirected redirections
2016-04-29 14:15:59 -07:00
Junio C Hamano
e0b5851907 Merge branch 'nd/test-helpers'
Sources to many test helper binaries (and the generated helpers)
have been moved to t/helper/ subdirectory to reduce clutter at the
top level of the tree.

* nd/test-helpers:
  test helpers: move test-* to t/helper/ subdirectory
  Makefile: clean *.o files we create
2016-04-29 12:59:06 -07:00
Dennis Kaarsemaker
ef642ff07c Makefile: remove dependency on git.spec
ab214331 (Makefile: stop pretending to support rpmbuild, 2016-04-04)
dropped support for rpmbuild using our own specfile by removing
git.spec.in, but forgot to remove the dependency of the dist target
on git.spec.

Signed-off-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-27 13:25:44 -07:00
Junio C Hamano
33e4ec89d9 Merge branch 'ky/imap-send-openssl-1.1.0'
Upcoming OpenSSL 1.1.0 will break compilation b updating a few APIs
we use in imap-send, which has been adjusted for the change.

* ky/imap-send-openssl-1.1.0:
  configure: remove checking for HMAC_CTX_cleanup
  imap-send: avoid deprecated TLSv1_method()
  imap-send: check NULL return of SSL_CTX_new()
  imap-send: use HMAC() function provided by OpenSSL
2016-04-22 15:45:08 -07:00
Vasco Almeida
045fac5845 i18n: git-parse-remote.sh: mark strings for translation
Change Makefile to include git-parse-remote.sh in LOCALIZED_SH.

TODO: remove 3rd argument of error_on_missing_default_upstream function
that is no longer required.

Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-19 12:07:49 -07:00
Nguyễn Thái Ngọc Duy
e6e7530d10 test helpers: move test-* to t/helper/ subdirectory
This keeps top dir a bit less crowded. And because these programs are
for testing purposes, it makes sense that they stay somewhere in t/

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-15 10:12:19 -07:00
Junio C Hamano
7897d84b82 Makefile: clean *.o files we create
The part that removes object files in the 'clean' target predates
various Makefile macros that list object files we create, and
instead removes the objects with shell glob, perpetually requiring
updates whenever a new location that builds object files is added.

Simplify the target by removing $(OBJECTS), which is supposed to
have all the objects we create during the build.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-15 10:07:35 -07:00
Junio C Hamano
8c9dec985e Merge branch 'jc/makefile-redirection-stderr'
A minor fix in the Makefile.

* jc/makefile-redirection-stderr:
  Makefile: fix misdirected redirections
2016-04-13 14:12:38 -07:00
Kazuki Yamaguchi
1245c74936 configure: remove checking for HMAC_CTX_cleanup
We don't need it, as we no longer use HMAC_CTX_cleanup() directly.

Signed-off-by: Kazuki Yamaguchi <k@rhe.jp>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-08 11:46:36 -07:00
Junio C Hamano
ab214331cf Makefile: stop pretending to support rpmbuild
Nobody in the active development community seems to watch breakages
in the rpmbuild target.  As most major RPM based distros use their
own specfile when packaging us, they aren't looking after us as
their pristine upstream tree, either.  At this point, it is turning
to be a disservice to the users to pretend that our tree natively
supports "make rpmbuild" target when we do not properly maintain it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-06 11:56:22 -07:00
Junio C Hamano
d55de70a1e Makefile: fix misdirected redirections
In general "echo 2>&1 $msg" to redirect a possible error message
that comes from 'echo' itself into the same standard output stream
$msg is getting written to does not make any sense; it is not like
we are expecting to see any errors out of 'echo' in these statements,
and even if it were the case, there is no reason to prevent the
error messages from being sent to the standard error stream.

These are clearly meant to send the argument given to echo to the
standard error stream as error messages.  Correctly redirect by
saying "send what is written to the standard output to the standard
error", i.e. "1>&2" aka ">&2".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-05 00:03:05 -07:00
Junio C Hamano
2df13639e7 Merge branch 'jc/sane-grep'
Recent versions of GNU grep is pickier than before to decide if a
file is "binary" and refuse to give line-oriented hits when we
expect it to, unless explicitly told with "-a" option.  As our
scripted Porcelains use sane_grep wrapper for line-oriented data,
even when the line may contain non-ASCII payload we took from
end-user data, use "grep -a" to implement sane_grep wrapper when
using an implementation of "grep" that takes the "-a" option.

* jc/sane-grep:
  rebase-i: clarify "is this commit relevant?" test
  sane_grep: pass "-a" if grep accepts it
2016-03-16 13:16:54 -07:00
Junio C Hamano
71b401032b sane_grep: pass "-a" if grep accepts it
Newer versions of GNU grep is reported to be pickier when we feed a
non-ASCII input and break some Porcelain scripts.  As we know we do
not feed random binary file to our own sane_grep wrapper, allow us
to always pass "-a" by setting SANE_TEXT_GREP=-a Makefile variable
to work it around.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-10 15:35:43 -08:00
Junio C Hamano
0f0dd370c8 Merge branch 'ls/makefile-cflags-developer-tweak'
There is a new DEVELOPER knob that enables many compiler warning
options in the Makefile.

* ls/makefile-cflags-developer-tweak:
  add DEVELOPER makefile knob to check for acknowledged warnings
2016-02-26 13:37:27 -08:00
Lars Schneider
658df95a4a add DEVELOPER makefile knob to check for acknowledged warnings
We assume Git developers have a reasonably modern compiler and recommend
them to enable the DEVELOPER makefile knob to ensure their patches are
clear of all compiler warnings the Git core project cares about.

Enable the DEVELOPER makefile knob in the Travis-CI build.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-25 12:49:45 -08:00
Junio C Hamano
4b589e5b28 Merge branch 'js/mingw-tests'
Test scripts have been updated to remove assumptions that are not
portable between Git for POSIX and Git for Windows, or to skip ones
with expectations that are not satisfiable on Git for Windows.

* js/mingw-tests: (21 commits)
  gitignore: ignore generated test-fake-ssh executable
  mingw: do not bother to test funny file names
  mingw: skip a test in t9130 that cannot pass on Windows
  mingw: handle the missing POSIXPERM prereq in t9124
  mingw: avoid illegal filename in t9118
  mingw: mark t9100's test cases with appropriate prereqs
  t0008: avoid absolute path
  mingw: work around pwd issues in the tests
  mingw: fix t9700's assumption about directory separators
  mingw: skip test in t1508 that fails due to path conversion
  tests: turn off git-daemon tests if FIFOs are not available
  mingw: disable mkfifo-based tests
  mingw: accomodate t0060-path-utils for MSYS2
  mingw: fix t5601-clone.sh
  mingw: let lstat() fail with errno == ENOTDIR when appropriate
  mingw: try to delete target directory before renaming
  mingw: prepare the TMPDIR environment variable for shell scripts
  mingw: factor out Windows specific environment setup
  Git.pm: stop assuming that absolute paths start with a slash
  mingw: do not trust MSYS2's MinGW gettext.sh
  ...
2016-02-17 10:13:29 -08:00
Johannes Schindelin
3064d5a38c mingw: fix t5601-clone.sh
Since baaf233 (connect: improve check for plink to reduce false
positives, 2015-04-26), t5601 writes out a `plink.exe` for testing that
is actually a shell script. So the assumption that the `.exe` extension
implies that the file is *not* a shell script is now wrong.

Since there was no love for the idea of allowing `.exe` files to be
shell scripts on Windows, let's go the other way round: *make*
`plink.exe` a real `.exe`.

This fixes t5601-clone.sh in Git for Windows' SDK.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-27 14:27:19 -08:00
Junio C Hamano
ce7da1d281 Merge branch 'ep/make-phoney'
A slight update to the Makefile.

* ep/make-phoney:
  Makefile: add missing phony target
2016-01-12 15:16:53 -08:00
Elia Pinto
e6be2655fc Makefile: add missing phony target
Add some missing phony target to Makefile.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Helped-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-16 12:01:10 -08:00
Junio C Hamano
b1cda70fff Merge branch 'dt/refs-backend-pre-vtable'
Code preparation for pluggable ref backends.

* dt/refs-backend-pre-vtable:
  refs: break out ref conflict checks
  files_log_ref_write: new function
  initdb: make safe_create_dir public
  refs: split filesystem-based refs code into a new file
  refs/refs-internal.h: new header file
  refname_is_safe(): improve docstring
  pack_if_possible_fn(): use ref_type() instead of is_per_worktree_ref()
  copy_msg(): rename to copy_reflog_msg()
  verify_refname_available(): new function
  verify_refname_available(): rename function
2015-12-08 14:14:49 -08:00
Junio C Hamano
b5d2d8eef0 Merge branch 'ad/sha1-update-chunked' into maint
Apple's common crypto implementation of SHA1_Update() does not take
more than 4GB at a time, and we now have a compile-time workaround
for it.

* ad/sha1-update-chunked:
  sha1: allow limiting the size of the data passed to SHA1_Update()
  sha1: provide another level of indirection for the SHA-1 functions
2015-12-08 14:05:03 -08:00
Junio C Hamano
4672123fe5 Merge branch 'ad/sha1-update-chunked'
Apple's common crypto implementation of SHA1_Update() does not take
more than 4GB at a time, and we now have a compile-time workaround
for it.

* ad/sha1-update-chunked:
  sha1: allow limiting the size of the data passed to SHA1_Update()
  sha1: provide another level of indirection for the SHA-1 functions
2015-12-04 11:19:10 -08:00
Michael Haggerty
7bd9bcf372 refs: split filesystem-based refs code into a new file
As another step in the move to pluggable reference backends, move the
code that is specific to the filesystem-based reference backend (i.e.,
the current system of storing references as loose and packed files) into
a separate file, refs/files-backend.c.

Aside from a tiny bit of file header boilerplate, this commit only moves
a subset of the code verbatim from refs.c to the new file, as can easily
be verified using patience diff:

    git diff --patience $commit^:refs.c $commit:refs.c
    git diff --patience $commit^:refs.c $commit:refs/files-backend.c

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 04:52:01 -05:00
Junio C Hamano
3897d2d906 Merge branch 'rp/link-curl-before-ssl' into maint
The linkage order of libraries was wrong in places around libcurl.

* rp/link-curl-before-ssl:
  configure.ac: detect ssl need with libcurl
  Makefile: make curl-config path configurable
  Makefile: link libcurl before zlib
2015-11-05 12:18:09 -08:00
Atousa Pahlevan Duprat
001fd7a90b sha1: allow limiting the size of the data passed to SHA1_Update()
Using the previous commit's inredirection mechanism for SHA1,
support a chunked implementation of SHA1_Update() that limits the
amount of data in the chunk passed to SHA1_Update().

This is enabled by using the Makefile variable SHA1_MAX_BLOCK_SIZE
to specify chunk size.  When using Apple's CommonCrypto library this
is set to 1GiB (the implementation cannot handle more 4GiB).

Signed-off-by: Atousa Pahlevan Duprat <apahlevan@ieee.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-05 10:35:11 -08:00
Junio C Hamano
ba5312da19 Merge branch 'jc/mailinfo-lib'
The implementation of "git mailinfo" was refactored so that a
mailinfo() function can be directly called from inside a process.

* jc/mailinfo-lib: (34 commits)
  mailinfo: remove calls to exit() and die() deep in the callchain
  mailinfo: handle charset conversion errors in the caller
  mailinfo: libify
  mailinfo: keep the parsed log message in a strbuf
  mailinfo: handle_commit_msg() shouldn't be called after finding patchbreak
  mailinfo: move content/content_top to struct mailinfo
  mailinfo: move [ps]_hdr_data to struct mailinfo
  mailinfo: move cmitmsg and patchfile to struct mailinfo
  mailinfo: move charset to struct mailinfo
  mailinfo: move transfer_encoding to struct mailinfo
  mailinfo: move check for metainfo_charset to convert_to_utf8()
  mailinfo: move metainfo_charset to struct mailinfo
  mailinfo: move use_scissors and use_inbody_headers to struct mailinfo
  mailinfo: move add_message_id and message_id to struct mailinfo
  mailinfo: move patch_lines to struct mailinfo
  mailinfo: move filter/header stage to struct mailinfo
  mailinfo: move global "FILE *fin, *fout" to struct mailinfo
  mailinfo: move keep_subject & keep_non_patch_bracket to struct mailinfo
  mailinfo: introduce "struct mailinfo" to hold globals
  mailinfo: move global "line" into mailinfo() function
  ...
2015-10-29 13:59:22 -07:00
Junio C Hamano
9627b0a49f Merge branch 'rp/link-curl-before-ssl'
The linkage order of libraries was wrong in places around libcurl.

* rp/link-curl-before-ssl:
  configure.ac: detect ssl need with libcurl
  Makefile: make curl-config path configurable
  Makefile: link libcurl before zlib
2015-10-29 13:59:20 -07:00
Junio C Hamano
a46dcfb840 Merge branch 'mr/worktree-list'
Add the "list" subcommand to "git worktree".

* mr/worktree-list:
  worktree: add 'list' command
  worktree: add details to the worktree struct
  worktree: add a function to get worktree details
  worktree: refactor find_linked_symref function
  worktree: add top-level worktree.c
2015-10-26 15:55:16 -07:00
Junio C Hamano
c6905e45f0 mailinfo: libify
Move the bulk of the code from builtin/mailinfo.c to mailinfo.c
so that new callers can start calling mailinfo() directly.

Note that a few calls to exit() and die() need to be cleaned up
for the API to be truly useful, which will come in later steps.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-21 15:59:34 -07:00
Remi Pommarel
f89158760d Makefile: make curl-config path configurable
There are situations, e.g. during cross compilation, where curl-config
program is not present in the PATH.

Make the makefile use a configurable curl-config program passed through
CURL_CONFIG variable which can be set through config.mak.

Also make this variable tunable through use of autoconf/configure. Configure
will set CURL_CONFIG variable in config.mak.autogen to whatever value has been
passed to ac_cv_prog_CURL_CONFIG.

Signed-off-by: Remi Pommarel <repk@triplefau.lt>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-21 12:43:31 -07:00
Remi Pommarel
9eaa78b0b0 Makefile: link libcurl before zlib
For static linking especially library order while linking is important. For
example, libcurl wants symbols from zlib when building http-push, http-fetch
and remote-curl. So for these programs libcurl has to be linked before zlib.

Signed-off-by: Remi Pommarel <repk@triplefau.lt>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-21 12:43:03 -07:00
Junio C Hamano
78891795df Merge branch 'jk/war-on-sprintf'
Many allocations that is manually counted (correctly) that are
followed by strcpy/sprintf have been replaced with a less error
prone constructs such as xstrfmt.

Macintosh-specific breakage was noticed and corrected in this
reroll.

* jk/war-on-sprintf: (70 commits)
  name-rev: use strip_suffix to avoid magic numbers
  use strbuf_complete to conditionally append slash
  fsck: use for_each_loose_file_in_objdir
  Makefile: drop D_INO_IN_DIRENT build knob
  fsck: drop inode-sorting code
  convert strncpy to memcpy
  notes: document length of fanout path with a constant
  color: add color_set helper for copying raw colors
  prefer memcpy to strcpy
  help: clean up kfmclient munging
  receive-pack: simplify keep_arg computation
  avoid sprintf and strcpy with flex arrays
  use alloc_ref rather than hand-allocating "struct ref"
  color: add overflow checks for parsing colors
  drop strcpy in favor of raw sha1_to_hex
  use sha1_to_hex_r() instead of strcpy
  daemon: use cld->env_array when re-spawning
  stat_tracking_info: convert to argv_array
  http-push: use an argv_array for setup_revisions
  fetch-pack: use argv_array for index-pack / unpack-objects
  ...
2015-10-20 15:24:01 -07:00
Junio C Hamano
47c566a4d6 Merge branch 'jk/make-findstring-makeflags-fix' into maint
Customization to change the behaviour with "make -w" and "make -s"
in our Makefile was broken when they were used together.

* jk/make-findstring-makeflags-fix:
  Makefile: fix MAKEFLAGS tests with multiple flags
2015-10-16 14:32:38 -07:00
Junio C Hamano
4d2a3011ee Merge branch 'jw/make-arflags-customizable' into maint
The Makefile always runs the library archiver with hardcoded "crs"
options, which was inconvenient for exotic platforms on which
people want to use programs with totally different set of command
line options.

* jw/make-arflags-customizable:
  Makefile: allow $(ARFLAGS) specified from the command line
2015-10-16 14:32:36 -07:00
Junio C Hamano
1e7ea4e7ab Merge branch 'jw/make-arflags-customizable'
The Makefile always runs the library archiver with hardcoded "crs"
options, which was inconvenient for exotic platforms on which
people want to use programs with totally different set of command
line options.

* jw/make-arflags-customizable:
  Makefile: allow $(ARFLAGS) specified from the command line
2015-10-05 12:30:23 -07:00
Junio C Hamano
96090283e0 Merge branch 'jk/make-findstring-makeflags-fix'
Customization to change the behaviour with "make -w" and "make -s"
in our Makefile was broken when they were used together.

* jk/make-findstring-makeflags-fix:
  Makefile: fix MAKEFLAGS tests with multiple flags
2015-10-05 12:30:20 -07:00
Junio C Hamano
65e1449614 Merge branch 'sb/submodule-helper'
The infrastructure to rewrite "git submodule" in C is being built
incrementally.  Let's polish these early parts well enough and make
them graduate to 'next' and 'master', so that the more involved
follow-up can start cooking on a solid ground.

* sb/submodule-helper:
  submodule: rewrite `module_clone` shell function in C
  submodule: rewrite `module_name` shell function in C
  submodule: rewrite `module_list` shell function in C
2015-10-05 12:30:19 -07:00
Jeff King
e23a91b047 Makefile: drop D_INO_IN_DIRENT build knob
Now that fsck has dropped its inode-sorting, there are no
longer any users of this knob, and it can go away.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-05 11:08:06 -07:00
Michael Rappazzo
ac6c561b59 worktree: add top-level worktree.c
worktree.c contains functions to work with and get information from
worktrees.  This introduction moves functions related to worktrees
from branch.c into worktree.c

Signed-off-by: Michael Rappazzo <rappazzo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-02 13:07:38 -07:00
Junio C Hamano
a9400b01df Merge branch 'sg/help-group'
* sg/help-group:
  Makefile: use SHELL_PATH when running generate-cmdlist.sh
2015-09-14 14:59:05 -07:00
Alejandro R. Sedeño
57cee8ac5f Makefile: use SHELL_PATH when running generate-cmdlist.sh
Non-POSIX shells, such as /bin/sh on SunOS, do not support $((...))
arithmetic expansion or $(...) command substitution needed by
generate-cmdlist.sh.  Make sure that we use a POSIX compliant shell
$(SHELL_PATH) when running generate-cmdlist.sh.

Signed-off-by: Alejandro R. Sedeño <asedeno@mit.edu>
Acked-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-10 17:49:00 -07:00
Junio C Hamano
ac179b4d9c Makefile: allow $(ARFLAGS) specified from the command line
We can do this because we have a very simple needs and run "ar"
exactly the same way everywhere ;-).

Requested-by: Jeffrey Walton
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-10 14:27:21 -07:00
John Keeping
ef49e05a64 Makefile: fix MAKEFLAGS tests with multiple flags
findstring is defined as $(findstring FIND,IN) so if multiple flags are
set these tests do the wrong thing unless $(MAKEFLAGS) is the second
argument.

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-10 09:26:23 -07:00
Junio C Hamano
969560bddc Merge branch 'sg/help-group' into maint
We rewrote one of the build scripts in Perl but this reimplements
in Bourne shell.

* sg/help-group:
  generate-cmdlist: re-implement as shell script
2015-09-03 19:17:51 -07:00
Stefan Beller
74703a1e4d submodule: rewrite module_list shell function in C
Most of the submodule operations work on a set of submodules.
Calculating and using this set is usually done via:

       module_list "$@" | {
           while read mode sha1 stage sm_path
           do
                # the actual operation
           done
       }

Currently the function `module_list` is implemented in the
git-submodule.sh as a shell script wrapping a perl script.
The rewrite is in C, such that it is faster and can later be
easily adapted when other functions are rewritten in C.

git-submodule.sh, similar to the builtin commands, will navigate
to the top-most directory of the repository and keep the
subdirectory as a variable. As the helper is called from
within the git-submodule.sh script, we are already navigated
to the root level, but the path arguments are still relative
to the subdirectory we were in when calling git-submodule.sh.
That's why there is a `--prefix` option pointing to an alternative
path which to anchor relative path arguments.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-03 14:12:40 -07:00
Junio C Hamano
5a4f07b322 Merge branch 'hv/submodule-config'
The gitmodules API accessed from the C code learned to cache stuff
lazily.

* hv/submodule-config:
  submodule: allow erroneous values for the fetchRecurseSubmodules option
  submodule: use new config API for worktree configurations
  submodule: extract functions for config set and lookup
  submodule: implement a config API for lookup of .gitmodules values
2015-08-31 15:38:52 -07:00
Junio C Hamano
678c5a49ee Merge branch 'sg/help-group'
We rewrote one of the build scripts in Perl but this reimplements
in Bourne shell.

* sg/help-group:
  generate-cmdlist: re-implement as shell script
2015-08-26 15:45:40 -07:00
Junio C Hamano
db86e61cbb Merge branch 'mh/tempfile'
The "lockfile" API has been rebuilt on top of a new "tempfile" API.

* mh/tempfile:
  credential-cache--daemon: use tempfile module
  credential-cache--daemon: delete socket from main()
  gc: use tempfile module to handle gc.pid file
  lock_repo_for_gc(): compute the path to "gc.pid" only once
  diff: use tempfile module
  setup_temporary_shallow(): use tempfile module
  write_shared_index(): use tempfile module
  register_tempfile(): new function to handle an existing temporary file
  tempfile: add several functions for creating temporary files
  prepare_tempfile_object(): new function, extracted from create_tempfile()
  tempfile: a new module for handling temporary files
  commit_lock_file(): use get_locked_file_path()
  lockfile: add accessor get_lock_file_path()
  lockfile: add accessors get_lock_file_fd() and get_lock_file_fp()
  create_bundle(): duplicate file descriptor to avoid closing it twice
  lockfile: move documentation to lockfile.h and lockfile.c
2015-08-25 14:57:09 -07:00
Eric Sunshine
82aec45b7d generate-cmdlist: re-implement as shell script
527ec39 (generate-cmdlist: parse common group commands, 2015-05-21)
replaced generate-cmdlist.sh with a more functional Perl version,
generate-cmdlist.perl. The Perl version gleans named tags from a new
"common groups" section in command-list.txt and recognizes those
tags in "command list" section entries in place of the old 'common'
tag. This allows git-help to, not only recognize, but also group
common commands.

Although the tests require Perl, 527ec39 creates an unconditional
dependence upon Perl in the build system itself, which can not be
overridden with NO_PERL. Such a dependency may be undesirable; for
instance, the 'git-lite' package in the FreeBSD ports tree is
intended as a minimal Git installation (which may, for example, be
useful on servers needing only local clone and update capability),
which, historically, has not depended upon Perl[1].

Therefore, revive generate-cmdlist.sh and extend it to recognize
"common groups" and its named tags. Retire generate-cmdlist.perl.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/275905/focus=276132

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-25 11:24:31 -07:00
Junio C Hamano
5a1ba6b48a Merge 'hv/submodule-config' to 'sb/submodule-helper'
* hv/submodule-config:
  submodule: allow erroneous values for the fetchRecurseSubmodules option
  submodule: use new config API for worktree configurations
  submodule: extract functions for config set and lookup
  submodule: implement a config API for lookup of .gitmodules values
2015-08-19 11:45:08 -07:00
Heiko Voigt
959b5455d0 submodule: implement a config API for lookup of .gitmodules values
In a superproject some commands need to interact with submodules. They
need to query values from the .gitmodules file either from the worktree
of from certain revisions. At the moment this is quite hard since a
caller would need to read the .gitmodules file from the history and then
parse the values. We want to provide an API for this so we have one
place to get values from .gitmodules from any revision (including the
worktree).

The API is realized as a cache which allows us to lazily read
.gitmodules configurations by commit into a runtime cache which can then
be used to easily lookup values from it. Currently only the values for
path or name are stored but it can be extended for any value needed.

It is expected that .gitmodules files do not change often between
commits. Thats why we lookup the .gitmodules sha1 from a commit and then
either lookup an already parsed configuration or parse and cache an
unknown one for each sha1. The cache is lazily build on demand for each
requested commit.

This cache can be used for all purposes which need knowledge about
submodule configurations. Example use cases are:

 * Recursive submodule checkout needs to lookup a submodule name from
   its path when a submodule first appears. This needs be done before
   this configuration exists in the worktree.

 * The implementation of submodule support for 'git archive' needs to
   lookup the submodule name to generate the archive when given a
   revision that is not checked out.

 * 'git fetch' when given the --recurse-submodules=on-demand option (or
   configuration) needs to lookup submodule names by path from the
   database rather than reading from the worktree. For new submodule it
   needs to lookup the name from its path to allow cloning new
   submodules into the .git folder so they can be checked out without
   any network interaction when the user does a checkout of that
   revision.

Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-19 11:43:09 -07:00
Michael Haggerty
1a9d15db25 tempfile: a new module for handling temporary files
A lot of work went into defining the state diagram for lockfiles and
ensuring correct, race-resistant cleanup in all circumstances.

Most of that infrastructure can be applied directly to *any* temporary
file. So extract a new "tempfile" module from the "lockfile" module.
Reimplement lockfile on top of tempfile.

Subsequent commits will add more users of the new module.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-10 12:57:14 -07:00
Paul Tan
783d7e865e builtin-am: remove redirection to git-am.sh
At the beginning of the rewrite of git-am.sh to C, in order to not break
existing test scripts that depended on a functional git-am, a
redirection to git-am.sh was introduced that would activate if the
environment variable _GIT_USE_BUILTIN_AM was not defined.

Now that all of git-am.sh's functionality has been re-implemented in
builtin/am.c, remove this redirection, and retire git-am.sh into
contrib/examples/.

Signed-off-by: Paul Tan <pyokagan@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-04 22:02:11 -07:00
Paul Tan
73c2779f42 builtin-am: implement skeletal builtin am
For the purpose of rewriting git-am.sh into a C builtin, implement a
skeletal builtin/am.c that redirects to $GIT_EXEC_PATH/git-am if the
environment variable _GIT_USE_BUILTIN_AM is not defined. Since in the
Makefile git-am.sh takes precedence over builtin/am.c,
$GIT_EXEC_PATH/git-am will contain the shell script git-am.sh, and thus
this allows us to fall back on the functional git-am.sh when running the
test suite for tests that depend on a working git-am implementation.

Since git-am.sh cannot handle any environment modifications by
setup_git_directory(), "am" is declared with no setup flags in git.c. On
the other hand, to re-implement git-am.sh in builtin/am.c, we need to
run all the git dir and work tree setup logic that git.c typically does
for us. As such, we work around this temporarily by copying the logic in
git.c's run_builtin(), which is roughly:

	prefix = setup_git_directory();
	trace_repo_setup(prefix);
	setup_work_tree();

This redirection should be removed when all the features of git-am.sh
have been re-implemented in builtin/am.c.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Paul Tan <pyokagan@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-04 22:02:11 -07:00
Junio C Hamano
5f02274e4c Merge branch 'pt/pull-builtin'
Reimplement 'git pull' in C.

* pt/pull-builtin:
  pull: remove redirection to git-pull.sh
  pull --rebase: error on no merge candidate cases
  pull --rebase: exit early when the working directory is dirty
  pull: configure --rebase via branch.<name>.rebase or pull.rebase
  pull: teach git pull about --rebase
  pull: set reflog message
  pull: implement pulling into an unborn branch
  pull: fast-forward working tree if head is updated
  pull: check if in unresolved merge state
  pull: support pull.ff config
  pull: error on no merge candidates
  pull: pass git-fetch's options to git-fetch
  pull: pass git-merge's options to git-merge
  pull: pass verbosity, --progress flags to fetch and merge
  pull: implement fetch + merge
  pull: implement skeletal builtin pull
  argv-array: implement argv_array_pushv()
  parse-options-cb: implement parse_opt_passthru_argv()
  parse-options-cb: implement parse_opt_passthru()
2015-08-03 11:01:17 -07:00
Junio C Hamano
7ebc8cbedd Merge branch 'kn/for-each-ref'
GSoC project to rebuild ref listing by branch and tag based on the
for-each-ref machinery.  This is its first part.

* kn/for-each-ref:
  ref-filter: make 'ref_array_item' use a FLEX_ARRAY for refname
  for-each-ref: introduce filter_refs()
  ref-filter: move code from 'for-each-ref'
  ref-filter: add 'ref-filter.h'
  for-each-ref: rename variables called sort to sorting
  for-each-ref: rename some functions and make them public
  for-each-ref: introduce 'ref_array_clear()'
  for-each-ref: introduce new structures for better organisation
  for-each-ref: rename 'refinfo' to 'ref_array_item'
  for-each-ref: clean up code
  for-each-ref: extract helper functions out of grab_single_ref()
2015-08-03 11:01:11 -07:00