Commit Graph

39597 Commits

Author SHA1 Message Date
Jeff King
b5f52f372e sha1_file: freshen pack objects before loose
When writing out an object file, we first check whether it
already exists and if so optimize out the write. Prior to
33d4221, we did this by calling has_sha1_file(), which will
check for packed objects followed by loose. Since that
commit, we check loose objects first.

For the common case of a repository whose objects are mostly
packed, this means we will make a lot of extra access()
system calls checking for loose objects. We should follow
the same packed-then-loose order that all of our other
lookups use.

Reported-by: Stefan Saasen <ssaasen@atlassian.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-20 13:09:38 -07:00
Jeff King
1385bb7ba3 reachable: only mark local objects as recent
When pruning and repacking a repository that has an
alternate object store configured, we may traverse a large
number of objects in the alternate. This serves no purpose,
and may be expensive to do. A longer explanation is below.

Commits d3038d2 and abcb865 taught prune and pack-objects
(respectively) to treat "recent" objects as tips for
reachability, so that we keep whole chunks of history. They
built on the object traversal in 660c889 (sha1_file: add
for_each iterators for loose and packed objects,
2014-10-15), which covers both local and alternate objects.

In both cases, covering alternate objects is unnecessary, as
both commands can only drop objects from the local
repository. In the case of prune, we traverse only the local
object directory. And in the case of repacking, while we may
or may not include local objects in our pack, we will never
reach into the alternate with "repack -d". The "-l" option
is only a question of whether we are migrating objects from
the alternate into our repository, or leaving them
untouched.

It is possible that we may drop an object that is depended
upon by another object in the alternate. For example,
imagine two repositories, A and B, with A pointing to B as
an alternate. Now imagine a commit that is in B which
references a tree that is only in A. Traversing from recent
objects in B might prevent A from dropping that tree. But
this case isn't worth covering. Repo B should take
responsibility for its own objects. It would never have had
the commit in the first place if it did not also have the
tree, and assuming it is using the same "keep recent chunks
of history" scheme, then it would itself keep the tree, as
well.

So checking the alternate objects is not worth doing, and
come with a significant performance impact. In both cases,
we skip any recent objects that have already been marked
SEEN (i.e., that we know are already reachable for prune, or
included in the pack for a repack). So there is a slight
waste of time in opening the alternate packs at all, only to
notice that we have already considered each object. But much
worse, the alternate repository may have a large number of
objects that are not reachable from the local repository at
all, and we end up adding them to the traversal.

We can fix this by considering only local unseen objects.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-20 13:09:27 -07:00
Lex Spoon
96b2d54aee git-p4: use -m when running p4 changes
Simply running "p4 changes" on a large branch can result in a "too
many rows scanned" error from the Perforce server. It is better to
use a sequence of smaller calls to "p4 changes", using the "-m"
option to limit the size of each call.

Signed-off-by: Lex Spoon <lex@lexspoon.org>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-20 12:36:00 -07:00
Matthieu Moy
0269f968b7 log -L: improve error message on malformed argument
The old message did not mention the :regex:file form.

To avoid overly long lines, split the message into two lines (in case
item->string is long, it will be the only part truncated in a narrow
terminal).

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-20 11:06:10 -07:00
Matthieu Moy
d349e0ee60 Documentation: change -L:<regex> to -L:<funcname>
The old wording was somehow implying that <start> and <end> were not
regular expressions. Also, the common case is to use a plain function
name here so <funcname> makes sense (the fact that it is a regular
expression is documented in line-range-format.txt).

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-20 11:05:50 -07:00
Junio C Hamano
1eb0545cce git-gui 0.20.0
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQCVAwUAVTI85mB90JXwhOSJAQIAwAP+Lcm1TuAKXdiCMcYJFLPHhSvHJJRHPiN8
 ydBIRP5UO6BJrmbo+w9tx3qAda4TS/crqRwSnRa5cXOWB+aDkuH8zm4IfkQKXIjI
 qG4Q2t53q8vLBtRd6Y4uNlDQ04mh/J64E9tJoCaIzMj2m6W/Xizo2zslnOfo6F2Z
 Z2rJhBMe1PY=
 =4pD7
 -----END PGP SIGNATURE-----

Merge tag 'gitgui-0.20.0' of http://repo.or.cz/r/git-gui

git-gui 0.20.0

* tag 'gitgui-0.20.0' of http://repo.or.cz/r/git-gui:
  git-gui: set version 0.20
  git-gui: sv.po: Update Swedish translation (547t0f0u)
  git-gui i18n: Updated Bulgarian translation (547t,0f,0u)
  git-gui: Makes chooser set 'gitdir' to the resolved path
  git-gui: Fixes chooser not accepting gitfiles
  git-gui: reinstate support for Tcl 8.4
  git-gui: fix problem with gui.maxfilesdisplayed
  git-gui: fix verbose loading when git path contains spaces.
  git-gui/gitk: Do not depend on Cygwin's "kill" command on Windows
  git-gui: add configurable tab size to the diff view
  git-gui: Make git-gui lib dir configurable at runime
  git-gui i18n: Updated Bulgarian translation (520t,0f,0u)
  L10n: vi.po (543t): Init translation for Vietnamese
  git-gui: align the new recursive checkbox with the radiobuttons.
  git-gui: Add a 'recursive' checkbox in the clone menu.
2015-04-18 18:35:48 -07:00
Nguyễn Thái Ngọc Duy
64f2589a09 t1509: update prepare script to be able to run t1509 in chroot again
Tested on Gentoo and OpenSUSE 13.1, both x86-64

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-18 17:51:04 -07:00
Nguyễn Thái Ngọc Duy
c6458e60ed index-pack: kill union delta_base to save memory
Once we know the number of objects in the input pack, we allocate an
array of nr_objects of struct delta_entry. On x86-64, this struct is
32 bytes long. The union delta_base, which is part of struct
delta_entry, provides enough space to store either ofs-delta (8 bytes)
or ref-delta (20 bytes).

Because ofs-delta encoding is more efficient space-wise and more
performant at runtime than ref-delta encoding, Git packers try to use
ofs-delta whenever possible, and it is expected that objects encoded
as ref-delta are minority.

In the best clone case where no ref-delta object is present, we waste
(20-8) * nr_objects bytes because of this union. That's about 38MB out
of 100MB for deltas[] with 3.4M objects, or 38%. deltas[] would be
around 62MB without the waste.

This patch attempts to eliminate that. deltas[] array is split into
two: one for ofs-delta and one for ref-delta. Many functions are also
duplicated because of this split. With this patch, ofs_deltas[] array
takes 51MB. ref_deltas[] should remain unallocated in clone case (0
bytes). This array grows as we see ref-delta. We save about half in
this case, or 25% of total bookkeeping.

The saving is more than the calculation above because some padding in
the old delta_entry struct is removed. ofs_delta_entry is 16 bytes,
including the 4 bytes padding. That's 13MB for padding, but packing
the struct could break platforms that do not support unaligned
access. If someone on 32-bit is really low on memory and only deals
with packs smaller than 2G, using 32-bit off_t would eliminate the
padding and save 27MB on top.

A note about ofs_deltas allocation. We could use ref_deltas memory
allocation strategy for ofs_deltas. But that probably just adds more
overhead on top. ofs-deltas are generally the majority (1/2 to 2/3) in
any pack. Incremental realloc may lead to too many memcpy. And if we
preallocate, say 1/2 or 2/3 of nr_objects initially, the growth rate
of ALLOC_GROW() could make this array larger than nr_objects, wasting
more memory.

Brought-up-by: Matthew Sporleder <msporleder@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-18 17:48:32 -07:00
Torsten Bögershausen
d6d6673e66 t0027: support NATIVE_CRLF platforms
t0027 expects the native end-of-lines to be a single line feed
character.  On Windows, however, we set it to a carriage return
character followed by a line feed character.  Thus, we have to
modify t0027 to expect different warnings depending on the
end-of-line markers.

Adjust the check of the warnings and use these macros:

  WILC:  Warn if LF becomes CRLF
  WICL:  Warn if CRLF becomes LF
  WAMIX: Mixed line endings: either CRLF->LF or LF->CRLF

Improve the information given by check_warning().

Use test_cmp to show which warning is missing (or shouldn't be
there).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-18 11:47:33 -07:00
Torsten Bögershausen
ad295bbca2 t0027: cleanup: rename functions; avoid non-leading TABs
Make more clear what the tests are doing:

  commit_check_warn():
    Commit files and checks for conversion warnings.
    Old name: create_file_in_repo()

  checkout_files():
    Checkout files from the repo and check if they have
    the appropriate line endings in the work space.
    Old name: check_files_in_ws()

Replace non-leading TABS with spaces

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-18 11:47:23 -07:00
Pat Thoyts
4498b3a50a git-gui: set version 0.20
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
2015-04-18 12:15:32 +01:00
Peter Krefting
5a5c11f19d git-gui: sv.po: Update Swedish translation (547t0f0u)
Signed-off-by: Peter Krefting <peter@softwolves.pp.se>
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
2015-04-18 12:03:50 +01:00
Alexander Shopov
de18648212 git-gui i18n: Updated Bulgarian translation (547t,0f,0u)
Signed-off-by: Alexander Shopov <ash@kambanaria.org>
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
2015-04-18 11:51:39 +01:00
Jeff King
b6e8a3b540 limit_list: avoid quadratic behavior from still_interesting
When we are limiting a rev-list traversal due to
UNINTERESTING refs, we have to walk down the tips (both
interesting and uninteresting) to find where they intersect.
We keep a queue of commits to examine, pop commits off
the queue one by one, and potentially add their parents.  The
size of the queue will naturally fluctuate based on the
"width" of the history graph; i.e., the number of
simultaneous lines of development. But for the most part it
will stay in the same ballpark as the initial number of tips
we fed, shrinking over time (as we hit common ancestors of
the tips). So roughly speaking, if we start with `N` tips,
we'll spend much of the time with a queue around `N` items.

For each UNINTERESTING commit we pop, we call
still_interesting to check whether marking its parents as
UNINTERESTING has made the whole queue uninteresting (in
which case we can quit early).  Because the queue is stored
as a linked list, this is `O(N)`, where `N` is the number of
items in the queue. So processing a queue with `N` commits
marked UNINTERESTING (and one or more interesting commits)
will take `O(N^2)`.

If you feed a lot of positive tips, this isn't a problem.
They aren't UNINTERESTING, so they don't incur the
still_interesting check.  It also isn't a problem if you
traverse from an interesting tip to some UNINTERESTING
bases. We order the queue by recency, so the interesting
commits stay at the front of the queue as we walk down them.
The linear check can exit early as soon as it sees one
interesting commit left in the queue.

But if you want to know whether an older commit is reachable
from a set of newer tips, we end up processing in the
opposite direction: from the UNINTERESTING ones down to the
interesting one. This may happen when we call:

  git rev-list $commits --not --all

in check_everything_connected after a fetch. If we fetched
something much older than most of our refs, and if we have a
large number of refs, the traversal cost is dominated by the
quadratic behavior.

These commands simulate the connectivity check of such a
fetch, when you have `$n` distinct refs in the receiver:

    # positive ref is 100,000 commits deep
    git rev-list --all | head -100000 | tail -1 >input

    # huge number of more recent negative refs
    git rev-list --all | head -$n | sed s/^/^/ >>input

    time git rev-list --stdin <input

Here are timings for various `n` on the linux.git
repository. The `n=1` case provides a baseline for just
walking the commits, which lets us see the still_interesting
overhead. The times marked with `+` subtract that baseline
to show just the extra time growth due to the large number
of refs. The `x` numbers show the slowdown of the adjusted
time versus the prior trial.

       n  | before                 | after
    --------------------------------------------------------
        1 | 0.991s                 | 0.848s
    10000 | 1.120s (+0.129s)       | 0.885s (+0.037s)
    20000 | 1.451s (+0.460s, 3.5x) | 0.923s (+0.075s, 2.0x)
    40000 | 2.731s (+1.740s, 3.8x) | 0.994s (+0.146s, 1.9x)
    80000 | 8.235s (+7.244s, 4.2x) | 1.123s (+0.275s, 1.9x)

Each trial doubles `n`, so you can see the quadratic (`4x`)
behavior before this patch. Afterwards, we have a roughly
linear relationship.

The implementation is fairly straightforward. Whenever we do
the linear search, we cache the interesting commit we find,
and next time check it before doing another linear search.
If that commit is removed from the list or becomes
UNINTERESTING itself, then we fall back to the linear
search. This is very similar to the trick used by fce87ae
(Fix quadratic performance in rewrite_one., 2008-07-12).

I considered and rejected several possible alternatives:

  1. Keep a count of UNINTERESTING commits in the queue.
     This requires managing the count not only when removing
     an item from the queue, but also when marking an item
     as UNINTERESTING. That requires touching the other
     functions which mark commits, and would require knowing
     quickly which commits are in the queue (lookup in the
     queue is linear, so we would need an auxiliary
     structure or to also maintain an IN_QUEUE flag in each
     commit object).

  2. Keep a separate list of interesting commits. Drop items
     from it when they are dropped from the queue, or if
     they become UNINTERESTING. This again suffers from
     extra complexity to maintain the list, not to mention
     CPU and memory.

  3. Use a better data structure for the queue. This is
     something that could help the fix in fce87ae, because
     we order the queue by recency, and it is about
     inserting quickly in recency order. So a normal
     priority queue would help there. But here, we cannot
     disturb the order of the queue, which makes things
     harder. We really do need an auxiliary index to track
     the flag we care about, which is basically option (2)
     above.

The "cache" trick is simple, and the numbers above show that
it works well in practice. This is because the length of
time it takes to find an interesting commit is proportional
to the length of time it will remain cached (i.e., if we
have to walk a long way to find it, it also means we have to
pop a lot of elements in the queue until we get rid of it
and have to find another interesting commit).

The worst case is still quadratic, though. We could have `N`
uninteresting commits at the front of the queue, followed by
`N` interesting commits, where commit `i` has parent `i+N`.
When we pop commit `i`, we will notice that the parent of
the next commit, `i+1+N` is still interesting and cache it.
But then handling commit `i+1`, we will mark its parent
`i+1+N` uninteresting, and immediately invalidate our cache.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-17 15:22:05 -07:00
Jeff King
b7994af0f9 type_from_string_gently: make sure length matches
When commit fe8e3b7 refactored type_from_string to allow
input that was not NUL-terminated, it switched to using
strncmp instead of strcmp. But this means we check only the
first "len" bytes of the strings, and ignore any remaining
bytes in the object_type_string. We should make sure that it
is also "len" bytes, or else we would accept "comm" as
"commit", and so forth.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-17 13:54:39 -07:00
Ossi Herrala
7e11052442 config: fix settings in default_user_config template
The name (not user) and email setting should be in config section
"user" and not in "core" as documented in Documentation/config.txt.

Signed-off-by: Ossi Herrala <oherrala@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-17 10:32:46 -07:00
Michael J Gruber
7348cdeb6c rev-list-options.txt: complete sentence about notes matching
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-17 10:30:51 -07:00
Matthieu Moy
7e9e048661 stash -p: demonstrate failure of split with mixed y/n
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 14:14:12 -07:00
Matthieu Moy
798a5b03fb t3904-stash-patch: factor PERL prereq at the top of the file
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 14:14:11 -07:00
Matthieu Moy
470b11e805 t3904-stash-patch: fix test description
The old description is rather clearly a wrong cut-and-paste from
t2016-checkout-patch.sh.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 14:14:11 -07:00
Matthieu Moy
1bf01040f0 add -p: demonstrate failure when running 'edit' after a split
The test passes if one replaces the 'e' command with a 'y' command in
the 'add -p' session.

Reported-by: Tanky Woo <wtq1990@gmail.com>
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 14:14:10 -07:00
Matthieu Moy
416145f07a t3701-add-interactive: simplify code
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 14:14:09 -07:00
Elia Pinto
de248e92c1 test-lib-functions.sh: fix the second argument to some helper functions
The second argument to test_path_is_file and test_path_is_dir
must be $2 and not $*, which instead would repeat the file
name in the error message.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Reviewed-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 13:31:35 -07:00
Junio C Hamano
27547e5fcc attr: skip UTF8 BOM at the beginning of the input file
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 11:35:38 -07:00
Junio C Hamano
599446dc32 config: use utf8_bom[] from utf.[ch] in git_parse_source()
Because the function reads one character at the time, unfortunately
we cannot use the easier skip_utf8_bom() helper, but at least we do
not have to duplicate the constant string this way.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 11:35:27 -07:00
Junio C Hamano
dde843e737 utf8-bom: introduce skip_utf8_bom() helper
With the recent change to ignore the UTF8 BOM at the beginning of
.gitignore files, we now have two codepaths that do such a skipping
(the other one is for reading the configuration files).

Introduce utf8_bom[] constant string and skip_utf8_bom() helper
and teach .gitignore code how to use it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 11:35:06 -07:00
Junio C Hamano
cb0abea870 add_excludes_from_file: clarify the bom skipping logic
Even though the previous step shifts where the "entry" begins, we
still iterate over the original buf[], which may begin with the
UTF-8 BOM we are supposed to be skipping.  At the end of the first
line, the code grabs the contents of it starting at "entry", so
there is nothing wrong per-se, but the logic looks really confused.

Instead, move the buf pointer and shrink its size, to truly
pretend that UTF-8 BOM did not exist in the input.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 11:26:29 -07:00
Carlos Martín Nieto
245e1c196d dir: allow a BOM at the beginning of exclude files
Some text editors like Notepad or LibreOffice write an UTF-8 BOM in
order to indicate that the file is Unicode text rather than whatever the
current locale would indicate.

If someone uses such an editor to edit a gitignore file, we are left
with those three bytes at the beginning of the file. If we do not skip
them, we will attempt to match a filename with the BOM as prefix, which
won't match the files the user is expecting.

Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 10:17:04 -07:00
Jeff King
a337292675 t1430: add another refs-escape test
In t1430, we check whether deleting the branch "../../foo"
will delete ".git/foo". However, this is not that
interesting a test; the precious file ".git/foo" does not
look like a ref, so even if we did not notice the "escape"
from the "refs/" hierarchy, we would fail for that reason
(i.e., if you turned refname_is_safe into a noop, the test
still passes).

Let's add an additional test for the same thing, but with a
file that actually looks like a ref. That will make sure we
are exercising the refname_is_safe code. While we're at it,
let's also make the code work a little harder by adding some
extra paths and some empty path components.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:06 -07:00
Jeff King
03afcbee9b read_packed_refs: avoid double-checking sane refs
Prior to d0f810f (refs.c: allow listing and deleting badly
named refs, 2014-09-03), read_packed_refs would barf on any
malformed refnames by virtue of calling create_ref_entry
with the "check" parameter set to 1. That commit loosened
our reading so that we call check_refname_format ourselves
and just set a REF_BAD_NAME flag.

We then call create_ref_entry with the check parameter set
to 0. That function learned to do an extra safety check even
when the check parameter is 0, so that we don't load any
dangerous refnames (like "../../../etc/passwd"). This is
implemented by calling refname_is_safe() in
create_ref_entry().

However, we can observe that refname_is_safe() can only be
true if check_refname_format() also failed. So in the common
case of a sanely named ref, we perform _both_ checks, even
though we know that the latter will never trigger. This has
a noticeable performance impact when the packed-refs file is
large.

Let's drop the refname_is_safe check from create_ref_entry(),
and make it the responsibility of the caller.  Of the three
callers that pass a check parameter of "0", two will have
just called check_refname_format(), and can check the
refname-safety only when it fails. The third case,
pack_if_possible_fn, is copying from an existing ref entry,
which must have previously passed our safety check.

With this patch, running "git rev-parse refs/heads/does-not-exist"
on a repo with a large (1.6GB) packed-refs file went from:

  real    0m6.768s
  user    0m6.340s
  sys     0m0.432s

to:

  real    0m5.703s
  user    0m5.276s
  sys     0m0.432s

for a wall-clock speedup of 15%.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:05 -07:00
Jeff King
0cc30e0e84 strbuf_getwholeline: use getdelim if it is available
We spend a lot of time in strbuf_getwholeline in a tight
loop reading characters from a stdio handle into a buffer.
The libc getdelim() function can do this for us with less
overhead. It's in POSIX.1-2008, and was a GNU extension
before that. Therefore we can't rely on it, but can fall
back to the existing getc loop when it is not available.

The HAVE_GETDELIM knob is turned on automatically for Linux,
where we have glibc. We don't need to set any new
feature-test macros, because we already define _GNU_SOURCE.
Other systems that implement getdelim may need to other
macros (probably _POSIX_C_SOURCE >= 200809L), but we can
address that along with setting the Makefile knob after
testing the feature on those systems.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):

  real    0m8.601s
  user    0m8.084s
  sys     0m0.524s

to:

  real    0m6.768s
  user    0m6.340s
  sys     0m0.432s

for a wall-clock speedup of 21%.

Based on a patch from Rasmus Villemoes <rv@rasmusvillemoes.dk>.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:05 -07:00
Jeff King
f80c153bea strbuf_getwholeline: avoid calling strbuf_grow
As with the recent speedup to strbuf_addch, we can avoid
calling strbuf_grow() in a tight loop of single-character
adds by instead checking strbuf_avail.

Note that we would instead call strbuf_addch directly here,
but it does more work than necessary: it will NUL-terminate
the result for each character read. Instead, in this loop we
read the characters one by one and then add the terminator
manually at the end.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from
(best-of-5):

  real    0m10.948s
  user    0m10.548s
  sys     0m0.412s

to:

  real    0m8.601s
  user    0m8.084s
  sys     0m0.524s

for a wall-clock speedup of 21%.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:05 -07:00
Jeff King
fec501dae8 strbuf_addch: avoid calling strbuf_grow
We mark strbuf_addch as inline, because we expect it may be
called from a tight loop. However, the first thing it does
is call the non-inline strbuf_grow(), which can handle
arbitrary-sized growth. Since we know that we only need a
single character, we can use the inline strbuf_avail() to
quickly check whether we need to grow at all.

Our check is redundant when we do call strbuf_grow(), but
that's OK. The common case is that we avoid calling it at
all, and we have made that case faster.

On a silly pathological case:

  perl -le '
    print "[core]";
    print "key$_ = value$_" for (1..1000000)
  ' >input
  git config -f input core.key1

this dropped the time to run git-config from:

  real    0m0.159s
  user    0m0.152s
  sys     0m0.004s

to:

  real    0m0.140s
  user    0m0.136s
  sys     0m0.004s

for a savings of 12%.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:05 -07:00
Jeff King
260d408e32 config: use getc_unlocked when reading from file
We read config files character-by-character from a stdio
handle using fgetc(). This incurs significant locking
overhead, even though we know that only one thread can
possibly access the handle. We can speed this up by taking
the lock ourselves, and then using getc_unlocked to read
each character.

On a silly pathological case:

  perl -le '
    print "[core]";
    print "key$_ = value$_" for (1..1000000)
  ' >input
  git config -f input core.key1

this dropped the time to run git-config from:

  real    0m0.263s
  user    0m0.260s
  sys     0m0.000s

to:

  real    0m0.159s
  user    0m0.152s
  sys     0m0.004s

for a savings of 39%.  Most config files are not this big,
but the savings should be proportional to the size of the
file (i.e., we always save 39%, just of a much smaller
number).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:05 -07:00
Jeff King
82912d1de8 strbuf_getwholeline: use getc_unlocked
strbuf_getwholeline calls getc in a tight loop. On modern
libc implementations, the stdio code locks the handle for
every operation, which means we are paying a significant
overhead.  We can get around this by locking the handle for
the whole loop and using the unlocked variant.

Running "git rev-parse refs/heads/does-not-exist" on a repo
with an extremely large (1.6GB) packed-refs file went from:

  real    0m18.900s
  user    0m18.472s
  sys     0m0.448s

to:

  real    0m10.953s
  user    0m10.384s
  sys     0m0.580s

for a wall-clock speedup of 42%. All times are best-of-3,
and done on a glibc 2.19 system.

Note that we call into strbuf_grow while holding the lock.
It's possible for that function to call other stdio
functions (e.g., printing to stderr when dying due to malloc
error); however, the POSIX.1-2001 definition of flockfile
makes it clear that the locks are per-handle, so we are fine
unless somebody else tries to read from our same handle.
This doesn't ever happen in the current code, and is
unlikely to be added in the future (we would have to do
something exotic like add a die_routine that tried to read
from stdin).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:04 -07:00
Jeff King
f43cce23ad git-compat-util: add fallbacks for unlocked stdio
POSIX.1-2001 specifies some functions for optimizing the
locking out of tight getc() loops. Not all systems are
POSIX, though, and even not all POSIX systems are required
to implement these functions. We can check for the
feature-test macro to see if they are available, and if not,
provide a noop implementation.

There's no Makefile knob here, because we should just detect
this automatically. If there are very bizarre systems, we
may need to add one, but it's not clear yet in which
direction:

  1. If a system defines _POSIX_THREAD_SAFE_FUNCTIONS but
     these functions are missing or broken, we would want a
     knob to manually turn them off.

  2. If a system has these functions but does not define
     _POSIX_THREAD_SAFE_FUNCTIONS, we would want a knob to
     manually turn them on.

We can add such a knob when we find a real-world system that
matches this.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:04 -07:00
Jeff King
3446a59b39 strbuf_getwholeline: use getc macro
strbuf_getwholeline calls fgetc in a tight loop. Using the
getc form, which can be implemented as a macro, should be
faster (and we do not care about it evaluating our argument
twice, as we just have a plain variable).

On my glibc system, running "git rev-parse
refs/heads/does-not-exist" on a file with an extremely large
(1.6GB) packed-refs file went from (best of 3 runs):

  real    0m19.383s
  user    0m18.876s
  sys     0m0.528s

to:

  real    0m18.900s
  user    0m18.472s
  sys     0m0.448s

for a wall-clock speedup of 2.5%.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16 08:15:04 -07:00
Junio C Hamano
3d6bc9a763 Revert "merge: pass verbosity flag down to merge-recursive"
This reverts commit 2bf15a3330, whose
intention was good, but the verbosity levels used in merge-recursive
turns out to be rather uneven.  For example, a merge of two branches
with conflicting submodule updates used to report CONFLICT: output
with --quiet but no longer (which *is* desired), while the final
"Automatic merge failed; fix conflicts and then commit" message is
still shown even with --quiet (which *is* inconsistent).

Originally reported by Bryan Turner; it is too early to declare what
the concensus is, but it seems that we would need to level the
verbosity levels used in merge strategy backends before we can go
forward.  In the meantime, we'd revert to the old behaviour until
that happens.

cf. $gmane/267245
2015-04-16 08:03:14 -07:00
Johannes Sixt
9a9a41db83 compat/mingw: stubs for getpgid() and tcgetpgrp()
Windows does not have process groups. It is, therefore, the simplest
to pretend that each process is in its own process group.

While here, move the getppid() stub from its old location (between
two sync related functions) next to the two new functions.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-15 11:50:34 -07:00
Luke Mewburn
85cb8906f0 progress: no progress in background
Disable the display of the progress if stderr is not the
current foreground process.
Still display the final result when done.

Signed-off-by: Luke Mewburn <luke@mewburn.net>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-15 11:50:24 -07:00
Junio C Hamano
f6e6362107 parse_date_basic(): let the system handle DST conversion
The function parses the input to compute the broken-down time in
"struct tm", and the GMT timezone offset.  If the timezone offset
does not exist in the input, the broken-down time is turned into the
number of seconds since epoch both in the current timezone and in
GMT and the offset is computed as their difference.

However, we forgot to make sure tm.tm_isdst is set to -1 (i.e. let
the system figure out if DST is in effect in the current timezone
when turning the broken-down time to the number of seconds since
epoch); it is done so at the beginning of the function, but a call
to match_digit() in the function can lead to a call to gmtime_r() to
clobber the field.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Diagnosed-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-15 10:25:32 -07:00
Junio C Hamano
7fcec48da9 parse_date_basic(): return early when given a bogus timestamp
When the input does not have GMT timezone offset, the code computes
it by computing the local and GMT time for the given timestamp. But
there is no point doing so if the given timestamp is known to be a
bogus one.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-15 10:25:05 -07:00
Junio C Hamano
e46fe3df01 Git 2.4.0-rc2
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-14 11:57:13 -07:00
Junio C Hamano
7a1aa0c288 Merge branch 'jk/colors'
"diff-highlight" (in contrib/) used to show byte-by-byte
differences, which meant that multi-byte characters can be chopped
in the middle.  It learned to pay attention to character boundaries
(assuming the UTF-8 payload).

* jk/colors:
  diff-highlight: do not split multibyte characters
2015-04-14 11:49:13 -07:00
Junio C Hamano
3cdff83fb0 Merge branch 'jk/merge-quiet'
"git merge --quiet" did not squelch messages from the underlying
merge-recursive strategy.

* jk/merge-quiet:
  merge: pass verbosity flag down to merge-recursive
2015-04-14 11:49:12 -07:00
Junio C Hamano
f8e593e9a7 Merge branch 'jk/pack-corruption-post-mortem'
Documentation update.

* jk/pack-corruption-post-mortem:
  howto: document more tools for recovery corruption
2015-04-14 11:49:11 -07:00
Junio C Hamano
fa9aaa8f10 Merge branch 'jc/update-instead-into-void'
A push into an unborn branch, with "receive.denyCurrentBranch" set
to "updateInstead", did not check out the working tree as expected.

* jc/update-instead-into-void:
  push-to-deploy: allow pushing into an unborn branch and updating it
2015-04-14 11:49:10 -07:00
Junio C Hamano
d2ae751b1c Merge branch 'sb/plug-streaming-leak'
* sb/plug-streaming-leak:
  streaming.c: fix a memleak
2015-04-14 11:49:09 -07:00
Junio C Hamano
0cb7054e75 Merge branch 'jn/doc-fast-import-no-16-octopus-limit'
Documentation update.

* jn/doc-fast-import-no-16-octopus-limit:
  fast-import doc: remove suggested 16-parent limit
2015-04-14 11:49:08 -07:00
Junio C Hamano
011cb58bfb Merge branch 'sb/plug-wt-shortstatus-tracking-leak'
* sb/plug-wt-shortstatus-tracking-leak:
  wt-status.c: fix a memleak
2015-04-14 11:49:07 -07:00