This puts delta resolving on each base on a separate thread, one base
cache per thread. Per-thread data is grouped in struct thread_local.
When running with nr_threads == 1, no pthreads calls are made. The
system essentially runs in non-thread mode.
An experiment on a Xeon 24 core machine with git.git shows that
performance does not increase proportional to the number of cores. So
by default, we use maximum 3 cores. Some numbers with --threads from 1
to 16:
1..4
real 0m8.003s 0m5.307s 0m4.321s 0m3.830s
user 0m7.720s 0m8.009s 0m8.133s 0m8.305s
sys 0m0.224s 0m0.372s 0m0.360s 0m0.360s
5..8
real 0m3.727s 0m3.604s 0m3.332s 0m3.369s
user 0m9.361s 0m9.817s 0m9.525s 0m9.769s
sys 0m0.584s 0m0.624s 0m0.540s 0m0.560s
9..12
real 0m3.036s 0m3.139s 0m3.177s 0m2.961s
user 0m8.977s 0m10.205s 0m9.737s 0m10.073s
sys 0m0.596s 0m0.680s 0m0.684s 0m0.680s
13..16
real 0m2.985s 0m2.894s 0m2.975s 0m2.971s
user 0m9.825s 0m10.573s 0m10.833s 0m11.361s
sys 0m0.788s 0m0.732s 0m0.904s 0m1.016s
On an Intel dual core and linux-2.6.git
1..4
real 2m37.789s 2m7.963s 2m0.920s 1m58.213s
user 2m28.415s 2m52.325s 2m50.176s 2m41.187s
sys 0m7.808s 0m11.181s 0m11.224s 0m10.731s
Thanks Ramsay Jones for troubleshooting and support on MinGW platform.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The second pass in parse_pack_objects() are split into
resolve_deltas(). The final phase, fixing thin pack or just seal the
pack, is now in conclude_pack() function. Main pack processing is now
a sequence of these functions:
- parse_pack_objects() reads through the input pack
- resolve_deltas() makes sure all deltas can be resolved
- conclude_pack() seals the output pack
- write_idx_file() writes companion index file
- final() moves the pack/index to proper place
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Update draft release notes to 1.7.9.3
CodingGuidelines: do not use 'which' in shell scripts
CodingGuidelines: Add a note about spaces after redirection
post-receive-email: match up $LOGBEGIN..$LOGEND pairs correctly
post-receive-email: remove unused variable
During the code review of a recent patch, it was noted that shell scripts
must not use 'which $cmd' to check the availability of the command $cmd.
The output of the command is not machine parseable and its exit code is
not reliable across platforms.
It is better to use 'type' to accomplish this task.
Signed-off-by: Tim Henigan <tim.henigan@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
During code review of some patches, it was noted that redirection operators
should have space before, but no space after them.
Signed-off-by: Tim Henigan <tim.henigan@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Without the magic line, prove shows lots and lots of errors:
% prove ./t9804-git-p4-label.sh
./t9804-git-p4-label.sh .. syntax error at ./t9804-git-p4-label.sh line 3, near ". ."
...
When #!/bin/sh is added, tests are skipped (I have no p4d).
Signed-off-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier, fba4f1 (grep -P: Fix matching ^ and $) fixed an ancient bug. Add
some tests to protect the change from future breakages; a slightly broken
version of this was a part of the originally submitted patch.
Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Signed-off-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
prep_for_email neither is passed a fourth argument nor uses it.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The option was already shown in -h output, so it should be documented
in the man page.
Signed-off-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Suggested-by: David A. Greene <greened@obbligato.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/gitweb-search-optim:
gitweb: Faster project search
gitweb: Option for filling only specified info in fill_project_list_info
gitweb: Refactor checking if part of project info need filling
* tr/maint-bundle-long-subject:
t5704: match tests to modern style
strbuf: improve strbuf_get*line documentation
bundle: use a strbuf to scan the log for boundary commits
bundle: put strbuf_readline_fd in strbuf.c with adjustments
* sp/smart-http-failure-to-push:
: Mask SIGPIPE on the command channel going to a transport helper
disconnect from remote helpers more gently
Conflicts:
transport-helper.c
* fc/push-prune:
push: add '--prune' option
remote: refactor code into alloc_delete_ref()
remote: reorganize check_pattern_match()
remote: use a local variable in match_push_refs()
Conflicts:
builtin/push.c
* jk/maint-avoid-streaming-filtered-contents:
do not stream large files to pack when filters are in use
teach dry-run convert_to_git not to require a src buffer
teach convert_to_git a "dry run" mode
It looks like commit 99fb6e04 (pack-objects: convert to use
parse_options(), 2012-02-01) moved the #ifdef NO_PTHREDS around but
hasn't noticed that the 'arg' variable no longer is available.
Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Acked-by: Nguyen Thai Ngoc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Document accumulated fixes since 1.7.9.2
Git 1.7.8.5
grep -P: Fix matching ^ and $
am: don't infloop for an empty input file
rebase -m: only call "notes copy" when rewritten exists and is non-empty
git-p4: remove bash-ism in t9800
git-p4: remove bash-ism in t9809
git-p4: fix submit regression with clientSpec and subdir clone
git-p4: set useClientSpec variable on initial clone
Makefile: add thread-utils.h to LIB_H
Conflicts:
RelNotes
t/t9809-git-p4-client-view.sh
When "git grep" is run with -P/--perl-regexp, it doesn't match ^ and $ at
the beginning/end of the line. This is because PCRE normally matches ^
and $ at the beginning/end of the whole text, not for each line, and "git
grep" passes a large chunk of text (possibly containing many lines) to
pcre_exec() and then splits the text into lines.
This makes "git grep -P" behave differently from "git grep -E" and also
from "grep -P" and "pcregrep":
$ cat file
a
b
$ git grep --no-index -P '^ ' file
$ git grep --no-index -E '^ ' file
file: b
$ grep -c -P '^ ' file
b
$ pcregrep -c '^ ' file
b
Reported-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-am.sh's check_patch_format function would attempt to preview
the patch to guess its format, but would go into an infinite loop
when the patch file happened to be empty. The solution: exit the
loop when "read" fails, not when the line var, "$l1" becomes empty.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This prevents a shell error complaining rebase-merge/rewritten doesn't exist.
Signed-off-by: Andrew Wong <andrew.kw.w@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This works in both bash and dash:
$ bash -c 'VAR=1 env' | grep VAR
VAR=1
$ dash -c 'VAR=1 env' | grep VAR
VAR=1
But environment variables assigned this way are not necessarily propagated
through a function in POSIX compliant shells:
$ bash -c 'f() { "$@"
}; VAR=1 f "env"' | grep VAR
VAR=1
$ dash -c 'f() { "$@"
}; VAR=1 f "env"' | grep VAR
Fix constructs like this, in particular, setting variables through
test_must_fail.
Based-on-patch-by: Vitor Antunes <vitor.hda@gmail.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Plain old $# works to count the number of arguments in
either bash or dash, even if the arguments have spaces.
Based-on-patch-by: Vitor Antunes <vitor.hda@gmail.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the --use-client-spec is given to clone, and the clone
path is a subset of the full tree as specified in the client,
future submits will go to the wrong place.
Factor out getClientSpec() so both clone/sync and submit can
use it. Introduce getClientRoot() that is needed for the client
spec case, and use it instead of p4Where().
Test the five possible submit behaviors (add, modify, rename,
copy, delete).
Reported-by: Laurent Charrière <lcharriere@promptu.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If --use-client-spec was given, set the matching configuration
variable. This is necessary to ensure that future submits
work properly.
The alternatives of requiring the user to set it, or providing
a command-line option on every submit, are error prone.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Starting with commit v1.7.8-165-g0579f91, grep.h includes
thread-utils.h, so the latter has to be added to LIB_H.
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
Acked-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Because git's object format requires us to specify the
number of bytes in the object in its header, we must know
the size before streaming a blob into the object database.
This is not a problem when adding a regular file, as we can
get the size from stat(). However, when filters are in use
(such as autocrlf, or the ident, filter, or eol
gitattributes), we have no idea what the ultimate size will
be.
The current code just punts on the whole issue and ignores
filter configuration entirely for files larger than
core.bigfilethreshold. This can generate confusing results
if you use filters for large binary files, as the filter
will suddenly stop working as the file goes over a certain
size. Rather than try to handle unknown input sizes with
streaming, this patch just turns off the streaming
optimization when filters are in use.
This has a slight performance regression in a very specific
case: if you have autocrlf on, but no gitattributes, a large
binary file will avoid the streaming code path because we
don't know beforehand whether it will need conversion or
not. But if you are handling large binary files, you should
be marking them as such via attributes (or at least not
using autocrlf, and instead marking your text files as
such). And the flip side is that if you have a large
_non_-binary file, there is a correctness improvement;
before we did not apply the conversion at all.
The first half of the new t1051 script covers these failures
on input. The second half tests the matching output code
paths. These already work correctly, and do not need any
adjustment.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we call convert_to_git in dry-run mode, it may still
want to look at the source buffer, because some CRLF
conversion modes depend on analyzing the source to determine
whether it is in fact convertible CRLF text.
However, the main motivation for convert_to_git's dry-run
mode is that we would decide which method to use to acquire
the blob's data (streaming versus in-core). Requiring this
source analysis creates a chicken-and-egg problem. We are
better off simply guessing that anything we can't analyze
will end up needing conversion.
This patch lets a caller specify a NULL src buffer when
using dry-run mode (and only dry-run mode). A non-zero
return value goes from "we would convert" to "we might
convert"; a zero return value remains "we would definitely
not convert".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some callers may want to know whether convert_to_git will
actually do anything before performing the conversion
itself (e.g., to decide whether to stream or handle blobs
in-core). This patch lets callers specify the dry run mode
by passing a NULL destination buffer. The return value,
instead of indicating whether conversion happened, will
indicate whether conversion would occur.
For readability, we also include a wrapper function which
makes it more obvious we are not actually performing the
conversion.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test did not adhere to the current style on several counts:
. empty lines around the test blocks, but within the test string
. ': > file' or even just '> file' with an extra space
. inconsistent indentation
. hand-rolled commits instead of using test_commit
Fix all of them.
There's a catch to the last point: test_commit creates a tag, which the
original test did not create. We still change it to test_commit, and
explicitly delete the tags, so as to highlight that the test relies on not
having them.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Clarify strbuf_getline() documentation, and add the missing documentation
for strbuf_getwholeline() and strbuf_getwholeline_fd().
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ld/git-p4-expanded-keywords:
: Teach git-p4 to unexpand $RCS$-like keywords that are embedded in
: tracked contents in order to reduce unnecessary merge conflicts.
git-p4: add initial support for RCS keywords
* jk/config-include:
: An assignment to the include.path pseudo-variable causes the named file
: to be included in-place when Git looks up configuration variables.
config: add include directive
config: eliminate config_exclusive_filename
config: stop using config_exclusive_filename
config: provide a version of git_config with more options
config: teach git_config_rename_section a file argument
config: teach git_config_set_multivar_in_file a default path
config: copy the return value of prefix_filename
t1300: add missing &&-chaining
docs/api-config: minor clarifications
docs: add a basic description of the config API
