fast-import was relying on the fact that on most systems mmap() and
write() are synchronized by the filesystem's buffer cache. We were
relying on the ability to mmap() 20 bytes beyond the current end
of the file, then later fill in those bytes with a future write()
call, then read them through the previously obtained mmap() address.
This isn't always true with some implementations of NFS, but it is
especially not true with our NO_MMAP=YesPlease build time option used
on some platforms. If fast-import was built with NO_MMAP=YesPlease
we used the malloc()+pread() emulation and the subsequent write()
call does not update the trailing 20 bytes of a previously obtained
"mmap()" (aka malloc'd) address.
Under NO_MMAP that behavior causes unpack_entry() in sha1_file.c to
be unable to read an object header (or data) that has been unlucky
enough to be written to the packfile at a location such that it
is in the trailing 20 bytes of a window previously opened on that
same packfile.
This bug has gone unnoticed for a very long time as it is highly data
dependent. Not only does the object have to be placed at the right
position, but it also needs to be positioned behind some other object
that has been accessed due to a branch cache invalidation. In other
words the stars had to align just right, and if you did run into
this bug you probably should also have purchased a lottery ticket.
Fortunately the workaround is a lot easier than the bug explanation.
Before we allow unpack_entry() to read data from a pack window
that has also (possibly) been modified through write() we force
all existing windows on that packfile to be closed. By closing
the windows we ensure that any new access via the emulated mmap()
will reread the packfile, updating to the current file content.
This comes at a slight performance degredation as we cannot reuse
previously cached windows when we update the packfile. But it
is a fairly minor difference as the window closes happen at only
two points:
- When the packfile is finalized and its .idx is generated:
At this stage we are getting ready to update the refs and any
data access into the packfile is going to be random, and is
going after only the branch tips (to ensure they are valid).
Our existing windows (if any) are not likely to be positioned
at useful locations to access those final tip commits so we
probably were closing them before anyway.
- When the branch cache missed and we need to reload:
At this point fast-import is getting change commands for the next
commit and it needs to go re-read a tree object it previously
had written out to the packfile. What windows we had (if any)
are not likely to cover the tree in question so we probably were
closing them before anyway.
We do try to avoid unnecessarily closing windows in the second case
by checking to see if the packfile size has increased since the
last time we called unpack_entry() on that packfile. If the size
has not changed then we have not written additional data, and any
existing window is still vaild. This nicely handles the cases where
fast-import is going through a branch cache reload and needs to read
many trees at once. During such an event we are not likely to be
updating the packfile so we do not cycle the windows between reads.
With this change in place t9301-fast-export.sh (which was broken
by c3b0dec509) finally works again.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We also move the assignment of -1 to the lock file descriptor
up, so that rollback_lock_file() can be called safely after a
possible attempt to fclose(). This matches the contents of
the 'if' statement just above testing success of fdopen().
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of calling close_lock_file() and commit_lock_file() directly,
which take a struct lock_file argument, add two new functions:
close_ref() and commit_ref(), which handle calling the previous
lock_file functions and modifying the ref_lock structure.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The lockfile API is a handy way to obtain a file that is cleaned
up if you die(). But sometimes you would need this sequence to
work:
1. hold_lock_file_for_update() to get a file descriptor for
writing;
2. write the contents out, without being able to decide if the
results should be committed or rolled back;
3. do something else that makes the decision --- and this
"something else" needs the lockfile not to have an open file
descriptor for writing (e.g. Windows do not want a open file
to be renamed);
4. call commit_lock_file() or rollback_lock_file() as
appropriately.
This adds close_lock_file() you can call between step 2 and 3 in
the above sequence.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We have nice set of placeholders, but nobody stepped in to fill
the gap in the API documentation, so I am doing it myself.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes write_ref_sha1() more careful: it actually checks the SHA1 of
the ref it is updating, and refuses to update a ref with an object that it
cannot find.
Perhaps more importantly, it also refuses to update a branch head with a
non-commit object. I don't quite know *how* the stable series maintainers
were able to corrupt their repository to have a HEAD that pointed to a tag
rather than a commit object, but they did. Which results in a totally
broken repository that cannot be cloned or committed on.
So make it harder for people to shoot themselves in the foot like that.
The test t1400-update-ref.sh is fixed at the same time, as it
assumed that the commands involved in the particular test would
not care about corrupted repositories whose refs point at
nonexistant bogus objects. That assumption does not hold true
anymore.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since having non-commits in branches is a no-no, and just means you cannot
commit on them, let's make fsck tell you when a branch is bad.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When creating the commit object, be a whole lot more careful about making
sure that the parent lines really are valid parent lines. Check things
like MERGE_HEAD having proper SHA1 lines in it, and double-check that all
the parents exist and are actually commits.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are two heuristics in Git to detect whether a file is binary
or text. One in xdiff-interface.c (which is taken from GNU diff)
relies on existence of the NUL byte at the beginning. However,
convert.c used a different heuristic, which relied on the percent
of non-printable symbols (less than 1% for text files).
Due to differences in detection whether a file is binary or not,
it was possible that a file that diff treats as binary could be
treated as text by CRLF conversion. This is very confusing for a
user who sees that 'git diff' shows the file as binary expects it
to be added as binary.
This patch makes is_binary to consider any file that contains at
least one NUL character as binary, to ensure that the heuristics
used for CRLF conversion is tighter than what is used by diff.
Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The codepath to prepare index files for the temporary and next
index file was closing file descriptor it obtained from the
lockfile API by hand, without letting the API know that the fd
should not be doubly closed.
This is not usually a problem (except it may get EBADFD) but if
we opened another fd for an entirely unrelated purpose (say, an
fd used to mmap a packfile) between the time we close the fd to
the index file and the time we commit or rollback the lockfile
(causing it to also try closing the recorded fd), the lockfile
API will close an incorrect file descriptor that is still used
for an entirely unrelated purpose.
There's four close(fd) calls in prepare_index() and they're all
incorrect. The open fd's are cleaned up in rollback_index_files() and
shouldn't be closed manually. The patch below gets rid of the extra
close() calls and should fix the problem.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch improves all of the popen calls in hg-to-git.py by specifying the
template 'hg log' should use instead of calling 'hg log' and grepping for the
desired data.
Signed-off-by: Mark Drago <markdrago@gmail.com>
Acked-by: Stelian Pop <stelian@popies.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These options are listed in the manpage (aliases for --tags/--heads) but they
were not handled.
Signed-off-by: Miklos Vajna <vmiklos@frugalware.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The command repeats "Rebasing (1/1)" many times even when
there is only one task remaining, because mark_action_done() is
called to skip comment and empty lines in the TODO file.
This should fix it.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rerere.enabled is _not_ on by default. The command is enabled if rr-cache
exists even when rerere.enabled is missing, and enabled or disabled by
explicitly setting the rerere.enabled variable.
This reverts commit 16335fdd7a.
We are calling overlay_tree_on_cache() which does use CE_UPDATE
flag to mark duplicated entries, which is the same as the
codepath in git-ls-files with its --with-tree option.
Because the pathname ce->name is given to path_list_insert()
which does not allow duplicates, there is no breakage either way
from the correctness point of view in this codepath, unlike the
one in ls-files. But avoiding unnecessary processing with a
single bit check is certainly better.
This hook thought to have found a conflict marker any time it saw
a 7-character combination of any of the characters '<>=' at the
beginning of a line, whereas it should only look for the *same*
character to appear repeatedly.
Also, restrict it to match exactly 7 times, to avoid matching the
underlining with '='-characters often used in documentation.
Signed-off-by: Jean-Luc Herren <jlh@gmx.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git cvsimport was the last tool to use repo-config instead of config. Update
it to use plain git config.
Signed-off-by: Dan McGee <dpmcgee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Switch all git command calls to use the git (space) command format, and
remove the use of git-repo-config in place of git config.
Signed-off-by: Dan McGee <dpmcgee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2888605c64 (builtin-commit: fix
partial-commit support) mindlessly cut and pasted from
builtin-ls-files.c, and included a part that was meant to
exclude redundant path after "ls-files --with-tree" overlayed
the HEAD commit on top of the index. This logic does not apply
to what git-commit does and should not have been copied, even
though it would not hurt.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running "git commit paths" to create a partial commit, we
used to carefully build the temporary index so that we do not
lose the cached stat information. The rewrite of the command in
C lost it by carelessly using read_tree().
This resurrects the earlier behaviour to keep the cached stat
information as much as possible by using one-tree merge logic.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "seen" variable is used by match_pathspec, and must have
as many elements as there are in the given pathspec. We
create the pathspec either from the command line arguments
_or_ from just the current prefix.
Thus allocating "seen" based upon just argc is wrong, since
if argc == 0, then we still have one pathspec, the prefix,
but we don't allocate any space in "seen".
Signed-off-by: Jeff King <peff@peff.net>
Tested-by: İsmail Dönmez <ismail@pardus.org.tr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Repositories generated by svnsync cannot be relied on to have
properly set revprops without newlines in UUIDs and URLs. There
may be broken versions of svnsync out there that append extra
newlines to UUIDs, or the revprops could've been changed by
repository administrators at any time, too.
At least one repository we've come across has an embedded
newline erroneously set in the svnsync-uuid prop. This is bad
because the trailing newline is taken as another record by the
Git.pm library, and the wantarray detection causes tmp_config()
to return an array with an empty-but-existing second element.
We will now strip leading and trailing whitespace both before
setting and after reading the uuid and url for svnsync values.
We will also force tmp_config to return a single scalar when
reading existing values.
SVN UUIDs should never have whitespace in them, and SVN
repository URLs should be URI-escaped, so neither of those
values we ever see in git-svn should actually have whitespace
in them.
Thanks to Dennis Schridde for the bug report and Junio for
helping diagnose this.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This documents the default values of gc.auto, gc.autopacklimit
fetch.unpacklimit, receive.unpacklimit and transfer.unpacklimit.
Signed-off-by: Michele Ballabio <barra_cuda@katamail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The po2msg.sh script and the .gitignore in the po directory have been
shamelessly copied from the current git-gui. This enables the top
level "make NO_MSGFMT" to work consistently for git across the git-gui
and gitk sub-projects.
This is the same effective patch that has previously been posted as a
git.git patch which more succinctly described the copying of
po/.gitignore and po/po2msg.sh from git-gui.
Signed-off-by: Charles Bailey <charles@hashpling.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We somehow called deflateEnd() on a stream that we have called
deflateEnd() on already.
In fact, the second deflateEnd() has always been returning
Z_STREAM_ERROR. We just never checked the error return from
that particular deflateEnd().
The first one returns 0 for success. We might want to tighten
the check even more to check that.
Noticed by Marco.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I noticed some unchecked writes. This fixes them.
* bundle.c (create_bundle): Die upon write failure.
* fast-import.c (keep_pack): Die upon write or close failure.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
prop_walk adds a leading / to all subdirectory paths. Unfortunately
this causes a problem when the remote repo lives in a subdirectory itself,
as the leading / causes subsequent PROPFIND calls to be executed on
the wrong path. Trimming the / before calling the PROPFIND fixes this problem.
Signed-off-by: Kevin Ballard <kevin@sb.org>
Acked-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
56752391a8 (Make "git gc" pack all
refs by default) changed the default of gc.packrefs to true, to
pack all refs by default in any repository. IOW, the users need
to disable it explicitly if they want to by setting the config
variable, since 1.5.3.
However, we forgot to update the documentation. This fixes it.
Signed-off-by: Florian La Roche <laroche@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "clear to end of line" sequence is used to nicely output the progress
indicator without leaving garbage on the terminal. However, this works
only on ANSI capable terminals. We use the same check as in color.c to
find out whether the terminal supports this feature and use a workaround
(a few spaces in a row) if it does not.
[jc: as an old fashoned git myself, and given the fact that the
possible prefix and suffix are small number of short constant strings,
I actually prefer a simpler-and-more-stupid approach. This is with
Nico's clean-up.]
Signed-off-by: Johannes Sixt <johannes.sixt@telecom.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* 'master' of git://git.kernel.org/pub/scm/gitk/gitk:
[PATCH] gitk: use user-configured background in view definition dialog
[PATCH] gitk: Update German translation
[PATCH] gitk: Update and fix Makefile
gitk: Restore some widget options whose defaults changed in Tk 8.5
gitk: Recode de.po to UTF-8
[PATCH] gitk i18n: Recode gitk from latin1 to utf8 so that the (c) copyright character is valid utf8.
[PATCH] gitk i18n: More markup -- various options menus
[PATCH] gitk i18n: Initial German translation
[PATCH] gitk i18n: Markup several strings for translation
[PATCH] gitk i18n: Import msgcat for message string translation; load translation catalogs
[PATCH] gitk i18n: Add Makefile with rules for po file creation and installation
Have the text fields in the view definition dialog (View->New view...)
use the background color as configured through the preferences, instead
of hard-coded 'white'.
This was suggested by Paul Wise through http://bugs.debian.org/457124
Signed-off-by: Gerrit Pape <pape@smarden.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
This Makefile uses the template provided at git.git/gitk-git/Makefile
by Junio and adds the rules for the i18n files.
Signed-off-by: Christian Stimming <stimming@tuhh.de>
Signed-off-by: Paul Mackerras <paulus@samba.org>
Don't set the needs-refresh flag when inserting a new file info, since
ewoc refreshes it upon insert already; this makes a full refresh twice
as fast.
Also make git-fileinfo-prettyprint a little faster by not retrieving
permission values twice.
Signed-off-by: Alexandre Julliard <julliard@winehq.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of just setting the state to up-to-date, retrieve the full
state again, so that the file type can be displayed properly.
Signed-off-by: Alexandre Julliard <julliard@winehq.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Skip non-zero stage files during git-ls-files -c, they are handled
later. Also fix git-insert-info-list to merge duplicate file names.
Signed-off-by: Alexandre Julliard <julliard@winehq.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running "git apply --check" while --whitespace=fix is
enabled (either from the command line or via the configuration),
we reported that "N line(s) applied after _fixing_", but --check
by itself does not apply and this message was alarming.
We could even reword the message to say "N line(s) would have
been applied after fixing...", but this patch does not go that
far. Instead, we just make it use the "N lines add whitespace
errors" warning, which happens to be a good diagnostic message a
user would expect from the --check option.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
