This is the base for making symlink detection in the middle fo a pathname
saner and (much) more efficient.
Under various loads, we want to verify that the full path leading up to a
filename is a real directory tree, and that when we successfully do an
'lstat()' on a filename, we don't get a false positive due to a symlink in
the middle of the path that git should have seen as a symlink, not as a
normal path component.
The 'has_symlink_leading_path()' function already did this, and cached
a single level of symlink information, but didn't cache the _lack_ of a
symlink, so the normal behaviour was actually the wrong way around, and we
ended up doing an 'lstat()' on each path component to check that it was a
real directory.
This caches the last detected full directory and symlink entries, and
speeds up especially deep directory structures a lot by avoiding to
lstat() all the directories leading up to each entry in the index.
[ This can - and should - probably be extended upon so that we eventually
never do a bare 'lstat()' on any path entries at *all* when checking the
index, but always check the full path carefully. Right now we do not
generally check the whole path for all our normal quick index
revalidation.
We should also make sure that we're careful about all the invalidation,
ie when we remove a link and replace it by a directory we should
invalidate the symlink cache if it matches (and vice versa for the
directory cache).
But regardless, the basic function needs to be sane to do that. The old
'has_symlink_leading_path()' was not capable enough - or indeed the code
readable enough - to really do that sanely. So I'm pushing this as not
just an optimization, but as a base for further work. ]
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
git-bisect: make "start", "good" and "skip" succeed or fail atomically
git-am: cope better with an empty Subject: line
Ignore leading empty lines while summarizing merges
bisect: squelch "fatal: ref HEAD not a symref" misleading message
builtin-apply: Show a more descriptive error on failure when opening a patch
Clarify documentation of git-cvsserver, particularly in relation to git-shell
* maint-1.5.4:
git-bisect: make "start", "good" and "skip" succeed or fail atomically
git-am: cope better with an empty Subject: line
Ignore leading empty lines while summarizing merges
bisect: squelch "fatal: ref HEAD not a symref" misleading message
builtin-apply: Show a more descriptive error on failure when opening a patch
Clarify documentation of git-cvsserver, particularly in relation to git-shell
When a patch can't be opened (it doesn't exist, there are permission
problems, etc.) we get the usage text, which is not a proper indication of
failure.
Signed-off-by: Alberto Bertogli <albertito@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
An earlier commit 4be6096 (apply --unidiff-zero: loosen sanity checks for
--unidiff=0 patches, 2006-09-17) made match_beginning and match_end
computed incorrectly. If a hunk inserts at the beginning, old position
recorded at the hunk is line 0, and if a hunk changes at the beginning, it
is line 1. The new test added to t4104 exposes that the old code did not
insist on matching at the beginning for a patch to add a line to an empty
file.
An even older 65aadb9 (apply: force matching at the beginning.,
2006-05-24) was equally wrong in that it tried to take hints from the
number of leading context lines, to decide if the hunk must match at the
beginning, but we can just look at the line number in the hunk to decide.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
setup_git_directory_gently() only modified the value of its *nongit_ok
argument if we were not in a git repository. Now it will always set it
to 0 when we are inside a repository.
Also remove now unnecessary initializations in the callers of this
function.
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/apply-whitespace:
ws_fix_copy(): move the whitespace fixing function to ws.c
apply: do not barf on patch with too large an offset
core.whitespace: cr-at-eol
git-apply --whitespace=fix: fix whitespace fuzz introduced by previous run
builtin-apply.c: pass ws_rule down to match_fragment()
builtin-apply.c: move copy_wsfix() function a bit higher.
builtin-apply.c: do not feed copy_wsfix() leading '+'
builtin-apply.c: simplify calling site to apply_line()
builtin-apply.c: clean-up apply_one_fragment()
builtin-apply.c: mark common context lines in lineinfo structure.
builtin-apply.c: optimize match_beginning/end processing a bit.
builtin-apply.c: make it more line oriented
builtin-apply.c: push match-beginning/end logic down
builtin-apply.c: restructure "offset" matching
builtin-apply.c: refactor small part that matches context
* lt/in-core-index:
lazy index hashing
Create pathname-based hash-table lookup into index
read-cache.c: introduce is_racy_timestamp() helper
read-cache.c: fix a couple more CE_REMOVE conversion
Also use unpack_trees() in do_diff_cache()
Make run_diff_index() use unpack_trees(), not read_tree()
Avoid running lstat(2) on the same cache entry.
index: be careful when handling long names
Make on-disk index representation separate from in-core one
Previously a patch that records too large a line number caused the
offset matching code in git-apply to overstep its internal buffer.
Noticed by Johannes Schindelin.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
CRLF conversion bears a slight chance of corrupting data.
autocrlf=true will convert CRLF to LF during commit and LF to
CRLF during checkout. A file that contains a mixture of LF and
CRLF before the commit cannot be recreated by git. For text
files this is the right thing to do: it corrects line endings
such that we have only LF line endings in the repository.
But for binary files that are accidentally classified as text the
conversion can corrupt data.
If you recognize such corruption early you can easily fix it by
setting the conversion type explicitly in .gitattributes. Right
after committing you still have the original file in your work
tree and this file is not yet corrupted. You can explicitly tell
git that this file is binary and git will handle the file
appropriately.
Unfortunately, the desired effect of cleaning up text files with
mixed line endings and the undesired effect of corrupting binary
files cannot be distinguished. In both cases CRLFs are removed
in an irreversible way. For text files this is the right thing
to do because CRLFs are line endings, while for binary files
converting CRLFs corrupts data.
This patch adds a mechanism that can either warn the user about
an irreversible conversion or can even refuse to convert. The
mechanism is controlled by the variable core.safecrlf, with the
following values:
- false: disable safecrlf mechanism
- warn: warn about irreversible conversions
- true: refuse irreversible conversions
The default is to warn. Users are only affected by this default
if core.autocrlf is set. But the current default of git is to
leave core.autocrlf unset, so users will not see warnings unless
they deliberately chose to activate the autocrlf mechanism.
The safecrlf mechanism's details depend on the git command. The
general principles when safecrlf is active (not false) are:
- we warn/error out if files in the work tree can modified in an
irreversible way without giving the user a chance to backup the
original file.
- for read-only operations that do not modify files in the work tree
we do not not print annoying warnings.
There are exceptions. Even though...
- "git add" itself does not touch the files in the work tree, the
next checkout would, so the safety triggers;
- "git apply" to update a text file with a patch does touch the files
in the work tree, but the operation is about text files and CRLF
conversion is about fixing the line ending inconsistencies, so the
safety does not trigger;
- "git diff" itself does not touch the files in the work tree, it is
often run to inspect the changes you intend to next "git add". To
catch potential problems early, safety triggers.
The concept of a safety check was originally proposed in a similar
way by Linus Torvalds. Thanks to Dimitry Potapov for insisting
on getting the naked LF/autocrlf=true case right.
Signed-off-by: Steffen Prohaska <prohaska@zib.de>
This new error mode allows a line to have a carriage return at the
end of the line when checking and fixing trailing whitespace errors.
Some people like to keep CRLF line ending recorded in the repository,
and still want to take advantage of the automated trailing whitespace
stripping. We still show ^M in the diff output piped to "less" to
remind them that they do have the CR at the end, but these carriage
return characters at the end are no longer flagged as errors.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When you have more than one patch series, an earlier one of which
tries to introduce whitespace breakages and a later one of which
has such a new line in its context, "git-apply --whitespace=fix"
will apply and fix the whitespace breakages in the earlier one,
making the resulting file not to match the context of the later
patch.
A short demonstration is in the new test, t4125.
For example, suppose the first patch is:
diff a/hello.txt b/hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -20,3 +20,3 @@
Hello world.$
-How Are you$
-Today?$
+How are you $
+today? $
to fix broken case in the string, but it introduces unwanted
trailing whitespaces to the result (pretend you are looking at
"cat -e" output of the patch --- '$' signs are not in the patch
but are shown to make the EOL stand out). And the second patch
is to change the wording of the greeting further:
diff a/hello.txt b/hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -18,5 +18,5 @@
Greetings $
-Hello world.$
+Hello, everybody. $
How are you $
-today? $
+these days? $
If you apply the first one with --whitespace=fix, you will get
this as the result:
Hello world.$
How are you$
today?$
and this does not match the preimage of the second patch, which
demands extra whitespace after "How are you" and "today?".
This series is about teaching "git apply --whitespace=fix" to
cope with this situation better. If the patch does not apply,
it rewrites the second patch like this and retries:
diff a/hello.txt b/hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -18,5 +18,5 @@
Greetings$
-Hello world.$
+Hello, everybody.$
How are you$
-today?$
+these days?$
This is done by rewriting the preimage lines in the hunk
(i.e. the lines that begin with ' ' or '-'), using the same
whitespace fixing rules as it is using to apply the patches, so
that it can notice what it did to the previous ones in the
series.
A careful reader may notice that the first patch in the example
did not touch the "Greetings" line, so the trailing whitespace
that is in the original preimage of the second patch is not from
the series. Is rewriting this context line a problem?
If you think about it, you will realize that the reason for the
difference is because the submitter's tree was based on an
earlier version of the file that had whitespaces wrong on that
"Greetings" line, and the change that introduced the "Greetings"
line was added independently of this two-patch series to our
tree already with an earlier "git apply --whitespace=fix".
So it may appear this logic is rewriting too much, it is not
so. It is just rewriting what we would have rewritten in the
past.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is necessary to allow match_fragment() to attempt a match
with a preimage that is based on a version before whitespace
errors were fixed.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "patch" parameter used to include leading '+' of an added
line in the patch, and the array was treated as 1-based. Make
it accept the contents of the line alone and simplify the code.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function apply_line() changed its behaviour depending on the
ws_error_action, whitespace_error and if the input was a context.
Make its caller responsible for such checking so that we can convert
the function to copy the contents of line while fixing whitespace
breakage more easily.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We had two pointer variables pointing to the same buffer and an
integer variable used to index into its tail part that was
active (old, oldlines and oldsize for the preimage, and their
'new' counterparts for the postimage).
To help readability, use 'oldlines' as the allocated pointer,
and use 'old' as the pointer to the tail that advances while the
code builds up the contents in the buffer. The size 'oldsize'
can be computed as (old-oldines).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This updates the way preimage and postimage in a patch hunk is
parsed and prepared for applying. By looking at image->line[n].flag,
the code can tell if it is a common context line that is the
same between the preimage and the postimage.
This matters when we actually start applying a patch with
contexts that have whitespace breakages that have already been
fixed in the target file.
Wnen the caller knows the hunk needs to match at the beginning
or at the end, there is no point starting from the line number
that is found in the patch and trying match with increasing
offset. The logic to find matching lines was made more line
oriented with the previous patch and this optimization is now
trivial.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This changes the way git-apply internally works to be more line
oriented. The logic to find where the patch applies with offset
used to count line numbers by always counting LF from the
beginning of the buffer, but it is simplified because we count
the line length of the target file and the preimage snippet
upfront now.
The ultimate motivation is to allow applying patches
whose preimage context has whitespace corruption that has
already been corrected in the local copy. For that purpose, we
introduce a table of line-hash that allows us to match lines
that differ only in whitespaces.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This moves the logic to force match at the beginning and/or at
the end of the buffer to the actual function that finds the
match from its caller. This is a necessary preparation for the
next step to allow matching disregarding certain differences,
such as whitespace changes.
We probably could optimize this even more by taking advantage of
the fact that match_beginning and match_end forces the match to
be at an exact location (anchored at the beginning and/or the
end), but that's for another commit.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This restructures code to find matching location with offset
in find_offset() function, so that there is need for only one
call site of match_fragment() function. There still isn't a
change in the logic of the program.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This moves three "if" conditions out of line from find_offset()
function, which is responsible for finding the matching place in
the preimage to apply the patch. There is no change in the
logic of the program.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This converts the index explicitly on read and write to its on-disk
format, allowing the in-core format to contain more flags, and be
simpler.
In particular, the in-core format is now host-endian (as opposed to the
on-disk one that is network endian in order to be able to be shared
across machines) and as a result we can dispense with all the
htonl/ntohl on accesses to the cache_entry fields.
This will make it easier to make use of various temporary flags that do
not exist in the on-disk format.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
When running "git apply --check" while --whitespace=fix is
enabled (either from the command line or via the configuration),
we reported that "N line(s) applied after _fixing_", but --check
by itself does not apply and this message was alarming.
We could even reword the message to say "N line(s) would have
been applied after fixing...", but this patch does not go that
far. Instead, we just make it use the "N lines add whitespace
errors" warning, which happens to be a good diagnostic message a
user would expect from the --check option.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix any sequence of 8 spaces in initial indent, not just the case where
the 8 spaces are the first thing on the line.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use 0 instead of -1 for the case where not tabs or spaces are found; it
will make some later math slightly simpler.
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For consistency, make the two tools report whitespace errors in the
same way (the output of "diff --check" has been tweaked to match
that of "git apply").
Note that although the textual content is basically the same only
"git diff --check" provides a colorized version of the problematic
lines; making "git apply" do colorization will require more extensive
changes (figuring out the diff colorization preferences of the user)
and so that will be a subject for another commit.
Signed-off-by: Wincent Colaiuta <win@wincent.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit unifies three separate places where whitespace checking was
performed:
- the whitespace checking previously done in builtin-apply.c is
extracted into a function in ws.c
- the equivalent logic in "git diff" is removed
- the emit_line_with_ws() function is also removed because that also
rechecks the whitespace, and its functionality is rolled into ws.c
The new function is called check_and_emit_line() and it does two things:
checks a line for whitespace errors and optionally emits it. The checking
is based on lines of content rather than patch lines (in other words, the
caller must strip the leading "+" or "-"); this was suggested by Junio on
the mailing list to allow for a future extension to "git show" to display
whitespace errors in blobs.
At the same time we teach it to report all classes of whitespace errors
found for a given line rather than reporting only the first found error.
Signed-off-by: Wincent Colaiuta <win@wincent.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/spht:
Use gitattributes to define per-path whitespace rule
core.whitespace: documentation updates.
builtin-apply: teach whitespace_rules
builtin-apply: rename "whitespace" variables and fix styles
core.whitespace: add test for diff whitespace error highlighting
git-diff: complain about >=8 consecutive spaces in initial indent
War on whitespace: first, a bit of retreat.
Conflicts:
cache.h
config.c
diff.c
The `core.whitespace` configuration variable allows you to define what
`diff` and `apply` should consider whitespace errors for all paths in
the project (See gitlink:git-config[1]). This attribute gives you finer
control per path.
For example, if you have these in the .gitattributes:
frotz whitespace
nitfol -whitespace
xyzzy whitespace=-trailing
all types of whitespace problems known to git are noticed in path 'frotz'
(i.e. diff shows them in diff.whitespace color, and apply warns about
them), no whitespace problem is noticed in path 'nitfol', and the
default types of whitespace problems except "trailing whitespace" are
noticed for path 'xyzzy'. A project with mixed Python and C might want
to have:
*.c whitespace
*.py whitespace=-indent-with-non-tab
in its toplevel .gitattributes file.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This adds description of core.whitespace to the manual page of git-config,
and updates the stale description of whitespace handling in the manual
page of git-apply.
Also demote "strip" to a synonym status for "fix" as the value of --whitespace
option given to git-apply.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We earlier introduced core.whitespace to allow users to tweak the
definition of what the "whitespace errors" are, for the purpose of diff
output highlighting. This teaches the same to git-apply, so that the
command can both detect (when --whitespace=warn option is given) and fix
(when --whitespace=fix option is given) as configured.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The variables were somewhat misnamed.
* "What to do when whitespace errors are detected" is now called
"ws_error_action" (used to be called "new_whitespace");
* The constants to denote the possible actions are "nowarn_ws_error",
"warn_on_ws_error", "die_on_ws_error", and "correct_ws_error". The
last one used to be "strip_whitespace", but we correct whitespace
error in indent (SP followed by HT) and "strip" is not quite an
accurate name for it.
Other than the renaming of variables and constants, there is no
functional change in this patch. While we are at it, it also fixes
overly long lines and multi-line comment styles (which of course do
not affect the generated code at all).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Solaris Workshop Compiler found a few unreachable statements.
Signed-off-by: Guido Ostkamp <git@ostkamp.fastmail.fm>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/maint-add-sync-stat:
t2200: test more cases of "add -u"
git-add: make the entry stat-clean after re-adding the same contents
ce_match_stat, run_diff_files: use symbolic constants for readability
Conflicts:
builtin-add.c
ce_match_stat() can be told:
(1) to ignore CE_VALID bit (used under "assume unchanged" mode)
and perform the stat comparison anyway;
(2) not to perform the contents comparison for racily clean
entries and report mismatch of cached stat information;
using its "option" parameter. Give them symbolic constants.
Similarly, run_diff_files() can be told not to report anything
on removed paths. Also give it a symbolic constant for that.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix size_t vs. unsigned long pointer mismatch warnings introduced
with the addition of strbuf_detach().
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* ph/strbuf: (44 commits)
Make read_patch_file work on a strbuf.
strbuf_read_file enhancement, and use it.
strbuf change: be sure ->buf is never ever NULL.
double free in builtin-update-index.c
Clean up stripspace a bit, use strbuf even more.
Add strbuf_read_file().
rerere: Fix use of an empty strbuf.buf
Small cache_tree_write refactor.
Make builtin-rerere use of strbuf nicer and more efficient.
Add strbuf_cmp.
strbuf_setlen(): do not barf on setting length of an empty buffer to 0
sq_quote_argv and add_to_string rework with strbuf's.
Full rework of quote_c_style and write_name_quoted.
Rework unquote_c_style to work on a strbuf.
strbuf API additions and enhancements.
nfv?asprintf are broken without va_copy, workaround them.
Fix the expansion pattern of the pseudo-static path buffer.
builtin-for-each-ref.c::copy_name() - do not overstep the buffer.
builtin-apply.c: fix a tiny leak introduced during xmemdupz() conversion.
Use xmemdupz() in many places.
...
* make strbuf_read_file take a size hint (works like strbuf_read)
* use it in a couple of places.
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For that purpose, the ->buf is always initialized with a char * buf living
in the strbuf module. It is made a char * so that we can sloppily accept
things that perform: sb->buf[0] = '\0', and because you can't pass "" as an
initializer for ->buf without making gcc unhappy for very good reasons.
strbuf_init/_detach/_grow have been fixed to trust ->alloc and not ->buf
anymore.
as a consequence strbuf_detach is _mandatory_ to detach a buffer, copying
->buf isn't an option anymore, if ->buf is going to escape from the scope,
and eventually be free'd.
API changes:
* strbuf_setlen now always works, so just make strbuf_reset a convenience
macro.
* strbuf_detatch takes a size_t* optional argument (meaning it can be
NULL) to copy the buffer's len, as it was needed for this refactor to
make the code more readable, and working like the callers.
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-am used "git apply -z --index-info" to find the original versions
of the files touched by the diff, to be able to do an inexpensive
three-way merge.
This operation makes only sense in a repository, since the index
information in the diff refers to blobs, which have to be present in
the current repository.
Therefore, teach "git apply" a mode to write out the result as an
index file to begin with, obviating the need for scripts to do it
themselves.
The sole user for --index-info is "git am" is converted to
use --build-fake-ancestor in this patch.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
