Several tests wanted to verify that files were actually modified by a
merge, which it would do by checking that the mtime was updated. In
order to avoid problems with the merge completing so fast that the mtime
at the beginning and end of the operation was the same, these tests
would first set the mtime of a file to something "old". This "old"
value was usually determined as current system clock minus one second,
truncated to the nearest integer. Unfortunately, it appears the system
clock and filesystem clock are different and comparing across the two
runs into race problems resulting in flaky tests.
From https://stackoverflow.com/questions/14392975/timestamp-accuracy-on-ext4-sub-millsecond:
date will call the gettimeofday system call which will always return
the most accurate time available based on the cached kernel time,
adjusted by the CPU cycle time if available to give nanosecond
resolution. The timestamps stored in the file system however, are
only based on the cached kernel time. ie The time calculated at the
last timer interrupt.
and from https://apenwarr.ca/log/20181113:
Does mtime get set to >= the current time?
No, this depends on clock granularity. For example, gettimeofday()
can return times in microseconds on my system, but ext4 rounds
timestamps down to the previous ~10ms (but not exactly 10ms)
increment, with the surprising result that a newly-created file is
almost always created in the past:
$ python -c "
import os, time
t0 = time.time()
open('testfile', 'w').close()
print os.stat('testfile').st_mtime - t0
"
-0.00234484672546
So, instead of trying to compare across what are effectively two
different clocks, just avoid using the system clock. Any new updates to
files have to give an mtime at least as big as what is already in the
file, so we could define "old" as one second before the mtime found in
the file before the merge starts. But, to avoid problems with leap
seconds, ntp updates, filesystems that only provide two second
resolution, and other such weirdness, let's just pick an hour before the
mtime found in the file before the merge starts.
Also, clarify in one test where we check the mtime of different files
that it really was intentional. I totally forgot the reasons for that
and assumed it was a bug when asked.
Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In t6022, we were testing for file being overwritten (or not) based on
an output message instead of checking for the file being overwritten.
Since we can check for the file being overwritten via mtime updates,
check that instead.
In t6046, we were largely checking for both the expected behavior and a
proxy for it, which is unnecessary. The calls to test-tool also were a
bit cryptic. Make them a little clearer.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make use of test_path_is_file, test_write_lines, and similar helpers
in these old test files.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Indent code, and include it inside test_expect* blocks.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change various tests that use an idiom of the form:
>expect &&
test_cmp expect actual
To instead use:
test_must_be_empty actual
The test_must_be_empty() wrapper was introduced in ca8d148daf ("test:
test_must_be_empty helper", 2013-06-09). Many of these tests have been
added after that time. This was mostly found with, and manually pruned
from:
git grep '^\s+>.*expect.* &&$' t
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rename detection logic in "diff" family that is used in "merge" has
learned to guess when all of x/a, x/b and x/c have moved to z/a,
z/b and z/c, it is likely that x/d added in the meantime would also
want to move to z/d by taking the hint that the entire directory
'x' moved to 'z'. A bug causing dirty files involved in a rename
to be overwritten during merge has also been fixed as part of this
work. Incidentally, this also avoids updating a file in the
working tree after a (non-trivial) merge whose result matches what
our side originally had.
* en/rename-directory-detection-reboot: (36 commits)
merge-recursive: fix check for skipability of working tree updates
merge-recursive: make "Auto-merging" comment show for other merges
merge-recursive: fix remainder of was_dirty() to use original index
merge-recursive: fix was_tracked() to quit lying with some renamed paths
t6046: testcases checking whether updates can be skipped in a merge
merge-recursive: avoid triggering add_cacheinfo error with dirty mod
merge-recursive: move more is_dirty handling to merge_content
merge-recursive: improve add_cacheinfo error handling
merge-recursive: avoid spurious rename/rename conflict from dir renames
directory rename detection: new testcases showcasing a pair of bugs
merge-recursive: fix remaining directory rename + dirty overwrite cases
merge-recursive: fix overwriting dirty files involved in renames
merge-recursive: avoid clobbering untracked files with directory renames
merge-recursive: apply necessary modifications for directory renames
merge-recursive: when comparing files, don't include trees
merge-recursive: check for file level conflicts then get new name
merge-recursive: add computation of collisions due to dir rename & merging
merge-recursive: check for directory level conflicts
merge-recursive: add get_directory_renames()
merge-recursive: make a helper function for cleanup for handle_renames
...
The can-working-tree-updates-be-skipped check has had a long and blemished
history. The update can be skipped iff:
a) The merge is clean
b) The merge matches what was in HEAD (content, mode, pathname)
c) The target path is usable (i.e. not involved in D/F conflict)
Traditionally, we split b into parts:
b1) The merged result matches the content and mode found in HEAD
b2) The merged target path existed in HEAD
Steps a & b1 are easy to check; we have always gotten those right. While
it is easy to overlook step c, this was fixed seven years ago with commit
4ab9a157d0 ("merge_content(): Check whether D/F conflicts are still
present", 2010-09-20). merge-recursive didn't have a readily available
way to directly check step b2, so various approximations were used:
* In commit b2c8c0a762 ("merge-recursive: When we detect we can skip
an update, actually skip it", 2011-02-28), it was noted that although
the code claimed it was skipping the update, it did not actually skip
the update. The code was made to skip it, but used lstat(path, ...)
as an approximation to path-was-tracked-in-index-before-merge.
* In commit 5b448b8530 ("merge-recursive: When we detect we can skip
an update, actually skip it", 2011-08-11), the problem with using
lstat was noted. It was changed to the approximation
path2 && strcmp(path, path2)
which is also wrong. !path2 || strcmp(path, path2) would have been
better, but would have fallen short with directory renames.
* In c5b761fb27 ("merge-recursive: ensure we write updates for
directory-renamed file", 2018-02-14), the problem with the previous
approximation was noted and changed to
was_tracked(path)
That looks close to what we were trying to answer, but was_tracked()
as implemented at the time should have been named is_tracked(); it
returned something different than what we were looking for.
* To make matters more complex, fixing was_tracked() isn't sufficient
because the splitting of b into b1 and b2 is wrong. Consider the
following merge with a rename/add conflict:
side A: modify foo, add unrelated bar
side B: rename foo->bar (but don't modify the mode or contents)
In this case, the three-way merge of original foo, A's foo, and B's
bar will result in a desired pathname of bar with the same
mode/contents that A had for foo. Thus, A had the right mode and
contents for the file, and it had the right pathname present (namely,
bar), but the bar that was present was unrelated to the contents, so
the working tree update was not skippable.
Fix this by introducing a new function:
was_tracked_and_matches(o, path, &mfi.oid, mfi.mode)
and use it to directly check for condition b.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Compared to 'test-chmtime -v +0 file' which prints the mtime and
and the file name, 'test-chmtime --get file' displays only the mtime.
If it is used in combination with (+|=|=+|=-|-)seconds, it changes
and prints the new value.
test-chmtime -v +0 file | sed 's/[^0-9].*$//'
is now equivalent to:
test-chmtime --get file
Signed-off-by: Paul-Sebastian Ungureanu <ungureanupaulsebastian@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The primary purpose of 't6022-merge-rename.sh' is to test 'git merge',
but one of the tests runs it upstream of a pipe, hiding its exit code.
Consequently, the test could continue even if 'git merge' exited with
error.
Use an intermediate file between 'git merge' and 'test_i18ngrep' to
catch a potential failure of the former.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These are tests which are missing a link in their &&-chain,
but during a setup phase. We may fail to notice failure in
commands that build the test environment, but these are
typically not expected to fail at all (but it's still good
to double-check that our test environment is what we
expect).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Mark strings in merge-recursive for translation.
Some tests would start to fail with GETTEXT_POISON turned on after
this update. Use test_i18ncmp and test_i18ngrep where appropriate
to mark strings that should only be checked in the C locale output
to avoid such issues.
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Reviewed-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge-recursive detects renames so that if one side modifies
"foo" and the other side moves it to "bar", the modification
is applied to "bar". However, our rename detection is based
on content analysis, it can be wrong (i.e., two files were
not intended as a rename, but just happen to have the same
or similar content).
This is quite rare if the files actually contain content,
since two unrelated files are unlikely to have exactly the
same content. However, empty files present a problem, in
that there is nothing to analyze. An uninteresting
placeholder file with zero bytes may or may not be related
to a placeholder file with another name.
The result is that adding content to an empty file may cause
confusion if the other side of a merge removed it; your
content may end up in another random placeholder file that
was added.
Let's err on the side of caution and not consider empty
files as renames. This will cause a modify/delete conflict
on the merge, which will let the user sort it out
themselves.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Often times, a potential conflict at a path is resolved by merge-recursive
by using the content that was already present at that location. In such
cases, we do not want to overwrite the content that is already present, as
that could trigger unnecessary recompilations. One of the patches earlier
in this series ("merge-recursive: When we detect we can skip an update,
actually skip it") fixed the cases that involved content merges, but there
were a few other cases as well.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I stumbled across a case, this one not involving a content merge, where
git currently rewrites a file unnecessarily. A quick audit uncovered two
additional situations (also not involving content merges) with the same
problem.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Calling update_stages() before update_file() can sometimes result in git
thinking the file being updated is untracked (whenever update_stages
moves it to stage 3). Reverse the call order, and add a big comment to
update_stages to hopefully prevent others from making the same mistake.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
modify/delete and rename/delete share a lot of similarities; we'd like all
the criss-cross and D/F conflict handling specializations to be shared
between the two.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 882fd11 (merge-recursive: Delay content merging for renames 2010-09-20),
there was code that checked for whether we could skip updating a file in
the working directory, based on whether the merged version matched the
current working copy. Due to the desire to handle directory/file conflicts
that were resolvable, that commit deferred content merging by first
updating the index with the unmerged entries and then moving the actual
merging (along with the skip-the-content-update check) to another function
that ran later in the merge process. As part moving the content merging
code, a bug was introduced such that although the message about skipping
the update would be printed (whenever GIT_MERGE_VERBOSITY was sufficiently
high), the file would be unconditionally updated in the working copy
anyway.
When we detect that the file does not need to be updated in the working
copy, update the index appropriately and then return early before updating
the working copy.
Note that there was a similar change in b2c8c0a (merge-recursive: When we
detect we can skip an update, actually skip it 2011-02-28), but it was
reverted by 6db4105 (Revert "Merge branch 'en/merge-recursive'"
2011-05-19) since it did not fix both of the relevant types of unnecessary
update breakages and, worse, it made use of some band-aids that caused
other problems. The reason this change works is due to the changes earlier
in this series to (a) record_df_conflict_files instead of just unlinking
them early, (b) allowing make_room_for_path() to remove D/F entries,
(c) the splitting of update_stages_and_entry() to have its functionality
called at different points, and (d) making the pathnames of the files
involved in the merge available to merge_content().
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Whenever there are merge conflicts in file contents, we would mark the
different sides of the conflict with the two branches being merged.
However, when there is a rename involved as well, the branchname is not
sufficient to specify where the conflicting content came from. In such
cases, mark the two sides of the conflict with branchname:filename rather
than just branchname.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When dealing with file merging and renames and D/F conflicts and possible
criss-cross merges (how's that for a corner case?), we did not do a
thorough job ensuring the index and working directory had the correct
contents. Fix the logic in merge_content() to handle this. Also,
correct some erroneous tests in t6022 that were expecting the wrong number
of unmerged index entries. These changes fix one of the tests in t6042
(and almost fix another one from t6042 as well).
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a testcase that was broken by b2c8c0a (merge-recursive: When we
detect we can skip an update, actually skip it 2011-02-28) and fixed by
6db4105 (Revert "Merge branch 'en/merge-recursive'" 2011-05-19). Include
this testcase to ensure we don't regress it again.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This testcase was part of en/merge-recursive that was reverted in 6db4105
(Revert "Merge branch 'en/merge-recursive'" 2011-05-19). While the other
changes in that series caused unfortunate breakage, this testcase is still
useful; reinstate it.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since this test later does a git add -A, we should clean out unnecessary
untracked files as part of our cleanup.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* en/merge-recursive: (41 commits)
t6022: Use -eq not = to test output of wc -l
merge-recursive:make_room_for_directories - work around dumb compilers
merge-recursive: Remove redundant path clearing for D/F conflicts
merge-recursive: Make room for directories in D/F conflicts
handle_delete_modify(): Check whether D/F conflicts are still present
merge_content(): Check whether D/F conflicts are still present
conflict_rename_rename_1to2(): Fix checks for presence of D/F conflicts
conflict_rename_delete(): Check whether D/F conflicts are still present
merge-recursive: Delay modify/delete conflicts if D/F conflict present
merge-recursive: Delay content merging for renames
merge-recursive: Delay handling of rename/delete conflicts
merge-recursive: Move handling of double rename of one file to other file
merge-recursive: Move handling of double rename of one file to two
merge-recursive: Avoid doubly merging rename/add conflict contents
merge-recursive: Update merge_content() call signature
merge-recursive: Update conflict_rename_rename_1to2() call signature
merge-recursive: Structure process_df_entry() to handle more cases
merge-recursive: Have process_entry() skip D/F or rename entries
merge-recursive: New function to assist resolving renames in-core only
merge-recursive: New data structures for deferring of D/F conflicts
...
Conflicts:
t/t6020-merge-df.sh
t/t6036-recursive-corner-cases.sh
When comparing numbers such as "3" to "$(wc -l)", we should check for
numerical equality using -eq instead of string equality using = because
some implementations of wc output extra whitespace.
Signed-off-by: Brian Gernhardt <brian@gernhardtsoftware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using 'return' in an attempt to end a test assertion can have
unpredictable results (probably escaping from test_run_ and breaking
its bookkeeping). Redo the control flow using helpers like
test_expect_code and git diff --exit-code, so each test assertion can
follow the usual form
command that should succeed &&
command that should succeed &&
command that should succeed &&
...
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If all the paths below some directory involved in a D/F conflict were not
removed during the rest of the merge, then the contents of the file whose
path conflicted needs to be recorded in file with an alternative filename.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This function is called from process_df_entry(), near the end of the merge.
Rather than just checking whether one of the sides of the merge had a
directory at the same path as one of our files, check whether that
directory is still present by this point of our merge.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If all the paths below some directory involved in a D/F conflict were not
removed during the rest of the merge, then the contents of the file whose
path conflicted needs to be recorded in file with an alternative filename.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the handling of content merging for renames from process_renames() to
process_df_entry().
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the handling of rename/rename conflicts where one file is renamed to
two different files, from process_renames() to process_df_entry().
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add tests where one file is renamed to two different paths in different
sides of history, and where each of the new files matches the name of a
directory from the opposite side of history. Include tests for both the
case where the merge results in those directories not being cleanly
removed, and where those directories are cleanly removed during the merge.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
An interesting testcase is having two files each in their own subdirectory
getting renamed to the toplevel at the directory pathname of the other.
Questions arise as to whether the order of operations matters and whether
the directories can correctly get out of the way and make room for the
new files.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Having the source of a rename be involved in a directory/file conflict does
not currently pose any difficulties to the current merge-recursive
algorithm (in contrast to destinations of renames and D/F conflicts).
However, combining the two seemed like good testcases to include for
completeness.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When merging two branches with some path involved in a D/F conflict, the
choice of which branch to merge into the other matters for (at least) two
reasons: (1) whether the working copy has a directory full of files that
is in the way of a file, or a file exists that is in the way of a
directory of files, (2) when the directory full of files does not disappear
due to the merge, what files at the same paths should be renamed to
(e.g. filename~HEAD vs. filename~otherbranch).
Add some tests that reverse the merge order of two other tests, and which
verify the contents are as expected (namely, that the results are identical
other than modified-for-uniqueness filenames involving branch names).
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add testing of the various ways that a renamed file to a path involved in
a directory/file conflict may be involved in. This includes whether or not
there are conflicts of the contents of the renamed file (if the file was
modified on both sides of history), and whether the directory from the
other side of the merge will disappear as a result of the merge or not.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In tests, call test_cmp rather than raw diff where possible (i.e. if
the output does not go to a pipe), to allow the use of, say, 'cmp'
when the default 'diff -u' is not compatible with a vendor diff.
When that is not possible, use $DIFF, as set in GIT-BUILD-OPTIONS.
Signed-off-by: Gary V. Vaughan <gary@thewrittenword.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Also leave a warning for future merge-recursive explorers.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
When a merge results in a creation of a path that did not exist
in HEAD, and if you already have that path on the working tree,
because the index has not been told about the working tree file,
read-tree happily removes it. The issue was brought up by Santi
Béjar on the list.
Signed-off-by: Junio C Hamano <junkio@cox.net>
In some setups (notably server setups) you do not need that dependency.
Gracefully handle the absence of python when NO_PYTHON is defined.
Signed-off-by: Johannes E. Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This changes the way the case two branches rename the same path
to different paths is handled. Earlier, the code removed the
original path and added both destinations to the index at
stage0. This commit changes it to leave the original path at
stage1, and two destination paths at stage2 and stage3,
respectively.
[jc: I am not really sure if this makes much difference in the
real life merge situations. What should happen when our branch
renames A to B and M to N, while their branch renames A to M?
That is, M remains in our tree as is.]
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds a couple of tests to cover the following renaming
merge cases:
- one side renames and the other side does not, with and without
content conflicts.
- both side rename to the same path, with and without content
conflicts.
The test setup also prepares a case in which both side rename to
different destination, but currently the code collapses these
destination paths and removes the original path, which may be
wrong. The outcome of this case is not checked by the tests in
this round.
Signed-off-by: Junio C Hamano <junkio@cox.net>
