The default file history simplification of "git log -- <path>" or
"git rev-list -- <path>" focuses on providing the smallest set of
commits that first contributed a change. The revision walk greatly
restricts the set of walked commits by visiting only the first
TREESAME parent of a merge commit, when one exists. This means
that portions of the commit-graph are not walked, which can be a
performance benefit, but can also "hide" commits that added changes
but were ignored by a merge resolution.
The --full-history option modifies this by walking all commits and
reporting a merge commit as "interesting" if it has _any_ parent
that is not TREESAME. This tends to be an over-representation of
important commits, especially in an environment where most merge
commits are created by pull request completion.
Suppose we have a commit A and we create a commit B on top that
changes our file. When we merge the pull request, we create a merge
commit M. If no one else changed the file in the first-parent
history between M and A, then M will not be TREESAME to its first
parent, but will be TREESAME to B. Thus, the simplified history
will be "B". However, M will appear in the --full-history mode.
However, suppose that a number of topics T1, T2, ..., Tn were
created based on commits C1, C2, ..., Cn between A and M as
follows:
A----C1----C2--- ... ---Cn----M------P1---P2--- ... ---Pn
\ \ \ \ / / / /
\ \__.. \ \/ ..__T1 / Tn
\ \__.. /\ ..__T2 /
\_____________________B \____________________/
If the commits T1, T2, ... Tn did not change the file, then all of
P1 through Pn will be TREESAME to their first parent, but not
TREESAME to their second. This means that all of those merge commits
appear in the --full-history view, with edges that immediately
collapse into the lower history without introducing interesting
single-parent commits.
The --simplify-merges option was introduced to remove these extra
merge commits. By noticing that the rewritten parents are reachable
from their first parents, those edges can be simplified away. Finally,
the commits now look like single-parent commits that are TREESAME to
their "only" parent. Thus, they are removed and this issue does not
cause issues anymore. However, this also ends up removing the commit
M from the history view! Even worse, the --simplify-merges option
requires walking the entire history before returning a single result.
Many Git users are using Git alongside a Git service that provides
code storage alongside a code review tool commonly called "Pull
Requests" or "Merge Requests" against a target branch. When these
requests are accepted and merged, they typically create a merge
commit whose first parent is the previous branch tip and the second
parent is the tip of the topic branch used for the request. This
presents a valuable order to the parents, but also makes that merge
commit slightly special. Users may want to see not only which
commits changed a file, but which pull requests merged those commits
into their branch. In the previous example, this would mean the
users want to see the merge commit "M" in addition to the single-
parent commit "C".
Users are even more likely to want these merge commits when they
use pull requests to merge into a feature branch before merging that
feature branch into their trunk.
In some sense, users are asking for the "first" merge commit to
bring in the change to their branch. As long as the parent order is
consistent, this can be handled with the following rule:
Include a merge commit if it is not TREESAME to its first
parent, but is TREESAME to a later parent.
These merges look like the merge commits that would result from
running "git pull <topic>" on a main branch. Thus, the option to
show these commits is called "--show-pulls". This has the added
benefit of showing the commits created by closing a pull request or
merge request on any of the Git hosting and code review platforms.
To test these options, extend the standard test example to include
a merge commit that is not TREESAME to its first parent. It is
surprising that that option was not already in the example, as it
is instructive.
In particular, this extension demonstrates a common issue with file
history simplification. When a user resolves a merge conflict using
"-Xours" or otherwise ignoring one side of the conflict, they create
a TREESAME edge that probably should not be TREESAME. This leads
users to become frustrated and complain that "my change disappeared!"
In my experience, showing them history with --full-history and
--simplify-merges quickly reveals the problematic merge. As mentioned,
this option is expensive to compute. The --show-pulls option
_might_ show the merge commit (usually titled "resolving conflicts")
more quickly. Of course, this depends on the user having the correct
parent order, which is backwards when using "git pull master" from a
topic branch.
There are some special considerations when combining the --show-pulls
option with --simplify-merges. This requires adding a new PULL_MERGE
object flag to store the information from the initial TREESAME
comparisons. This helps avoid dropping those commits in later filters.
This is covered by a test, including how the parents can be simplified.
Since "struct object" has already ruined its 32-bit alignment by using
33 bits across parsed, type, and flags member, let's not make it worse.
PULL_MERGE is used in revision.c with the same value (1u<<15) as
REACHABLE in commit-graph.c. The REACHABLE flag is only used when
writing a commit-graph file, and a revision walk using --show-pulls
does not happen in the same process. Care must be taken in the future
to ensure this remains the case.
Update Documentation/rev-list-options.txt with significant details
around this option. This requires updating the example in the
History Simplification section to demonstrate some of the problems
with TREESAME second parents.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As we are working to rewrite some of the revision-walk machinery,
there could easily be some interesting interactions between the
options that force topological constraints (--topo-order,
--date-order, and --author-date-order) along with specifying a
path.
Add extra tests to t6012-rev-list-simplify.sh to add coverage of
these interactions. To ensure interesting things occur, alter the
repo data shape to have different orders depending on topo-, date-,
or author-date-order.
When testing using GIT_TEST_COMMIT_GRAPH, this assists in covering
the new logic for topo-order walks using generation numbers. The
extra tests can be added indepently.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Switch all uses of $_x40 to $OID_REGEX so that they work correctly with
larger hashes. This commit was created by using the following sed
command to modify all files in the t directory except t/test-lib.sh:
sed -i 's/\$_x40/$OID_REGEX/g'
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While it makes sense to allow merging unrelated histories of two
projects that started independently into one, in the way "gitk" was
merged to "git" itself aka "the coolest merge ever", such a merge is
still an unusual event. Worse, if somebody creates an independent
history by starting from a tarball of an established project and
sends a pull request to the original project, "git merge" however
happily creates such a merge without any sign of something unusual
is happening.
Teach "git merge" to refuse to create such a merge by default,
unless the user passes a new "--allow-unrelated-histories" option to
tell it that the user is aware that two unrelated projects are
merged.
Because such a "two project merge" is a rare event, a configuration
option to always allow such a merge is not added.
We could add the same option to "git pull" and have it passed
through to underlying "git merge". I do not have a fundamental
opposition against such a feature, but this commit does not do so
and instead leaves it as low-hanging fruit for others, because such
a "two project merge" would be done after fetching the other project
into some location in the working tree of an existing project and
making sure how well they fit together, it is sufficient to allow a
local merge without such an option pass-through from "git pull" to
"git merge". Many tests that are updated by this patch does the
pass-through manually by turning:
git pull something
into its equivalent:
git fetch something &&
git merge --allow-unrelated-histories FETCH_HEAD
If somebody is inclined to add such an option, updated tests in this
change need to be adjusted back to:
git pull --allow-unrelated-histories something
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests call test_cmp, and if it fails show the actual
output generated. This is mostly pointless, as test_cmp will
already show a diff between the expected and actual output.
It also fools --chain-lint by putting an "||" in the middle
of the chain, so we'd rather not use this construct.
Note that these cases actually show a pre-processed version
of the data, rather than exactly what test_cmp would show.
However, test_cmp's output is generally good for pointing
the user in the right direction, and they can then dig in
the trash directory themselves if they want to see more
details.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When using pathspec filtering in combination with diff-based log
output, parent simplification happens before the diff is computed.
The diff is therefore against the *simplified* parents.
This works okay, arguably by accident, in the normal case:
simplification reduces to one parent as long as the commit is TREESAME
to it. So the simplified parent of any given commit must have the
same tree contents on the filtered paths as its true (unfiltered)
parent.
However, --full-diff breaks this guarantee, and indeed gives pretty
spectacular results when comparing the output of
git log --graph --stat ...
git log --graph --full-diff --stat ...
(--graph internally kicks in parent simplification, much like
--parents).
To fix it, store a copy of the parent list before simplification (in a
slab) whenever --full-diff is in effect. Then use the stored parents
instead of the simplified ones in the commit display code paths. The
latter do not actually check for --full-diff to avoid duplicated code;
they just grab the original parents if save_parents() has not been
called for this revision walk.
For ordinary commits it should be obvious that this is the right thing
to do.
Merge commits are a bit subtle. Observe that with default
simplification, merge simplification is an all-or-nothing decision:
either the merge is TREESAME to one parent and disappears, or it is
different from all parents and the parent list remains intact.
Redundant parents are not pruned, so the existing code also shows them
as a merge.
So if we do show a merge commit, the parent list just consists of the
rewrite result on each parent. Running, e.g., --cc on this in
--full-diff mode is not very useful: if any commits were skipped, some
hunks will disagree with all sides of the merge (with one side,
because commits were skipped; with the others, because they didn't
have those changes in the first place). This triggers --cc showing
these hunks spuriously.
Therefore I believe that even for merge commits it is better to show
the diffs wrt. the original parents.
Reported-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reimplement commit 4b7f53da on top of the new simplify-merges
infrastructure, tightening the condition to only consider root parents;
the original version incorrectly dropped parents that were TREESAME to
anything.
Original log message follows.
The merge simplification rule stated in 6546b59 (revision traversal:
show full history with merge simplification, 2008-07-31) still
treated merge commits too specially. Namely, in a history with this
shape:
---o---o---M
/
x---x---x
where three 'x' were on a history completely unrelated to the main
history 'o' and do not touch any of the paths we are following, we
still said that after simplifying all of the parents of M, 'x'
(which is the leftmost 'x' that rightmost 'x simplifies down to) and
'o' (which would be the last commit on the main history that touches
the paths we are following) are independent from each other, and
both need to be kept.
That is incorrect; when the side branch 'x' never touches the paths,
it should be removed to allow M to simplify down to the last commit
on the main history that touches the paths.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kevin Bracey <kevin@bracey.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Kevin Bracey reports that the change regresses a case shown in the
user manual.
Trading one fix with another breakage is not worth it. Just keep
the test to document the existing breakage, and revert the change
for now.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The --simplify-merges logic did not cull irrelevant parents from a
merge that is otherwise not interesting with respect to the paths
we are following.
This touches a fairly core part of the revision traversal
infrastructure; even though I think this change is correct, please
report immediately if you find any unintended side effect.
* jc/remove-treesame-parent-in-simplify-merges:
simplify-merges: drop merge from irrelevant side branch
The merge simplification rule stated in 6546b59 (revision traversal:
show full history with merge simplification, 2008-07-31) still
treated merge commits too specially. Namely, in a history with this
shape:
---o---o---M
/
x---x---x
where three 'x' were on a history completely unrelated to the main
history 'o' and do not touch any of the paths we are following, we
still said that after simplifying all of the parents of M, 'x'
(which is the leftmost 'x' that rightmost 'x simplifies down to) and
'o' (which would be the last commit on the main history that touches
the paths we are following) are independent from each other, and
both need to be kept.
That is incorrect; when the side branch 'x' never touches the paths,
it should be removed to allow M to simplify down to the last commit
on the main history that touches the paths.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a topic branch workflow, you often want to find the latest commit that
merged a side branch that touched a particular area of the system, so that
a new topic branch to work on that area can be forked from that commit.
For example, I wanted to find an appropriate fork-point to queue Luke's
changes related to git-p4 in contrib/fast-import/.
"git log --first-parent" traverses the first-parent chain, and "-m --stat"
shows the list of paths touched by commits including merge commits. We
could ask the question this way:
# What is the latest commit that touched that path?
$ git log --first-parent --oneline -m --stat master |
sed -e '/^ contrib\/fast-import\/git-p4 /q' | tail
The above finds that 8cbfc11 (Merge branch 'pw/p4-view-updates',
2012-01-06) was such a commit.
But a more natural way to spell this question is this:
$ git log --first-parent --oneline -m --stat -1 master -- \
contrib/fast-import/git-p4
Unfortunately, this does not work. It finds ecb7cf9 (git-p4: rewrite view
handling, 2012-01-02). This commit is a part of the merged topic branch
and is _not_ on the first-parent path from the 'master':
$ git show-branch 8cbfc11ecb7cf9
! [8cbfc11] Merge branch 'pw/p4-view-updates'
! [ecb7cf9] git-p4: rewrite view handling
--
- [8cbfc11] Merge branch 'pw/p4-view-updates'
+ [8cbfc11^2] git-p4: view spec documentation
++ [ecb7cf9] git-p4: rewrite view handling
The problem is caused by the merge simplification logic when it inspects
the merge commit 8cbfc11. In this case, the history leading to the tip of
'master' did not touch git-p4 since 'pw/p4-view-updates' topic forked, and
the result of the merge is simply a copy from the tip of the topic branch
in the view limited by the given pathspec. The merge simplification logic
discards the history on the mainline side of the merge, and pretends as if
the sole parent of the merge is its second parent, i.e. the tip of the
topic. While this simplification is correct in the general case, it is at
least surprising if not outright wrong when the user explicitly asked to
show the first-parent history.
Here is an attempt to fix this issue, by not allowing us to compare the
merge result with anything but the first parent when --first-parent is in
effect, to avoid the history traversal veering off to the side branch.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes the algorithm more honest about what it is doing.
We start from an already limited, topo-sorted list, and postprocess
it by simplifying the irrelevant merges away.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
