git-commit-vandalism/Documentation
Derrick Stolee 4f6d26b167 list-objects: consume sparse tree walk
When creating a pack-file using 'git pack-objects --revs' we provide
a list of interesting and uninteresting commits. For example, a push
operation would make the local topic branch be interesting and the
known remote refs as uninteresting. We want to discover the set of
new objects to send to the server as a thin pack.

We walk these commits until we discover a frontier of commits such
that every commit walk starting at interesting commits ends in a root
commit or unintersting commit. We then need to discover which
non-commit objects are reachable from  uninteresting commits. This
commit walk is not changing during this series.

The mark_edges_uninteresting() method in list-objects.c iterates on
the commit list and does the following:

* If the commit is UNINTERSTING, then mark its root tree and every
  object it can reach as UNINTERESTING.

* If the commit is interesting, then mark the root tree of every
  UNINTERSTING parent (and all objects that tree can reach) as
  UNINTERSTING.

At the very end, we repeat the process on every commit directly
given to the revision walk from stdin. This helps ensure we properly
cover shallow commits that otherwise were not included in the
frontier.

The logic to recursively follow trees is in the
mark_tree_uninteresting() method in revision.c. The algorithm avoids
duplicate work by not recursing into trees that are already marked
UNINTERSTING.

Add a new 'sparse' option to the mark_edges_uninteresting() method
that performs this logic in a slightly different way. As we iterate
over the commits, we add all of the root trees to an oidset. Then,
call mark_trees_uninteresting_sparse() on that oidset. Note that we
include interesting trees in this process. The current implementation
of mark_trees_unintersting_sparse() will walk the same trees as
the old logic, but this will be replaced in a later change.

Add a '--sparse' flag in 'git pack-objects' to call this new logic.
Add a new test script t/t5322-pack-objects-sparse.sh that tests this
option. The tests currently demonstrate that the resulting object
list is the same as the old algorithm. This includes a case where
both algorithms pack an object that is not needed by a remote due to
limits on the explored set of trees. When the sparse algorithm is
changed in a later commit, we will add a test that demonstrates a
change of behavior in some cases.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17 13:44:39 -08:00
..
config index: make index.threads=true enable ieot and eoie 2018-11-21 16:46:54 +09:00
howto Merge branch 'uk/merge-subtree-doc-update' into maint 2018-11-21 22:58:08 +09:00
RelNotes RelNotes 2.20: drop spurious double quote 2018-12-04 11:26:40 +09:00
technical Merge branch 'mw/doc-typofixes' into maint 2018-11-21 22:58:03 +09:00
.gitattributes
.gitignore add a script to diff rendered documentation 2018-08-06 12:30:23 -07:00
asciidoc.conf
asciidoctor-extensions.rb
blame-options.txt
build-docdep.perl
cat-texi.perl
cmd-list.perl
CodingGuidelines Merge branch 'jc/how-to-document-api' 2018-10-19 13:34:05 +09:00
config.txt Merge branch 'nd/packobjectshook-doc-fix' into maint 2018-11-21 22:58:01 +09:00
date-formats.txt
diff-format.txt doc: update diff-format.txt for removed ellipses in --raw 2018-11-26 15:59:14 +09:00
diff-generate-patch.txt
diff-options.txt Merge branch 'es/diff-color-moved-fix' 2018-08-15 15:08:22 -07:00
doc-diff Merge branch 'es/worktree-forced-ops-fix' 2018-09-17 13:53:59 -07:00
docbook-xsl.css
docbook.xsl
everyday.txto
fetch-options.txt Merge branch 'ab/fetch-tags-noclobber' 2018-09-17 13:54:00 -07:00
fix-texi.perl
git-add.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-am.txt Merge branch 'nd/rebase-show-current-patch' 2018-03-06 14:54:02 -08:00
git-annotate.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-apply.txt Merge branch 'nd/diff-apply-ita' 2018-06-25 13:22:36 -07:00
git-archimport.txt git-archimport.1: specify what kind of Arch we're talking about 2018-09-21 09:28:58 -07:00
git-archive.txt
git-bisect-lk2009.txt doc: fix ASCII art tab spacing 2018-10-23 12:23:09 +09:00
git-bisect.txt Merge branch 'ak/bisect-doc-typofix' 2018-04-25 13:28:56 +09:00
git-blame.txt
git-branch.txt Merge branch 'jk/branch-l-1-repurpose' 2018-09-17 13:53:50 -07:00
git-bundle.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-cat-file.txt cat-file: support "unordered" output for --batch-all-objects 2018-08-13 13:48:31 -07:00
git-check-attr.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-check-ignore.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-check-mailmap.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-check-ref-format.txt
git-checkout-index.txt
git-checkout.txt doc: fix ASCII art tab spacing 2018-10-23 12:23:09 +09:00
git-cherry-pick.txt
git-cherry.txt
git-citool.txt
git-clean.txt
git-clone.txt Merge branch 'nd/doc-header' 2018-05-23 14:38:22 +09:00
git-column.txt git-column.1: clarify initial description, provide examples 2018-09-21 09:32:15 -07:00
git-commit-graph.txt Doc: refer to the "commit-graph file" with dash 2018-09-27 15:29:12 -07:00
git-commit-tree.txt
git-commit.txt commit: add support for --fixup <commit> -m"<extra message>" 2017-12-22 13:10:24 -08:00
git-config.txt worktree: add per-worktree config files 2018-10-22 13:17:04 +09:00
git-count-objects.txt
git-credential-cache--daemon.txt
git-credential-cache.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-credential-store.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-credential.txt
git-cvsexportcommit.txt
git-cvsimport.txt
git-cvsserver.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-daemon.txt daemon: add --log-destination=(stderr|syslog|none) 2018-02-05 10:30:44 -08:00
git-describe.txt git-describe.1: clarify that "human readable" is also git-readable 2018-09-21 09:32:21 -07:00
git-diff-files.txt
git-diff-index.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-diff-tree.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-diff.txt doc: fix a typo and clarify a sentence 2018-10-11 09:39:15 +09:00
git-difftool.txt
git-fast-export.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-fast-import.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-fetch-pack.txt Documentation: fix several one-character-off spelling errors 2018-04-09 14:15:02 +09:00
git-fetch.txt fetch: make the --prune-tags work with <url> 2018-02-09 13:10:13 -08:00
git-filter-branch.txt Merge branch 'nd/doc-header' 2018-05-23 14:38:22 +09:00
git-fmt-merge-msg.txt config.txt: move fmt-merge-msg-config.txt to config/ 2018-10-29 10:17:01 +09:00
git-for-each-ref.txt Merge branch 'jk/ui-color-always-to-auto' 2018-08-15 15:08:19 -07:00
git-format-patch.txt format-patch: do not let its diff-options affect --range-diff 2018-11-30 13:47:55 +09:00
git-fsck-objects.txt
git-fsck.txt fsck: verify commit-graph 2018-06-27 10:29:10 -07:00
git-gc.txt gc doc: mention the commit-graph in the intro 2018-10-11 15:40:27 +09:00
git-get-tar-commit-id.txt
git-grep.txt grep: add -r/--[no-]recursive 2018-10-03 21:25:57 -07:00
git-gui.txt
git-hash-object.txt
git-help.txt Merge branch 'rv/alias-help' 2018-10-26 14:22:13 +09:00
git-http-backend.txt
git-http-fetch.txt http-fetch: make -a standard behaviour 2018-04-24 10:55:02 +09:00
git-http-push.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-imap-send.txt git-imap-send.txt: move imap.* to a separate file 2018-10-29 10:17:02 +09:00
git-index-pack.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-init-db.txt
git-init.txt
git-instaweb.txt
git-interpret-trailers.txt Merge branch 'jk/trailer-fixes' into maint 2018-11-21 22:57:42 +09:00
git-log.txt git-[short]log.txt: unify quoted standalone -- 2018-04-18 12:49:26 +09:00
git-ls-files.txt Merge branch 'ah/misc-doc-updates' 2018-05-23 14:38:23 +09:00
git-ls-remote.txt Merge branch 'bw/server-options' 2018-05-23 14:38:15 +09:00
git-ls-tree.txt
git-mailinfo.txt
git-mailsplit.txt
git-merge-base.txt doc: fix ASCII art tab spacing 2018-10-23 12:23:09 +09:00
git-merge-file.txt
git-merge-index.txt
git-merge-one-file.txt
git-merge-tree.txt
git-merge.txt config.txt: move merge-config.txt to config/ 2018-10-29 10:17:03 +09:00
git-mergetool--lib.txt
git-mergetool.txt mergetool: accept -g/--[no-]gui as arguments 2018-10-25 14:01:10 +09:00
git-mktag.txt
git-mktree.txt Documentation: normalize spelling of 'normalised' 2018-04-09 14:15:07 +09:00
git-multi-pack-index.txt multi-pack-index: add 'verify' verb 2018-09-17 13:49:38 -07:00
git-mv.txt
git-name-rev.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-notes.txt Documentation: spelling and grammar fixes 2018-06-22 14:26:23 -07:00
git-p4.txt git-p4: fully support unshelving changelists 2018-10-16 13:28:49 +09:00
git-pack-objects.txt list-objects: consume sparse tree walk 2019-01-17 13:44:39 -08:00
git-pack-redundant.txt
git-pack-refs.txt
git-parse-remote.txt
git-patch-id.txt
git-prune-packed.txt
git-prune.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-pull.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-push.txt push doc: add spacing between two words 2018-09-19 12:43:50 -07:00
git-quiltimport.txt
git-range-diff.txt range-diff doc: add a section about output stability 2018-11-12 12:05:38 +09:00
git-read-tree.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-rebase.txt Merge branch 'en/rebase-consistency' 2018-12-04 12:49:39 +09:00
git-receive-pack.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-reflog.txt reflog expire: cover reflog from all worktrees 2018-10-22 13:32:54 +09:00
git-remote-ext.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-remote-fd.txt
git-remote-helpers.txto
git-remote-testgit.txt
git-remote.txt Merge branch 'nd/remote-update-doc' 2018-06-04 21:39:49 +09:00
git-repack.txt Merge branch 'cc/delta-islands' 2018-09-17 13:53:55 -07:00
git-replace.txt replace: introduce --convert-graft-file 2018-04-30 11:12:30 +09:00
git-request-pull.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-rerere.txt rerere: add note about files with existing conflict markers 2018-08-29 09:03:29 -07:00
git-reset.txt Merge branch 'ma/reset-doc-rendering-fix' 2018-12-01 21:41:43 +09:00
git-rev-list.txt
git-rev-parse.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-revert.txt
git-rm.txt
git-send-email.txt Merge branch 'jw/send-email-no-auth' 2018-11-06 15:50:20 +09:00
git-send-pack.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-sh-i18n--envsubst.txt
git-sh-i18n.txt
git-sh-setup.txt
git-shell.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-shortlog.txt git-[short]log.txt: unify quoted standalone -- 2018-04-18 12:49:26 +09:00
git-show-branch.txt doc: fix small typo in git show-branch 2018-10-18 12:26:51 +09:00
git-show-index.txt show-index: update documentation for index v2 2018-05-29 00:28:22 +09:00
git-show-ref.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-show.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-stage.txt
git-stash.txt git-stash.txt: remove extra square bracket 2018-03-27 19:09:13 -07:00
git-status.txt Documentation: spelling and grammar fixes 2018-06-22 14:26:23 -07:00
git-stripspace.txt
git-submodule.txt Merge branch 'pc/submodule-helper-foreach' 2018-06-25 13:22:35 -07:00
git-svn.txt Documentation: spelling and grammar fixes 2018-06-22 14:26:23 -07:00
git-symbolic-ref.txt
git-tag.txt doc: fix descripion for 'git tag --format' 2018-10-23 12:23:09 +09:00
git-tools.txt
git-unpack-file.txt
git-unpack-objects.txt
git-update-index.txt Merge branch 'jc/update-index-doc' 2018-08-20 11:33:50 -07:00
git-update-ref.txt Merge branch 'ah/doc-updates' into maint 2018-11-21 22:58:07 +09:00
git-update-server-info.txt
git-upload-archive.txt
git-upload-pack.txt doc: fix inappropriate monospace formatting 2018-10-23 12:23:09 +09:00
git-var.txt doc: keep first level section header in upper case 2018-05-02 17:03:33 +09:00
git-verify-commit.txt
git-verify-pack.txt
git-verify-tag.txt
git-web--browse.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
git-whatchanged.txt
git-worktree.txt Merge branch 'nd/per-worktree-ref-iteration' 2018-11-13 22:37:26 +09:00
git-write-tree.txt
git.txt Merge branch 'ah/doc-updates' into maint 2018-11-21 22:58:07 +09:00
gitattributes.txt doc: fix inappropriate monospace formatting 2018-10-23 12:23:09 +09:00
gitcli.txt Use proper syntax for replaceables in command docs 2018-05-25 17:16:47 +09:00
gitcore-tutorial.txt
gitcredentials.txt doc: clarify gitcredentials path component matching 2018-09-27 15:24:50 -07:00
gitcvs-migration.txt
gitdiffcore.txt
giteveryday.txt Documentation: spelling and grammar fixes 2018-06-22 14:26:23 -07:00
gitglossary.txt
githooks.txt git-p4: add the p4-pre-submit hook 2018-08-01 13:37:18 -07:00
gitignore.txt Merge branch 'nd/wildmatch-double-asterisk' 2018-11-13 22:37:19 +09:00
gitk.txt doc: convert [\--] to [--] 2018-04-18 12:49:26 +09:00
gitmodules.txt doc: fix inappropriate monospace formatting 2018-10-23 12:23:09 +09:00
gitnamespaces.txt
gitremote-helpers.txt Merge branch 'bw/protocol-v2' 2018-05-08 15:59:16 +09:00
gitrepository-layout.txt doc: move extensions.worktreeConfig to the right place 2018-11-16 14:10:31 +09:00
gitrevisions.txt push doc: correct lies about how push refspecs work 2018-08-31 14:04:06 -07:00
gitsubmodules.txt doc: fix inappropriate monospace formatting 2018-10-23 12:23:09 +09:00
gittutorial-2.txt
gittutorial.txt
gitweb.conf.txt doc: fix indentation of listing blocks in gitweb.conf.txt 2018-10-23 12:23:09 +09:00
gitweb.txt
gitworkflows.txt Merge branch 'km/doc-workflows-typofix' 2018-06-18 10:18:42 -07:00
glossary-content.txt Documentation: spelling and grammar fixes 2018-06-22 14:26:23 -07:00
howto-index.sh
i18n.txt
install-doc-quick.sh
install-webdoc.sh
line-range-format.txt
lint-gitlink.perl
mailmap.txt
Makefile Merge branch 'ts/doc-build-manpage-xsl-quietly' into maint 2018-11-21 22:57:53 +09:00
manpage-1.72.xsl
manpage-base-url.xsl.in
manpage-base.xsl
manpage-bold-literal.xsl
manpage-normal.xsl
manpage-quote-apos.xsl
manpage-suppress-sp.xsl
merge-options.txt merge: allow fast-forward when merging a tracked tag 2018-02-16 11:22:43 -08:00
merge-strategies.txt merge: add merge.renames config setting 2018-05-08 16:19:41 +09:00
pretty-formats.txt gpg-interface.c: obtain primary key fingerprint as well 2018-10-23 08:00:43 +09:00
pretty-options.txt
pull-fetch-param.txt fetch doc: correct grammar in --force docs 2018-09-20 09:40:03 -07:00
rev-list-options.txt Merge branch 'md/exclude-promisor-objects-fix' 2018-11-06 15:50:21 +09:00
revisions.txt Merge branch 'wc/find-commit-with-pattern-on-detached-head' 2018-07-24 14:50:49 -07:00
sequencer.txt
SubmittingPatches SubmittingPatches: mention doc-diff 2018-08-21 12:54:33 -07:00
texi.xsl
transfer-data-leaks.txt
urls-remotes.txt
urls.txt
user-manual.conf
user-manual.txt