7b0f229222
Before this change the "commit-graph write" command didn't report any progress. On my machine this command takes more than 10 seconds to write the graph for linux.git, and around 1m30s on the 2015-04-03-1M-git.git[1] test repository (a test case for a large monorepository). Furthermore, since the gc.writeCommitGraph setting was added ind5d5d7b641("gc: automatically write commit-graph files", 2018-06-27), there was no indication at all from a "git gc" run that anything was different. This why one of the progress bars being added here uses start_progress() instead of start_delayed_progress(), so that it's guaranteed to be seen. E.g. on my tiny 867 commit dotfiles.git repository: $ git -c gc.writeCommitGraph=true gc Enumerating objects: 2821, done. [...] Computing commit graph generation numbers: 100% (867/867), done. On larger repositories, such as linux.git the delayed progress bar(s) will kick in, and we'll show what's going on instead of, as was previously happening, printing nothing while we write the graph: $ git -c gc.writeCommitGraph=true gc [...] Annotating commits in commit graph: 1565573, done. Computing commit graph generation numbers: 100% (782484/782484), done. Note that here we don't show "Finding commits for commit graph", this is because under "git gc" we seed the search with the commit references in the repository, and that set is too small to show any progress, but would e.g. on a smaller repo such as git.git with --stdin-commits: $ git rev-list --all | git -c gc.writeCommitGraph=true write --stdin-commits Finding commits for commit graph: 100% (162576/162576), done. Computing commit graph generation numbers: 100% (162576/162576), done. With --stdin-packs we don't show any estimation of how much is left to do. This is because we might be processing more than one pack. We could be less lazy here and show progress, either by detecting that we're only processing one pack, or by first looping over the packs to discover how many commits they have. I don't see the point in doing that work. So instead we get (on 2015-04-03-1M-git.git): $ echo pack-<HASH>.idx | git -c gc.writeCommitGraph=true --exec-path=$PWD commit-graph write --stdin-packs Finding commits for commit graph: 13064614, done. Annotating commits in commit graph: 3001341, done. Computing commit graph generation numbers: 100% (1000447/1000447), done. No GC mode uses --stdin-packs. It's what they use at Microsoft to manually compute the generation numbers for their collection of large packs which are never coalesced. The reason we need a "report_progress" variable passed down from "git gc" is so that we don't report this output when we're running in the process "git gc --auto" detaches from the terminal. Since we write the commit graph from the "git gc" process itself (as opposed to what we do with say the "git repack" phase), we'd end up writing the output to .git/gc.log and reporting it to the user next time as part of the "The last gc run reported the following[...]" error, see329e6e8794("gc: save log from daemonized gc --auto and print it next time", 2015-09-19). So we must keep track of whether or not we're running in that demonized mode, and if so print no progress. See [2] and subsequent replies for a discussion of an approach not taken in compute_generation_numbers(). I.e. we're saying "Computing commit graph generation numbers", even though on an established history we're mostly skipping over all the work we did in the past. This is similar to the white lie we tell in the "Writing objects" phase (not all are objects being written). Always showing progress is considered more important than accuracy. I.e. on a repository like 2015-04-03-1M-git.git we'd hang for 6 seconds with no output on the second "git gc" if no changes were made to any objects in the interim if we'd take the approach in [2]. 1. https://github.com/avar/2015-04-03-1M-git 2. <c6960252-c095-fb2b-e0bc-b1e6bb261614@gmail.com> (https://public-inbox.org/git/c6960252-c095-fb2b-e0bc-b1e6bb261614@gmail.com/) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
67 lines
1.7 KiB
C
67 lines
1.7 KiB
C
#ifndef COMMIT_GRAPH_H
|
|
#define COMMIT_GRAPH_H
|
|
|
|
#include "git-compat-util.h"
|
|
#include "repository.h"
|
|
#include "string-list.h"
|
|
#include "cache.h"
|
|
|
|
struct commit;
|
|
|
|
char *get_commit_graph_filename(const char *obj_dir);
|
|
|
|
/*
|
|
* Given a commit struct, try to fill the commit struct info, including:
|
|
* 1. tree object
|
|
* 2. date
|
|
* 3. parents.
|
|
*
|
|
* Returns 1 if and only if the commit was found in the packed graph.
|
|
*
|
|
* See parse_commit_buffer() for the fallback after this call.
|
|
*/
|
|
int parse_commit_in_graph(struct repository *r, struct commit *item);
|
|
|
|
/*
|
|
* It is possible that we loaded commit contents from the commit buffer,
|
|
* but we also want to ensure the commit-graph content is correctly
|
|
* checked and filled. Fill the graph_pos and generation members of
|
|
* the given commit.
|
|
*/
|
|
void load_commit_graph_info(struct repository *r, struct commit *item);
|
|
|
|
struct tree *get_commit_tree_in_graph(struct repository *r,
|
|
const struct commit *c);
|
|
|
|
struct commit_graph {
|
|
int graph_fd;
|
|
|
|
const unsigned char *data;
|
|
size_t data_len;
|
|
|
|
unsigned char hash_len;
|
|
unsigned char num_chunks;
|
|
uint32_t num_commits;
|
|
struct object_id oid;
|
|
|
|
const uint32_t *chunk_oid_fanout;
|
|
const unsigned char *chunk_oid_lookup;
|
|
const unsigned char *chunk_commit_data;
|
|
const unsigned char *chunk_large_edges;
|
|
};
|
|
|
|
struct commit_graph *load_commit_graph_one(const char *graph_file);
|
|
|
|
void write_commit_graph_reachable(const char *obj_dir, int append,
|
|
int report_progress);
|
|
void write_commit_graph(const char *obj_dir,
|
|
struct string_list *pack_indexes,
|
|
struct string_list *commit_hex,
|
|
int append, int report_progress);
|
|
|
|
int verify_commit_graph(struct repository *r, struct commit_graph *g);
|
|
|
|
void free_commit_graph(struct commit_graph *);
|
|
|
|
#endif
|