undef/git-commit-vandalism
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Allow frontends to bidirectionally communicate with fast-import

Browse Source 
The existing checkpoint command is very useful to force fast-import
to dump the branches out to disk so that standard Git tools can
access them and the objects they refer to.  However there was not a
way to know when fast-import had finished executing the checkpoint
and it was safe to read those refs.

The progress command can be used to make fast-import output any
message of the frontend's choosing to standard out.  The frontend
can scan for these messages using select() or poll() to monitor a
pipe connected to the standard output of fast-import.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This commit is contained in:
Shawn O. Pearce 2007-08-01 10:23:08 -04:00
parent 1fdb649c6a
commit ac053c0202
3 changed files with 84 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

39
Documentation/git-fast-import.txt
View File

@ -298,6 +298,11 @@ and control the current import process. More detailed discussion
This command is optional and is not needed to perform
an import.
`progress`::
Causes fast-import to echo the entire line to its own
standard output. This command is optional and is not needed
to perform an import.
`commit`
~~~~~~~~
Create or update a branch with a new commit, recording one logical
@ -775,6 +780,31 @@ explicit checkpointing may not be necessary.
The `LF` after the command is optional (it used to be required).
`progress`
~~~~~~~~~~
Causes fast-import to print the entire `progress` line unmodified to
its standard output channel (file descriptor 1) when the command is
processed from the input stream. The command otherwise has no impact
on the current import, or on any of fast-import's internal state.
....
'progress' SP <any> LF
LF?
....
The `<any>` part of the command may contain any sequence of bytes
that does not contain `LF`. The `LF` after the command is optional.
Callers may wish to process the output through a tool such as sed to
remove the leading part of the line, for example:
====
frontend | git-fast-import | sed 's/^progress //'
====
Placing a `progress` command immediately after a `checkpoint` will
inform the reader when the `checkpoint` has been completed and it
can safely access the refs that fast-import updated.
Tips and Tricks
---------------
The following tips and tricks have been collected from various
@ -867,6 +897,15 @@ This will take longer, but will also produce a smaller packfile.
You only need to expend the effort once, and everyone using your
project will benefit from the smaller repository.
Include Some Progress Messages
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Every once in a while have your frontend emit a `progress` message
to fast-import. The contents of the messages are entirely free-form,
so one suggestion would be to output the current month and year
each time the current commit date moves into the next month.
Your users will feel better knowing how much of the data stream
has been processed.
Packfile Optimization
---------------------

14
fast-import.c
View File

@ -8,6 +8,7 @@ Format of STDIN stream:
| new_tag
| reset_branch
| checkpoint
| progress
;
new_blob ::= 'blob' lf
@ -53,6 +54,9 @@ Format of STDIN stream:
checkpoint ::= 'checkpoint' lf
lf?;
progress ::= 'progress' sp not_lf* lf
lf?;
# note: the first idnum in a stream should be 1 and subsequent
# idnums should not have gaps between values as this will cause
# the stream parser to reserve space for the gapped values. An
@ -2125,6 +2129,14 @@ static void cmd_checkpoint(void)
skip_optional_lf();
}
static void cmd_progress(void)
{
fwrite(command_buf.buf, 1, command_buf.len - 1, stdout);
fputc('\n', stdout);
fflush(stdout);
skip_optional_lf();
}
static void import_marks(const char *input_file)
{
char line[512];
@ -2235,6 +2247,8 @@ int main(int argc, const char **argv)
cmd_reset_branch();
else if (!strcmp("checkpoint", command_buf.buf))
cmd_checkpoint();
else if (!prefixcmp(command_buf.buf, "progress "))
cmd_progress();
else
die("Unsupported command: %s", command_buf.buf);
}

31
t/t9300-fast-import.sh
View File

@ -885,4 +885,35 @@ test_expect_success \
git log --reverse --pretty=oneline O3 | sed s/^.*z// >actual &&
git diff expect actual'
cat >input <<INPUT_END
commit refs/heads/O4
committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
data <<COMMIT
zstring
COMMIT
commit refs/heads/O4
committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
data <<COMMIT
zof
COMMIT
progress Two commits down, 2 to go!
commit refs/heads/O4
committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
data <<COMMIT
zempty
COMMIT
progress Three commits down, 1 to go!
commit refs/heads/O4
committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
data <<COMMIT
zcommits
COMMIT
progress I'm done!
INPUT_END
test_expect_success \
'O: progress outputs as requested by input' \
'git-fast-import <input >actual &&
grep "progress " <input >expect &&
git diff expect actual'
test_done