Not only does it prevent accidentally losing older bundles, but it
also fixes a subtle bug: when writing into an existing bundle,
git-pack-objects would not truncate the bundle. Therefore,
fetching from the bundle would trigger an error in unpack-objects:
"fatal: pack has junk at the end".
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The earlier patch tried to be nice by just warning, but it seems
more likely that the user wants to adjust the parameters.
Also, it prevents a bundle containing _all_ revisions in the case
when the user only gave one ref, but also rev-list options which
excluded the ref.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The patch to make the packs in a bundle thin forgot the receiving side.
D'oh.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* 'master' of git://repo.or.cz/git/fastimport:
Allow fast-import frontends to reload the marks table
Use atomic updates to the fast-import mark file
Preallocate memory earlier in fast-import
Thin packs are way smaller, but they rely on the receiving end to have the
base objects. However, Git's pack protocol also uses thin packs by
default. So make the packs contained in bundles thin, since bundles are
just another transport.
The patch looks a bit bigger than intended, mainly because --thin
_implies_ that pack-objects should run its own rev-list. Therefore, this
patch removes all the stuff we used to roll rev-list ourselves.
This commit also changes behaviour slightly: since we now know early
enough if a specified ref is _not_ contained in the pack, we can avoid
putting that ref into the pack. So, we don't die() here, but warn()
instead, and skip that ref.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
When saying something like "--since=1.day.ago" or "--max-count=5",
git-bundle finds the boundary commits which are recorded as
prerequisites. However, it failed to tell pack-objects _not_ to
pack the objects which are in these.
Fix that. And add a test for that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I'm giving fast-import a lesson on how to reload the marks table
using the same format it outputs with --export-marks. This way
a frontend can reload the marks table from a prior import, making
incremental imports less painful.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When we allow fast-import frontends to reload a mark file from a
prior session we want to let them use the same file as they exported
the marks to. This makes it very simple for the frontend to save
state across incremental imports.
But we don't want to lose the old marks table if anything goes wrong
while writing our current marks table. So instead of truncating and
overwriting the path specified to --export-marks we use the standard
lockfile code to write the current marks out to a temporary file,
then rename it over the old marks table.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Bill Lear pointed out that it is easy to send out notifications of
changes with the update hook, but successful execution of the update
hook does not necessarily mean that the ref was actually updated.
Lock contention on the ref or being unable to append to the reflog
may prevent the ref from being changed. Sending out notifications
prior to the ref actually changing is very misleading.
To help this situation I am introducing two new hooks to the
receive-pack flow: pre-receive and post-receive. These new hooks
are invoked only once per receive-pack execution and are passed
three arguments per ref (refname, old-sha1, new-sha1).
The new post-receive hook is ideal for sending out notifications,
as it has the complete list of all refnames that were successfully
updated as well as the old and new SHA-1 values. This allows more
interesting notifications to be sent. Multiple ref updates could
be easily summarized into one email, for example.
The new pre-receive hook is ideal for logging update attempts, as it
is run only once for the entire receive-pack operation. It can also
be used to verify multiple updates happen at once, e.g. an update
to the `maint` head must also be accompained by a new annotated tag.
Lots of documentation improvements for receive-pack are included
in this change, as we want to make sure the new hooks are clearly
explained.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I discovered we did not send an ng line in the report-status feedback
if the ref was not updated because the repository has the config
option receive.denyNonFastForwards enabled. I think the reason this
happened is that it is simply too easy to forget to set error_string
when returning back a failure from update()
We now return an ng line for a non-fastforward update, which in
turn will cause send-pack to exit with a non-zero exit status.
Hence the modified test.
This refactoring changes update to return a const char* describing
the error, which execute_commands always loads into error_string.
The result is what I think is cleaner code, and allows us to
initialize the error_string member to NULL when we read_head_info.
I want error_string to be NULL in all commands before we call
execute_commands, so that we can reuse the run_hook function to
execute a new pre-receive hook.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a simple refactoring of run_update_hook to allow the function
to be passed the name of the hook it runs and also to build the
argument list from a list of struct commands, rather than just one
struct command.
The refactoring is to support new pre-receive and post-receive
hooks that will be given the entire list of struct commands,
rather than just one struct command. These new hooks will follow
in another patch.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
There is little point in executing the post-update hook if all refs
had an error and were unable to be updated. In this case nothing
new is reachable within the repository, and there is no state change
for the post-update hook to be interested in.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
As the post-update hook is meant to run after we have completed the
receipt of the pushed changes, and it might actually try to kick off
a `repack -a -d`, we should delay on invoking it until after we have
removed the *.keep file on the uploaded pack (if we kept the pack).
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I'm about to teach fast-import how to reload the marks file created
by a prior session. The general approach that I want to use is to
immediately parse the marks file when the specific argument is found
in argv, thereby allowing the caller to supply multiple marks files,
as the mark space can be sparsely populated.
To make that work out we need to allocate our object tables before
we parse the command line options. Since none of these tables
depend on the command line options, we can easily relocate them.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some systems have sizeof(off_t) == 8 while sizeof(size_t) == 4.
This implies that we are able to access and work on files whose
maximum length is around 2^63-1 bytes, but we can only malloc or
mmap somewhat less than 2^32-1 bytes of memory.
On such a system an implicit conversion of off_t to size_t can cause
the size_t to wrap, resulting in unexpected and exciting behavior.
Right now we are working around all gcc warnings generated by the
-Wshorten-64-to-32 option by passing the off_t through xsize_t().
In the future we should make xsize_t on such problematic platforms
detect the wrapping and die if such a file is accessed.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Always use an off_t value in pack-objects anytime we are dealing
with an offset to some data within a packfile.
Also fixed a minor uintmax_t that was incorrectly defined before.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Not all platforms have declared 'unsigned long' to be a 64 bit value,
but we want to support a 64 bit packfile (or close enough anyway)
in the near future as some projects are getting large enough that
their packed size exceeds 4 GiB.
By using off_t, the POSIX type that is declared to mean an offset
within a file, we support whatever maximum file size the underlying
operating system will handle. For most modern systems this is up
around 2^60 or higher.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
As we technically try to support up to a maximum of 2**32-1 objects
in a single packfile we should act like it and use unsigned 32 bit
integers for all of our object counts and progress output.
This change does not modify everything in pack-objects that probably
needs to change to fully support the maximum of 2**32-1 objects.
I'm intentionally breaking the improvements into slightly smaller
commits to make them easier to follow.
No logic change should be occuring here, with the exception that
some comparsions will now work properly when the number of objects
exceeds 2**31-1.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
As we permit up to 2^32-1 objects in a single packfile we cannot
use a signed int to represent the object offset within a packfile,
after 2^31-1 objects we will start seeing negative indexes and
error out or compute bad addresses within the mmap'd index.
This is a minor cleanup that does not introduce any significant
logic changes. It is roach free.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We shouldn't attempt to assign constant strings into char*, as the
string is not writable at runtime. Likewise we should always be
treating unsigned values as unsigned values, not as signed values.
Most of these are very straightforward. The only exception is the
(unnecessary) xstrdup/free in builtin-branch.c for the detached
head case. Since this is a user-level interactive type program
and that particular code path is executed no more than once, I feel
that the extra xstrdup call is well worth the easy elimination of
this warning.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If an index is corrupt, or is simply too new for us to understand,
we were leaking the mmap that held the entire content of the index.
This could be a considerable size on large projects, given that
the index is at least 24 bytes * nr_objects.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Because we are currently cheating and never supplying the delta base
for an OBJ_OFS_DELTA we get a random SHA-1 in the delta base field.
Instead lets clear the hash out so its at least all 0's. This is
somewhat more obvious that something fishy is going on, like we
don't actually have the SHA-1 of the base handy. :)
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This patch adds support to archimport for remapping the branch
names to match those used in git more closely. This is useful
for projects that migrate to git (as opposed to users that want
to use git on Arch-based projects). For example, one can choose
an Arch branch name and call it "master".
The new command-line syntax works even if there is a colon in
a branch name, since only the part after the last colon is taken
to be the git name (git does not allow colons in branch names).
The new feature is implemented so that archives rotated every
year can also be remapped into a single git archive.
Signed-off-by: Paolo Bonzini <bonzini@gnu.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
They test the behaviour with just a URL in the command line.
Signed-off-by: Santi B,Ai(Bjar <sbejar@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This failure to catch the failure of write_ref_sha1 was noticed
by Bill Lear. The ref will not update if the log file could not
be appended to (due to file permissions problems). Such a failure
should be flagged as a failure to update the ref, so that the client
knows the push did not succeed.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If read_header() fails, it returns <0, not 0. Further, an open(/dev/null)
was not checked for errors.
Also, this adds two tests to make sure that the bundle file looks
correct, by checking if it has the header has the expected form, and that
the pack contains the right amount of objects.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
There is no need to escape HTML tag's attributes in CGI.pm
HTML methods (like CGI::a()), because CGI.pm does attribute
escaping automatically.
$cgi->a({ ... -attribute => atribute_value }, tag_contents)
is translated to
<a ... attribute="attribute_value">tag_contents</a>
The rules for escaping attribute values (which are string contents) are
different. For example you have to take care about escaping embedded '"'
and "'" characters; CGI::a() does that for us automatically.
CGI::a() does not HTML escape tag_contents; we would need to write
<a href="URL">some <b>bold</b> text</a>
for example. So we use esc_html (or esc_path) to escape tag_contents
as needed.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
- print output file name for .c files
- suppress output of the names of subdirectories when make changes into them
- use GEN prefix for makefile generation in perl/
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The test was recently broken to expect sed to leave the
incomplete line at the end without newline.
POSIX says that output of the pattern space is to be followed by
a newline, while GNU adds the newline back only when it was
stripped when input. GNU behaviour is arguably more intuitive
and nicer, but we should not depend on it.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
When the list is truly limited and get_revision_1() returned NULL,
the code incorrectly returned it without switching to boundary emiting
mode. Silly.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Change to use explicitly function call cgi->escapHTML().
This fix the problem on some systems that escapeHTML() is not
functioning, as default CGI is not setting 'escape' parameter.
Signed-off-by: Li Yang <leoli@freescale.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This moves the code to set SHOWN on the commit from get_revision_1()
back to get_revision(), so that the bit means what it originally
meant: this commit has been given back to the caller.
Also it fixes the --reverse breakage Dscho pointed out.
Signed-off-by: Junio C Hamano <junkio@cox.net>
verify_bundle() returned with an error early only when all
prerequisite commits were missing. It should error out much
earlier when some are missing.
When the rev-list is limited in ways other than revision range
(e.g. --max-count or --max-age), create_bundle() listed all
positive refs given from the command line as if they are
available, but resulting pack may not have some of them. Add a
logic to make sure all of them are included, and error out
otherwise.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This removes the flag internally used by revision traversal to
decide which commits are indeed boundaries and renames it to
CHILD_SHOWN. builtin-bundle uses the symbol for its
verification, but I think the logic it uses it is wrong. The
flag is still useful but it is local to the git-bundle, so it is
renamed to PREREQ_MARK.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This cleans up the boundary processing in the commit walker. It
- rips out the boundary logic from the commit walker. Placing
"negative" commits in the revs->commits list was Ok if all we
cared about "boundary" was the UNINTERESTING limiting case,
but conceptually it was wrong.
- makes get_revision_1() function to walk the commits and return
the results as if there is no funny postprocessing flags such
as --reverse, --skip nor --max-count.
- makes get_revision() function the postprocessing phase:
If reverse is given, wait for get_revision_1() to give
everything that it would normally give, and then reverse it
before consuming.
If skip is given, skip that many before going further.
If max is given, stop when we gave out that many.
Now that we are about to return one positive commit, mark
the parents of that commit to be potential boundaries
before returning, iff we are doing the boundary processing.
Return the commit.
- After get_revision() finishes giving out all the positive
commits, if we are doing the boundary processing, we look at
the parents that we marked as potential boundaries earlier,
see if they are really boundaries, and give them out.
It loses more code than it adds, even when the new gc_boundary()
function, which is purely for early optimization, is counted.
Note that this patch is purely for eyeballing and discussion
only. It breaks git-bundle's verify logic because the logic
does not use BOUNDARY_SHOW flag for its internal computation
anymore. After we correct it not to attempt to affect the
boundary processing by setting the BOUNDARY_SHOW flag, we can
remove BOUNDARY_SHOW from revision.h and use that bit assignment
for the new CHILD_SHOWN flag.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Per Junio's suggestion we are setting 'make' to be quiet by default,
with `make V=1` available to force GNU make back to its default
behavior of showing each command it is running.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I find it difficult to see compiler warnings amongst the massive
spewing produced by GNU make as it works through our productions.
This is especially true if CFLAGS winds up being rather long, due
to a large number of -W options being enabled and due to a number
of -D options being configured/required by my platform.
By defining QUIET_MAKE (e.g. make QUIET_MAKE=YesPlease) during
compilation users will get a less verbose output, such as:
...
CC builtin-grep.c
builtin-grep.c:187: warning: 'external_grep' defined but not used
CC builtin-init-db.c
CC builtin-log.c
CC builtin-ls-files.c
CC builtin-ls-tree.c
...
The verbose (normal make) output is still the default.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* maint:
git-gui: Make 'make' quieter by default
git-gui: Remove unnecessary /dev/null redirection.
git-gui: Don't create empty (same tree as parent) commits.
git-gui: Add Reset to the Branch menu.
git-gui: Relocate the menu/transport menu code.
* 'master' of git://repo.or.cz/git-gui:
git-gui: Make 'make' quieter by default
git-gui: Remove unnecessary /dev/null redirection.
git-gui: Don't create empty (same tree as parent) commits.
git-gui: Add Reset to the Branch menu.
git-gui: Relocate the menu/transport menu code.
The handcrafted built-in rev-list lookalike forgot to mark the trees
and blobs contained in the boundary commits uninteresting, resulting
in unnecessary objects in the pack.
Signed-off-by: Junio C Hamano <junkio@cox.net>
To fit nicely into the output of the git.git project's own quieter
Makefile, we want to make the git-gui Makefile nice and quiet too.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
