If the stderr of "git credential-cache" is redirected to a
pipe, the reader on the other end of a pipe may be surprised
that the pipe remains open long after the process exits.
This happens because we may auto-spawn a daemon which is
long-lived, and which keeps stderr open.
We can solve this by redirecting the daemon's stderr to
/dev/null once we are ready to go into our event loop. We
would not want to do so before then, because we may want to
report errors about the setup (e.g., failure to establish
the listening socket).
This does mean that we will not report errors we encounter
for specific clients. That's acceptable, as such errors
should be rare (e.g., clients sending buggy requests).
However, we also provide an escape hatch: if you want to see
these later messages, you can provide the "--debug" option
to keep stderr open.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since commit 81c5cf7 (mailinfo: skip bogus UNIX From line inside
body, 2006-05-21), we have treated lines like ">From" in the body as
headers. This makes "git am" work for people who erroneously paste
the whole output from format-patch:
From 12345abcd...fedcba543210 Mon Sep 17 00:00:00 2001
From: them
Subject: [PATCH] whatever
into their email body (assuming that an mbox writer then quotes
"From" as ">From", as otherwise we would actually mailsplit on the
in-body line).
However, this has false positives if somebody actually has a commit
body that starts with "From "; in this case we erroneously remove
the line entirely from the commit message. We can make this check
more robust by making sure the line actually looks like a real mbox
"From" line.
Inspect the line that begins with ">From " a more carefully to only
skip lines that match the expected pattern (note that the datestamp
part of the format-patch output is designed to be kept constant to
help those who write magic(5) entries).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Record the URL of the intended recipient for a push (after
anonymizing it if it has authentication material) on a new "pushee
URL" header. Because the networking configuration (SSH-tunnels,
proxies, etc.) on the pushing user's side varies, the receiving
repository may not know the single canonical URL all the pushing
users would refer it as (besides, many sites allow pushing over
ssh://host/path and https://host/path protocols to the same
repository but with different local part of the path). So this
value may not be reliably used for replay-attack prevention
purposes, but this will still serve as a human readable hint to
identify the repository the certificate refers to.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the interim protocol, we used to send the update commands even
though we already send a signed copy of the same information when
push certificate is in use. Update the send-pack/receive-pack pair
not to do so.
The notable thing on the receive-pack side is that it makes sure
that there is no command sent over the traditional protocol packet
outside the push certificate. Otherwise a pusher can claim to be
pushing one set of ref updates in the signed certificate while
issuing commands to update unrelated refs, and such an update will
evade later audits.
Finally, start documenting the protocol.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We would want to update the interim protocol so that we do not send
the usual update commands when the push certificate feature is in
use, as the same information is in the certificate. Once that
happens, the push-cert packet may become the only protocol command,
but then there is no packet to put the feature request behind, like
we always did.
As we have prepared the receiving end that understands the push-cert
feature to accept the feature request on the first protocol packet
(other than "shallow ", which was an unfortunate historical mistake
that has to come before everything else), we can give the feature
request on the push-cert packet instead of the first update protocol
packet, in preparation for the next step to actually update to the
final protocol.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reusing the GPG signature check helpers we already have, verify
the signature in receive-pack and give the results to the hooks
via GIT_PUSH_CERT_{SIGNER,KEY,STATUS} environment variables.
Policy decisions, such as accepting or rejecting a good signature by
a key that is not fully trusted, is left to the hook and kept
outside of the core.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While signed tags and commits assert that the objects thusly signed
came from you, who signed these objects, there is not a good way to
assert that you wanted to have a particular object at the tip of a
particular branch. My signing v2.0.1 tag only means I want to call
the version v2.0.1, and it does not mean I want to push it out to my
'master' branch---it is likely that I only want it in 'maint', so
the signature on the object alone is insufficient.
The only assurance to you that 'maint' points at what I wanted to
place there comes from your trust on the hosting site and my
authentication with it, which cannot easily audited later.
Introduce a mechanism that allows you to sign a "push certificate"
(for the lack of better name) every time you push, asserting that
what object you are pushing to update which ref that used to point
at what other object. Think of it as a cryptographic protection for
ref updates, similar to signed tags/commits but working on an
orthogonal axis.
The basic flow based on this mechanism goes like this:
1. You push out your work with "git push --signed".
2. The sending side learns where the remote refs are as usual,
together with what protocol extension the receiving end
supports. If the receiving end does not advertise the protocol
extension "push-cert", an attempt to "git push --signed" fails.
Otherwise, a text file, that looks like the following, is
prepared in core:
certificate version 0.1
pusher Junio C Hamano <gitster@pobox.com> 1315427886 -0700
7339ca65... 21580ecb... refs/heads/master
3793ac56... 12850bec... refs/heads/next
The file begins with a few header lines, which may grow as we
gain more experience. The 'pusher' header records the name of
the signer (the value of user.signingkey configuration variable,
falling back to GIT_COMMITTER_{NAME|EMAIL}) and the time of the
certificate generation. After the header, a blank line follows,
followed by a copy of the protocol message lines.
Each line shows the old and the new object name at the tip of
the ref this push tries to update, in the way identical to how
the underlying "git push" protocol exchange tells the ref
updates to the receiving end (by recording the "old" object
name, the push certificate also protects against replaying). It
is expected that new command packet types other than the
old-new-refname kind will be included in push certificate in the
same way as would appear in the plain vanilla command packets in
unsigned pushes.
The user then is asked to sign this push certificate using GPG,
formatted in a way similar to how signed tag objects are signed,
and the result is sent to the other side (i.e. receive-pack).
In the protocol exchange, this step comes immediately before the
sender tells what the result of the push should be, which in
turn comes before it sends the pack data.
3. When the receiving end sees a push certificate, the certificate
is written out as a blob. The pre-receive hook can learn about
the certificate by checking GIT_PUSH_CERT environment variable,
which, if present, tells the object name of this blob, and make
the decision to allow or reject this push. Additionally, the
post-receive hook can also look at the certificate, which may be
a good place to log all the received certificates for later
audits.
Because a push certificate carry the same information as the usual
command packets in the protocol exchange, we can omit the latter
when a push certificate is in use and reduce the protocol overhead.
This however is not included in this patch to make it easier to
review (in other words, the series at this step should never be
released without the remainder of the series, as it implements an
interim protocol that will be incompatible with the final one).
As such, the documentation update for the protocol is left out of
this step.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Everywhere else we use PKT-LINE to denote the pkt-line formatted
data, but "shallow/deepen" messages are described with PKT_LINE().
Fix them.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our signed-tag objects set the standard format used by Git to store
GPG-signed payload (i.e. the payload followed by its detached
signature) [*1*], and it made sense to have a helper to find the
boundary between the payload and its signature in tag.c back then.
Newer code added later to parse other kinds of objects that learned
to use the same format to store GPG-signed payload (e.g. signed
commits), however, kept using the helper from the same location.
Move it to gpg-interface; the helper is no longer about signed tag,
but it is how our code and data interact with GPG.
[Reference]
*1* http://thread.gmane.org/gmane.linux.kernel/297998/focus=1383
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier, ffb6d7d5 (Move commit GPG signature verification to
commit.c, 2013-03-31) moved this helper that used to be in pretty.c
(i.e. the output code path) to commit.c for better reusability.
It was a good first step in the right direction, but still suffers
from a myopic view that commits will be the only thing we would ever
want to sign---we would actually want to be able to reuse it even
wider.
The function interprets what GPG said; gpg-interface is obviously a
better place. Move it there.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We use it to make sure that the feature request is sent only once on
the very first request packet (ignoring the "shallow " line, which
was an unfortunate mistake we cannot retroactively fix with existing
receive-pack already deployed in the field) and we set it to "true"
with cmds_sent++, not because we care about the actual number of
updates sent but because it is merely an idiomatic way.
Set it explicitly to one to clarify that the code that uses this
variable only cares about its zero-ness.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The main loop over remote_refs list inspects the ref status
to see if we need to generate pack data (i.e. a delete-only push
does not need to send any additional data), resets it to "expecting
the status report" state, and formats the actual update commands
to be sent.
Split the former two out of the main loop, as it will become
conditional in later steps.
Besides, we should have code that does real thing here, before the
"Finally, tell the other end!" part ;-)
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The variable counts how many non-deleting command is being sent, but
is only checked with 0-ness to decide if we need to send the pack
data.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similar to the previous one for send-pack, make it easier and
cleaner to add to capability advertisement.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A run of 'var ? " var" : ""' fed to a long printf string in a deeply
nested block was hard to read. Move it outside the loop and format
it into a strbuf.
As an added bonus, the trick to add "agent=<agent-name>" by using
two conditionals is replaced by a more readable version.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We tried to avoid sending one extra byte, NUL and nothing behind it
to signal there is no protocol capabilities being sent, on the first
command packet on the wire, but it just made the code look ugly.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A new helper function ref_update_to_be_sent() decides for each ref
if the update is to be sent based on the status previously set by
set_ref_status_for_push() and also if this is a mirrored push.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
20e8b465 (refactor ref status logic for pushing, 2010-01-08)
restructured the code to set status for each ref to be pushed, but
did not quite go far enough. We inspect the status set earlier by
set_refs_status_for_push() and then perform yet another update to
the status of a ref with an otherwise OK status to be deleted to
mark it with REF_STATUS_REJECT_NODELETE when the protocol tells us
never to delete.
Split the latter into a separate loop that comes before we enter the
per-ref loop. This way we would have one less condition to check in
the main loop.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make a helper function to accept a line of a protocol message and
queue an update command out of the code from read_head_info().
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This piece of code reads object names of shallow boundaries, not
old_sha1[], i.e. the current value the ref points at, which is to be
replaced by what is in new_sha1[].
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ideally, we should have also allowed the first "shallow" to carry
the feature request trailer, but that is water under the bridge
now. This makes the next step to factor out the queuing of commands
easier to review.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
An "update" command in the protocol exchange consists of 40-hex old
object name, SP, 40-hex new object name, SP, and a refname, but the
first instance is further followed by a NUL with feature requests.
The command structure, which has a flex-array member that stores the
refname at the end, was allocated based on the whole length of the
update command, without excluding the trailing feature requests.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
CodingGuidelines states that the first #include in C files should be
git-compat-util.h or another header file that includes it, such as
cache.h or builtin.h.
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
glibc has deprecated the use of _BSD_SOURCE define
warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
To make it easier to maintain a cross platform source code, that
warning can be suppressed by _DEFAULT_SOURCE.
Define both _BSD_SOURCE and _DEFAULT_SOURCE to clean-up the build.
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Call the functions behind git prune-packed and git update-server-info
directly instead of using run_command(). This is shorter, easier and
quicker.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We allocate pack information in a static global list but
never clean it up. This leaks memory, and means that calling
update_server_info twice will generate a buggy file (it will
have duplicate entries).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since "git update-server-info" may be called automatically
as part of a push or a "gc --auto", we should be robust
against two processes trying to update it simultaneously.
However, we currently use a fixed tempfile, which means that
two simultaneous writers may step on each other's toes and
end up renaming junk into place.
Let's instead switch to using a unique tempfile via mkstemp.
We do not want to use a lockfile here, because it's OK for
two writers to simultaneously update (one will "win" the
rename race, but that's OK; they should be writing the same
information).
While we're there, let's clean up a few other things:
1. Detect write errors. Report them and abort the update
if any are found.
2. Free path memory rather than leaking it (and clean up
the tempfile when necessary).
3. Use the pathdup functions consistently rather than
static buffers or manually calculated lengths.
This last one fixes a potential overflow of "infofile" in
update_info_packs (e.g., by putting large junk into
$GIT_OBJECT_DIRECTORY). However, this overflow was probably
not an interesting attack vector for two reasons:
a. The attacker would need to control the environment to
do this, in which case it was already game-over.
b. During its setup phase, git checks that the directory
actually exists, which means it is probably shorter
than PATH_MAX anyway.
Because both update_info_refs and update_info_packs share
these same failings (and largely duplicate each other), this
patch factors out the improved error-checking version into a
helper function.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We form all of our directories in a strbuf, but never release it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On my Debian 7 system, this fixes annoying warnings when the output
of "git svn" commands are redirected:
Unable to get Terminal Size. The TIOCGWINSZ ioctl didn't work.
The COLUMNS and LINES environment variables didn't work. The
resize program didn't work.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Allow -B and -A to act as short aliases for --before and --after
options respectively. This reduces typing and hopefully allows
reuse of muscle memory for grep(1) users.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Git no longer seems to use these flags or their associated config keys;
when they are present, git-svn outputs a message indicating that they
are being ignored.
Signed-off-by: Lawrence Velázquez <vq@larryv.me>
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Calling "git svn info $(pwd)" would hit:
"Reading from filehandle failed at ..."
errors due to improper prefixing and canonicalization.
Strip the toplevel path from absolute filesystem paths to ensure
downstream canonicalization routines are only exposed to paths
tracked in git (or SVN).
v2:
Thanks to Andrej Manduch for originally noticing the issue
and fixing my original version of this to handle
more corner cases such as "/path/to/top/../top" and
"/path/to/top/../top/file" as shown in the new test cases.
v3:
Fix pathname portability problems pointed out by Johannes Sixt
with a hint from brian m. carlson.
Cc: Johannes Sixt <j6t@kdbg.org>
Cc: "brian m. carlson" <sandals@crustytoothpaste.net>
Signed-off-by: Andrej Manduch <amanduch@gmail.com>
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Commands such as "git svn init/fetch/dcommit" do not prompt for client
certificate/password if they are stored in SVN config file. Make
"git svn branch" consistent with the other commands, as SVN::Client is
capable of building its own authentication baton from information in the
SVN config directory.
Signed-off-by: Monard Vong <travelingsoul86@gmail.com>
Signed-off-by: Eric Wong <normalperson@yhbt.net>
test_cmp is intended to produce diff output for human consumption. The
input in one instance in t9300-fast-import.sh are binary files, however.
Use test_cmp_bin to compare the files.
This was noticed because on Windows we have a special implementation of
test_cmp in pure bash code (to ignore differences due to intermittent CR
in actual output), and bash runs into an infinite loop due to the binary
nature of the input.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our filesystem ref storage does not allow D/F conflicts; so
if "refs/heads/a/b" exists, we do not allow "refs/heads/a"
to exist (and vice versa). This falls out naturally for
loose refs, where the filesystem enforces the condition. But
for packed-refs, we have to make the check ourselves.
We do so by iterating over the entire packed-refs namespace
and checking whether each name creates a conflict. If you
have a very large number of refs, this is quite inefficient,
as you end up doing a large number of comparisons with
uninteresting bits of the ref tree (e.g., we know that all
of "refs/tags" is uninteresting in the example above, yet we
check each entry in it).
Instead, let's take advantage of the fact that we have the
packed refs stored as a trie of ref_entry structs. We can
find each component of the proposed refname as we walk
through the trie, checking for D/F conflicts as we go. For a
refname of depth N (i.e., 4 in the above example), we only
have to visit N nodes. And at each visit, we can binary
search the M names at that level, for a total complexity of
O(N lg M). ("M" is different at each level, of course, but
we can take the worst-case "M" as a bound).
In a pathological case of fetching 30,000 fresh refs into a
repository with 8.5 million refs, this dropped the time to
run "git fetch" from tens of minutes to ~30s.
This may also help smaller cases in which we check against
loose refs (which we do when renaming a ref), as we may
avoid a disk access for unrelated loose directories.
Note that the tests we add appear at first glance to be
redundant with what is already in t3210. However, the early
tests are not robust; they are run with reflogs turned on,
meaning that we are not actually testing
is_refname_available at all! The operations will still fail
because the reflogs will hit D/F conflicts in the
filesystem. To get a true test, we must turn off reflogs
(but we don't want to do so for the entire script, because
the point of turning them on was to cover some other cases).
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With "hash-object --literally", write a tag object that is not
supposed to pass one of the new checks added to "fsck", and make
sure that the new check catches the breakage.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* js/fsck-tag-validation:
Make sure that index-pack --strict checks tag objects
Add regression tests for stricter tag fsck'ing
fsck: check tag objects' headers
Make sure fsck_commit_buffer() does not run out of the buffer
fsck_object(): allow passing object data separately from the object itself
Refactor type_from_string() to allow continuing after detecting an error
One of the most important use cases for the strict tag object checking
is when transfer.fsckobjects is set to true to catch invalid objects
early on. This new regression test essentially tests the same code path
by directly calling 'index-pack --strict' on a pack containing an
tag object without a 'tagger' line.
Technically, this test is not enough: it only exercises a code path that
*warns*, not one that *fails*. The reason is that hash-object and
pack-objects both insist on parsing the tag objects and would fail on
invalid tag objects at this time.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fsck tries hard to detect missing objects, and will complain
(and exit non-zero) about any inter-object links that are
missing. However, it will not exit non-zero for any missing
ref tips, meaning that a severely broken repository may
still pass "git fsck && echo ok".
The problem is that we use for_each_ref to iterate over the
ref tips, which hides broken tips. It does at least print an
error from the refs.c code, but fsck does not ever see the
ref and cannot note the problem in its exit code. We can solve
this by using for_each_rawref and noting the error ourselves.
In addition to adding tests for this case, we add tests for
all types of missing-object links (all of which worked, but
which we were not testing).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce CONFIG_REGEX_NONE as a more explicit sentinel value to say
"we do not want to replace any existing entry" and use it in the
implementation of "git config --add".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
githooks(5) suggests:
Information about why the push is rejected may be sent to the user
by writing to standard error.
So follow that advice in the sample.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This allows "hash-object --stdin" to just hash any garbage into a
"loose object" that may not pass the standard object parsing check
or fsck, so that different kind of corrupt objects we may encounter
in the field can be imitated in our test suite. That would in turn
allow us to test features that catch these corrupt objects.
Note that "cat-file" may need to learn "--literally" option to allow
us peek into a truly broken object.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The intent of the new test case is to catch general breakages in
the fsck_tag() function, not so much to test it extensively, trying to
strike the proper balance between thoroughness and speed.
While it *would* have been nice to test the code path where fsck_object()
encounters an invalid tag object, this is not possible using git fsck: tag
objects are parsed already before fsck'ing (and the parser already fails
upon such objects).
Even worse: we would not even be able write out invalid tag objects
because git hash-object parses those objects, too, unless we resorted to
really ugly hacks such as using something like this in the unit tests
(essentially depending on Perl *and* Compress::Zlib):
hash_invalid_object () {
contents="$(printf '%s %d\0%s' "$1" ${#2} "$2")" &&
sha1=$(echo "$contents" | test-sha1) &&
suffix=${sha1#??} &&
mkdir -p .git/objects/${sha1%$suffix} &&
echo "$contents" |
perl -MCompress::Zlib -e 'undef $/; print compress(<>)' \
> .git/objects/${sha1%$suffix}/$suffix &&
echo $sha1
}
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of forcing callers of lower level functions write
(write_object ? HASH_WRITE_OBJECT : 0), prepare the flag to be
passed down in the callchain from the command line parser.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of the knobs that affect helper functions called from
cmd_hash_object() were passed to them as parameters already, and the
only effect of having them as file-scope statics was to make the
reader wonder if the parameters are hiding the file-scope global
values by accident. Adjust their initialisation and make them
function-local variables.
The only exception was no_filters hash_stdin_paths() peeked from the
file-scope global, which was converted to a parameter to the helper
function.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
