Most of the callers of packet_read_line just read into a
static 1000-byte buffer (callers which handle arbitrary
binary data already use LARGE_PACKET_MAX). This works fine
in practice, because:
1. The only variable-sized data in these lines is a ref
name, and refs tend to be a lot shorter than 1000
characters.
2. When sending ref lines, git-core always limits itself
to 1000 byte packets.
However, the only limit given in the protocol specification
in Documentation/technical/protocol-common.txt is
LARGE_PACKET_MAX; the 1000 byte limit is mentioned only in
pack-protocol.txt, and then only describing what we write,
not as a specific limit for readers.
This patch lets us bump the 1000-byte limit to
LARGE_PACKET_MAX. Even though git-core will never write a
packet where this makes a difference, there are two good
reasons to do this:
1. Other git implementations may have followed
protocol-common.txt and used a larger maximum size. We
don't bump into it in practice because it would involve
very long ref names.
2. We may want to increase the 1000-byte limit one day.
Since packets are transferred before any capabilities,
it's difficult to do this in a backwards-compatible
way. But if we bump the size of buffer the readers can
handle, eventually older versions of git will be
obsolete enough that we can justify bumping the
writers, as well. We don't have plans to do this
anytime soon, but there is no reason not to start the
clock ticking now.
Just bumping all of the reading bufs to LARGE_PACKET_MAX
would waste memory. Instead, since most readers just read
into a temporary buffer anyway, let's provide a single
static buffer that all callers can use. We can further wrap
this detail away by having the packet_read_line wrapper just
use the buffer transparently and return a pointer to the
static storage. That covers most of the cases, and the
remaining ones already read into their own LARGE_PACKET_MAX
buffers.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Having the packet sizes defined near the packet read/write
functions makes more sense.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The packets sent during ref negotiation are all terminated
by newline; even though the code to chomp these newlines is
short, we end up doing it in a lot of places.
This patch teaches packet_read_line to auto-chomp the
trailing newline; this lets us get rid of a lot of inline
chomping code.
As a result, some call-sites which are not reading
line-oriented data (e.g., when reading chunks of packfiles
alongside sideband) transition away from packet_read_line to
the generic packet_read interface. This patch converts all
of the existing callsites.
Since the function signature of packet_read_line does not
change (but its behavior does), there is a possibility of
new callsites being introduced in later commits, silently
introducing an incompatibility. However, since a later
patch in this series will change the signature, such a
commit would have to be merged directly into this commit,
not to the tip of the series; we can therefore ignore the
issue.
This is an internal cleanup and should produce no change of
behavior in the normal case. However, there is one corner
case to note. Callers of packet_read_line have never been
able to tell the difference between a flush packet ("0000")
and an empty packet ("0004"), as both cause packet_read_line
to return a length of 0. Readers treat them identically,
even though Documentation/technical/protocol-common.txt says
we must not; it also says that implementations should not
send an empty pkt-line.
By stripping out the newline before the result gets to the
caller, we will now treat the newline-only packet ("0005\n")
the same as an empty packet, which in turn gets treated like
a flush packet. In practice this doesn't matter, as neither
empty nor newline-only packets are part of git's protocols
(at least not for the line-oriented bits, and readers who
are not expecting line-oriented packets will be calling
packet_read directly, anyway). But even if we do decide to
care about the distinction later, it is orthogonal to this
patch. The right place to tighten would be to stop treating
empty packets as flush packets, and this change does not
make doing so any harder.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Originally we had a single function for reading packetized
data: packet_read_line. Commit 46284dd grew a more "gentle"
form, packet_read, that returns an error instead of dying
upon reading a truncated input stream. However, it is not
clear from the names which should be called, or what the
difference is.
Let's instead make packet_read be a generic public interface
that can take option flags, and update the single callsite
that uses it. This is less code, more clear, and paves the
way for introducing more options into the generic interface
later. The function signature is changed, so there should be
no hidden conflicts with topics in flight.
While we're at it, we'll document how error conditions are
handled based on the options, and rename the confusing
"return_line_fail" option to "gentle_on_eof". While we are
cleaning up the names, we can drop the "return_line_fail"
checks in packet_read_internal entirely. They look like
this:
ret = safe_read(..., return_line_fail);
if (return_line_fail && ret < 0)
...
The check for return_line_fail is a no-op; safe_read will
only ever return an error value if return_line_fail was true
in the first place.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is just write_or_die by another name. The one
distinction is that write_or_die will treat EPIPE specially
by suppressing error messages. That's fine, as we die by
SIGPIPE anyway (and in the off chance that it is disabled,
write_or_die will simulate it).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The comment describing the packet writing interface was
originally written above packet_write, but migrated to be
above safe_write in f3a3214, probably because it is meant to
generally describe the packet writing interface and not a
single function. Let's move it into the header file, where
users of the interface are more likely to see it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The write_or_die function will always die on an error,
including EPIPE. However, it currently treats EPIPE
specially by suppressing any error message, and by exiting
with exit code 0.
Suppressing the error message makes some sense; a pipe death
may just be a sign that the other side is not interested in
what we have to say. However, exiting with a successful
error code is not a good idea, as write_or_die is frequently
used in cases where we want to be careful about having
written all of the output, and we may need to signal to our
caller that we have done so (e.g., you would not want a push
whose other end has hung up to report success).
This distinction doesn't typically matter in git, because we
do not ignore SIGPIPE in the first place. Which means that
we will not get EPIPE, but instead will just die when we get
a SIGPIPE. But it's possible for a default handler to be set
by a parent process, or for us to add a callsite inside one
of our few SIGPIPE-ignoring blocks of code.
This patch converts write_or_die to actually raise SIGPIPE
when we see EPIPE, rather than exiting with zero. This
brings the behavior in line with the "normal" case that we
die from SIGPIPE (and any callers who want to check why we
died will see the same thing). We also give the same
treatment to other related functions, including
write_or_whine_pipe and maybe_flush_or_die.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current parsing scheme for upload-archive is to pack
arguments into a fixed-size buffer, separated by NULs, and
put a pointer to each argument in the buffer into a
fixed-size argv array.
This works fine, and the limits are high enough that nobody
reasonable is going to hit them, but it makes the code hard
to follow. Instead, let's just stuff the arguments into an
argv_array, which is much simpler. That lifts the "all
arguments must fit inside 4K together" limit.
We could also trivially lift the MAX_ARGS limitation (in
fact, we have to keep extra code to enforce it). But that
would mean a client could force us to allocate an arbitrary
amount of memory simply by sending us "argument" lines. By
limiting the MAX_ARGS, we limit an attacker to about 4
megabytes (64 times a maximum 64K packet buffer). That may
sound like a lot compared to the 4K limit, but it's not a
big deal compared to what git-archive will actually allocate
while working (e.g., to load blobs into memory). The
important thing is that it is bounded.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
According to the comment, enter_repo will modify its input.
However, this has not been the case since 1c64b48
(enter_repo: do not modify input, 2011-10-04). Drop the
now-useless copy.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This code predates prefixcmp, so it used memcmp along with
static sizes. Replacing these memcmps with prefixcmp makes
the code much more readable, and the lack of static sizes
will make refactoring it in future patches simpler.
Note that we used to be unnecessarily liberal in parsing the
"unpack" status line, and would accept "unpack ok\njunk". No
version of git has ever produced that, and it violates the
BNF in Documentation/technical/pack-protocol.txt. Let's take
this opportunity to tighten the check by converting the
prefix comparison into a strcmp.
While we're in the area, let's also fix a vague error
message that does not follow our usual conventions (it
writes directly to stderr and does not use the "error:"
prefix).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we read acks from the remote, we expect either:
ACK <sha1>
or
ACK <sha1> <multi-ack-flag>
We parse the "ACK <sha1>" bit from the line, and then start
looking for the flag strings at "line+45"; if we don't have
them, we assume it's of the first type. But if we do have
the first type, then line+45 is not necessarily inside our
string at all!
It turns out that this works most of the time due to the way
we parse the packets. They should come in with a newline,
and packet_read puts an extra NUL into the buffer, so we end
up with:
ACK <sha1>\n\0
with the newline at offset 44 and the NUL at offset 45. We
then strip the newline, putting a NUL at offset 44. So
when we look at "line+45", we are looking past the end of
our string; but it's OK, because we hit the terminator from
the original string.
This breaks down, however, if the other side does not
terminate their packets with a newline. In that case, our
packet is one character shorter, and we start looking
through uninitialized memory for the flag. No known
implementation sends such a packet, so it has never come up
in practice.
This patch tightens the check by looking for a short,
flagless ACK before trying to parse the flag.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If you set the GIT_DEBUG_SEND_PACK environment variable,
upload-pack will dump lines it receives in the receive_needs
phase to a descriptor. This debugging harness is a strict
subset of what GIT_TRACE_PACKET can do. Let's just drop it
in favor of that.
A few tests used GIT_DEBUG_SEND_PACK to confirm which
objects get sent; we have to adapt them to the new output
format.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the client tells us it has a shallow object via
"shallow <sha1>", we make sure we have the object, mark it
with a flag, then add it to a dynamic array of shallow
objects. This means that a client can get us to allocate
arbitrary amounts of memory just by flooding us with shallow
lines (whether they have the objects or not). You can
demonstrate it easily with:
yes '0035shallow e83c5163316f89bfbde7d9ab23ca2e25604af290' |
git-upload-pack git.git
We already protect against duplicates in want lines by
checking if our flag is already set; let's do the same thing
here. Note that a client can still get us to allocate some
amount of memory by marking every object in the repo as
"shallow" (or "want"). But this at least bounds it with the
number of objects in the repository, which is not under the
control of an upload-pack client.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we receive a line like "shallow <sha1>" from the
client, we feed the <sha1> part to get_sha1. This is a
mistake, as the argument on a shallow line is defined by
Documentation/technical/pack-protocol.txt to contain an
"obj-id". This is never defined in the BNF, but it is clear
from the text and from the other uses that it is meant to be
a hex sha1, not an arbitrary identifier (and that is what
fetch-pack has always sent).
We should be using get_sha1_hex instead, which doesn't allow
the client to request arbitrary junk like "HEAD@{yesterday}".
Because this is just marking shallow objects, the client
couldn't actually do anything interesting (like fetching
objects from unreachable reflog entries), but we should keep
our parsing tight to be on the safe side.
Because get_sha1 is for the most part a superset of
get_sha1_hex, in theory the only behavior change should be
disallowing non-hex object references. However, there is
one interesting exception: get_sha1 will only parse
a 40-character hex sha1 if the string has exactly 40
characters, whereas get_sha1_hex will just eat the first 40
characters, leaving the rest. That means that current
versions of git-upload-pack will not accept a "shallow"
packet that has a trailing newline, even though the protocol
documentation is clear that newlines are allowed (even
encouraged) in non-binary parts of the protocol.
This never mattered in practice, though, because fetch-pack,
contrary to the protocol documentation, does not include a
newline in its shallow lines. JGit follows its lead (though
it correctly is strict on the parsing end about wanting a
hex object id).
We do not adjust fetch-pack to send newlines here, as it
would break communication with older versions of git (and
there is no actual benefit to doing so, except for
consistency with other parts of the protocol).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ob/imap-send-ssl-verify:
imap-send: support subjectAltName as well
imap-send: the subject of SSL certificate must match the host
imap-send: move #ifdef around
Check not only the common name of the certificate subject, but also
check the subject alternative DNS names as well, when verifying that
the certificate matches that of the host we are trying to talk to.
Signed-off-by: Oswald Buddenhagen <ossi@kde.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We did not check a valid certificate's subject at all, and would
have happily talked with a wrong host after connecting to an
incorrect address and getting a valid certificate that does not
belong to the host we intended to talk to.
Signed-off-by: Oswald Buddenhagen <ossi@kde.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Html documents will be installed to root dir (/) no matter what prefix
is set, if run these commands before `make` and `make install-html`:
$ make configure
$ ./configure --prefix=<PREFIX>
After the installation, all the html documents will copy to rootdir (/),
and:
$ git --html-path
<PREFIX>
$ git help -w something
fatal: '<PREFIX>': not a documentation directory.
This is because the variable "htmldir" points to a undefined variable
"$(docdir)" in file "config.mak.autogen", which is generated by running
`./configure`. By default $(docdir) generated by configure is supposed
be set this way:
datarootdir='${prefix}/share'
htmldir='${docdir}'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
but since fc1c5415d6 (Honor configure's htmldir switch, 2013-02-02),
we only set and export htmldir without doing so for PACKAGE_TARNAME
(which is set to 'git' by the configure script).
Add the required two variables "PACKAGE_TARNAME" and "docdir" to file
"config.mak.in" will work this issue around.
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Usually we do not pass an empty string to the function hash_name()
because we almost always ask for hash values for a path that is a
candidate to be added to the index. However, check-ignore (and most
likely check-attr, but I didn't check) apparently has a callchain
to ask the hash value for an empty path when it was given a "." from
the top-level directory to ask "Is the path . excluded by default?"
Make sure that hash_name() does not overrun the end of the given
pathname even when it is empty.
Remove a sweep-the-issue-under-the-rug conditional in check-ignore
that avoided to pass an empty string to the callchain while at it.
It is a valid question to ask for check-ignore if the top-level is
set to be ignored by default, even though the answer is most likely
no, if only because there is currently no way to specify such an
entry in the .gitignore file. But it is an unusual thing to ask and
it is not worth optimizing for it by special casing at the top level
of the call chain.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If you try and update a submodule with a dirty working directory, you
get an error message like:
$ git submodule update
error: Your local changes to the following files would be overwritten by checkout:
...
Please, commit your changes or stash them before you can switch branches.
Aborting
...
Mention this in the submodule notes. The previous phrase was short
enough that I originally thought it might have been referring to the
reflog note (obviously, uncommitted changes will not show up in the
reflog either ;).
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Less work and more error checking (e.g. does a merge base exist?).
Add an explicit push before request-pull to satisfy request-pull,
which checks to make sure the references are publically available.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I think this interface is often more convenient than extended cherry
picking or using 'git format-patch'. In fact, I removed the
cherry-pick section entirely. The entry-level suggestions for
rerolling are now:
1. git commit --amend
2. git format-patch origin
git reset --hard origin
...edit and reorder patches...
git am *.patch
3. git rebase -i origin
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
test_expect_success_multi() helper function warrants some explanation,
since at first sight it may seem like generic test framework plumbing,
but is in fact specific to testing check-ignore, and allows more
thorough testing of the various output formats without significantly
increase the size of t0008.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, git will append two newlines to every message supplied via
the -m switch. The purpose of this is to allow -m to be supplied
multiple times and have each supplied string become a paragraph in the
resulting commit message.
Normally, this does not cause a problem since any trailing newlines will
be removed by the cleanup operation. If cleanup=verbatim for example,
then the trailing newlines will not be removed and will survive into the
resulting commit message.
Instead, let's ensure that the string supplied to -m is newline terminated,
but only append a second newline when appending additional messages.
Fixes the test in t7502.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This test attempts to verify that a commit message supplied to 'git
commit' via the -m switch was used in full as the commit message for a
commit when --cleanup=verbatim was used.
But, this test has been broken since it was introduced. Since the
commit message containing trailing newlines was supplied to 'git commit'
using a command substitution, the trailing newlines were removed by the
shell. This means that a string without any trailing newlines was
actually supplied to 'git commit'.
The test was able to complete successfully since internally, git appends
two newlines to each string supplied via the -m switch. So, the two
newlines removed by the shell were then re-added by git, and the
resulting commit matched what was expected.
So, let's move the initial creation of the commit message string out
from within a previous test so that it stands alone. Assign the desired
commit message to a variable using literal newlines. Then populate the
expect file from the contents of the commit message variable. This way
the shell variable becomes the authoritative source of the commit
message and can be supplied via the -m switch with the trailing newlines
intact.
Mark this test as failing, since it is not handled correctly by git.
As described above, git appends two extra newlines to every string
supplied via -m, even to the ones that already end with a newline.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This test attempts to verify that a commit in "verbatim" mode, when
supplied a commit template, produces a commit in which the commit
message matches exactly the template that was supplied. But, since the
commit operation appends additional instructions for the user as
comments in the commit buffer, which would cause the comparison to fail,
this test decided to compare only the first three lines (the length of
the template) of the resulting commit message to the original template
file.
This has two problems.
1. It does not allow the template to be lengthened or shortened
without also modifying the number of lines that are considered
significant (i.e. the argument to 'head -n').
2. It will not catch a bug in git that causes git to append additional
lines to the commit message.
So, let's use the --no-status option to 'git commit' which will cause
git to refrain from appending the lines of instructional text to the
commit message. This will allow the entire resulting commit message to
be compared against the expected value.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of adding an early return to the inside of the
ssl_socket_connect() function for NO_OPENSSL compilation, split it
into a separate stub function.
No functional change, but the next change to extend ssl_socket_connect()
will become easier to read this way.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We stopped mentioning `tracking` is a deprecated but supported
synonym for `upstream` in pull.default even though we have no
intention of removing the support for it.
* jc/mention-tracking-for-pull-default:
doc: mention tracking for pull.default
Also migration path for the default behaviour of "git add -u/-A" run
in a subdirectory is worth mentioning.
Both pointed out by Matthieu Moy.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Minor clean up of if-then nesting in checks for environment variables
and config options. No functional changes.
Signed-off-by: Martin Erik Werner <martinerikwerner@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
user-manual: use -o latest.tar.gz to create a gzipped tarball
user-manual: use 'git config --global user.*' for setup
user-manual: mention 'git remote add' for remote branch config
user-manual: give 'git push -f' as an alternative to +master
user-manual: use 'remote add' to setup push URLs
This functionality was introduced by 0e804e09 (archive: provide
builtin .tar.gz filter, 2011-07-21) for v1.7.7.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A simple command line call is easier than spawning an editor,
especially for folks new to ideas like the "command line" and "text
editors". This is also the approach suggested by 'git commit' if you
try and commit without having configured user.name or user.email.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I hardly ever setup remote.<name>.url using 'git config'. While it
may be instructive to do so, we should also point out 'git remote
add'.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This mirrors existing language in the description of 'git fetch'.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There is no need to use here documents to setup this configuration.
It is easier, less confusing, and more robust to use `git remote add`
directly.
Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
