At present we do headers unfolding (see RFC822 3.1.1. LONG HEADER FIELDS) for
all fields except 'From' (always) and 'Subject' (when keep_subject is set)
Not unfolding 'From' is a bug -- see above-mentioned RFC link.
Signed-off-by: Kirill Smelkov <kirr@landau.phys.spbu.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When native language (RU) is in use, subject header usually contains several
parts, e.g.
Subject: [Navy-patches] [PATCH]
=?utf-8?b?0JjQt9C80LXQvdGR0L0g0YHQv9C40YHQvtC6INC/0LA=?=
=?utf-8?b?0LrQtdGC0L7QsiDQvdC10L7QsdGF0L7QtNC40LzRi9GFINC00LvRjyA=?=
=?utf-8?b?0YHQsdC+0YDQutC4?=
This exposes several bugs in builtin-mailinfo.c:
1. decode_b_segment: do not append explicit NUL -- explicit NUL was preventing
correct header construction on parts concatenation via strbuf_addbuf in
decode_header_bq. Fixes:
-Subject: Изменён список пакетов необходимых для сборки
+Subject: Изменён список па
Then
2. Do not emit '\n' between "encoded-word" where RFC2046 says that linear
white space between them are ignored when displaying. Fixes:
-Subject: Изменён список пакетов необходимых для сборки
+Subject: Изменён список па кетов необходимых для сборки
Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In handle_from, we calculate the end boundary of a section
to remove from a strbuf using strcspn like this:
el = strcspn(buf, set_of_end_boundaries);
strbuf_remove(&sb, start, el + 1);
This works fine if "el" is the offset of the boundary
character, meaning we remove up to and including that
character. But if the end boundary didn't match (that is, we
hit the end of the string as the boundary instead) then we
want just "el". Asking for "el+1" caught an out-of-bounds
assertion in the strbuf library.
This manifested itself when we got a 'From' header that had
just an email address with nothing else in it (the end of
the string was the end of the address, rather than, e.g., a
trailing '>' character), causing git-mailinfo to barf.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Recent changes to is_multipart_boundary() caused git-mailinfo to segfault.
The reason was after handling the end of the boundary the code tried to look
for another boundary. Because the boundary list was empty, dereferencing
the pointer to the top of the boundary caused the program to go boom.
The fix is to check to see if the list is empty and if so go on its merry
way instead of looking for another boundary.
I also fixed a couple of increments and decrements that didn't look correct
relating to content_top.
The boundary test case was updated to catch future problems like this again.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After finding a MIME multi-part message boundary line, the handle_body()
function is supposed to first flush any accumulated contents from the
previous part to the output stream. However, the code mistakenly output
the boundary line it found.
The old code that used one global, fixed-length buffer line[] used an
alternate static buffer newline[] for keeping track of this accumulated
contents and flushed newline[] upon seeing the boundary; when 3b6121f
(git-mailinfo: use strbuf's instead of fixed buffers, 2008-07-13)
converted a fixed-length buffer in this program to use strbuf,these two
buffers were converted to "line" and "prev" (the latter of which now has a
much more sensible name) strbufs, but the code mistakenly flushed "line"
(which contains the boundary we have just found), instead of "prev".
This resulted in the first boundary to be output in front of the first
line of the message.
The rewritten implementation of handle_boundary() lost the terminating
newline; this would then result in the second line of the message to be
stuck with the first line.
The is_multipart_boundary() was designed to catch both the internal
boundary and the terminating one (the one with trailing "--"); this also
was broken with the rewrite, and the code in the handle_boundary() to
handle the terminating boundary was never triggered.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When using git-rebase, author fields containing a ')' at the last position
had the close-parens character removed; the removal should be done only
when it is of this form:
user@host (User Name)
i.e. the remainder after stripping the e-mail address part is enclosed in
a parentheses pair as a whole, not for addresses like this:
User Name (me) <user@host>
Signed-off-by: Philippe Bruhat (BooK) <book@cpan.org>
Acked-by: Lukas Sandström <lukass@etek.chalmers.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A patch title "[PATCH] 1" was sanitized by the original code by stripping
the "[PATCH]" from the front, but after the conversion to use strbuf this
behaviour was broken due to a counting error.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ls/mailinfo:
git-mailinfo: use strbuf's instead of fixed buffers
Add some useful functions for strbuf manipulation.
Make some strbuf_*() struct strbuf arguments const.
Conflicts:
builtin-mailinfo.c
* sb/dashless:
Make usage strings dash-less
t/: Use "test_must_fail git" instead of "! git"
t/test-lib.sh: exit with small negagive int is ok with test_must_fail
Conflicts:
builtin-blame.c
builtin-mailinfo.c
builtin-mailsplit.c
builtin-shortlog.c
git-am.sh
t/t4150-am.sh
t/t4200-rerere.sh
"Subject: " isn't in the static array "header", and thus
memcmp("Subject:", header[i], 7) will never match.
Even if it did so, hdr_data[] may not have been allocated if there weren't
a "Subject: " in-body when we process "[PATCH]" in the affected codepath.
Signed-off-by: Lukas Sandström <lukass@etek.chalmers.se>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When you misuse a git command, you are shown the usage string.
But this is currently shown in the dashed form. So if you just
copy what you see, it will not work, when the dashed form
is no longer supported.
This patch makes git commands show the dash-less version.
For shell scripts that do not specify OPTIONS_SPEC, git-sh-setup.sh
generates a dash-less usage string now.
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When handling a MIME multipart message, multi-part boundary lines are eaten
by a call to handle_boundary() function from the main loop of handle_body(),
and after that happens, we should update the line length correctly, because
handle_boundary() udpates line[] with new data.
This was caused by a thinko in 9aa2309 (mailinfo: apply the same fix not
to lose NULs in BASE64 and QP codepaths, 2008-05-25).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function fgets() has a big problem with NUL characters: it reads
them, but nobody will know if the NUL comes from the file stream, or
was appended at the end of the line.
So implement a custom read_line_with_nul() function.
Noticed by Tommy Thorn.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git_config() only had a function parameter, but no callback data
parameter. This assumes that all callback functions only modify
global variables.
With this patch, every callback gets a void * parameter, and it is hoped
that this will help the libification effort.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function is intended to be fed one logical line at a time to
inspect, but a QP encoded raw input line can have more than one
lines, just like BASE64 encoded one.
Quoting LF as =0A may be unusual but RFC2045 allows it.
The issue was noticed and fixed by Jay Soffian. JC added a test
to protect the fix from regressing later.
Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes rebase/am keep the original commit log message
better, even when it does not conform to "single line paragraph
to say what it does, then explain and defend why it is a good
change in later paragraphs" convention.
This change is a two-edged sword. While the earlier behaviour
would make such commit log messages more friendly to readers who
expect to get the birds-eye view with oneline summary formats,
users who primarily use git as a way to interact with foreign
SCM systems would not care much about the convenience of oneline
git log tools, but care more about preserving their own
convention. This changes their commits less useful to readers
who read them with git tools while keeping them more consistent
with the foreign SCM systems they interact with.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For some reason, I got this error message. Maybe it does not make sense,
but then we should not really try to convert the text when it is not
necessary.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
At least in the kernel development community, we're generally slowly
converting to UTF-8 everywhere, and the old default of Latin1 in emails is
being supplanted by UTF-8, and it doesn't necessarily show up as such in
the mail headers (because, quite frankly, when people send patches
around, they want the email client to do as little as humanly possible
about the patch)
Despite that, it's often the case that email addresses etc still have
Latin1, so I've seen emails where this is a mixed bag, with Signed-off
parts being copied from email (and containing Latin1 characters), and the
rest of the email being a patch in UTF-8.
So this suggests a very natural change: if the target character set is
utf-8 (the default), and if the source already looks like utf-8, just
assume that it doesn't need any conversion at all.
Only assume that it needs conversion if it isn't already valid utf-8, in
which case we (for historical reasons) will assume it's Latin1.
Basically no really _valid_ latin1 will ever look like utf-8, so while
this changes our historical behaviour, it doesn't do so in practice, and
makes the default behaviour saner for the case where the input was already
in proper format.
We could do a more fancy guess, of course, but this correctly handled a
series of patches I just got from Andrew that had a mixture of Latin1 and
UTF-8 (in different emails, but without any character set indication).
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Don't translate the patch to UTF-8, instead preserve the data as
is. This also reverts a test case that was included in the
original patch series.
Also allow overwriting the authorship and title information we
gather from RFC2822 mail headers with additional in-body
headers, which was pointed out by Linus.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I have come across many emails that use long strings of '-'s as separators
for ideas. This patch below limits the separator to only 3 '-', with the
intent that long string of '-'s will stay in the commit msg and not in the
patch file.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I am working on a project that required parsing through regular
mboxes that didn't necessarily have patches embedded in them. I
started by creating my own modified copy of git-am and working
from there. Very quickly, I noticed git-mailinfo wasn't able to
handle a big chunk of my email.
After hacking up numerous solutions and running into more
limitations, I decided it was just easier to rewrite a big chunk
of it. The following patch has a bunch of fixes and features
that I needed in order for me do what I wanted.
Note: I'm didn't follow any email rfc papers but I don't think
any of the changes I did required much knowledge (besides the
boundary stuff).
List of major changes/fixes:
- can't create empty patch files fix
- empty patch files don't fail, this failure will come inside git-am
- multipart boundaries are now handled
- only output inbody headers if a patch exists otherwise assume those
headers are part of the reply and instead output the original headers
- decode and filter base64 patches correctly
- various other accidental fixes
I believe I didn't break any existing functionality or
compatibility (other than what I describe above, which is really
only the empty patch file).
I tested this through various mailing list archives and
everything seemed to parse correctly (a couple thousand emails).
[jc: squashed in another patch from Don's five patch series to
fix the test case, as this patch exposes the bug in the test.]
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We shouldn't attempt to assign constant strings into char*, as the
string is not writable at runtime. Likewise we should always be
treating unsigned values as unsigned values, not as signed values.
Most of these are very straightforward. The only exception is the
(unnecessary) xstrdup/free in builtin-branch.c for the detached
head case. Since this is a user-level interactive type program
and that particular code path is executed no more than once, I feel
that the extra xstrdup call is well worth the easy elimination of
this warning.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* maint:
git-apply: do not fix whitespaces on context lines.
diff --cc: integer overflow given a 2GB-or-larger file
mailinfo: do not get confused with logical lines that are too long.
It basically considers all the continuation lines to be lines of their
own, and if the total line is bigger than what we can fit in it, we just
truncate the result rather than stop in the middle and then get confused
when we try to parse the "next" line (which is just the remainder of the
first line).
[jc: added test, and tightened boundary a bit per list discussion.]
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This mechanically converts strncmp() to use prefixcmp(), but only when
the parameters match specific patterns, so that they can be verified
easily. Leftover from this will be fixed in a separate step, including
idiotic conversions like
if (!strncmp("foo", arg, 3))
=>
if (!(-prefixcmp(arg, "foo")))
This was done by using this script in px.perl
#!/usr/bin/perl -i.bak -p
if (/strncmp\(([^,]+), "([^\\"]*)", (\d+)\)/ && (length($2) == $3)) {
s|strncmp\(([^,]+), "([^\\"]*)", (\d+)\)|prefixcmp($1, "$2")|;
}
if (/strncmp\("([^\\"]*)", ([^,]+), (\d+)\)/ && (length($1) == $3)) {
s|strncmp\("([^\\"]*)", ([^,]+), (\d+)\)|(-prefixcmp($2, "$1"))|;
}
and running:
$ git grep -l strncmp -- '*.c' | xargs perl px.perl
Signed-off-by: Junio C Hamano <junkio@cox.net>
It is plausible for somebody to want to view the commit log in a
different encoding from i18n.commitencoding -- the project's
policy may be UTF-8 and the user may be using a commit message
hook to run iconv to conform to that policy (and either not have
i18n.commitencoding to default to UTF-8 or have it explicitly
set to UTF-8). Even then, Latin-1 may be more convenient for
the usual pager and the terminal the user uses.
The new variable i18n.logoutputencoding is used in preference to
i18n.commitencoding to decide what encoding to recode the log
output in when git-log and friends formats the commit log message.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a mechanical clean-up of the way *.c files include
system header files.
(1) sources under compat/, platform sha-1 implementations, and
xdelta code are exempt from the following rules;
(2) the first #include must be "git-compat-util.h" or one of
our own header file that includes it first (e.g. config.h,
builtin.h, pkt-line.h);
(3) system headers that are included in "git-compat-util.h"
need not be included in individual C source files.
(4) "git-compat-util.h" does not have to include subsystem
specific header files (e.g. expat.h).
Signed-off-by: Junio C Hamano <junkio@cox.net>
builtin-mailinfo.c has its own hexval implementaiton but it can
share the table-lookup one recently implemented in sha1_file.c
Signed-off-by: Junio C Hamano <junkio@cox.net>
[jc: I needed to hand merge the changes to the updated codebase,
so the result needs to be checked.]
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This changes the calling convention of built-in commands and
passes the "prefix" (i.e. pathname of $PWD relative to the
project root level) down to them.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Mail I get sometimes has multiple From lines, like this:
From Majordomo@vger.kernel.org Thu Jul 27 16:39:36 2006
>From mtsirkin Thu Jul 27 16:39:36 2006
Received: from yok.mtl.com [10.0.8.11]
...
which confuses git-mailinfo since that does not recognize >From
as a valid header line.
This patch makes it recognize >From XXX as a valid header line.
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Junio C Hamano <junkio@cox.net>
When the input mbox does not identify what encoding it is in,
and already have RFC2047 stripped away, we cannot tell what
encoding the header text is in. For body text, when the message
does not say what charset it is in, we fall back to assume
latin-1 input when converting to utf8. This should be done
consistently to the header as well.
Signed-off-by: Junio C Hamano <junkio@cox.net>
