Merge branch 'mm/multimail-1.5'

Update "git multimail" from the upstream.

* mm/multimail-1.5:
  git-multimail: update to release 1.5.0
This commit is contained in:
Junio C Hamano 2019-01-18 13:49:55 -08:00
commit 41db137234
8 changed files with 281 additions and 57 deletions

View File

@ -1,3 +1,59 @@
Release 1.5.0
=============
Backward-incompatible change
----------------------------
The name of classes for environment was misnamed as `*Environement`.
It is now `*Environment`.
New features
------------
* A Thread-Index header is now added to each email sent (except for
combined emails where it would not make sense), so that MS Outlook
properly groups messages by threads even though they have a
different subject line. Unfortunately, even adding this header the
threading still seems to be unreliable, but it is unclear whether
this is an issue on our side or on MS Outlook's side (see discussion
here: https://github.com/git-multimail/git-multimail/pull/194).
* A new variable multimailhook.ExcludeMergeRevisions was added to send
notification emails only for non-merge commits.
* For gitolite environment, it is now possible to specify the mail map
in a separate file in addition to gitolite.conf, using the variable
multimailhook.MailaddressMap.
Internal changes
----------------
* The testsuite now uses GIT_PRINT_SHA1_ELLIPSIS where needed for
compatibility with recent Git versions. Only tests are affected.
* We don't try to install pyflakes in the continuous integration job
for old Python versions where it's no longer available.
* Stop using the deprecated cgi.escape in Python 3.
* New flake8 warnings have been fixed.
* Python 3.6 is now tested against on Travis-CI.
* A bunch of lgtm.com warnings have been fixed.
Bug fixes
---------
* SMTPMailer logs in only once now. It used to re-login for each email
sent which triggered errors for some SMTP servers.
* migrate-mailhook-config was broken by internal refactoring, it
should now work again.
This version was tested with Python 2.6 to 3.7. It was tested with Git
1.7.10.406.gdc801, 2.15.1 and 2.20.1.98.gecbdaf0.
Release 1.4.0
=============

View File

@ -4,9 +4,8 @@ Contributing
git-multimail is an open-source project, built by volunteers. We would
welcome your help!
The current maintainers are Matthieu Moy
<matthieu.moy@grenoble-inp.fr> and Michael Haggerty
<mhagger@alum.mit.edu>.
The current maintainers are `Matthieu Moy <http://matthieu-moy.fr>`__ and
`Michael Haggerty <https://github.com/mhagger>`__.
Please note that although a copy of git-multimail is distributed in
the "contrib" section of the main Git project, development takes place
@ -33,6 +32,29 @@ mailing list`_.
Please CC emails regarding git-multimail to the maintainers so that we
don't overlook them.
Help needed: testers/maintainer for specific environments/OS
------------------------------------------------------------
The current maintainer uses and tests git-multimail on Linux with the
Generic environment. More testers, or better contributors are needed
to test git-multimail on other real-life setups:
* Mac OS X, Windows: git-multimail is currently not supported on these
platforms. But since we have no external dependencies and try to
write code as portable as possible, it is possible that
git-multimail already runs there and if not, it is likely that it
could be ported easily.
Patches to improve support for Windows and OS X are welcome.
Ideally, there would be a sub-maintainer for each OS who would test
at least once before each release (around twice a year).
* Gerrit, Stash, Gitolite environments: although the testsuite
contains tests for these environments, a tester/maintainer for each
environment would be welcome to test and report failure (or success)
on real-life environments periodically (here also, feedback before
each release would be highly appreciated).
.. _`git-multimail repository on GitHub`: https://github.com/git-multimail/git-multimail
.. _`Git mailing list`: git@vger.kernel.org

View File

@ -6,10 +6,10 @@ website:
https://github.com/git-multimail/git-multimail
The version in this directory was obtained from the upstream project
on August 17 2016 and consists of the "git-multimail" subdirectory from
on January 07 2019 and consists of the "git-multimail" subdirectory from
revision
07b1cb6bfd7be156c62e1afa17cae13b850a869f refs/tags/1.4.0
04e80e6c40be465cc62b6c246f0fcb8fd2cfd454 refs/tags/1.5.0
Please see the README file in this directory for information about how
to report bugs or contribute to git-multimail.

View File

@ -1,4 +1,4 @@
git-multimail version 1.4.0
git-multimail version 1.5.0
===========================
.. image:: https://travis-ci.org/git-multimail/git-multimail.svg?branch=master
@ -20,8 +20,8 @@ GPLv2 (see the COPYING file for details).
Please note: although, as a convenience, git-multimail may be
distributed along with the main Git project, development of
git-multimail takes place in its own, separate project. See section
"Getting involved" below for more information.
git-multimail takes place in its own, separate project. Please, read
`<CONTRIBUTING.rst>`__ for more information.
By default, for each push received by the repository, git-multimail:
@ -89,6 +89,10 @@ Requirements
the multimailhook.mailer configuration variable below for how to
configure git-multimail to send emails via an SMTP server.
* git-multimail is currently tested only on Linux. It may or may not
work on other platforms such as Windows and Mac OS. See
`<CONTRIBUTING.rst>`__ to improve the situation.
Invocation
----------
@ -369,7 +373,7 @@ multimailhook.mailer
unset, then the value of multimailhook.from is used.
multimailhook.smtpServerTimeout
Timeout in seconds.
Timeout in seconds. Default is 10.
multimailhook.smtpEncryption
Set the security type. Allowed values: ``none``, ``ssl``, ``tls`` (starttls).
@ -419,8 +423,20 @@ multimailhook.from, multimailhook.fromCommit, multimailhook.fromRefchange
If config values are unset, the value of the From: header is
determined as follows:
1. (gitolite environment only) Parse gitolite.conf, looking for a
block of comments that looks like this::
1. (gitolite environment only)
1.a) If ``multimailhook.MailaddressMap`` is set, and is a path
to an existing file (if relative, it is considered relative to
the place where ``gitolite.conf`` is located), then this file
should contain lines like::
username Firstname Lastname <email@example.com>
git-multimail will then look for a line where ``$GL_USER``
matches the ``username`` part, and use the rest of the line for
the ``From:`` header.
1.b) Parse gitolite.conf, looking for a block of comments that
looks like this::
# BEGIN USER EMAILS
# username Firstname Lastname <email@example.com>
@ -436,6 +452,11 @@ multimailhook.from, multimailhook.fromCommit, multimailhook.fromRefchange
3. Use the value of multimailhook.envelopeSender.
multimailhook.MailaddressMap
(gitolite environment only)
File to look for a ``From:`` address based on the user doing the
push. Defaults to unset. See ``multimailhook.from`` for details.
multimailhook.administrator
The name and/or email address of the administrator of the Git
repository; used in FOOTER_TEMPLATE. Default is
@ -484,6 +505,11 @@ multimailhook.maxCommitEmails
mailbombing, for example on an initial push. To disable commit
emails limit, set this option to 0. The default is 500.
multimailhook.excludeMergeRevisions
When sending out revision emails, do not consider merge commits (the
functional equivalent of `rev-list --no-merges`).
The default is `false` (send merge commit emails).
multimailhook.emailStrictUTF8
If this boolean option is set to `true`, then the main part of the
email body is forced to be valid UTF-8. Any characters that are

View File

@ -46,6 +46,15 @@ and add::
config multimailhook.mailingList = # Where emails should be sent
config multimailhook.from = # From address to use
Note that by default, gitolite forbids ``<`` and ``>`` in variable
values (for security/paranoia reasons, see
`compensating for UNSAFE_PATT
<http://gitolite.com/gitolite/git-config/index.html#compensating-for-unsafe95patt>`__
in gitolite's documentation for explanations and a way to disable
this). As a consequence, you will not be able to use ``First Last
<First.Last@example.com>`` as recipient email, but specifying
``First.Last@example.com`` alone works.
Obviously, you can customize all parameters on a per-repository basis by
adding these ``config multimailhook.*`` lines in the section
corresponding to a repository or set of repositories.

View File

@ -1,6 +1,6 @@
#! /usr/bin/env python
__version__ = '1.4.0'
__version__ = '1.5.0'
# Copyright (c) 2015-2016 Matthieu Moy and others
# Copyright (c) 2012-2014 Michael Haggerty and others
@ -64,7 +64,9 @@ except ImportError:
# Python < 2.6 do not have ssl, but that's OK if we don't use it.
pass
import time
import cgi
import uuid
import base64
PYTHON3 = sys.version_info >= (3, 0)
@ -73,7 +75,7 @@ if sys.version_info <= (2, 5):
for element in iterable:
if not element:
return False
return True
return True
def is_ascii(s):
@ -108,6 +110,12 @@ if PYTHON3:
return out.decode(sys.getdefaultencoding())
except UnicodeEncodeError:
return out.decode(ENCODING)
import html
def html_escape(s):
return html.escape(s)
else:
def is_string(s):
try:
@ -130,6 +138,10 @@ else:
def next(it):
return it.next()
import cgi
def html_escape(s):
return cgi.escape(s, True)
try:
from email.charset import Charset
@ -190,6 +202,7 @@ Content-Transfer-Encoding: 8bit
Message-ID: %(msgid)s
From: %(fromaddr)s
Reply-To: %(reply_to)s
Thread-Index: %(thread_index)s
X-Git-Host: %(fqdn)s
X-Git-Repo: %(repo_shortname)s
X-Git-Refname: %(refname)s
@ -322,6 +335,7 @@ From: %(fromaddr)s
Reply-To: %(reply_to)s
In-Reply-To: %(reply_to_msgid)s
References: %(reply_to_msgid)s
Thread-Index: %(thread_index)s
X-Git-Host: %(fqdn)s
X-Git-Repo: %(repo_shortname)s
X-Git-Refname: %(refname)s
@ -763,6 +777,9 @@ class GitObject(object):
def __eq__(self, other):
return isinstance(other, GitObject) and self.sha1 == other.sha1
def __ne__(self, other):
return not self == other
def __hash__(self):
return hash(self.sha1)
@ -852,7 +869,7 @@ class Change(object):
if html_escape_val:
for k in values:
if is_string(values[k]):
values[k] = cgi.escape(values[k], True)
values[k] = html_escape(values[k])
for line in template.splitlines(True):
yield line % values
@ -909,7 +926,7 @@ class Change(object):
raise NotImplementedError()
def generate_email_body(self):
def generate_email_body(self, push):
"""Generate the main part of the email body, a line at a time.
The text in the body might be truncated after a specified
@ -936,7 +953,7 @@ class Change(object):
yield "<pre style='margin:0'>\n"
for line in lines:
yield cgi.escape(line)
yield html_escape(line)
yield '</pre>\n'
else:
@ -1011,7 +1028,7 @@ class Change(object):
fgcolor = '404040'
# Chop the trailing LF, we don't want it inside <pre>.
line = cgi.escape(line[:-1])
line = html_escape(line[:-1])
if bgcolor or fgcolor:
style = 'display:block; white-space:pre;'
@ -1060,6 +1077,10 @@ class Revision(Change):
self.author = read_git_output(['log', '--no-walk', '--format=%aN <%aE>', self.rev.sha1])
self.recipients = self.environment.get_revision_recipients(self)
# -s is short for --no-patch, but -s works on older git's (e.g. 1.7)
self.parents = read_git_lines(['show', '-s', '--format=%P',
self.rev.sha1])[0].split()
self.cc_recipients = ''
if self.environment.get_scancommitforcc():
self.cc_recipients = ', '.join(to.strip() for to in self._cc_recipients())
@ -1090,6 +1111,7 @@ class Revision(Change):
oneline = oneline[:max_subject_length - 6] + ' [...]'
values['rev'] = self.rev.sha1
values['parents'] = ' '.join(self.parents)
values['rev_short'] = self.rev.short
values['change_type'] = self.change_type
values['refname'] = self.refname
@ -1097,6 +1119,7 @@ class Revision(Change):
values['short_refname'] = self.reference_change.short_refname
values['refname_type'] = self.reference_change.refname_type
values['reply_to_msgid'] = self.reference_change.msgid
values['thread_index'] = self.reference_change.thread_index
values['num'] = self.num
values['tot'] = self.tot
values['recipients'] = self.recipients
@ -1244,6 +1267,23 @@ class ReferenceChange(Change):
old=old, new=new, rev=rev,
)
@staticmethod
def make_thread_index():
"""Return a string appropriate for the Thread-Index header,
needed by MS Outlook to get threading right.
The format is (base64-encoded):
- 1 byte must be 1
- 5 bytes encode a date (hardcoded here)
- 16 bytes for a globally unique identifier
FIXME: Unfortunately, even with the Thread-Index field, MS
Outlook doesn't seem to do the threading reliably (see
https://github.com/git-multimail/git-multimail/pull/194).
"""
thread_index = b'\x01\x00\x00\x12\x34\x56' + uuid.uuid4().bytes
return base64.standard_b64encode(thread_index).decode('ascii')
def __init__(self, environment, refname, short_refname, old, new, rev):
Change.__init__(self, environment)
self.change_type = {
@ -1257,6 +1297,7 @@ class ReferenceChange(Change):
self.new = new
self.rev = rev
self.msgid = make_msgid()
self.thread_index = self.make_thread_index()
self.diffopts = environment.diffopts
self.graphopts = environment.graphopts
self.logopts = environment.logopts
@ -1276,6 +1317,7 @@ class ReferenceChange(Change):
values['refname'] = self.refname
values['short_refname'] = self.short_refname
values['msgid'] = self.msgid
values['thread_index'] = self.thread_index
values['recipients'] = self.recipients
values['oldrev'] = str(self.old)
values['oldrev_short'] = self.old.short
@ -1941,6 +1983,9 @@ class Mailer(object):
def __init__(self, environment):
self.environment = environment
def close(self):
pass
def send(self, lines, to_addrs):
"""Send an email consisting of lines.
@ -2054,6 +2099,7 @@ class SMTPMailer(Mailer):
self.username = smtpuser
self.password = smtppass
self.smtpcacerts = smtpcacerts
self.loggedin = False
try:
def call(klass, server, timeout):
try:
@ -2130,20 +2176,30 @@ class SMTPMailer(Mailer):
% (self.smtpserver, sys.exc_info()[1]))
sys.exit(1)
def __del__(self):
def close(self):
if hasattr(self, 'smtp'):
self.smtp.quit()
del self.smtp
def __del__(self):
self.close()
def send(self, lines, to_addrs):
try:
if self.username or self.password:
self.smtp.login(self.username, self.password)
if not self.loggedin:
self.smtp.login(self.username, self.password)
self.loggedin = True
msg = ''.join(lines)
# turn comma-separated list into Python list if needed.
if is_string(to_addrs):
to_addrs = [email for (name, email) in getaddresses([to_addrs])]
self.smtp.sendmail(self.envelopesender, to_addrs, msg)
except socket.timeout:
self.environment.get_logger().error(
'*** Error sending email ***\n'
'*** SMTP server timed out (timeout is %s)\n'
% self.smtpservertimeout)
except smtplib.SMTPResponseException:
err = sys.exc_info()[1]
self.environment.get_logger().error(
@ -2171,7 +2227,8 @@ class OutputMailer(Mailer):
SEPARATOR = '=' * 75 + '\n'
def __init__(self, f):
def __init__(self, f, environment=None):
super(OutputMailer, self).__init__(environment=environment)
self.f = f
def send(self, lines, to_addrs):
@ -2382,6 +2439,7 @@ class Environment(object):
self.html_in_footer = False
self.commitBrowseURL = None
self.maxcommitemails = 500
self.excludemergerevisions = False
self.diffopts = ['--stat', '--summary', '--find-copies-harder']
self.graphopts = ['--oneline', '--decorate']
self.logopts = []
@ -2621,6 +2679,8 @@ class ConfigOptionsEnvironmentMixin(ConfigEnvironmentMixin):
self.commitBrowseURL = config.get('commitBrowseURL')
self.excludemergerevisions = config.get('excludeMergeRevisions')
maxcommitemails = config.get('maxcommitemails')
if maxcommitemails is not None:
try:
@ -3152,7 +3212,10 @@ class GitoliteEnvironmentHighPrecMixin(Environment):
return self.osenv.get('GL_USER', 'unknown user')
class GitoliteEnvironmentLowPrecMixin(Environment):
class GitoliteEnvironmentLowPrecMixin(
ConfigEnvironmentMixin,
Environment):
def get_repo_shortname(self):
# The gitolite environment variable $GL_REPO is a pretty good
# repo_shortname (though it's probably not as good as a value
@ -3162,6 +3225,16 @@ class GitoliteEnvironmentLowPrecMixin(Environment):
super(GitoliteEnvironmentLowPrecMixin, self).get_repo_shortname()
)
@staticmethod
def _compile_regex(re_template):
return (
re.compile(re_template % x)
for x in (
r'BEGIN\s+USER\s+EMAILS',
r'([^\s]+)\s+(.*)',
r'END\s+USER\s+EMAILS',
))
def get_fromaddr(self, change=None):
GL_USER = self.osenv.get('GL_USER')
if GL_USER is not None:
@ -3174,18 +3247,42 @@ class GitoliteEnvironmentLowPrecMixin(Environment):
GL_CONF = self.osenv.get(
'GL_CONF',
os.path.join(GL_ADMINDIR, 'conf', 'gitolite.conf'))
mailaddress_map = self.config.get('MailaddressMap')
# If relative, consider relative to GL_CONF:
if mailaddress_map:
mailaddress_map = os.path.join(os.path.dirname(GL_CONF),
mailaddress_map)
if os.path.isfile(mailaddress_map):
f = open(mailaddress_map, 'rU')
try:
# Leading '#' is optional
re_begin, re_user, re_end = self._compile_regex(
r'^(?:\s*#)?\s*%s\s*$')
for l in f:
l = l.rstrip('\n')
if re_begin.match(l) or re_end.match(l):
continue # Ignore these lines
m = re_user.match(l)
if m:
if m.group(1) == GL_USER:
return m.group(2)
else:
continue # Not this user, but not an error
raise ConfigurationException(
"Syntax error in mail address map.\n"
"Check file {}.\n"
"Line: {}".format(mailaddress_map, l))
finally:
f.close()
if os.path.isfile(GL_CONF):
f = open(GL_CONF, 'rU')
try:
in_user_emails_section = False
re_template = r'^\s*#\s*%s\s*$'
re_begin, re_user, re_end = (
re.compile(re_template % x)
for x in (
r'BEGIN\s+USER\s+EMAILS',
re.escape(GL_USER) + r'\s+(.*)',
r'END\s+USER\s+EMAILS',
))
re_begin, re_user, re_end = self._compile_regex(
r'^\s*#\s*%s\s*$')
for l in f:
l = l.rstrip('\n')
if not in_user_emails_section:
@ -3195,8 +3292,8 @@ class GitoliteEnvironmentLowPrecMixin(Environment):
if re_end.match(l):
break
m = re_user.match(l)
if m:
return m.group(1)
if m and m.group(1) == GL_USER:
return m.group(2)
finally:
f.close()
return super(GitoliteEnvironmentLowPrecMixin, self).get_fromaddr(change)
@ -3228,7 +3325,7 @@ class StashEnvironmentHighPrecMixin(Environment):
self.__repo = repo
def get_pusher(self):
return re.match('(.*?)\s*<', self.__user).group(1)
return re.match(r'(.*?)\s*<', self.__user).group(1)
def get_pusher_email(self):
return self.__user
@ -3262,7 +3359,7 @@ class GerritEnvironmentHighPrecMixin(Environment):
if self.__submitter.find('<') != -1:
# Submitter has a configured email, we transformed
# __submitter into an RFC 2822 string already.
return re.match('(.*?)\s*<', self.__submitter).group(1)
return re.match(r'(.*?)\s*<', self.__submitter).group(1)
else:
# Submitter has no configured email, it's just his name.
return self.__submitter
@ -3615,6 +3712,9 @@ class Push(object):
for (num, sha1) in enumerate(sha1s):
rev = Revision(change, GitObject(sha1), num=num + 1, tot=len(sha1s))
if len(rev.parents) > 1 and change.environment.excludemergerevisions:
# skipping a merge commit
continue
if not rev.recipients and rev.cc_recipients:
change.environment.log_msg('*** Replacing Cc: with To:')
rev.recipients = rev.cc_recipients
@ -3664,11 +3764,14 @@ def run_as_post_receive_hook(environment, mailer):
changes.append(
ReferenceChange.create(environment, oldrev, newrev, refname)
)
if changes:
push = Push(environment, changes)
if not changes:
mailer.close()
return
push = Push(environment, changes)
try:
push.send_emails(mailer, body_filter=environment.filter_body)
if hasattr(mailer, '__del__'):
mailer.__del__()
finally:
mailer.close()
def run_as_update_hook(environment, mailer, refname, oldrev, newrev, force_send=False):
@ -3687,10 +3790,14 @@ def run_as_update_hook(environment, mailer, refname, oldrev, newrev, force_send=
refname,
),
]
if not changes:
mailer.close()
return
push = Push(environment, changes, force_send)
push.send_emails(mailer, body_filter=environment.filter_body)
if hasattr(mailer, '__del__'):
mailer.__del__()
try:
push.send_emails(mailer, body_filter=environment.filter_body)
finally:
mailer.close()
def check_ref_filter(environment):
@ -3860,7 +3967,7 @@ def build_environment_klass(env_name):
low_prec_mixin = known_env['lowprec']
environment_mixins.append(low_prec_mixin)
environment_mixins.append(Environment)
klass_name = env_name.capitalize() + 'Environement'
klass_name = env_name.capitalize() + 'Environment'
environment_klass = type(
klass_name,
tuple(environment_mixins),
@ -4057,21 +4164,21 @@ class Logger(object):
environment, 'git_multimail.error', environment.error_log_file, logging.ERROR)
self.loggers.append(error_log_file)
def info(self, msg):
def info(self, msg, *args, **kwargs):
for l in self.loggers:
l.info(msg)
l.info(msg, *args, **kwargs)
def debug(self, msg):
def debug(self, msg, *args, **kwargs):
for l in self.loggers:
l.debug(msg)
l.debug(msg, *args, **kwargs)
def warning(self, msg):
def warning(self, msg, *args, **kwargs):
for l in self.loggers:
l.warning(msg)
l.warning(msg, *args, **kwargs)
def error(self, msg):
def error(self, msg, *args, **kwargs):
for l in self.loggers:
l.error(msg)
l.error(msg, *args, **kwargs)
def main(args):
@ -4189,7 +4296,7 @@ def main(args):
show_env(environment, sys.stderr)
if options.stdout or environment.stdout:
mailer = OutputMailer(sys.stdout)
mailer = OutputMailer(sys.stdout, environment)
else:
mailer = choose_mailer(config, environment)
@ -4234,5 +4341,6 @@ def main(args):
sys.stderr.write(msg)
sys.exit(1)
if __name__ == '__main__':
main(sys.argv[1:])

View File

@ -110,11 +110,12 @@ def is_section_empty(section, local):
try:
read_output(
['git', 'config']
+ local_option
+ ['--get-regexp', '^%s\.' % (section,)]
['git', 'config'] +
local_option +
['--get-regexp', '^%s\.' % (section,)]
)
except CommandError, e:
except CommandError:
t, e, traceback = sys.exc_info()
if e.retcode == 1:
# This means that no settings were found.
return True
@ -188,7 +189,9 @@ def migrate_config(strict=False, retain=False, overwrite=False):
sys.stderr.write(
'...copying "%s.%s" to "%s.%s"\n' % (old.section, name, new.section, name)
)
new.set_recipients(name, old.get_recipients(name))
old_recipients = old.get_all(name, default=None)
old_recipients = ', '.join(o.strip() for o in old_recipients)
new.set_recipients(name, old_recipients)
if strict:
sys.stderr.write(

View File

@ -30,7 +30,6 @@ script's behavior could be changed or customized.
"""
import sys
import os
# If necessary, add the path to the directory containing
# git_multimail.py to the Python path as follows. (This is not
@ -86,6 +85,7 @@ mailer = git_multimail.choose_mailer(config, environment)
# Use Python's smtplib to send emails. Both arguments are required.
#mailer = git_multimail.SMTPMailer(
# environment=environment,
# envelopesender='git-repo@example.com',
# # The smtpserver argument can also include a port number; e.g.,
# # smtpserver='mail.example.com:25'