Purpose ======= We want to be able to force the FROM headers when we sent email in SMTP. So we can avoid the emails to be considered as spam. Specifications ============== This is done with 2 system parameters. If the system parameter `mail.force.smtp.from` is set we encapsulate all outgoing email from with the given value. If the previous system parameter is not set and if both `mail.dynamic.smtp.from` and `mail.catchall.domain` are set, we encapsulate the FROM only if the domain of the email is not the same as the domain of the catchall parameter. Otherwise we do not encapsulate the email (same behavior as before this commit). Task 2367946 See odoo/odoo/pull/61853 closes odoo/odoo#70854 X-original-commit: 044532aed6f796cd93e37556b4a8ce47d2c23a7c Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com> Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
Python 3 before 3.8 has a bug that causes the email.policy classes to incorrectly fold and RFC2047-encode "identification fields" in email messages. This mainly applies to Message-Id, References, and In-Reply-To fields. We are impacted by this bug since odoo/odoo#35929 where we switched to using the "modern" email.message API. RFC2047 section 5 clearly states that those headers/fields are not to be encoded, and that would violate RFC5322. Further, such a folded Message-Id is considered non-RFC-conformant by popular MTAs (GMail, Outlook), which will then generate *another* Message-Id field, causing the original threading information to be lost. Replies to such a modified message will reference the new, unknown Message-Id, and won't be attached to the original thread. The solution we adopt here is to monkey-patch the SMTP policies to special-case those identification fields and deactivate the automatic folding, until the bug is properly and fully fixed in the standard lib. Some considerations taken into account for this patch: - `email.policy.SMTP` is being monkey-patched globally to make sure we fix all possible places where Messages are being encoded/folded - the fix is **not** made version-specific, considering that even in Python 3.8 the official bugfix only applies to Message-Id, but still fails to protect other identification fields, like *References* and *In-Reply-To*. The author specifically noted that shortcoming [2]. The fix wouldn't break anything on Python 3.8 anyway. - the `noFoldPolicy` trick for preventing folding is done with no max line length at all. RFC5322, section 2.1.1 states [3] that the maximum length is 998 due to legacy implementations, but there is no provision to wrap identification fields that are longer than that. Wrapping at 998 chars would corrupt the header anyway. We'll just count on the fact that we don't usually need 1k+ chars in those headers. The invalid folding/encoding in action on Python 3.6 (in Python 3.8 only the second header gets folded): ```py >>> msg = email.message.EmailMessage(policy=email.policy.SMTP) >>> msg['Message-Id'] = '<929227342217024.1596730490.324691772460938-example-30661-some.reference@test-123.example.com>' >>> msg['In-Reply-To'] = '<92922734221723.1596730568.324691772460444-another-30661-parent.reference@test-123.example.com>' >>> print(msg.as_string()) Message-Id: =?utf-8?q?=3C929227342217024=2E1596730490=2E324691772460938-exam?= =?utf-8?q?ple-30661-some=2Ereference=40test-123=2Eexample=2Ecom=3E?= In-Reply-To: =?utf-8?q?=3C92922734221723=2E1596730568=2E324691772460444-anot?= =?utf-8?q?her-30661-parent=2Ereference=40test-123=2Eexample=2Ecom=3E?= ``` and the expected result after the fix: ```py >>> msg = email.message.EmailMessage(policy=email.policy.SMTP) >>> msg['Message-Id'] = '<929227342217024.1596730490.324691772460938-example-30661-some.reference@test-123.example.com>' >>> msg['In-Reply-To'] = '<92922734221723.1596730568.324691772460444-another-30661-parent.reference@test-123.example.com>' >>> print(msg.as_string()) Message-Id: <929227342217024.1596730490.324691772460938-example-30661-some.reference@test-123.example.com> In-Reply-To: <92922734221723.1596730568.324691772460444-another-30661-parent.reference@test-123.example.com> ``` [1] bpo-35805: https://bugs.python.org/issue35805 [2] https://github.com/python/cpython/pull/13397#issuecomment-493618544 [3] https://tools.ietf.org/html/rfc5322#section-2.1.1 closes odoo/odoo#55656 X-original-commit: 02b78770147e2eda65a76d29c6f2fc3278581b22 Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
PURPOSE When displaying an email in a list the mail client (gmail, outlook...) computes a preview based on the content. This preview is generally not well computed as it contains a lot of garbage and tags while it should contain only relevant text. SPECIFICATIONS To build this preview, all mail clients read the content of the email. The only way to be able to customize the preview is to add an invisible HTML element at the beginning of the email with the wanted preview text. We add at the end of the preview `‌` (zero-width non-joiner) to fill the end of the preview in order to not have the beginning of the mail at the end of the preview. It doesn't work with simple space as the mail clients trim each HTML element content. Task ID-2172125 PR #49886
Since a recent commit[1], we have a utility method in tools named 'is_html_empty' that checks whether the given html content is void(containing only formatting tags) or not. However, the re from this method does not consider the case of self closing tags, fox ex `<br/>`. In such cases, the method returns Falsy value even if the content is void. This commit fixes the issue by considering self-closing void tags in the regular expression. commit[1] - https://github.com/odoo/odoo/commit/974f512f5f5d3b9f80a8c3fcde290e4f55cf1230 Task - 2267689 closes odoo/odoo#53167 X-original-commit: 0fd6c9388dc959a9b6b7828850683175edb05b9d Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>
It has been a recurrent request from customers to be able to send email messages to email addresses containing non-ascii characters. [IDNA] is a domain extension to allow unicode characters in domain names. [SMTPUTF8] is a SMTP extension to allow unicode in any header. IDNA defines the [punycode] encoding which translates unicode to an ascii representation. This encoding MUST be used to encode domains. SMTPUTF8 is an SMTP extension that allow utf-8 in all headers on the envelope. [IDNA] https://tools.ietf.org/html/rfc5890 [SMTPUTF8] https://tools.ietf.org/html/rfc6531 [punycode] https://tools.ietf.org/html/rfc3492 Task: 2116928 opw-2229906 opw-2248251 closes odoo/odoo#47709 Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
A plaintext email is displayed in a `<pre/>` tag to conserve spacing. But since there is no escaping, if in this text there was XML tags or HTML entities, they would appear as HTML in browser which is not wanted. Do note that this was not a security issue since the content will still be subjected to the checks and foundling of HTML emails. Without the change, the added test would fail because character &,<,> were not escaped. opw-2242323 closes #50003 closes odoo/odoo#50123 X-original-commit: 932532b5b59e0b71c8e16dadfb2ff36c38764208 Signed-off-by: Nicolas Lempereur (nle) <nle@odoo.com> Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
In odoo empty html is not always completely empty. This tool method allow to consider html holding void tags as empty html. This happens notably when using the editor that may store <p><br><p> even if displayed content is void. Task ID 2126509 PR #41827
Problem description: Using the chatter, send an email to someone having non-ascii characters in their name: the body of the received email looks like it contains a mix of the original body and headers. Python encodes the name into either base64 or quoted-printable but leaves some redundant carriage returns at the end of the header. Those redundant carriage returns are not RFC conformant but are interpreted as newlines by some email clients, thus are read like a headers/body separator and the next headers are considered part of the body, corrupting the email structure. This problem is internal to Python and has been fixed in 3.8, cfr bpo-34424 [1], and later backported to 3.7.4. As we also support 3.6 and earlier 3.7 and it is not possible to easily monkey-patch the function, we fixed it by squashing duplicate carriage returns. (There is no case where a series of bare CRs can occur in a normal RFC5322 email message) Task: 2003936 [1]: https://bugs.python.org/issue34424
Python 2 `email.tools.formataddr` doesn't encode not RFC-2822 compliant realname to base64 or quoted-printable. This is the wanted behavior to output formatted email addresses on screen. Python 3 `email.tools.formataddr` does encode the realname to be RFC-2822 compliant. This is the wanted behavior when connecting to a smtp server to send emails. That method has been deprecated by pep-594 as the new python email API is capable of automatically formatting email address in a RFC compliant way. As the method was still in use in a lot of modules as the preferred way to format email addresses, a refined P3-like implementation as been included in the tool suite. This commit changes the default behavior so it mimics P2 implementation with an easy way to use the P3 behavior. Task: 2003936
It is possible to get to a situation where Odoo would try to send an email without a `From:` header address. In such case, you're unlucky if you don't have access to the underlying deployment, or if you use multiple databases in a single Odoo instance and each of them uses a different mail configuration. To make this configuration easier to use and cover those use cases, here I add support for a new ICP: `mail.default.from`. It will be used when present, so it shouldn't affect existing deployments. When present, it will allow a admin to configure the default sending address just with Odoo itself. closes odoo/odoo#38874 X-original-commit: 5010ce630a7f0e01d45d99c9083318b50f9f624a Signed-off-by: Thibault Delavallee (tde) <tde@openerp.com>