This is an old compatibility code. It is time to remove it as it is not used sinces ages.
The purpose of this commit is to clean a bit the code related to email routing in the mailgateway: message_route and message_route_verify. message_route is now clearly separeted into two categories. First one handles emails considered as answers to existing threads. Second one handles emails that have to create new threads, either through alias or default behavior of the mailgateway. message_route_verify has been a bit facotrized to simplify the various overrides that appeared lately in the codebase. This commit only contains code cleaning. No functional change should occur due to this commit.
mail.mail and mail.thread have methods to decode headers. This commit move them in mail tools. It is indeed simplier to have all email tools in the same file. Moreover they have been renamed to avoid confusion with standard library methods: decode and decode_header already exist. * decode in mail_message is now decode_smtp_header * decode_header in mail_thread is now decode_message_header A method to data from references is also moved into tools. A regex was already defined in tools. Now the method using it is already put in tools. Mailgateway will now simply call this method.
Mako uses pipe character in several occasions like |safe to tell an email is safe. However the sanitizer will encode it, leading to template not correctly rendered. Like other special characters used in mako, it is now decoded after sanitize.
Now having * sanitize: run the sanitizer to clean the html (removing javascripts, unwanted tags, ...) * sanitize_tags: only a subset of tags is allowed in html content. Unwelcomed tags are remove dand their content stripped. * sanitize_attributes: only a subset of attributes is allowed. * sanitize_style: only a subset of style attributes is allowed. Style attributes are parsed to keep only a white list. * strip_style: all style is removed. It bypasses sanitize_style as there is no need to sanitize something that is removed. * strip_classes: remove class attributes Fields parameters have also been updated to match the sanitize options. Html fields by default are sanitized with sanitize_tags activated but without any further options. All addons have been updated to match the new options according to their previous behavior.
This commit protects tags having an alt attribute containing an @. Like cite on blockquote, alt on image for example could contain an @. Do not escape it as it could break images.
Remove html_email_clean in tools. Remove shorten in mail.
Quote detection is now done in the sanitizer itself. It tags nodes that are inside quotes (signature, text quotes, email quotes). The purpose is to remove html_email_clean and have all the html cleaning / sanitizing inside a single function. When a node is tagged, data-o-mail-quote is set on the node. This attribute is added in the whitelist of valid attributes for the sanitizer. [REM] Support of shortening messages. The read more / read less will only display or hide detected quotes and signatures. Shortening messages above a given amount of characters is not supported anymore. Indeed it adds much complexity to the sanitizer without adding much value to the result. The primary purpose of the sanitizer is indeed to remove noise and unnecessary content. [TESTS] a lot of test are not necessary anymore, as read more / read less display will be moved in the front-end and as the shortening has been removed.Tests have therefore been cleaned and simplified. [DEMO] mail: small demo update to include a bit quote detection
Style attribute is now parsed and sanitized. Only a while list of accepted properties is kept. The behavior is implemented directly in the cleaner itself. As there is no easy possible inheritance, the styling cleaning is appended directly after the legacy processing in __call__. The styling sanitizer is called only if the style attribute is kept. If the sanitizer is called with strip_style=True, the styling is removed and therefore no sanitizing is performed. Tests about html fields have been updated. Indeed the styling is now sanitized and the test was not correct anymore, as the test html was stripped. It now contains styling that is kept. The strip_classes is also tested.