[IMP] base: prevent SHA-1 collision

SHA-1 is a cryptographic hash function that have weaknesses known since
2005, it has been deprecated by the NIST [1] about 10 years ago in 2011
and Google [2] have been able to perform a collision attack in 2017.

We use SHA-1 as filename for the attachments stored in our filestore.

Although practical attacks still requires quite a lot of computational
resources, it is time to prevent SHA-1 collisions on our filestore.

To prevent such attacks, when uploading a file that has the same SHA-1
as a stored file, we perform a SHA-2 computation on both files to verify
their signature. If their SHA-2 signature is different, an error is
raised.

We have selected the SHA-512 variant of the SHA-2 algorithm because on
64 bits platform, SHA-512 is the fastest SHA-2 variant, it is only ~1.5x
slower than SHA-1. [3]

We have not used SHA-3 because at the moment of writing, it is too slow
(~3x slower than SHA-1) [3] and it is not guaranteed to be available
with the Python 3.5 `hashlib` module.

[1]: https://csrc.nist.gov/projects/hash-functions/nist-policy-on-hash-functions
[2]: https://shattered.io/
[3]: http://bench.cr.yp.to/results-hash.html
[4]: http://www.commitstrip.com/en/2017/02/27/the-sha-1-alternative/

closes odoo/odoo#36430

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>

1 files changed, 18 insertions, 1 deletions

diff --git a/odoo/addons/base/models/ir_attachment.py b/odoo/addons/base/models/ir_attachment.py
index 27b52033ab5..a58ba54c448 100644
--- a/odoo/addons/base/models/ir_attachment.py
+++ b/odoo/addons/base/models/ir_attachment.py
@@ -11,7 +11,7 @@ from collections import defaultdict
 import uuid
 
 from odoo import api, fields, models, tools, _
-from odoo.exceptions import AccessError, ValidationError, MissingError
+from odoo.exceptions import AccessError, ValidationError, MissingError, UserError
 from odoo.tools import config, human_size, ustr, html_escape
 from odoo.tools.mimetypes import guess_mimetype
 
@@ -90,6 +90,9 @@ class IrAttachment(models.Model):
         dirname = os.path.dirname(full_path)
         if not os.path.isdir(dirname):
             os.makedirs(dirname)
+        # prevent sha-1 collision
+        if os.path.isfile(full_path) and not self._same_content(bin_data, full_path):
+            raise UserError("The attachment is colliding with an existing file.")
         return fname, full_path
 
     @api.model
@@ -226,6 +229,20 @@ class IrAttachment(models.Model):
         # an empty file has a checksum too (for caching)
         return hashlib.sha1(bin_data or b'').hexdigest()
 
+    @api.model
+    def _same_content(self, bin_data, filepath):
+        BLOCK_SIZE = 1024
+        with open(filepath, 'rb') as fd:
+            i = 0
+            while True:
+                data = fd.read(BLOCK_SIZE)
+                if data != bin_data[i * BLOCK_SIZE:(i + 1) * BLOCK_SIZE]:
+                    return False
+                if not data:
+                    break
+                i += 1
+        return True
+
     def _compute_mimetype(self, values):
         """ compute the mimetype of the given values
             :param values : dict of values to create or write an ir_attachment