summaryrefslogtreecommitdiff
path: root/addons/web
diff options
context:
space:
mode:
authorNicolas Martinelli <nim@odoo.com>2019-08-07 07:28:57 +0000
committerNicolas Martinelli <nim@odoo.com>2019-08-07 08:25:53 +0000
commitfde3b696562a12dc3ec99fcb85004a6a40e040f0 (patch)
tree8ea7907475b0a63f98f3ee291310d9fd4e728bf6 /addons/web
parent71af1a710ab6892d09156e4a6c61c63820532dee (diff)
[FIX] tools: epsilon magnitude
The following rounding is incorrect: ``` >>> float_round(6.6 * 0.175, precision_digits=2) 1.15 ``` Indeed, 6.6 * 0.175 = 1.155 ≈ 1.16. In this specific case, the `epsilon` computed is not sufficient. A precision of 53 gives: normalized_value = 115.49999999999997 epsilon = 1.2823075934420547e-14 => new normalized_value = 115.49999999999999 Bad luck, this is just not enough to tip the value in the right direction. However, a precision of 52 is sufficient: normalized_value = 115.49999999999997 epsilon = 2.5646151868841094e-14 => new normalized_value = 115.5 The value of 53 was chosen from the `binary64` number format precision. In case of Python, the corresponding machine epsilon is 2^-52 [1]. Therefore, using 52 instead of 53 does make sense. It is worth noting that the value of the machine epsilon 2^-52 = 2.2204460492503131e-16, which is still 2 orders of magnitude below our dynamic estimation. [1] https://en.wikipedia.org/wiki/Machine_epsilon [2] `numpy.finfo(float).eps = 2.2204460492503131e-16` opw-2047368 closes odoo/odoo#35521 Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
Diffstat (limited to 'addons/web')
-rw-r--r--addons/web/static/src/js/core/utils.js2
1 files changed, 1 insertions, 1 deletions
diff --git a/addons/web/static/src/js/core/utils.js b/addons/web/static/src/js/core/utils.js
index a21c4241864..231ad97d202 100644
--- a/addons/web/static/src/js/core/utils.js
+++ b/addons/web/static/src/js/core/utils.js
@@ -309,7 +309,7 @@ var utils = {
}
var normalized_value = value / precision;
var epsilon_magnitude = Math.log(Math.abs(normalized_value))/Math.log(2);
- var epsilon = Math.pow(2, epsilon_magnitude - 53);
+ var epsilon = Math.pow(2, epsilon_magnitude - 52);
normalized_value += normalized_value >= 0 ? epsilon : -epsilon;
/**