diff options
| author | Nicolas Martinelli <nim@odoo.com> | 2019-08-07 07:28:57 +0000 |
|---|---|---|
| committer | Nicolas Martinelli <nim@odoo.com> | 2019-08-08 12:06:31 +0000 |
| commit | 0f878e4839c36cd73fe8096ec1364f71af6bd8be (patch) | |
| tree | b3ec2943e5bfbf96cefd86c7421f561bfaeac6ae /odoo/tools/float_utils.py | |
| parent | 1e4de19cd49c005ebe3b14558e8eca6a1f55e72f (diff) | |
[FIX] tools: epsilon magnitude
The following rounding is incorrect:
```
>>> float_round(6.6 * 0.175, precision_digits=2)
1.15
```
Indeed, 6.6 * 0.175 = 1.155 ≈ 1.16.
In this specific case, the `epsilon` computed is not sufficient. A
precision of 53 gives:
normalized_value = 115.49999999999997
epsilon = 1.2823075934420547e-14
=> new normalized_value = 115.49999999999999
Bad luck, this is just not enough to tip the value in the right
direction.
However, a precision of 52 is sufficient:
normalized_value = 115.49999999999997
epsilon = 2.5646151868841094e-14
=> new normalized_value = 115.5
The value of 53 was chosen from the `binary64` number format precision.
In case of Python, the corresponding machine epsilon is 2^-52 [1].
Therefore, using 52 instead of 53 does make sense.
It is worth noting that the value of the machine epsilon
2^-52 = 2.2204460492503131e-16, which is still 2 orders of magnitude
below our dynamic estimation.
[1] https://en.wikipedia.org/wiki/Machine_epsilon
[2] `numpy.finfo(float).eps = 2.2204460492503131e-16`
opw-2047368
closes odoo/odoo#35565
Signed-off-by: Nicolas Martinelli (nim) <nim@odoo.com>
Diffstat (limited to 'odoo/tools/float_utils.py')
| -rw-r--r-- | odoo/tools/float_utils.py | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/odoo/tools/float_utils.py b/odoo/tools/float_utils.py index 3e13f9ee0bc..3b6e78a8f3b 100644 --- a/odoo/tools/float_utils.py +++ b/odoo/tools/float_utils.py @@ -73,7 +73,7 @@ def float_round(value, precision_digits=None, precision_rounding=None, rounding_ normalized_value = value / rounding_factor # normalize sign = math.copysign(1.0, normalized_value) epsilon_magnitude = math.log(abs(normalized_value), 2) - epsilon = 2**(epsilon_magnitude-53) + epsilon = 2**(epsilon_magnitude-52) # TIE-BREAKING: UP/DOWN (for ceiling[resp. flooring] operations) # When rounding the value up[resp. down], we instead subtract[resp. add] the epsilon value |