<feed xmlns='http://www.w3.org/2005/Atom'>
<title>Odoo/odoo/tools/cache.py, branch 14.0</title>
<subtitle>The current checkout of upstream Odoo</subtitle>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/'/>
<entry>
<title>[IMP] core: replace dedicated uid cache by ormcache</title>
<updated>2020-08-14T23:03:27+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2019-05-21T11:58:28+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=083c70bbb63f27839b2c9a4b549e216947bc4dd1'/>
<id>083c70bbb63f27839b2c9a4b549e216947bc4dd1</id>
<content type='text'>
Before this, invalidations to the UID cache is not synchronised
between workers because it's an ad-hoc solution (so a user changing
their password or an admin disabling a user would only lock out an
attacker currently using the API of one of possibly several
workers). Shift the entire thing to ormcache which already has proper
support for synchronising cache invalidation between workers.

Also simplify the cache invalidation mess in Users.write because the
caches have been unified into a single registry-level LRU, so the
half-dozen cache clears on specific ormcached methods &amp; models is
pretty much the same as repeatedly calling clear_caches on the current
model.

**However** registry.cache is trivially accessible from server actions
and safe_eval as long as they provide access to a model (through
`model.pool.cache`). Which is common, and an issue given we're very
much putting sensible data in there.

Fix this by renaming `Registry.cache` to `Registry.__cache`, this
requires few editions and mangled names are not accessible from
safe_eval contexts.

The alternative would have been to add more bespoke handling of the
uid cache to hook it into the cache invalidation propagation
machinery.

After discussion with (@)odony, fixing LRU access and using that seems
cleaner and less error-prone.

Note on lazy_property
=====================

Make Registry.cache / Registry.__cache into a regular attribute: the
overhead of the LRU is not that high (compared to that of the registry
itself), it's rare that we *don't* need it, and it's assumed to be a
persisted attribute (it's not just a cache) so making it a normal
attribute seems fine; and lazy_property doesn't work for mangled
names: the name of the property is mangled using the name of the
definition class, but the name of the symbol (fget) is not mangled so
lazy_property would set the __cache attribute but then Python would
lookup _Registry__cache, creating a new cache every access.

And we can't (always) mangle things correctly on `__get__(obj,
owner)`: `owner` is just `type(obj)`, meaning in the case of
inheritance the type we get is the type through which the property is
accessed rather than the one it's defined on. So it would work in the
cases where no inheritance is involved (such as Registry.__cache) but
not in general (lest we want to play around walking the MRO ourselves
to find the definition source, which doesn't seem worth it).

lazy_property *could* be made to work properly on Python 3.6+: the
descriptor protocol gains `__set_name__(name, owner)`, which is called
with the properly mangled name — and with the definition class to boot
(though there might still be issues when overriding lazy properties as
the override will be mangled &amp; named differently... or maybe that's a
feature?). However we're still supporting 3.5 at this point, AFAIK, so
that's not an option. Plus it feels unnecessary / not very useful.

However add an assertion to `lazy_property` so it signals when we try
to use it on a mangled method (as otherwise it kinda sorta work in the
sense that the property / object is accessible but is in effect a
slower way to write a regular property).</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Before this, invalidations to the UID cache is not synchronised
between workers because it's an ad-hoc solution (so a user changing
their password or an admin disabling a user would only lock out an
attacker currently using the API of one of possibly several
workers). Shift the entire thing to ormcache which already has proper
support for synchronising cache invalidation between workers.

Also simplify the cache invalidation mess in Users.write because the
caches have been unified into a single registry-level LRU, so the
half-dozen cache clears on specific ormcached methods &amp; models is
pretty much the same as repeatedly calling clear_caches on the current
model.

**However** registry.cache is trivially accessible from server actions
and safe_eval as long as they provide access to a model (through
`model.pool.cache`). Which is common, and an issue given we're very
much putting sensible data in there.

Fix this by renaming `Registry.cache` to `Registry.__cache`, this
requires few editions and mangled names are not accessible from
safe_eval contexts.

The alternative would have been to add more bespoke handling of the
uid cache to hook it into the cache invalidation propagation
machinery.

After discussion with (@)odony, fixing LRU access and using that seems
cleaner and less error-prone.

Note on lazy_property
=====================

Make Registry.cache / Registry.__cache into a regular attribute: the
overhead of the LRU is not that high (compared to that of the registry
itself), it's rare that we *don't* need it, and it's assumed to be a
persisted attribute (it's not just a cache) so making it a normal
attribute seems fine; and lazy_property doesn't work for mangled
names: the name of the property is mangled using the name of the
definition class, but the name of the symbol (fget) is not mangled so
lazy_property would set the __cache attribute but then Python would
lookup _Registry__cache, creating a new cache every access.

And we can't (always) mangle things correctly on `__get__(obj,
owner)`: `owner` is just `type(obj)`, meaning in the case of
inheritance the type we get is the type through which the property is
accessed rather than the one it's defined on. So it would work in the
cases where no inheritance is involved (such as Registry.__cache) but
not in general (lest we want to play around walking the MRO ourselves
to find the definition source, which doesn't seem worth it).

lazy_property *could* be made to work properly on Python 3.6+: the
descriptor protocol gains `__set_name__(name, owner)`, which is called
with the properly mangled name — and with the definition class to boot
(though there might still be issues when overriding lazy properties as
the override will be mangled &amp; named differently... or maybe that's a
feature?). However we're still supporting 3.5 at this point, AFAIK, so
that's not an option. Plus it feels unnecessary / not very useful.

However add an assertion to `lazy_property` so it signals when we try
to use it on a mangled method (as otherwise it kinda sorta work in the
sense that the property / object is accessible but is in effect a
slower way to write a regular property).</pre>
</div>
</content>
</entry>
<entry>
<title>[FIX] tools: LRU doesn't have keys() method anymore</title>
<updated>2020-08-06T10:54:07+00:00</updated>
<author>
<name>Jeremy Kersten</name>
<email>jke@odoo.com</email>
</author>
<published>2020-07-17T12:38:57+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=7e8a1f7a0693c6b6af7b075b5bbae1f02a48c259'/>
<id>7e8a1f7a0693c6b6af7b075b5bbae1f02a48c259</id>
<content type='text'>
Since commit 1b30a78b, LRU is now based on OrderedDict and keys can be found
on dictionnary 'd' directly.

closes odoo/odoo#55541

X-original-commit: 3f093da75765d66ed1e2d828e8b41ed05d936d0b
Signed-off-by: Xavier Morel (xmo) &lt;xmo@odoo.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Since commit 1b30a78b, LRU is now based on OrderedDict and keys can be found
on dictionnary 'd' directly.

closes odoo/odoo#55541

X-original-commit: 3f093da75765d66ed1e2d828e8b41ed05d936d0b
Signed-off-by: Xavier Morel (xmo) &lt;xmo@odoo.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>[FIX] core: iterations on LRU</title>
<updated>2020-04-06T09:45:29+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2020-04-06T09:39:47+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=9d37cecfca8e6b8d4c0e2880f422673c526a3aef'/>
<id>9d37cecfca8e6b8d4c0e2880f422673c526a3aef</id>
<content type='text'>
Iteration methods on LRU were removed because they were not
thread-safe and it's not clear that making them thread-safe is the
correct thing to do, so not providing them seems saner.

I thought I'd looked for usages of the LRU but apparently didn't look
hard enough as I missed that it's used by the cron workers (apparently
using the threaded server we only run crons for dbs currently living
in the registry cache, the more you know).

Convert these to iterating on the LRU's internal mapping, and also
don't iterate on the LRU to clear its entries one by one when we can
just clear the entire thing safely, although Registry.delete_all
really seems completely unused.

closes odoo/odoo#49023

Signed-off-by: Xavier Morel (xmo) &lt;xmo@odoo.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Iteration methods on LRU were removed because they were not
thread-safe and it's not clear that making them thread-safe is the
correct thing to do, so not providing them seems saner.

I thought I'd looked for usages of the LRU but apparently didn't look
hard enough as I missed that it's used by the cron workers (apparently
using the threaded server we only run crons for dbs currently living
in the registry cache, the more you know).

Convert these to iterating on the LRU's internal mapping, and also
don't iterate on the LRU to clear its entries one by one when we can
just clear the entire thing safely, although Registry.delete_all
really seems completely unused.

closes odoo/odoo#49023

Signed-off-by: Xavier Morel (xmo) &lt;xmo@odoo.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>[FIX] *: remove old deprecated modules/functions</title>
<updated>2019-09-17T11:36:42+00:00</updated>
<author>
<name>Julien Castiaux</name>
<email>juc@odoo.com</email>
</author>
<published>2019-09-11T09:45:31+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=4f03a5f136ab928ae74322bf1c34a1b6a6baf979'/>
<id>4f03a5f136ab928ae74322bf1c34a1b6a6baf979</id>
<content type='text'>
PEP-594 is deprecating a bunch of modules. As part of the cleanup, we
are also dealing with long deprecated modules, functions and aliases.

* `assert_` -&gt; `assertTrue`
* `assertEquals` -&gt; `assertEqual`
* `assertNotEquals` -&gt; `assertNotEqual`
* `assertAlmostEquals` -&gt; `assertAlmostEqual`
* `assertRaisesRegexp` -&gt; `assertRaisesRegex`
* `assertRegexpMatches` -&gt; `assertRegex`
* `base64.encodestring` -&gt; `base64.encodebytes`
* `base64.decodestring` -&gt; `base64.decodebytes`
* `inspect.getargspec` -&gt; `inspect.signature`
* `inspect.formatargspec` -&gt; `inspect.signature`
* `logging.warn` -&gt; `logging.warning`

closes odoo/odoo#36863

Task: 2003936
Signed-off-by: Raphael Collet (rco) &lt;rco@openerp.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
PEP-594 is deprecating a bunch of modules. As part of the cleanup, we
are also dealing with long deprecated modules, functions and aliases.

* `assert_` -&gt; `assertTrue`
* `assertEquals` -&gt; `assertEqual`
* `assertNotEquals` -&gt; `assertNotEqual`
* `assertAlmostEquals` -&gt; `assertAlmostEqual`
* `assertRaisesRegexp` -&gt; `assertRaisesRegex`
* `assertRegexpMatches` -&gt; `assertRegex`
* `base64.encodestring` -&gt; `base64.encodebytes`
* `base64.decodestring` -&gt; `base64.decodebytes`
* `inspect.getargspec` -&gt; `inspect.signature`
* `inspect.formatargspec` -&gt; `inspect.signature`
* `logging.warn` -&gt; `logging.warning`

closes odoo/odoo#36863

Task: 2003936
Signed-off-by: Raphael Collet (rco) &lt;rco@openerp.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>[IMP] doc: remove calls to @api.guess and @api.noguess</title>
<updated>2019-08-01T13:16:37+00:00</updated>
<author>
<name>mreficent</name>
<email>miquel.raich@eficent.com</email>
</author>
<published>2019-07-31T11:49:37+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=64773140b03f7a3d1c8e3d55ef39d79c29cc2493'/>
<id>64773140b03f7a3d1c8e3d55ef39d79c29cc2493</id>
<content type='text'>
Those two decorators are removed/deprecated since recent commits but
some references remained in the documentation.

api.guess and api.noguess is deprecated and removed since c552fb7a618

closes odoo/odoo#35332

Signed-off-by: Martin Trigaux (mat) &lt;mat@odoo.com&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Those two decorators are removed/deprecated since recent commits but
some references remained in the documentation.

api.guess and api.noguess is deprecated and removed since c552fb7a618

closes odoo/odoo#35332

Signed-off-by: Martin Trigaux (mat) &lt;mat@odoo.com&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>[FIX] base: ir_rule don't not cache _compute_domain</title>
<updated>2019-06-24T12:29:41+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2019-06-24T12:08:17+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=5fd75da0e944952b2f8c9c4de19d84b3b78f6187'/>
<id>5fd75da0e944952b2f8c9c4de19d84b3b78f6187</id>
<content type='text'>
Following the revamp / contextualisation of multi company, the list of
allowed companies from the context is now used as part of the ir_rule
caching key.

Sadly that list is a list, and thus not hashable, meaning
_compute_domain was not cached anymore. Sadlier, type errors would be
silently ignored (just increasing the error counter of the cache stats).

Fix both issues:

* Store lists from the context as tuple. Technically could be a
frozenset but that might not always be the case so it doesn't feel
future-proof, a tuple might lead to slightly less hits but seems safer
* Log a proper warning in case of an ormcache TypeError (non-hashable
type used as cache key). It should never routinely happen.

closes odoo/odoo#34330

Signed-off-by: Xavier Morel (xmo) &lt;xmo@odoo.com&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Following the revamp / contextualisation of multi company, the list of
allowed companies from the context is now used as part of the ir_rule
caching key.

Sadly that list is a list, and thus not hashable, meaning
_compute_domain was not cached anymore. Sadlier, type errors would be
silently ignored (just increasing the error counter of the cache stats).

Fix both issues:

* Store lists from the context as tuple. Technically could be a
frozenset but that might not always be the case so it doesn't feel
future-proof, a tuple might lead to slightly less hits but seems safer
* Log a proper warning in case of an ormcache TypeError (non-hashable
type used as cache key). It should never routinely happen.

closes odoo/odoo#34330

Signed-off-by: Xavier Morel (xmo) &lt;xmo@odoo.com&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>[REF] *: replace deprecated pycompat helpers for builtins</title>
<updated>2018-11-29T09:28:17+00:00</updated>
<author>
<name>Adrian Torres</name>
<email>adt@odoo.com</email>
</author>
<published>2018-11-08T15:27:30+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=52f5528cfb6d1152ee226e268247dc685a8ba329'/>
<id>52f5528cfb6d1152ee226e268247dc685a8ba329</id>
<content type='text'>
This commit replaces calls to pycompat helpers that were intended for
python 2 &lt;-&gt; python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.

This includes:
    * calls to imap/izip/ifilter replaced by map/zip/filter
    * uses of text_type replaced by str
    * uses of unichr replaced by chr
    * calls to implements_to_string, implements_iterator removed
    * string_types and integer_types replaced by str, int respectively
    * calls to to_native replaced by calls to to_text

This is done in preparation to the removal of these deprecated helpers
in the following commit.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This commit replaces calls to pycompat helpers that were intended for
python 2 &lt;-&gt; python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.

This includes:
    * calls to imap/izip/ifilter replaced by map/zip/filter
    * uses of text_type replaced by str
    * uses of unichr replaced by chr
    * calls to implements_to_string, implements_iterator removed
    * string_types and integer_types replaced by str, int respectively
    * calls to to_native replaced by calls to to_text

This is done in preparation to the removal of these deprecated helpers
in the following commit.</pre>
</div>
</content>
</entry>
<entry>
<title>[MERGE] forward port branch 11.0 up to 263b388c504</title>
<updated>2018-11-22T16:52:57+00:00</updated>
<author>
<name>Christophe Simonis</name>
<email>chs@odoo.com</email>
</author>
<published>2018-11-22T16:52:57+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=b09f624f20781d65baf2060df40383898e6f86e5'/>
<id>b09f624f20781d65baf2060df40383898e6f86e5</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>[FIX] cache: ormcache logger</title>
<updated>2018-11-21T10:16:06+00:00</updated>
<author>
<name>Raphael Collet</name>
<email>rco@odoo.com</email>
</author>
<published>2018-11-21T10:16:06+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=4894419b61a62daf66452ea6c798e6867ea2ef88'/>
<id>4894419b61a62daf66452ea6c798e6867ea2ef88</id>
<content type='text'>
The adaptation to Python 3 was done with two mistakes: the iteration on the lru
cache does not return keys, and functions are not sortable.

closes odoo/odoo#28870</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The adaptation to Python 3 was done with two mistakes: the iteration on the lru
cache does not return keys, and functions are not sortable.

closes odoo/odoo#28870</pre>
</div>
</content>
</entry>
<entry>
<title>[IMP] tools: Add note to ormcache regarding closed cursors</title>
<updated>2017-11-08T14:30:59+00:00</updated>
<author>
<name>Dave Lasley</name>
<email>dave@laslabs.com</email>
</author>
<published>2017-08-28T16:13:54+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=aa3834b9fac30875f498c5d09b7c03a020bbe372'/>
<id>aa3834b9fac30875f498c5d09b7c03a020bbe372</id>
<content type='text'>
Returning a recordset from a cached method will raise an
`psycopg2.OperationalError` once the cursor is closed.

Add documentation to `ormcache` method indicating such
See OCA/stock-logistics-barcode#93 and #8795 for example of issues

Closes #19113
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Returning a recordset from a cached method will raise an
`psycopg2.OperationalError` once the cursor is closed.

Add documentation to `ormcache` method indicating such
See OCA/stock-logistics-barcode#93 and #8795 for example of issues

Closes #19113
</pre>
</div>
</content>
</entry>
</feed>
