[IMP] core: improve mapped and filtered performance

Previously, mapped was following a very naive approach, which was simply
calling the field name passed as input for every record in a recordset,
sequentially.

The problem with this approach is that we will potentially recompute the
same fields multiple times for differents records, when this could be
done once per field for ALL records, and store this value in cache for
further access.

Another potential problem is that we don't take advantage of the ORM's
prefetching to fetch all the records that are not in cache at once,
instead of doing the same query for every record in the recordset.

Yet another problem is the conversion of each cache value to a record
format and then combining all of the individual records into a single
recordset, which, depending on the size of the recordset, can take an
unbelievable amount of CPU time.

With this new implementation of `mapped()` we take care of all of these
problems:

This is done by first delegating `mapped()` from the model to the field,
this mapped takes a recordset as input and it will try to batch compute
and prefetch as much as possible for the entire recordset, but it will
not keep these values for the actual output, it just stores everything
in cache and then at the end, retrieves everything from the cache to
guarantee the same order.

After the mapped, the conversion from cache format to record format is
delegated to the new `convert_to_record_multi` which will fetch all the
ids and then perform a single browse to encapsulate all of the records
into a single recordset with the least amount of overhead possible.

Part of Task 2170344

closes odoo/odoo#42611

Signed-off-by: Raphael Collet (rco) <rco@openerp.com>

1 files changed, 15 insertions, 0 deletions

diff --git a/odoo/api.py b/odoo/api.py
index cf56718b032..b6a7f1acb8a 100644
--- a/odoo/api.py
+++ b/odoo/api.py
@@ -827,6 +827,21 @@ class Cache(object):
             except KeyError:
                 pass
 
+    def get_until_miss(self, records, field):
+        """ Return the cached values of ``field`` for ``records`` until a value is not found. """
+        field_cache = self._data[field]
+        key = records.env.cache_key(field) if field.depends_context else None
+        vals = []
+        for record_id in records._ids:
+            try:
+                if key is not None:
+                    vals.append(field_cache[record_id][key])
+                else:
+                    vals.append(field_cache[record_id])
+            except KeyError:
+                break
+        return vals
+
     def get_records_different_from(self, records, field, value):
         """ Return the subset of ``records`` that has not ``value`` for ``field``. """
         field_cache = self._data[field]