summaryrefslogtreecommitdiff
path: root/odoo/sql_db.py
AgeCommit message (Collapse)Author
2020-10-20[FIX] core: Ensure that rollback hooks are called when the cursor is closedLaurent Mignon (ACSONE)
Before this change the rollback hooks were never called since rollback() was never explicitely called by the framework. At the end of a transaction, if no error occurs the famework call commit() on the odoo cursor. In all cases, the transaction ends with a call to close(). Into the implementation of _close() rollback() is called on the underlying connection to ensure that not committed changes are rollbacked. That's the reason why despite the fact that rollback() was not called on the Odoo cursor, changes are not committed into the db in case of exception. To keep the same behaviour and avoid to have to explicitely call rollback() on the odoo cursor to trigger the execution of registered rollback hooks, these hooks are now processed in _close(). Since the list of registered hooks is emptied if commit() is called, we are sure that rollback hooks are only executed in case of rollback. OPW #2294911 closes odoo/odoo#60334 X-original-commit: dce9a05f3a5d37fab711c8ca3c5444941f98e814 Signed-off-by: Raphael Collet (rco) <rco@openerp.com> Co-authored-by: Raphael Collet <rco@odoo.com>
2020-10-09[FIX] core: preferably flush() with an env with a real uidRaphael Collet
This fixes a crash in some controllers with `auth='None'` where some updates are flushed with an environment where `uid=None`. When there is an environment with a real uid, preferably use it. closes odoo/odoo#59650 X-original-commit: 2795c86df54389b858a454cbc1782b1f34f3ba4f Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-09-03[IMP] core: deprecate method cr.after()Raphael Collet
Use the more convenient `cr.postcommit.add()` instead. closes odoo/odoo#56990 Related: odoo/enterprise#12914 Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-09-03[FIX] core: cursor hooks API and implementationRaphael Collet
Python 3.8 changed the equality rules for bound methods to be based on the *identity* of the receiver (`__self__`) rather than its *equality*. This means that in 3.7, methods from different instances will compare (and hash) equal, thereby landing in the same map "slot", but that isn't the case in 3.8. While it's usually not relevant, it's an issue for `GroupCalls` which is indexed by a function: in 3.7, that being a method from recordsets comparing equal will deduplicate them, but not anymore in 3.8, leading to duplicated callbacks (exactly the thing GroupCalls aims to avoid). Also, the API of `GroupCalls` turned out to be unusual and weird. The bug above is fixed by using a plain list for callbacks, thereby avoiding comparisons between registered functions. The API is now: callbacks.add(func) # add func to callbacks callbacks.run() # run all callbacks in addition order callbacks.clear() # remove all callbacks In order to handle aggregated data, the `callbacks` object provides a dictionary `callbacks.data` that any callback function can freely use. For the sake of consistency, the `callbacks.data` dict is automatically cleared upon execution of callbacks. Discovered by @william-andre Related to odoo#56583 References: * https://bugs.python.org/issue1617161 * python/cpython#7848 * https://docs.python.org/3/whatsnew/changelog.html#python-3-8-0-alpha-1 (no direct link because individual entries are not linkable, look for bpo-1617161) X-original-commit: a986f4929bab95759086cc77ca5e60795f82e91a
2020-06-11[FIX] core: flushing an environment should not leave things to computeRaphael Collet
Upon cursor `commit()`, the pending computations are performed with method `flush()`. However the latter method leaves fields to compute on new records. As the `commit` ends the current transaction, we can clear those pending computations. X-original-commit: d27e98d9048d0cb1250522205286ebc7b7d5ea39
2020-04-01[FIX] sql_db: flush when no `uid` in environmentToufik Ben Jaa
- A bug has been introduced by the PR https://github.com/odoo/odoo/pull/46719 This issue prevented flushes to database when no `uid` is set in the current environment. This happens when database changes are made in `auth="none"` routes. In a MonoDB setup this was working because Odoo consider `None`, `False` as `null` and was always bound to a database. Nothing prevents to write `null` in the columns `create_uid` and `write_uid` in database. The PR that introduced the bug wanted to block the cases where `uid` is an instance of the class `RequestUID` to avoid `Cannot adapt type` errors. So testing that `uid` is an integer isn't enough. Two fixes were possible here, either check if `uid` is an instance of the class `RequestUID` or if it is either an integer, `False` or `None`. To avoid noise and imports from `odoo`, we test that `uid` is an instance of `None`. The case where it is `False` is already tested in the python code: ```python isinstance(env.uid, int) ``` (manual) forward-port of odoo/odoo#48685 closes odoo/odoo#48693 Signed-off-by: Toufik Benjaa (tbe) <tbe@odoo.com> Signed-off-by: Olivier Dony (odo) <odo@openerp.com>
2020-03-27[FIX] core: do not flush() with an environment using RequestUIDRaphael Collet
When routing an HTTP request, the dispatcher uses an environment with a placeholder for the user (the object `RequestUID`). This environment is not supposed to be used after routing has been done. However, the flushing function takes any environment with a given cursor, and it crashes when this environment is chosen (`psycopg2` cannot adapt the type `RequestUID` for `write_uid`), and simply does not make sense. closes odoo/odoo#48495 X-original-commit: da61a64bf6731a08557d23680eabf4d910035ebd Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2020-03-13[FIX] sql_db: remove unused __closerJulien Castiaux
The `__closer` attribute is filled with the origin of a call to `cr.close()`. That information is reused in the `check` decotator that ensure the user is not using a closed cursor. In case he is using a closed cursor, an error is raised with that optional origin in case of `--log-sql`. The attribute is a dundler so it is only accessible within its class, as the decorator has been moved outside of the class by 058cf208a8e7 it is not directly accessible thus `__getattr__` is called as a fallback. The `__getattr__` itself is protected by the `check` decorator thus they call each other in a infinite recursion. As the related dundler is no more used internally, it has been decided to remove it entirely. closes odoo/odoo#47616 Task: 2199895 X-original-commit: 3ebf7cb9cca37b05f44aaffc0b1f289cc65f300f Signed-off-by: Julien Castiaux <Julien00859@users.noreply.github.com>
2020-02-05[ADD] sql_db: pre/post-commit/rollback hooksRaphael Collet
2019-10-08[FIX] Log failed queries decodedJairo Llopis
Starting with Python 3, queries sent by psyocopg2 are stored as `bytes()` objects. Logging those raw makes them appear unformatted, harder to read than in v10 or lower Odoo versions (i.e. `\n` instead of a raw newline character). Decoding the query into unicode makes it easier to read in the logs. closes odoo/odoo#38074 X-original-commit: 329accde2ed6d4dbfdde6d3d99d71e21d15396b7 Signed-off-by: Christophe Simonis <chs@odoo.com>
2019-09-30[IMP] Support psycopg2.sql moduleJairo Llopis
[`psycopg2` includes a submodule called `sql`][1] which provides more safety and comfort when dealing with raw SQL queries. As of today, it works fine with Odoo unless SQL query logging is enabled. This patch fixes that problem, allowing usage of that module. [1]: http://initd.org/psycopg/docs/sql.html closes odoo/odoo#37561 Signed-off-by: Christophe Simonis <chs@odoo.com>
2019-09-20[FIX] sql_db: retrieve correct environmentMartin Trigaux
When creating a new database from the database manager, with no open cursor, the wrong env was retrieved during the cr.commit() call. 2019-09-09 18:16:07,454 19886 ERROR None odoo.service.db: CREATE DATABASE failed: Traceback (most recent call last): File "/home/odoo/odoo/service/db.py", line 60, in _initialize_db cr.commit() File "/home/odoo/odoo/sql_db.py", line 165, in wrapper return f(self, *args, **kwargs) File "/home/odoo/odoo/sql_db.py", line 389, in commit env = get_env(currentframe(), 2) File "/home/odoo/odoo/sql_db.py", line 72, in get_env env = getattr(frame.f_locals.get('self'), 'env', SENTINEL) File "/home/odoo/odoo/http.py", line 268, in env self._env = odoo.api.Environment(self.cr, self.uid, self.context) File "/home/odoo/odoo/http.py", line 238, in cr raise RuntimeError('request not bound to a database') get_env method checks every frame to find a variable called 'env' As none was found, it used the one of the WebRequest object. In a WebRequest, env is a property that will try to create a new Environment, which fails as no database is available (yet). At eafdf1812d3 the iteration on Environment.envs was removed as envs is a weakset and it introduced potential errors where the content was garbage-collected during the iteration. Go back to the simpler iteration but convert the set to list to avoid garbage-collection issues. Fixes odoo/odoo#36588 closes odoo/odoo#37005 Signed-off-by: Martin Trigaux (mat) <mat@odoo.com> Co-authored-by: Raphael Collet <rco@odoo.com>
2019-08-26[FIX] sql_db: add flush() in savepoint()Raphael Collet
closes odoo/odoo#36060 Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-08-26[FIX] sql_db: environment retrieval in cursorRaphael Collet
2019-08-26[FIX] sql_db: revert ee03fc37704Christophe Monniez
ee03fc37704 introduced a non deterministic bug by choosing an environment in a set, so the environment is not always the good one. closes odoo/odoo#36056 Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-08-23[FIX] sql_db: flush in savepoint, following ↵Denis Ledoux
odoo/odoo@9920f20e4c7753bc17bea71dea3a90f7de687196 closes odoo/odoo#35986 Signed-off-by: Raphael Collet (rco) <rco@openerp.com>
2019-08-20[IMP] models: ORM speedupRaphael Collet
This branch is the combination of several optimizations in the ORM: * store field values once in the cache: the cache reflects more faithfully the database, only fields that explicitly depend on the context have an extra indirection in the cache; * delay recomputations by default: use method `recompute` to explicitly flush out pending recomputations; * delay updates in method `write`: updates are stored in a data structure that can be flushed efficiently to the database with method `flush` (which also flush out recomputations); * make method `modified` take advantage of inverse fields to inverse dependencies; * filter records by evaluating a domain on records in Python; * a computed field with `readonly=False` behaves like a normal field with an onchange method; * computed fields are computed in superuser mode by default. Work done by Toufik Ben Jaa, Raphael Collet, Denis Ledoux and Fabien Pinckaers. closes odoo/odoo#35659 Signed-off-by: Denis Ledoux <beledouxdenis@users.noreply.github.com>
2019-07-03[IMP] odoo: remove LazyCursorMartin Trigaux
Was specifically created for decimal.precision at f0646cb51b236a3afd5 Should no longer be needed
2018-11-29[REF] *: replace deprecated pycompat helpers for builtinsAdrian Torres
This commit replaces calls to pycompat helpers that were intended for python 2 <-> python 3 interoperability for python 3 builtins, as python 2 is no longer officially supported by Odoo. This includes: * calls to imap/izip/ifilter replaced by map/zip/filter * uses of text_type replaced by str * uses of unichr replaced by chr * calls to implements_to_string, implements_iterator removed * string_types and integer_types replaced by str, int respectively * calls to to_native replaced by calls to to_text This is done in preparation to the removal of these deprecated helpers in the following commit.
2018-09-07[IMP] http,server: add query count and request times in logsXavierDo
When there is a performance issue, it's sometimes difficult to discover which request increased the query count or its duration. With this commit, the query count, the query time and "python and io" time are displayed in the logs at the end of each werkzeug request line. Co-authored-by: Christophe Monniez <moc@odoo.com>
2018-08-06[REF] *: use native date/datetime for Date/Datetime fieldsRaphael Collet
From this commit onwards, Date fields will return datetime.date objects and Datetime fields will return datetime.datetime objects, this implies a number of things that are clearly explained both in the ORM API for master. This commit also introduces a number of helper functions for dates and datetimes that are exposed in tools.date_utils and fields.Date[time], explained in the documentation as well. Task-ID: 47189
2018-04-26[FIX] sql_db: log-level debug_sql log parametersNicolas Lempereur
Since 1a58e108 (on 10.0 and over) parameters are no longer present when logging sql queries. This removes some of the usefulness of the feature, and this commit adds them back. Since mogrify returns a byte string in the database encoding, we decode it using the corresponding python encoding. opw-1839241 closes #24446
2018-02-12[REF] tests: `TestCursor` is now a proxy to a real `Cursor`Raphael Collet
This allows rpc requests in `HttpCase` to use the cursor `self.cr`, which is now shared between the Python test and the rpc requests. This simplifies code to prepare a JS test, and code to check the result of a JS tour. Fixes #12237
2017-10-25[MERGE] forward port branch saas-16 up to 7bfde6e05d1Christophe Simonis
2017-10-25[MERGE] forward port branch saas-15 up to d516d39948fChristophe Simonis
2017-10-25[MERGE] forward port branch 10.0 up to 598ac60a776Christophe Simonis
2017-10-25[MERGE] forward port branch 9.0 up to 63661504846Christophe Simonis
2017-09-28[IMP] core: allow to connect to database server with a secure SSL TCP/IP ↵Christophe Simonis
connection
2017-09-11[FIX] sql_db: use pycompat string_typesJulien Legros
2017-08-20[FIX] P3: remove pycompat.{keys,items,values} helpersOlivier Dony
Now that we're closer to switching to P3 for good, these helpers have outlived their usefulness, and mostly add noise. All remaining dict.iter*() or dict.view*() must be converted to the normal keys(), values() or items() calls. Whenever the result is likely to be used for more than the scope of a loop, or when the dict needs to be modified during iteration, the calls must be wrapped in a ``list()``, to protect the new P3 semantics. Those cases are very exceptional. Also removed some dead code or improved the API to remove unnecessary conversions.
2017-07-26[MERGE] forward port branch saas-16 up to 46e1104ce12Christophe Simonis
2017-07-26[MERGE] forward port branch saas-15 up to 681f03e3097Christophe Simonis
2017-07-26[IMP] core: log bad queries as ERRORChristophe Simonis
Also adapt default config and config mapping.
2017-06-26[FIX] P3: __nonzero__ -> __bool__Xavier Morel
Also ~removed bool-testing on cursors
2017-06-15[MERGE] forward port branch saas-16 up to acfebff8807Christophe Simonis
2017-06-15[FIX] *: correct PY3 compatibility introduced by previous forward-portsChristophe Simonis
2017-06-15[MERGE] forward port branch saas-16 up to 1b50c829ef3Christophe Simonis
2017-06-15[MERGE] forward port branch saas-15 up to 9713dc2e1f5Christophe Simonis
2017-06-15[MERGE] forward port branch 10.0 up to dc8b473a033Christophe Simonis
2017-06-15[FIX] sql_db: port fix from psycopg/psycopg2#459Olivier Dony
NUL characters must not be used in query parameters, as they will be ignored by libpq, being end-of-string characters. Preventing NULs avoids unexpected results from queries. It is only necessary with psycopg2 versions before 2.7, which includes the upstream fix.
2017-05-15[FIX] P3: urllib, urllib2 and urlparseXavier Morel
In Python 3, all of these were "consolidated" under urllib(.request, .parse, .errors) which is inconvenient. Since we already have hard dependencies on requests and werkzeug(.urls, which is a backport of Python 3's unicode-aware urllib.parse) migrate *everything* to that. A sticking point is urllib2.URLError, those were (mostly) replaced by the slightly more general IOError which URLError extends.
2017-05-10[FIX] P3: list -> iterable builtins (#16811)xmo-odoo
In Python 3: * various builtins and dict methods were changed to return view/iterable objects rather than lists * and the separate Python 2 view/iterable builtins and methods were removed altogether This is problematic when using these items as list (which the happens repeatedly in Odoo), but more viciously when iterating *multiple times* over them (which also happens, which I've messed up multiple times while writing this, and which is a pain to debug even when you've just created the issue). Convert all code using these to semantics-matching cross-version helper functions to get the LCD behaviour between P2 and P3, and forbid the builtins via lint. issue #8530
2017-04-28[FIX] Various P3-related import changesxmo-odoo
* LDAP import: python-ldap is not python3-compatible, pyldap is Warning: only supported from debian Stretch (current testing)? https://packages.debian.org/search?searchon=names&keywords=pyldap * implicitly relative imports * imports of moved or removed stdlib modules issue #8530
2017-04-27[FIX] builtins removed from Python 3xmo-odoo
* Reverse wrapper courtesy of @rco-odoo's original P3 branch * thin compat module stripped down from werkzeug (to augment as needed) issue 8530
2016-11-17[IMP] sql_db: Improve error message logging on database querriesDevendra Kavthekar
Purpose: When we have a SQL error, it's not very easy to identify the root of the crash. For example if a mandatory field is not set on a row creation, it difficult to see which field isn't correctly set as a lot of fields are not required and a lot a NULL values are present in the INSERT querry. Specification: Show a proper log message, with the reason (cause) of the sql error. Behavior Modification Examples: Some sample log after applied fix. The new line are those beginning by '+' 2016-10-25 09:59:40,208 21161 INFO ent_master odoo.sql_db: bad query: INSERT INTO "res_partner_category" ("id", "active", "color", "create_uid", "write_uid", "create_date", "write_date") VALUES(nextval('res_partner_category_id_seq'), true, 10, 1, 1, (now() at time zone 'UTC'), (now() at time zone 'UTC')) RETURNING id +REASON: null value in column "name" violates not-null constraint +DETAIL: Failing row contains (27, null, null, 1, null, 10, 1, null, 2016-10-25 09:59:40.207202, t, 2016-10-25 09:59:40.207202). 2016-10-25 10:00:29,710 21161 INFO ent_master odoo.sql_db: bad query: UPDATE "res_partner" SET "parent_id"=1001,"type"='contact',"name"='Course Script',"company_name"=NULL,"write_uid"=1,"write_date"=(now() at time zone 'UTC') WHERE id IN (1) +REASON: insert or update on table "res_partner" violates foreign key constraint "res_partner_parent_id_fkey" +DETAIL: Key (parent_id)=(1001) is not present in table "res_partner". 2016-10-25 10:23:28,690 21336 INFO ent_master odoo.sql_db: bad query: DELETE FROM res_country WHERE id IN (235) +REASON: null value in column "country_id" violates not-null constraint +DETAIL: Failing row contains (9, 1, AL, Alabama, null, 1, 2016-10-24 10:19:58.765552, 2016-10-21 09:32:26.939607). +CONTEXT: SQL statement "UPDATE ONLY "public"."res_country_state" SET "country_id" = NULL WHERE $1 OPERATOR(pg_catalog.=) "country_id""
2016-09-02[FIX] odoo: rename imports and adapt import hooksRaphael Collet
2016-09-02[REF] openerp: move `openerp` to `odoo`Raphael Collet