diff options
Diffstat (limited to 'doc/reference/testing.rst')
| -rw-r--r-- | doc/reference/testing.rst | 259 |
1 files changed, 239 insertions, 20 deletions
diff --git a/doc/reference/testing.rst b/doc/reference/testing.rst index d14b3c25838..9196fd876c9 100644 --- a/doc/reference/testing.rst +++ b/doc/reference/testing.rst @@ -10,9 +10,9 @@ Testing Odoo There are many ways to test an application. In Odoo, we have three kinds of tests -- python unit tests: useful for testing model business logic -- js unit tests: this is necessary to test the javascript code in isolation -- tours: this is a form of integration testing. The tours ensure that the +- Python unit tests (see `Testing Python code`_): useful for testing model business logic +- JS unit tests (see `Testing JS code`_): useful to test the javascript code in isolation +- Tours (see `Integration Testing`_): tours simulate a real situation. They ensures that the python and the javascript parts properly talk to each other. Testing Python code @@ -257,35 +257,242 @@ Run only the tests from stock or tagged as slow: Testing JS code =============== +Testing a complex system is an important safeguard to prevent regressions and to +guarantee that some basic functionality still works. Since Odoo has a non trivial +codebase in Javascript, it is necessary to test it. In this section, we will +discuss the practice of testing JS code in isolation: these tests stay in the +browser, and are not supposed to reach the server. + Qunit test suite ---------------- -Odoo Web includes means to unit-test both the core code of -Odoo Web and your own javascript modules. On the javascript side, -unit-testing is based on QUnit_ with a number of helpers and -extensions for better integration with Odoo. +The Odoo framework uses the QUnit_ library testing framework as a test runner. +QUnit defines the concepts of *tests* and *modules* (a set of related tests), +and gives us a web based interface to execute the tests. -To see what the runner looks like, find (or start) an Odoo server -with the web client enabled, and navigate to ``/web/tests`` -This will show the runner selector, which lists all modules with javascript -unit tests, and allows starting any of them (or all javascript tests in all -modules at once). +For example, here is what a pyUtils test could look like: -.. image:: ./images/runner.png - :align: center +.. code-block:: javascript + + QUnit.module('py_utils'); + + QUnit.test('simple arithmetic', function (assert) { + assert.expect(2); -Clicking any runner button will launch the corresponding tests in the -bundled QUnit_ runner: + var result = pyUtils.py_eval("1 + 2"); + assert.strictEqual(result, 3, "should properly evaluate sum"); + result = pyUtils.py_eval("42 % 5"); + assert.strictEqual(result, 2, "should properly evaluate modulo operator"); + }); + +The main way to run the test suite is to have a running Odoo server, then +navigate a web browser to ``/web/tests``. The test suite will then be executed +by the web browser Javascript engine. .. image:: ./images/tests.png :align: center -Writing a test case -------------------- +The web UI has many useful features: it can run only some submodules, or +filter tests that match a string. It can show every assertions, failed or passed, +rerun specific tests, ... -This section will be updated as soon as possible. -.. _qunit: http://qunitjs.com/ +Testing Infrastructure +---------------------- + +Here is a high level overview of the most important parts of the testing +infrastructure: + +- there is an asset bundle named `web.js_tests_assets`_. This bundle contains + the main code (assets common + assets backend), some libraries, the QUnit test + runner, and some additional helper code + +- another asset bundle, `web.qunit_suite`_, contains all the tests (and the + js_tests_assets code). Almost all the test files should be added to this + bundle + +- there is a `controller`_ in web, mapped to the route */web/tests*. This controller + simply renders the *web.qunit_suite* template. + +- to execute the tests, one can simply point its browser to the route */web/tests*. + In that case, the browser will download all assets, and QUnit will take over. + +- there is some code in `qunit_config.js`_ which logs in the console some + information when a test passes or fails. + +- we want the runbot to also run these tests, so there is a test (in `test_js.py`_) + which simply spawns a browser and points it to the *web/tests* url. Note that + the phantom_js method does not spawn phantom_js, but Chrome headless instead. + + +Modularity and testing +---------------------- + +With the way Odoo is designed, any addon can modify the behaviour of other parts +of the system. For example, the *voip* addon can modify the *FieldPhone* widget +to use extra features. This is not really good from the perspective of the +testing system, since this means that a test in the addon web will fail whenever +the voip addon is installed (note that the runbot runs the tests with all addons +installed). + +At the same time, our testing sytem is good, because it can detect whenever +another module breaks some core functionality. There is no complete solution to +this issue. For now, we solve this on a case by case basis. + +Usually, it is not a good idea to modify some other behaviour. For our voip +example, it is certainly cleaner to add a new *FieldVOIPPhone* widget and +modify the few views that needs it. This way, the *FieldPhone* widget is not +impacted, and both can be tested. + +Adding a new test case +---------------------- + +Let us assume that we are maintaining an addon *my_addon*, and that we +want to add a test for some javascript code (for example, some utility function +myFunction, located in *my_addon.utils*). The process to add a new test case is +the following: + +1. create a new file *my_addon/static/tests/utils_tests.js*. This file contains the basic code to + add a QUnit module *my_addon > utils*. + + .. code-block:: javascript + + odoo.define('my_addon.utils_tests', function (require) { + "use strict"; + + var utils = require('my_addon.utils'); + + QUnit.module('my_addon', {}, function () { + + QUnit.module('utils'); + + }); + }); + + +2. In *my_addon/assets.xml*, add the file to the main test assets: + + .. code-block:: xml + + <?xml version="1.0" encoding="utf-8"?> + <odoo> + <template id="qunit_suite" name="my addon tests" inherit_id="web.qunit_suite"> + <xpath expr="//script[last()]" position="after"> + <script type="text/javascript" src="/my_addon/static/tests/utils_tests.js"/> + </xpath> + </template> + </odoo> + +3. Restart the server and update *my_addon*, or do it from the interface (to + make sure the new test file is loaded) + +4. Add a test case after the definition of the *utils* sub test suite: + + .. code-block:: javascript + + QUnit.test("some test case that we want to test", function (assert) { + assert.expect(1); + + var result = utils.myFunction(someArgument); + assert.strictEqual(result, expectedResult); + }); + +5. Visit */web/tests/* to make sure the test is executed + +Helper functions and specialized assertions +------------------------------------------- + +Without help, it is quite difficult to test some parts of Odoo. In particular, +views are tricky, because they communicate with the server and may perform many +rpcs, which needs to be mocked. This is why we developed some specialized +helper functions, located in `test_utils.js`_. + +- Mock test functions: these functions help setting up a test environment. The + most important use case is mocking the answers given by the Odoo server. These + functions use a `mock server`_. This is a javascript class that simulates + answers to the most common model methods: read, search_read, nameget, ... + +- DOM helpers: useful to simulate events/actions on some specific target. For + example, testUtils.dom.click performs a click on a target. Note that it is + safer than doing it manually, because it also checks that the target exists, + and is visible. + +- create helpers: they are probably the most important functions exported by + `test_utils.js`_. These helpers are useful to create a widget, with a mock + environment, and a lot of small detail to simulate as much as possible the + real conditions. The most important is certainly `createView`_. + +- `qunit assertions`_: QUnit can be extended with specialized assertions. For + Odoo, we frequently test some DOM properties. This is why we made some + assertions to help with that. For example, the *containsOnce* assertion takes + a widget/jQuery/HtmlElement and a selector, then checks if the target contains + exactly one match for the css selector. + +For example, with these helpers, here is what a simple form test could look like: + +.. code-block:: javascript + + QUnit.test('simple group rendering', function (assert) { + assert.expect(1); + + var form = testUtils.createView({ + View: FormView, + model: 'partner', + data: this.data, + arch: '<form string="Partners">' + + '<group>' + + '<field name="foo"/>' + + '</group>' + + '</form>', + res_id: 1, + }); + + assert.containsOnce(form, 'table.o_inner_group'); + + form.destroy(); + }); + +Notice the use of the testUtils.createView helper and of the containsOnce +assertion. Also, the form controller was properly destroyed at the end of +the test. + +Best Practices +-------------- + +In no particular order: + +- all test files should be added in *some_addon/static/tests/* +- for bug fixes, make sure that the test fails without the bug fix, and passes + with it. This ensures that it actually works. +- try to have the minimal amount of code necessary for the test to work. +- usually, two small tests are better than one large test. A smaller test is + easier to understand and to fix. +- always cleanup after a test. For example, if your test instantiates a widget, + it should destroy it at the end. +- no need to have full and complete code coverage. But adding a few tests helps + a lot: it makes sure that your code is not completely broken, and whenever a + bug is fixed, it is really much easier to add a test to an existing test suite. +- if you want to check some negative assertion (for example, that a HtmlElement + does not have a specific css class), then try to add the positive assertion in + the same test (for example, by doing an action that changes the state). This + will help avoid the test to become dead in the future (for example, if the css + class is changed). + +Tips +---- + +- running only one test: you can (temporarily!) change the *QUnit.test(...)* + definition into *QUnit.only(...)*. This is useful to make sure that QUnit + only runs this specific test. +- debug flag: most create utility functions have a debug mode (activated by the + debug: true parameter). In that case, the target widget will be put in the DOM + instead of the hidden qunit specific fixture, and more information will be + logged. For example, all mocked network communications will be available in the + console. +- when working on a failing test, it is common to add the debug flag, then + comment the end of the test (in particular, the destroy call). With this, it + is possible to see the state of the widget directly, and even better, to + manipulate the widget by clicking/interacting with it. Integration Testing @@ -297,3 +504,15 @@ write another kind of test: tours. A tour is a mini scenario of some interestin business flow. It explains a sequence of steps that should be followed. The test runner will then create a phantom_js browser, point it to the proper url and simulate the click and inputs, according to the scenario. + + +.. _qunit: http://qunitjs.com/ +.. _qunit_config.js: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/static/tests/helpers/qunit_config.js#L49 +.. _web.js_tests_assets: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/views/webclient_templates.xml#L427 +.. _web.qunit_suite: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/views/webclient_templates.xml#L509 +.. _controller: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/controllers/main.py#L637 +.. _test_js.py: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/tests/test_js.py#L13 +.. _test_utils.js: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/static/tests/helpers/test_utils.js +.. _mock server: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/static/tests/helpers/mock_server.js +.. _qunit assertions: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/static/tests/helpers/qunit_asserts.js +.. _createView: https://github.com/odoo/odoo/blob/51ee0c3cb59810449a60dae0b086b49b1ed6f946/addons/web/static/tests/helpers/test_utils_create.js#L267
\ No newline at end of file |