<feed xmlns='http://www.w3.org/2005/Atom'>
<title>Odoo/addons/base_import/tests, branch 14.0</title>
<subtitle>The current checkout of upstream Odoo</subtitle>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/'/>
<entry>
<title>[FIX] base_import: handling of scientific notation</title>
<updated>2020-01-02T13:31:09+00:00</updated>
<author>
<name>jerome hanke (jhk)</name>
<email>jhk@odoo.com</email>
</author>
<published>2019-12-26T12:43:36+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=7757a63fc24cbe84016cb68ad8b1a8ca54006fd4'/>
<id>7757a63fc24cbe84016cb68ad8b1a8ca54006fd4</id>
<content type='text'>
Steps to reproduce:
-install sales
-try to import a product file with volumes set to scientific notation
(9.2e-05 for example)

Previous behavior:
scientific notation is not recognized by the base_import module
and raises a small warning

Current behavior:
scientific notation is converted to decimal notation on the fly
when possible

opw-2162353

closes odoo/odoo#42591

X-original-commit: 9acd76c5d116abd07207af3b548658ad983ff82d
Signed-off-by: Jorge Pinna Puissant (jpp) &lt;jpp@odoo.com&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Steps to reproduce:
-install sales
-try to import a product file with volumes set to scientific notation
(9.2e-05 for example)

Previous behavior:
scientific notation is not recognized by the base_import module
and raises a small warning

Current behavior:
scientific notation is converted to decimal notation on the fly
when possible

opw-2162353

closes odoo/odoo#42591

X-original-commit: 9acd76c5d116abd07207af3b548658ad983ff82d
Signed-off-by: Jorge Pinna Puissant (jpp) &lt;jpp@odoo.com&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>[ADD] base_import: batching (&amp; some other features)</title>
<updated>2019-09-13T09:45:29+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2019-08-28T13:02:23+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=f58368210c9c962c84cdcd2e2a799c5fc6183a13'/>
<id>f58368210c9c962c84cdcd2e2a799c5fc6183a13</id>
<content type='text'>
various UI changes
------------------

* renamed "test import" button to "test"
* move relation fields thing to debug mode
* remove "Defer parent/child computation" option as it was deprecated
  / removed from the backend in
  80f1ac3599ee630e7556ff4fcf3eb192931eccd8, turns out this checkbox
  existed inactive for longer than it's been of any use (added in
  68cb2ade09206b0abb3f7c8dccb5d35a6499f802 on 2017-11-29, made
  non-operating on 2018-01-24, that's so sad)

batching
--------

* add support for batching imports (skip &amp; limit parameters)
* modify client to use batched imports &amp; properly adapt responses so
  it still looks like a single import for the client (more or less)
  e.g. update row numbers in error messages, etc...
* properly handle partial imports though
* disable usual loading throbber to have a single progress
  notification displayed continuously throughout all the batches: the
  normal throbber only shows after 3s of waiting for an RPC response,
  so it would keep flashing in and out (appear 3s into a batch's
  import then disappear at the end only to reappear 3s into the next
  batch's loading)

NOTE: the limit is row-wise. If a record straddles the limit (because
of nested O2M records), the record is imported in full and the "next
row" is whatever row follows the record. This means a limit of 10 can
lead to an import of 17 lines, and as the progress indicator is in
records# the increments can jump around.

Task 2059448</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
various UI changes
------------------

* renamed "test import" button to "test"
* move relation fields thing to debug mode
* remove "Defer parent/child computation" option as it was deprecated
  / removed from the backend in
  80f1ac3599ee630e7556ff4fcf3eb192931eccd8, turns out this checkbox
  existed inactive for longer than it's been of any use (added in
  68cb2ade09206b0abb3f7c8dccb5d35a6499f802 on 2017-11-29, made
  non-operating on 2018-01-24, that's so sad)

batching
--------

* add support for batching imports (skip &amp; limit parameters)
* modify client to use batched imports &amp; properly adapt responses so
  it still looks like a single import for the client (more or less)
  e.g. update row numbers in error messages, etc...
* properly handle partial imports though
* disable usual loading throbber to have a single progress
  notification displayed continuously throughout all the batches: the
  normal throbber only shows after 3s of waiting for an RPC response,
  so it would keep flashing in and out (appear 3s into a batch's
  import then disappear at the end only to reappear 3s into the next
  batch's loading)

NOTE: the limit is row-wise. If a record straddles the limit (because
of nested O2M records), the record is imported in full and the "next
row" is whatever row follows the record. This means a limit of 10 can
lead to an import of 17 lines, and as the progress indicator is in
records# the increments can jump around.

Task 2059448</pre>
</div>
</content>
</entry>
<entry>
<title>[FIX] base_import: tests failing with older chardet</title>
<updated>2019-05-06T09:16:25+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2019-05-06T09:16:25+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=e12b6013d5e52bfbad93dc7fad40f17658490475'/>
<id>e12b6013d5e52bfbad93dc7fad40f17658490475</id>
<content type='text'>
Some distributions still bundle chardet 2.3, which have some guessing
divergences / incompatibilities with python (resolved in 3.x):

1. UTF-{16,32} with BOM is guessed as LE/BE, which when used to decode
   the string doesn't strip out the BOM. Handle this by checking if
   the BOM is present and converting the encoding name to the
   non-marked version in that case.
2. The ISO-8859-1 test string is guessed as ISO-8859-2 (TBF the
   decoding does make some sense). Allow multiple targets/guesses to
   "fix" that.

closes odoo/odoo#33179

Signed-off-by: Xavier Morel (xmo) &lt;xmo@odoo.com&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Some distributions still bundle chardet 2.3, which have some guessing
divergences / incompatibilities with python (resolved in 3.x):

1. UTF-{16,32} with BOM is guessed as LE/BE, which when used to decode
   the string doesn't strip out the BOM. Handle this by checking if
   the BOM is present and converting the encoding name to the
   non-marked version in that case.
2. The ISO-8859-1 test string is guessed as ISO-8859-2 (TBF the
   decoding does make some sense). Allow multiple targets/guesses to
   "fix" that.

closes odoo/odoo#33179

Signed-off-by: Xavier Morel (xmo) &lt;xmo@odoo.com&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>[IMP] base_import: date-parsing patterns</title>
<updated>2018-08-09T08:58:52+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2018-08-09T08:58:52+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=be96188f2fbf99918ba37f605c8090d248db1abe'/>
<id>be96188f2fbf99918ba37f605c8090d248db1abe</id>
<content type='text'>
* expand auto-detected date and time patterns (e.g. %b, %I, ...)
* try to make date-pattern-detection clearer
* add a select2 dropdown for date patterns (with a bunch of
  preselected patterns) rather than just an input
* also try to improve other column-matching bits (e.g. less reliance
  on exceptions, attempts to avoid redundant work)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* expand auto-detected date and time patterns (e.g. %b, %I, ...)
* try to make date-pattern-detection clearer
* add a select2 dropdown for date patterns (with a bunch of
  preselected patterns) rather than just an input
* also try to improve other column-matching bits (e.g. less reliance
  on exceptions, attempts to avoid redundant work)
</pre>
</div>
</content>
</entry>
<entry>
<title>[IMP] base_import: attempt to automatically guess the separator</title>
<updated>2018-08-07T14:46:29+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2018-08-07T14:34:11+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=36ee8a78ece08c3fd18ec2cd4904e464641e0b5d'/>
<id>36ee8a78ece08c3fd18ec2cd4904e464641e0b5d</id>
<content type='text'>
It would probably be even better to iterate the file content and get
the non-quoted non-alphanumeric characters as separator
candidates (instead of a hard-coded list) however Python does not seem
to have a decoding iterator (taking bytes and yielding an iterator of
codepoints or even grapheme clusters) — incidentally uniseg seems to
require up-front decoding as well — so that's not really convenient as
we may be dealing with large-ish files and not want to load it
entirely in memory.

An alternative would be to use TextIOWrapper and iterate the file by
buffers of a few ks, and classify that based on either codepoints or
grapheme clusters.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
It would probably be even better to iterate the file content and get
the non-quoted non-alphanumeric characters as separator
candidates (instead of a hard-coded list) however Python does not seem
to have a decoding iterator (taking bytes and yielding an iterator of
codepoints or even grapheme clusters) — incidentally uniseg seems to
require up-front decoding as well — so that's not really convenient as
we may be dealing with large-ish files and not want to load it
entirely in memory.

An alternative would be to use TextIOWrapper and iterate the file by
buffers of a few ks, and classify that based on either codepoints or
grapheme clusters.
</pre>
</div>
</content>
</entry>
<entry>
<title>[IMP] base_import: reintroduce encoding selector &amp; override guess</title>
<updated>2018-08-07T11:07:05+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2018-08-07T11:07:05+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=5aba5ac99e6663f0b8d329f7607e44068d1174e4'/>
<id>5aba5ac99e6663f0b8d329f7607e44068d1174e4</id>
<content type='text'>
* if an encoding is explicitly specified, use it and don't guess
* otherwise guess and return the guessed encoding so it can be
  displayed in the configuration UI
* fix less-than-stellar configuration &amp; behaviour of select2 inputs
  to properly reflect underlying values as they get modified, to
  correctly handle future configuration guesses
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* if an encoding is explicitly specified, use it and don't guess
* otherwise guess and return the guessed encoding so it can be
  displayed in the configuration UI
* fix less-than-stellar configuration &amp; behaviour of select2 inputs
  to properly reflect underlying values as they get modified, to
  correctly handle future configuration guesses
</pre>
</div>
</content>
</entry>
<entry>
<title>[IMP] base_import: parsing/cleanup of floats</title>
<updated>2018-08-06T10:03:12+00:00</updated>
<author>
<name>Xavier Morel</name>
<email>xmo@odoo.com</email>
</author>
<published>2018-07-20T08:06:23+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=cb9c99b9370e8464b240c6da4831a13e98a7c8c7'/>
<id>cb9c99b9370e8464b240c6da4831a13e98a7c8c7</id>
<content type='text'>
* Handle a leading + in import data, some contexts (e.g. bank
  statements) will mark positive sums explicitly for clarity
* Add basic grouping/decimal separator inference for people importing
  data from many localisations or to avoid them *having* to configure
  their separators if we can handle it for them, currently very basic
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Handle a leading + in import data, some contexts (e.g. bank
  statements) will mark positive sums explicitly for clarity
* Add basic grouping/decimal separator inference for people importing
  data from many localisations or to avoid them *having* to configure
  their separators if we can handle it for them, currently very basic
</pre>
</div>
</content>
</entry>
<entry>
<title>[IMP] import/export UIs</title>
<updated>2018-07-17T13:19:05+00:00</updated>
<author>
<name>Jigar Patel</name>
<email>jat@openerp.com</email>
</author>
<published>2018-07-17T13:19:05+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=b60de0db41bbe4af3c2f6e2369eda5d483702d76'/>
<id>b60de0db41bbe4af3c2f6e2369eda5d483702d76</id>
<content type='text'>
Task 40692

Various changes to import/export (mainly) UIs:

* default to excel &amp; "full" (non-import-compatible) export
* auto-detect encoding of CSV using chardet
* remember column -&gt; field mapping after having imported a file (useful
   for repeated imports where auto-matching failed)
* better handle localised booleans &amp; column names
* automatically select source list view's fields when exporting
* better integrate import templates feature and add a number of templates</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Task 40692

Various changes to import/export (mainly) UIs:

* default to excel &amp; "full" (non-import-compatible) export
* auto-detect encoding of CSV using chardet
* remember column -&gt; field mapping after having imported a file (useful
   for repeated imports where auto-matching failed)
* better handle localised booleans &amp; column names
* automatically select source list view's fields when exporting
* better integrate import templates feature and add a number of templates</pre>
</div>
</content>
</entry>
<entry>
<title>[FIX] base_import: activate advanced mode</title>
<updated>2018-04-12T12:54:25+00:00</updated>
<author>
<name>Nicolas Martinelli</name>
<email>nim@odoo.com</email>
</author>
<published>2018-04-12T11:32:17+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=72425445062bc595d0718d83580de9901023aaf3'/>
<id>72425445062bc595d0718d83580de9901023aaf3</id>
<content type='text'>
The commit 1ba4fbe640cf22abe did not match the specs of the task, but
was merged anyway. Reverting the commit, and check by default so the
user is not confused with missing fields.

opw-1824074
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The commit 1ba4fbe640cf22abe did not match the specs of the task, but
was merged anyway. Reverting the commit, and check by default so the
user is not confused with missing fields.

opw-1824074
</pre>
</div>
</content>
</entry>
<entry>
<title>[MERGE] forward port branch saas-16 up to dc2a6c6cd2a</title>
<updated>2017-10-20T17:27:04+00:00</updated>
<author>
<name>Christophe Simonis</name>
<email>chs@odoo.com</email>
</author>
<published>2017-10-20T17:27:04+00:00</published>
<link rel='alternate' type='text/html' href='http://erp.ceuconsult.com:9000/Odoo/commit/?id=10128fa7e8c743e9c838743204fb8116bcdeefd6'/>
<id>10128fa7e8c743e9c838743204fb8116bcdeefd6</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
