| Age | Commit message (Collapse) | Author |
|---|
|
When an import failed because of a specific cell, the error was
Error cell found while reading XLS/XLSX file: #N/A
Add the line and column index in the error for better debugging
Translate error message
Cherry-pick of odoo/odoo#30729 in master
Closes #30729
closes odoo/odoo#31677
Signed-off-by: Martin Trigaux (mat) <mat@odoo.com>
|
|
Check that it makes custom binary fields into attachment as that's the
main reason for the change: when users create binary fields via Studio,
they're necessarily db-stored (as the interface doesn't allow altering
the attachment attribute and it's unclear how we'd handle users
switching it on/off every time), which significantly bloats their
database (and burns storage & backup space), especially as the primary
use case for binary fields is adding images and documents to records.
* check that binary fields are properly created as attachment=True
* add attachment=False on fields where that seems relevant (most but not
all of the fields previously using the default)
* remove occurrences of attachment=True
closes odoo/odoo#29308
|
|
|
|
This commit replaces calls to pycompat helpers that were intended for
python 2 <-> python 3 interoperability for python 3 builtins, as python
2 is no longer officially supported by Odoo.
This includes:
* calls to imap/izip/ifilter replaced by map/zip/filter
* uses of text_type replaced by str
* uses of unichr replaced by chr
* calls to implements_to_string, implements_iterator removed
* string_types and integer_types replaced by str, int respectively
* calls to to_native replaced by calls to to_text
This is done in preparation to the removal of these deprecated helpers
in the following commit.
|
|
* actually pass the flags as flags in re.sub, passing re.IGNORECASE
as *count* doesn't actually ignore case
* ensure the entire string is matched by the pattern, or we'll get
shortest-matching-pattern which is *not* what the later strptime
will do
closes odoo/odoo#27925
|
|
Do not propagate cache invalidations to other workers when changes must be
discarded, because of an import error or a dry run, which are both handled as
successful transactions.
|
|
* reduce number of patterns: using localised month names is useless
since we're not setting the locale so it always uses the server's own
* avoid going through the entire strptime machinery: lift the bits of
TimeRE we're interested in to get regex bits out of strptime patterns
and just to an re.match to check whether value & pattern match
Further possible optimisation: cache REs and only compile user-provided
patterns dynamically.
|
|
Purpose of this commit is to give description more "business oriented"
because those descriptions appears in Odoo Studio which is supposed to be used by end users, not only by developers.
Related Task ID : 37311
|
|
Adds a checkbox to import columns (in debug mode) allowing a user to
create records M2O and M2M records not found (via name_search).
Task ID: 1850633
* uses a context key to avoid altering basically all the import callstack
* attempted to lift the creation in the `_str_to_*` functions and create
m2m via commands, but that doesn't really work out
|
|
Turns out %s is not the pattern for sub-minute seconds, that'd be
%S (note the casing). The former is the number of seconds since
epoch, which tends not to match sub-minute second patterns (sounds
like bull to me but there you are), and so importing datetimes was a
bit broken since the previous improvements.
Fix that.
|
|
Before, once reaching level 0 all of an O2M's fields would still be
selected, so we would really be selecting depth+1 levels of fields,
which makes for an extremely busy list as it's a combinatorial
explosion.
Change this so at level 0 the only field selected from a field is an
xid. Not sure that's even necessary though, do xids really make sense
for o2ms?
|
|
* expand auto-detected date and time patterns (e.g. %b, %I, ...)
* try to make date-pattern-detection clearer
* add a select2 dropdown for date patterns (with a bunch of
preselected patterns) rather than just an input
* also try to improve other column-matching bits (e.g. less reliance
on exceptions, attempts to avoid redundant work)
|
|
It would probably be even better to iterate the file content and get
the non-quoted non-alphanumeric characters as separator
candidates (instead of a hard-coded list) however Python does not seem
to have a decoding iterator (taking bytes and yielding an iterator of
codepoints or even grapheme clusters) — incidentally uniseg seems to
require up-front decoding as well — so that's not really convenient as
we may be dealing with large-ish files and not want to load it
entirely in memory.
An alternative would be to use TextIOWrapper and iterate the file by
buffers of a few ks, and classify that based on either codepoints or
grapheme clusters.
|
|
Various bits would lead to odd tracebacks or a complete lack of
actionable feedback.
Discover that a binary field set to b'' will return None when read back,
handle that, then handle reading an empty CSV file, then handle the
resulting iterator having no lines whatsoever when trying to match
headers.
At least for CSVs, we're now properly telling users that their file
seems to have no content when they literally upload an empty file.
|
|
* if an encoding is explicitly specified, use it and don't guess
* otherwise guess and return the guessed encoding so it can be
displayed in the configuration UI
* fix less-than-stellar configuration & behaviour of select2 inputs
to properly reflect underlying values as they get modified, to
correctly handle future configuration guesses
|
|
* Handle a leading + in import data, some contexts (e.g. bank
statements) will mark positive sums explicitly for clarity
* Add basic grouping/decimal separator inference for people importing
data from many localisations or to avoid them *having* to configure
their separators if we can handle it for them, currently very basic
|
|
Task 32132
Currently, importing image requires either uploading them
one-by-one (after the initial import), scripting that upload, or
converting all images to base64 and putting that in the import
file (leading to a huge unreadable file) before actually doing the
import.
This is problematic for non-technical users, especially when they're
deploying or synchronising e-commerce products.
This change matches URLs in image fields and goes to read the URL &
load that as image data. It performs the read inline which is reactive
but means the imports will be very slow (wallclock) as each URL will
have to be fetched and encoded.
Configuration: via config file only
* import_image_timeout (seconds, default 3), initial connection timeout (not
overall file download)
* import_image_maxbytes (bytes, default 10MB): maximum image size to
download, checked before download (Content-Length) and validated
during (in case Content-Length is missing)
* import_image_regex (str), matches field content as "probably an
image", by default checks that the field value starts with
http|https and ends with an image-type extension (e.g. bmp, png, jpg)
Hard Limits:
* only works on fields actually called "icon", "image", "logo" or
"picture"
* the image can't be more than 42MP (checked after all data is
downloaded)
Hooks:
* _can_import_remote_urls (on res.users) checks whether the current
user is allowed to import via URLs
* _import_image_by_url (on base_import.import) performs the actual
download, could be overridden to store urls to import and
batch/queue the actual imports
|
|
Task 40692
Various changes to import/export (mainly) UIs:
* default to excel & "full" (non-import-compatible) export
* auto-detect encoding of CSV using chardet
* remember column -> field mapping after having imported a file (useful
for repeated imports where auto-matching failed)
* better handle localised booleans & column names
* automatically select source list view's fields when exporting
* better integrate import templates feature and add a number of templates
|
|
|
|
|
|
The commit 1ba4fbe640cf22abe did not match the specs of the task, but
was merged anyway. Reverting the commit, and check by default so the
user is not confused with missing fields.
opw-1824074
|
|
|
|
|
|
When a user imports a file with a datetime field set to a blank space,
the value is computed as 1900-01-01 00:00:00. However, when converted in
the user's TZ, this can lead to a value < 1900-01-01, which is not
recognized at the JS level as a valid value.
This makes the access to the record impossible, and possibly makes the
access to an app impossible. It is for example the case with Purchase,
where the order date is displayed on the default's action view (list
view).
We strip the value before testing if the value exists to avoid this
case.
opw-1826344
|
|
|
|
|
|
Empty cells containing only spaces may have unexpected results (not
shown traceback and broken import) when being imported and they were
intended for a float or monetary field.
To avoid the error we could either, ignore left/right whitespace when
parsing a float value (so same behavior as en emtpy cell) or prevent the
error and get an friendly import report error:
' ' does not seem to be a number for field 'Name of Field'
This change choose the first way, so a float ' ' (just whitespace) will
be considered as '' (empty).
opw-1816784
closes #23105
closes #23355
Co-authored-by: Humberto Arocha <hbto@vauxoo.com>
|
|
Simplify the implementation of parent_left/parent_right by removing this
optimization.
|
|
|
|
|
|
|
|
Typo in the currency removal method.
opw-787873
|
|
OPW 777366
|
|
|
|
|
|
Currently when importing files, the parent left and right are recomputed on each record creation/update, which could lead to a lot a request.
This commit will defer the parent field recomputation to the end on the `load` function by recomputing the values from the MPTT and inserting directly the new values on the records.
Tested on a customer instance, this will reduce the execution time when importing a thousand stock.location from 15 minutes to several seconds.
|
|
Use case to reproduce:
-> Import a file with a float using a coma related to the model
(for example the product.supplierinfo/price)
It happens because the method that parse the date and float field
only compare them to the main imported model.
This commit complete the method by using a recursion on relational
model in order to also parse the relational subfields.
|
|
|
|
|
|
|
|
Steps to reproduce the bug:
1. Import a bank statement with the shown dateformad DDMMYYYY (without headers)
2. Specify this date format in the advanced options
3. Try to find the date column to assign.
Bug:
The date format is not suggested for the fields with format DDMMYYYY.
opw:769118
|
|
|
|
|
|
|
|
- Import a CSV file without header
- Untick the option 'The first row contains the label of the column'
An error is reported.
If no header, the method `_match_headers` should return consistent
object types.
opw-769117
|
|
|
|
|
|
Now that we're closer to switching to P3 for good, these helpers have
outlived their usefulness, and mostly add noise.
All remaining dict.iter*() or dict.view*() must be converted to the
normal keys(), values() or items() calls.
Whenever the result is likely to be used for more than the scope of a
loop, or when the dict needs to be modified during iteration, the calls
must be wrapped in a ``list()``, to protect the new P3 semantics.
Those cases are very exceptional.
Also removed some dead code or improved the API to remove unnecessary
conversions.
|
|
|
|
|
