Merge lp:~openerp-dev/openobject-server/trunk-apiculture-rco into lp:openobject-server

[IMP] orm: improve str representation of model instances

[IMP] orm: support the creation of non-scoped records

* By default, method browse(ids) returns scoped records (backwards compatible
  behavior), while browse(ids, scoped=False) returns non-scoped records.
* The decorator @returns generates non-scoped records.

[IMP] ir.model.access: automatically invalidate the record caches when access rights are modified

[IMP] test_api: improve tests on scopes

[IMP] orm: better error handling in new code

[IMP] orm: when converting exception, attach existing traceback

[IMP] api: new record-map convention: map results into a list instead of a dict

[FIX] test_impex: change calls to name_get() to follow new record-map convention

[FIX] workflow: when evaluating a trigger expression, handle the case of a single output

[MERGE] from trunk

[IMP] orm tests: move test on related fields in openerp/tests/addons/test_fields

[IMP] openerp.osv.fields2: new module with new-style fields

[IMP] new fields: record.X and record['X'] are now equivalent

[IMP] orm: method read() now handles new-style fields
 - existing method read() has been renamed _read();
 - method _read() takes care of old-style fields only;
 - new higher-level read() handles both field styles.

[IMP] orm: add automatic recomputation of new-style function fields

[IMP] orm: improve methods invalidate_cache()

[IMP] orm tests: improve test on new-style fields

[IMP] new fields: improve field recomputation, and handle stored fields as well
 - redesign the recomputation algorithm such that it handles field dependencies
 - use it to compute fields for existing records when column is created in db

[FIX] orm: invalidate the whole cache upon record delete, because of ondelete='cascade', etc.

[IMP] orm: improve names of method that implement record.field, and check for deleted records

[IMP] new fields: add test for a regular field (no compute method)

[IMP] orm: internally recordsets use a tuple of ids instead of a list
 - this is only internal stuff, and does not break anything
 - this enforces the idea that a recordset is an immutable collection

[IMP] new fields: small improvement when deriving string attribute

[IMP] new fields: add test for compute method defined on recordset

[MERGE] from trunk

[FIX] new fields: in recompute_spec(), avoid returning useless specs (empty ids)

[FIX] orm: rewrite method read() to always check access rights before doing anything else

[IMP] orm: make method is_null() more robust

[IMP] new fields: add method _set_field() in BaseModel

[IMP] new fields: improve field copy, and fix check for non-attribute fields

[IMP] new fields: add new-style fields to interface all columns

[ADD] new fields: add boolean and float fields

[ADD] new fields: add char, text, html, date and datetime fields

[IMP] new fields: refactor methods from_column and to_column

[IMP] new fields: check value format when assigning a field

[ADD] new fields: add binary and selection fields

[IMP] new fields: small refactoring of field methods

[IMP] new fields: add missing option 'translate' on string fields

[IMP] orm: make null instances a subset of records

[IMP] new fields: fix recomputation (spec stored in thread-local instead of scope proxy), and improve tests

[ADD] new fields: add relational fields (many2one, one2many, many2many)

[ADD] new fields: add reference fields + fix small error in selection fields

[MERGE] from trunk

[IMP] orm: browse() always returns instances attached to a scope

[IMP] orm: refactor record and recordset instances creation
 - add explicit conversion method to_record(), to_recordset()
 - null(), record(), recordset() return instances attached to the current scope
 - consequently, all BaseModel instances (except the registry one) are attached
   to a scope
 - adapt existing code using those methods

[IMP] orm: recordset implementation now refers to records instead of ids

[IMP] openerp.osv.scope: improve api of scope proxy

[IMP] orm: method recordset() called without params returns an empty recordset

[MERGE] from trunk

[IMP] orm: remove instance property scope, and add method to attach instances to the current scope

[IMP] orm: in the record cache, store values at the record level

[IMP] orm, scope: change the implementation of the record cache
 - the cache is a dictionary like {model_name: {record_id: record, ...}, ...}
 - each record holds its own cache: record._data = {field_name: value, ...}
 - the code to get/set fields on records has been adapted

[FIX] new fields: attach model instances to the same scope as a record's cache

[IMP] new fields: improve test methods

[IMP] orm: small code improvements

[IMP] orm: avoid methods record() and recordset() looking up for records in the cache

[IMP] orm: make method to_record() more tolerant, and method one() to be strict

[FIX] orm: when assigning record field with setattr, call write() in record's scope

[IMP] orm: interface all _columns with new-style fields

[FIX] new fields tests: improve selection of partners when assigning parents!

[FIX] reports: fix method setLang() so that it 're-scopes' the objects in a clean way

[IMP] orm: rename instance attributes (_id -> _record_id, _data -> _record_cache)

[IMP] orm: improve error handling in method _get_field()

[IMP] orm, new fields: small code cleanup

[IMP] orm: introduce draft records

[IMP] orm: add pseudo-class Draft for draft records

[IMP] fields: make method read_to_cache() more robust

[IMP] orm: rewrite method default_get() in order to simplify refactoring with draft records

[IMP] orm: small improvements for draft records, and add __setitem__ method

[FIX] new fields: forgot method record_to_read() on one2many and many2many fields

[ADD] new fields: add related fields (non-stored ones)

[IMP] orm: interface inherited fields with new-style related fields
+fix the field 'date' in res.currency to avoid cache inconsistencies

[FIX] scope cache: never remove records from the cache, as it breaks an implementation property

This fixes a subtle issue that may happen after a cache invalidation. If you
remove a record from the scope cache, you may still refer and read that record,
but the values read from the database will be stored in another record instance!

In fact this issue did not cause a bug, because the record was "forced" into the
cache before fetching data from the database. But the "prefetching" effect (read
all records of the same model in the cache) was broken if you try to read other
records that were also removed from the cache!

[IMP] scope cache: fix the consistency check of the cache (because of the modified behavior of read())

[IMP] orm: code simplification, using the fact that all fields are interfaced with new-style fields

[IMP] fields2: determine inverse fields, and improve cache invalidation and recomputation

[IMP] orm: put all conversion method from/to cache in new fields classes

[IMP] orm: change implementation of draft records, add them a specific cache

[IMP] orm: reimplement default_get with draft records

[IMP] orm: optimize method add_default_value() by caching defaults from ir_values

[IMP] rename new API test module for clarity

[ADD] bunch of todos in doc (for both implementation and documentation of new API)

[ADD] start of onchange implementation

* fix _get_field for non-stored fields on draft records: computed
  fields will be stored in _record_draft, not _record_cache

* add handling of cache invalidation to draft fields: when setting a
  value on a field, invalidate the stuff which depends on it (nb: not
  currently recursive)

The current contract for Model.onchange is simple: give it the field
change which triggered the call and all values[0], onchange will build
a draft record with all values *but* the trigger in its cache (as if
they were defaults or had been read from the db) then set the changed
field via the normal UI.

Then iterate all fields and see what changed from the values provided,
this should yield only the fields which 1. were recomputed 2. to a
value different than the one they previously had.

[0] this may be a very, very bad idea for huge & complex objects full
    of o2ms & m2ms

[DOC] more todos

[FIX] forgot to add onchange tests

[FIX] apparently we still need to manually mark methods as api.record

[FIX] log_access fields

[REM] _protected attribute has been unused since revision 921 (2008)...

[FIX] m2o fields and conversion to columns

* Fix handling of ondelete when converting to a column

* Allow Field -> Column overwriting when the field was created
  explicitly (not field.interface), otherwise when overwriting a field
  in an _inherit the corresponding Column object is outdated and
  incorrect.

[FIX] correctly get default value from new-style fields when adding required attribute/constraint

[IMP] un-generalize getattr: since all _columns are also set up as fields, it's only a source of errors

in case of e.g. method name typos it'll go into _get_field then blow up

[FIX] fields_view_get for purely computed new-style fields

* generate a function column if store=False

* handle all setting on old-style (base and generated) fields

* slighly improve Field.to_column flow

* bypass db write on setattr if store=False

* remove more special handling of concurrency field (it's now a basic
  Field@store=False)

[IMP] fields2: add domain attribute to x2x.

[MERGE] orm: store model attributes shared by all instance in the class instead of the instances

[REF] orm: refactor the addition of a new-style field on a model's class

[IMP] orm: turn some internal methods into class methods, and make them private

[REF] orm: introduce 'id' as an explicit field (with special getter)

[IMP] enable debug mode post_mortem on CLI use of the server, not just rpc

[FIX] generalized symbol_* handling on function fields

[IMP] default values for ir_model datetime fields: use fields.datetime.now

[IMP] query building in create: string concatenation is 'orrible

[IMP] extract addition of defaults to a column so they stay in sync between new column and old-column-becoming-required

[IMP] creation of access fields & handling of concurrency computation methods, ensure access rights fields can't be written by a user

[IMP] remove creation of ir.ui.menu table in SQL, move security stuff around to get it to work correctly

[REM] useless creation of workflow tables in raw SQL

[IMP] orm: reimplement fields_get with new-style fields

[IMP] new fields: do not create columns for non-stored fields

[FIX] orm: duplicate model attributes like _columns to avoid sharing their default value!

[IMP] use unittest2.TestCase.addCleanup for rollback global alteration to registry/fields in tests

[IMP] scope: change the cache structure (to ease upcoming changes)

[IMP] orm: in draft records, use a sentinel for a missing value

[IMP] orm: remove attribute _record_cache from record instances

[IMP] ir_model: introduce right scope when creating instance in registry

[IMP] api: improve internal code, and rename decorators: @recordset -> @multi, @record -> @one

[IMP] api: improve code for extracting context from arguments

[IMP] orm: NEW DESIGN - now all model instances are recordsets

[FIX] service: fix test on model returned from registry

[MERGE] from trunk

[REV] revid:<email address hidden>
this changeset was not correct:
 - column wkf_instance.uid still used in openerp/workflow/instance.py
 - table wkf_witm_trans still used in openerp/workflow/workitem.py
one has to clean up the code first...

[FIX] test_impex: fix exports by reimplementing __export_row()

[MERGE] from trunk

[IMP] scope: do not use class Scope in non-ORM code

[IMP] openerp: improve imports for module code

[REM] orm: remove pseudo-classes Recordset, Record, Null

[IMP] cache invalidation: simplify the 'spec' parameter

[IMP] field recomputation: do not search for records in the case of non-stored fields

[FIX] scope: handle the case of calls like scope(cr, uid, None)

[IMP] orm: new implementation of draft records that fits better with the design

[MERGE] from trunk

[IMP] orm: reimplement new-style method onchange

[ADD] orm: union, intersection, difference of recordset + set comparison

[IMP] orm: test error cases for set operations

[IMP] orm: improve record cache management
 - better prefetching: browse() introduces cache entries for its records
 - quicker access: memoize access to the cache in model instances

[IMP] fields attribute cleanup: f.model, f.comodel and f.inverse_field are the objects instead of their names

[IMP] scope: improve the cache invalidation API

[REF] orm: improve management of recomputation of stored function fields
 - make fields manage their dependencies and recomputation triggers
 - introduce a global recomputation manager object (simpler than "specs")

[IMP] scope: simplification of API and implementation
 - scopes are no longer iterable, use scope.args to retrieve cr, uid, context
 - use named args when calling a scope: scope(user=42), scope(context={...})

[MERGE] from trunk

[IMP] orm, fields: small code improvements (avoid __class__ and such)

[IMP] new fields: when searching for records to recompute, make sure to retrieve all of them

[IMP] orm: add automatic cache invalidation for non-stored fields.function

[IMP] orm: delegate compute/read fields on fields themselves

[IMP] orm: change API of draft records

[REF] orm: improve the representation of model instances
 - each instance refers directly to its records' cache dictionaries
 - the 'id' of each record is in its cache dictionary (except new records)
 - VirtualID is no longer needed
 - the access to a record's cache is faster

[IMP] record cache: better implementation for cache invalidation
 - the record cache is no longer a built-in dict
 - the model cache keeps track of which fields are present in record caches
 - invalidating an already-invalidated field simply does nothing

[IMP] new fields: when dealing with modified fields, invalidate the cache in a more direct way

[IMP] scope: improve code for checking the cache

[IMP] new fields: add helper function to provide a constant default for a field

[IMP] orm: separate concepts of 'new' and 'draft' instances

[IMP] orm: fix the mess with 'id' in record cache

[IMP] scope: use better exception type when checking the cache

[IMP] fields: when computing the value of a field, do it for all records in cache

[IMP] orm: improve the recomputation manager to hide some hack

[IMP] record cache: small code cleanup

[FIX] orm: fix instance comparison, which was not independent from scope!

[IMP] record cache: add special values to missing default values, fields being computed, etc.

[IMP] new fields: make date/datetime conversions more robust

[FIX] orm: fix computation of field __last_update

[FIX] new fields: add missing method redirection in related fields

[IMP] new fields: refactor conversion methods to the cache level

[IMP] orm: small code cleanup

[ADD] new fields: add inverse computation of fields

[ADD] osv/cache: make the record cache read/write-through

[IMP] record cache: improve implementation, use named tuples for cache slots

[IMP] osv cache: improve documentation

[MERGE] from trunk

[ADD] new fields: add search on fields
 - add an option to rewrite a condition on a field
 - in expression.py, rename local variables to fit new terminology

[FIX] test_impex: make code consistent with values in selection function (ints not strings)

[FIX] test_workflow: use 'class' instead of type() to create model classes,
because type() registers the class under the wrong module name (openerp.osv.api)

[IMP] api: improve documentation of attributes set on methods

[ADD] orm: add new-style Python constraints with a decorator

[IMP] orm: fix and extend a bit the generic onchange method

[FIX] test_new_api: make test pass when test_new_api_extend is installed

[IMP] new fields: reimplement related fields as compute fields

[IMP] new fields: when setting up a related field, complete the setup of a field before getting its attributes

[MERGE] from trunk

[FIX] registry: buggy non-zero test on model

[IMP] orm: make result of onchange() backwards-compatible

[IMP] fields2: reorganize code, and ensure that _ready is reset upon copy

[IMP] base/ir.module.module: optimize state_update() (minimize calls to write)

[FIX] new fields: on related selection fields, the selection must be computed by the target field

[MERGE] from trunk

[IMP] fields2: remove pudb invocation

[FIX] scope: make scope() more robust when None is given as context

[FIX] orm: introduce scope explicitly in method scoped()

[FIX] fields2: bypass access rights to compute related fields

[FIX] fields2: catch exceptions when computing related fields

[FIX] orm: apparently, some code in mail needs explicit cache invalidation

[IMP] cache: simplify implementation of cache slots

[FIX] orm, registry: add hooks on models before and after a registry update
This fixes a bug where some new fields were keeping a reference to a model
that has been updated in the registry (Many2one's comodel, for instance).

[FIX] new fields: in *2many values, one may have a dict in the list of commands

[IMP] fields2: add attributes 'states' to new fields

[MERGE] from trunk

[IMP] orm: make read(), create() and write() tolerate unknown fields (with warning)

[IMP] orm: improve warning message when read/create/write with unknown fields

[MERGE] from trunk

[IMP] new fields: in case a field depends on '*', it should not depend on itself

[IMP] new fields: small code reorganization

[IMP] orm: add field display_name, and reimplement name_{get,create,search} in terms of it

[ADD] fields2: add option delegate=True in many2one fields to include field in dict _inherits

[IMP] orm: small simplification in method _set_magic_fields()

[MERGE] trunk-apiculture-snakecase-rco: use snake-casing for model names

[FIX] fields: fix _symbol_set for fields.model

[IMP] fields2: do not compute fields on non-existing records
 - This reduces the need to handle exceptions when computing fields.

[IMP] orm: make inverse and search on display_name more robust

[FIX] base: in ir_module_module, snakecase model names explicitly

[MERGE] from trunk

[FIX] openerp.tools: make snake_dict raise KeyError when looking up with a non-string

[IMP] api: reduce methods wrappers to a single layer; this eases debugging

[MERGE] from trunk

[FIX] test_workflow: remove unexpected argument that is mistakenly interpreted as a context

[FIX] orm: avoid doing None + list in name_search()

[FIX] ir_ui_view: snake-case model name in sql query

[IMP] fields2: small code cleanup

[IMP] openerp/osv/*: remove unused imports

[FIX] orm: make resolve_2many_commands() robust when argument is False

[FIX] orm, fields2: make field 'id' non-overridable
The implementation fields2.Id is critical to make "record.id" work properly.

[MERGE] from trunk

[IMP] fields2: cleanup of Field.interface

[IMP] fields2: improve code about digits in field Float

[IMP] fields, fields2: cleanup of code converting one to another

[IMP] fields2: uniformize internal API to export field attributes

[FIX] orm: in _add_missing_default_values, do not ask default values for magic fields

[IMP] openerp/tools: optimize lazy_property.reset_all()

[REVERT] snake-casing of model names: not worth the bugs and performance penalty

[MERGE] trunk-apiculture-stw: make logs from mail server less noisy

[FIX][TMP] Use a fields.char instead of the fields.function(selection) for the ir.ui.view#type field

[FIX] Authorize the modules to define new kind of views.

[IMP] ir_ui_view: improve method name

[MERGE] trunk-apiculture-recname: avoid non-null constraint if _rec_name is required

[FIX] ir_ui_menu: put the menu cache on the model's class, instead of an instance

[FIX] In the ORM, the functions who start with 'set_' are used by the
default_get method. But it's not the case for set_default_value_on_column.
We rename this method to avoid this mistake.

[FIX] ir_model_data: put the cache dictionary 'loads' on the class instead of an instance

[IMP] orm: refactor method create_instance()

[IMP] orm: remove method __getattr__ from BaseModel; it is bug-prone and not really useful

[IMP] scope: small code refactoring

[IMP] ir_sequence: small code cleanup to help API converters

[IMP] orm, scope: move the 'draft' flag on the scope instead of the records

[IMP] scope: draft mode is now global to all scopes

[FIX] openerp/report: use_global_header is a transient field on the ir.action.report.xml object. We use it to avoid a change in the API.

[FIX] test_mail: Rewrite the XSS test

[IMP] orm: simplify the records cache (remove slots)

[IMP] api, fields: @depends can now be given a function to compute dependencies
For field 'display_name', this optimizes triggers, since dependencies were given
as '*', which adds a trigger on *all* fields of the model!

[MERGE] from trunk

[MERGE] from trunk

[FIX] orm: insert the instance in the registry sooner (fixes missing fields in _inherits_reload())

[IMP] Add a new test for the inheritance computing.

[IMP] Add a test for the new api. This test will create a new model and checks
that we can add a new attribute on the instance of a model.

Sometimes, by example, in the report engine, we add a new attribute on the
instance. By the way, we can add this attribute without any write to the
database.

[FIX] Add a new attribue into the instance of the ir.actions.report.xml instance
because we need of a flag to use or not the right header in the report engine.

The previous implementation was wrong! Sorry

[REF] Rewrite the fields_get method of the ORM. Especially the part for the
translation. This new implementation is just a refactoring and does not change
the behavior the function.

[IMP] orm: improve __sub__ and __and__ on records, avoid comparing dict contents

[IMP] base: always use method read() with multiple ids (this is necessary to change its API)

[FIX] res_company: fix call to read()

[IMP] new fields: decorator @one enables recursive computation of fields

[IMP] new fields: enable recursive computation of stored fields, too

[IMP] orm: reimplement method read() using the new API
 - call chain:
    record.read() uses new API to get and convert values
    -> field.__get__() retrieves value from cache or compute it
    -> field.determine_value() determines whether to compute or read value
    -> record._prefetch_field() determines which records and fields to read
    -> record._read_into_cache() actually reads data and store it in cache
 - exceptions are stored as values in cache, and raised upon reading cache
 - added a new exception for missing records
 - ISSUE: we cannot reproduce the behavior of read() with a single id; it now
   returns a list of dicts instead of a dict

[FIX] base: The default value for the name of a currency rate is now a
field.datetime, and thus in this case, we have to use the default value for this
kind of a field.

[FIX] openerp/osv/orm: Check if the fields parameter of the read method is not
None or False. In this case, the method will use the _fields attribute of the
current object. Otherwise, this code will crash because a boolean is not
iterable.

[IMP] scope: optimize API of scope.invalidate(), resulting in 5% to 10% speedup
 - every call to proxy.invalidate iterates over all scopes (which is costly to access)
 - the new api groups many calls into a single one, which leads to less access to all_scopes

[IMP] orm: rename _read_cache into _get_cache to make it less confusing

[IMP] orm: change structure of the cache, resulting in 5-10% speedup
 - new cache is indexed by field object, then by record id
 - cache access is a bit slower, but invalidation is way faster

[IMP] orm: private method _update_cache now updates all records in self

[REF] fields2: generalize FailedValues to special values stored in the cache
 - SpecialValue encapsulates null values in new records missing default values
 - FailedValue encapsulates exceptions for various kinds of failures to report
 - These special values transmit information from the point where a field value
   is determined to the point where the cache is actually read.

[FIX] fields2: undefined variable

[FIX] base: missing dependency on computed field

[MERGE] from trunk

[IMP] scope: rename method SUDO() into sudo()

[FIX] new fields: fix scope mess in related fields, and evaluate name_get() in sudo mode in convert_to_read()

[FIX] fields2: add cache converter on text fields, and remove it from binary fields
 - this fixes two broken tests in test_impex
 - change type of field 'domain' on 'ir.rule' to 'binary' (value is a Python list)

[FIX] fields: add missing comma in tuple

[FIX] orm: method name_get() must evaluate display_name in the current scope, not self's own scope

[REF] orm: make AccessError and MissingError children classes of except_orm
Why? Because some modules handle access right errors by catching except_orm!

[FIX] orm: before getting records in cache, always make sure the record to read is in cache!

[IMP] orm: improve method check_field_access_rights()

[FIX] fields2: when setting related fields, do not assign null records

[IMP] orm: add properties _cr, _uid, _context on recordsets (bw compatibility w/ browse records)

[IMP] Keep the backward compatibility with the old version of OpenERP

[FIX] Pass the rights parameter to the right method in the wrapper

[MERGE] from trunk

[IMP] orm: simplify the query for reading 'classic_write' columns

[FIX] On OSX, the system has a lot of Bitmap fonts, and in this case, Reportlab
can not load the 'head' table from the structure of the TTF file. There is no
good way to check if a TTF file is an old style or new style.

[FIX] orm: add access rights check for fields in read()

[IMP] api: add method decorators 'old' and 'new' to manually define api implementations
This allows to remove the hack that handles the case of method 'read': its old-style
implementation is given explicitly.

[FIX] fields: decorate functions used in fields (selection, domain) to make them callable with both apis

[IMP] api: make scope getter function more specific

[MERGE] from trunk

[MERGE] from trunk

[FIX] fields: add api wrapper for selection function in function fields

[IMP] orm: improve code of method create()

[MERGE] from trunk

[IMP] orm: modify the chain of calls for reading records such that overridden read() is taken into account
 - motivation: if method read() is overridden, field.__get__() uses it!
 - records.read() calls records._read_from_database() to store values in cache,
   then it retrieves values from the cache (included computed fields)
 - field.__get__() retrieves stored fields by calling record._prefetch_field(),
   which calls records.read() for stored fields only (no call loop!)

[FIX] orm: in _read_from_database(), avoid getting post fields with empty ids list

[FIX] scope: make the draft switch more robust in case of exceptions

[IMP] api: make the decorators 'old' and 'new' more intuitive

[IMP] orm scope: use a weak dictionary to store the set of scopes