Performance & convenience improvements to integration tests, part1
Summary
=======
As integration tests are now stable enough, the next step is to make them pass faster and their internal architecture to be less redundant.
Motivation
=========
While the number of tests is relatively small, the major part of time spent running dsvm-integration job is due to preparing devstack environment. As number of tests grow (and hopefully, it will), the fraction of time spent actually running tests will increase and then the performance issues will arise. Let's fix some of these issues before they hit us.
Also there is a number of items (such as simplifying selectors) eluded from the scope of previous blueprint, integration-
Description
=========
1. Every table wrapper currently holds both selector and table name. Table selector could be easily calculated using table name, so 2 should be reduced to 1.
2. [MOVED OUT OF THE SCOPE OF THIS BLUEPRINT] The constructors for almost every region and control element in testing Page Objects are put inside method marked with @property decorator, which means that every time you access that element or a region, its constructor is called again, which may take a significant time for complex regions including many nested elements. Caching the results of such constructors would save us a lot of time running tests. On the other hand, in some cases (especially, if some fields appear only after providing an input in previous fields) it may be desired to re-run constructors, so the caching feature should be switchable (by means of config). Also, the proposed approach is to first add the caching facility and then start adding it gradually to different categories of properties.
3. Table columns are matched by their index. This will require to change tests when additional columns are inserted into the beginning of a table. To avoid this, columns should be matched by their id/name and not by index.
4. Existing Page Objects have a very serious flaw: many forms/views that appear on them as a result of some actions being initiated by user/test are put inside @property and thus seem to be accessible all the time. This creates false expectations for the test writers who will be using these Page Objects and makes the code using these @properties quite fragile (as the success of using such properties depends on some previous actions being done). Instead of this we need to return such Page Regions as a result of table/row actions and never put them inside Page Object properties. To achieve this goal, current implementation of actions in integration tests has to be heavily reworked. As a final outcome we'll have each action defined explicitly as a method (so we could put the logic for creation new Page Regions into it) and being tied to an HTML control element by means of @decorators.
5. For every passing test we should also ensure in test that an expected success message was shown and no error messages were shown. This check could be combined with expected messages dismissing in order to make tests pass faster (no need to wait until the message is auto-dismissed).
6. Since submit.click() is replaced with submit() action, it seems natural to put waiting for spinner to vanish (which always happens after submitting a form) into this action, thus reducing the amount of boilerplate .wait_for_
UX
===
The end user experience should be unaffected.
Wireframes, Mocks, Videos and UI Markup
-------
N/A
Testing
======
N/A
Outside Dependencies
==================
N/A
Requirements Update Required
=======
N/A
Doc Impact
=========
The changes will be reflected in an upcoming documentation for integration tests writers. Actually, the necessity to document some of idiosyncrasies of existing tests led me to an idea of rewriting the tests machinery (easier to fix it than to explain it).
Blueprint information
- Status:
- Complete
- Approver:
- David Lyle
- Priority:
- Undefined
- Drafter:
- Timur Sufiev
- Direction:
- Approved
- Assignee:
- Timur Sufiev
- Definition:
- Approved
- Series goal:
- None
- Implementation:
- Implemented
- Milestone target:
- None
- Started by
- Timur Sufiev
- Completed by
- Timur Sufiev
Related branches
Related bugs
Sprints
Whiteboard
Gerrit topic: https:/
Addressed by: https:/
Move integration_
Addressed by: https:/
Simplify tables page regions
Addressed by: https:/
Add @cached_property decorator to integration tests
Addressed by: https:/
In integration tests reference table columns by name
Addressed by: https:/
Rework table actions in integration tests
Addressed by: https:/
Reduce number of calls to Selenium for form fields
Addressed by: https:/
Improve working with messages in integration tests
Addressed by: https:/
Don't wait for spinner to disappear explicitly in i9n tests
Addressed by: https:/
Remove redundant dashboard.login_url setting from horizon.conf
Addressed by: https:/
Introduce field mappings for FormRegion descendants in i9n tests
I'm moving (2) @cached_property out of the scope of this blueprint, because possible performance gain is not clear, while complexity is obviously increased.