Improving crossplatform capabilities of current UI testing
With the need of doing UI testing in UE, we are looking for a set of existing test cases that can be used for that, unity test suite is an ideal candidate as a starting point.
Blueprint information
- Status:
- Complete
- Approver:
- Pete Graner
- Priority:
- Essential
- Drafter:
- Gema Gomez
- Direction:
- Needs approval
- Assignee:
- Canonical Platform QA Team
- Definition:
- Approved
- Series goal:
- Accepted for raring
- Implementation:
-
Implemented
- Milestone target:
- None
- Started by
- Gema Gomez
- Completed by
- Gema Gomez
Whiteboard
There are two aspects of Autopilot, one is the tool and another is the current unity test cases. UE would like to reuse the existing test cases and use them in an integration testing manner. For this, the test cases will need to be reliable crossplatform, so it needs to be discussed if changes will be required for this to happen.
There are two aspects of Autopilot, one is the tool and another is the current unity test cases.
UE would like to reuse the existing test cases and use them in an integration testing manner.
For this, the test cases will need to be reliable crossplatform, so it needs to be discussed if changes will be required for this to happen.
Current status of the UI (unity) testing
358 test cases, they run daily+ on three different HW flavours (different graphic chipsets)
Problems identified (if any!)
- The failure rate is very high (25%-30% fail)
* Some fail due to genuine bugs (we want to keep these)
* some of this problems are hidden to users
* Some of the issues are timing related
e.g. User pushing super key, verify the dash opens
eventually feature to tackle the issue, but still not reliable
* little test isolation between tests (some things happen after the test has finished and cause trouble to the next test)
* Poorly written test cases
99% of the work should be unit tests
Solutions
Unity tests are to be fixed in the coming 3 weeks. Goal is to use autopilot results as release criteria for unity
* unit tests for code checkins
* autopilot tests for submissions to -proposed
Autopilot + Xpresser can be used together
Profiling support for compiz to be added
Documentation
http://
Work Items
Work items for ubuntu-13.04:
[mrazik] to create a team of autopilot reviewers: DONE
[gema] Add a new BP for unity testability: POSTPONED
[thomir-
[nuclearbob] Make sure utah works with the autopilot tests: POSTPONED
[mrazik] To keep the platform QA team in the loop of the fixes, so that we know what to expect: DONE
[thomir-
