Android LEB Validation and Test Suite

Registered by Zach Pfeffer

This discussion will focus on LEB validation and test suites. Topics will include: Monkey, CTS, Monkeyrunner, unit-tests, continuous integration merge tests.

Discussion will include:
1. Now we have basic, monkey, 0xbench, cts, on the going, What more validation should we have for Android?
2. The requirement for validation framework on Android.

Possible Action Items from the discussion:

1. Integrate Monkey.

2. Integrate Monkey Runner and tests that use it.

3. Integrate tests from partners.

4. Create and integrate low-level tests.

5. Create and integrate native-level tests.

6. Create a list if things we need to validate and test.

7. Create a way to summarize and communicate test results across platforms.
7.1 Create a visualization mock-up for Paul.

8. Export tests for local execution.

9. Write tests using LTT and perf.
9.1 Integrate LTT on Android.

10. Run CTS
10.1 Break CTS up for debug and test parallelization.

11. Define and setup peripherals we will test with.

-Zach

Session notes: http://summit.ubuntu.com/uds-o/meeting/linaro-android-o-validation-and-testsuite/

Blueprint information

Status:
Not started
Approver:
Zach Pfeffer
Priority:
Essential
Drafter:
Jeremy Chang
Direction:
Needs approval
Assignee:
Jeremy Chang
Definition:
New
Series goal:
Accepted for 2011q2
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Overall Goal:
The goal of this Blueprint is to enhance validation for all builds. This includes adding more tests, making tests run more efficiently, communicating test results and making test modification easy. Linaro has an opportunity to shine with our validation offering as most community projects fail to run or document the quality of their builds.

Work items (month 1):
[pfefferz] Create a list of things we need to validat and test: TODO
[jeremychang] Integrate monkey into 0xbench: TODO
[jeremychang] Write a test that starts a browser, goes to www.google.com and quits using Monkeyrunner: TODO
[jeremychang] Integrate browser test into 0xbench: TODO
Write other monkeyrunner tests (many tasks - will expand): TODO

Work items (month 2):
Find and integrate tests from partners (many tasks - will expand): TODO
Create and integrate low-level tests (many tasks - will expand): TODO
Create and integrate native-level tests (many tasks - will expand): TODO
Create tool-chain tests: TODO
Integrate and run CTS: TODO
Break up CTS and identify parts that can run individually: TODO

Work items (month 3):
Create a test summary visualization mock-up for Paul: TODO
Document and enable all tests locally: TODO
Define peripherals we need to test with: TODO
Setup peripherals: TODO

Work items (month 4):
Integrate LTT into Android kernel: TODO
Adapt LTT client for Android: TODO
Integrate LTT into validation: TODO

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.