Testsuites and Profilers

Registered by James Westby on 2010-04-27

Investigate, decide on, and prioritize the testsuites and profiling tools that would be most useful to run on ARM devices in an automated testing environment.

Blueprint information

Status:
Started
Approver:
Scott Bambrough
Priority:
Medium
Drafter:
Paul Larson
Direction:
Needs approval
Assignee:
Paul Larson
Definition:
Approved
Series goal:
Accepted for maverick
Implementation:
Started
Milestone target:
None
Started by
Jamie Bennett on 2010-07-08

Related branches

Sprints

Whiteboard

[asac Jun 10]: plars, could you do some milestone planning for the blueprint and the work items to make this even better?
[pwlars 08 Jul 10]: milestones set, but starting this has been delayed due to focus on the automated testing framework blueprint
[pwlars 14 Jul 10]: I did a run of LTP syscalls today, a few expected failures exist. Some are due to environment issues, some may need some fixing on ARM. The entire run of syscalls only, which would make a useful test, took about 20 min. Also, I've got it defined in Abrek to install and run from there, results parsing will come later after that code is checked in.
[pwlars 12 Aug 10] - openposix: looks useful, runs in a reasonable amount of time, but will need swapspace on small memory devices. Looked at openmoko tests, they have some decent tests but are mostly device specific for neo, not much that I think we can use from there. Looked at Ofono, there are some tests, but due to the state of ofono development and support in images, I think it's best to wait on automation efforts here. If ofono shows up in images, it should go through some manual testing until things stabilize, then automation should be revisited.

Status:
[pwlars 29 Sep 10] - openposix, gmpbench, and others have been done for abrek, more on the way. Will continue to keep phoronix open as more tests are investigated. Many of the autobench tests seem to be dups of other tests we're seeing in phoronix or other places, may revisit again later if needed.

Work Items (maverick-alpha-3):
[pwlars] Establish LTP set of tests to run: DONE
[pwlars] Baseline results and time for LTP: DONE

Work items (ubuntu-10.10-beta):
[pwlars] Investigate Openmoko Testsuite: DONE
[pwlars] Investigate OpenPosix Testsuite: DONE
[pwlars] Investigate Ofono Testsuite: DONE
[dave-martin-arm] Feedback on any testsuites or benchmarks ARM has which are relevant (conclusion - nothing significant available): DONE
[dave-martin-arm] Feedback on additional ARM sanity checks we might want to create (conclusion - nothing significant): DONE

Work items (ubuntu-10.10):
[pwlars] Baseline results and time Phoronix: INPROGRESS
[pwlars] Establish set of Phoronix tests to run: TODO
[deeptik] Add a test definition file for gmpbench to Abrek: DONE
[deeptik] Add a test definition file for OpenPosix to Abrek: DONE
[pwlars] Investigate autobench tests: DONE

(?)

Work Items