Integate interesting workloads in the PM QA testsuite to measure against
Work with partners to consolidate a set of interesting workloads that are important to test energy consumption against.
Linaro's ability to improve power management is contingent on our ability to measure improvement (or degradation) on a regular basis. This blueprint suggests a number workloads for consideration.
The goal of the power measurement infrastructure is to measure relative performance between builds on each device: we are not planning to compare performance between devices. (Given the diversity of development board functionality, this is not really feasible in any case.)
Assumptions
- Our initial focus is on mobile computing workloads. (We don't preclude future server-based workloads.)
- Workloads should constructed as generically as possible, but, for the sake of expediency, we may initially focus work on a single target distro
- Android's current popularity with OEMs makes it a candidate for that target distro
- The Android LEBs appear to provide us suitable environments to get up and running quickly.
These assumptions may be revisited in future - howerver, the key here is to have a useful set of workloads integrated during the 11.11 sprint.
Environmental factors and device settings
Workloads may require a number of parameters. However, in order to reduce the complexity workloads, we recommend that parameters applicable to mulitiple workloads are broken out into 'environmental settings,' that may be configured by the automation environment.
These factors include:
- Display brightness
- Volume
- Server bandwidth (e.g. to web / email servers)
We propose the following workloads as interesting:
- Idle
- Device in 'standby' (display off)
- Idle at homescreen (no animated wallpaper)
- Idle at homescreen (animated wallpaper)
- Web browsing
- iterate through a set of web pages, with suitable 'idle' time between pages
o some pages may scroll during the idle time
o content may be stored locally, or on a locally available server
o server may be configured to limit bandwidth as an environmental setting
o initial tests assume wifi or ethernet connectivity
(WAN access is important, but difficult to set up inexpensively in a controled environment)
- Email
- Push email
- IMAP client
- Audio playback
- low power local media playback
- Video playback
- local media playback: 'baseline' profile. (Expecting HW assist.)
- Gaming
- 'Casual' 2D gaming
- 3D gaming / benchmark
- Multitasking
- e.g. local audio playback while web browsing
The following workloads are interesting, are not considered for this sprint:
- Phone calls
- Test messaging
- Camera
- Bluetooth
- audio streaming
- GPS-based services
- mapping
- route tracking
- yelp / foursquare
Android applications and Android-specific infrastructure
While we try to maximize the portability of workloads, we have selected several Android applications in order to make progress quickly. They also reflect the popularity of Android in the marketplace, and are generally well-integrated with SoC features (such as accelerators) by SiPs / OEMs.
- Android browser
- Android email client
- Media player
- Photo viewer?
- MonkeyRunner for UI automation
- Remote display capture for headless systems
Blueprint information
- Status:
- Complete
- Approver:
- Amit Kucheria
- Priority:
- Undefined
- Drafter:
- Bobby Batacharia
- Direction:
- Needs approval
- Assignee:
- Amit Kucheria
- Definition:
- Obsolete
- Series goal:
- Accepted for trunk
- Implementation:
- Informational
- Milestone target:
- None
- Started by
- Completed by
- Amit Kucheria
Related branches
Related bugs
Sprints
Whiteboard
ARM's workload testsuite makes this obsolete.
Work Items
Work items:
[amitk] Dummy work item to show up in status.linaro.org: TODO