Integate interesting workloads in the PM QA testsuite to measure against

Registered by Amit Kucheria

Work with partners to consolidate a set of interesting workloads that are important to test energy consumption against.

Linaro's ability to improve power management is contingent on our ability to measure improvement (or degradation) on a regular basis. This blueprint suggests a number workloads for consideration.

The goal of the power measurement infrastructure is to measure relative performance between builds on each device: we are not planning to compare performance between devices. (Given the diversity of development board functionality, this is not really feasible in any case.)

Assumptions
 - Our initial focus is on mobile computing workloads. (We don't preclude future server-based workloads.)
 - Workloads should constructed as generically as possible, but, for the sake of expediency, we may initially focus work on a single target distro
 - Android's current popularity with OEMs makes it a candidate for that target distro
 - The Android LEBs appear to provide us suitable environments to get up and running quickly.

These assumptions may be revisited in future - howerver, the key here is to have a useful set of workloads integrated during the 11.11 sprint.

Environmental factors and device settings

Workloads may require a number of parameters. However, in order to reduce the complexity workloads, we recommend that parameters applicable to mulitiple workloads are broken out into 'environmental settings,' that may be configured by the automation environment.

These factors include:
 - Display brightness
 - Volume
 - Server bandwidth (e.g. to web / email servers)

We propose the following workloads as interesting:
 - Idle
    - Device in 'standby' (display off)
    - Idle at homescreen (no animated wallpaper)
    - Idle at homescreen (animated wallpaper)
 - Web browsing
    - iterate through a set of web pages, with suitable 'idle' time between pages
       o some pages may scroll during the idle time
       o content may be stored locally, or on a locally available server
       o server may be configured to limit bandwidth as an environmental setting
       o initial tests assume wifi or ethernet connectivity
          (WAN access is important, but difficult to set up inexpensively in a controled environment)
 - Email
    - Push email
    - IMAP client
 - Audio playback
    - low power local media playback
 - Video playback
    - local media playback: 'baseline' profile. (Expecting HW assist.)
 - Gaming
    - 'Casual' 2D gaming
    - 3D gaming / benchmark
 - Multitasking
    - e.g. local audio playback while web browsing

The following workloads are interesting, are not considered for this sprint:
 - Phone calls
 - Test messaging
 - Camera
 - Bluetooth
    - audio streaming
 - GPS-based services
    - mapping
    - route tracking
    - yelp / foursquare

Android applications and Android-specific infrastructure

While we try to maximize the portability of workloads, we have selected several Android applications in order to make progress quickly. They also reflect the popularity of Android in the marketplace, and are generally well-integrated with SoC features (such as accelerators) by SiPs / OEMs.

 - Android browser
 - Android email client
 - Media player
 - Photo viewer?

 - MonkeyRunner for UI automation
 - Remote display capture for headless systems

Blueprint information

Status:
Complete
Approver:
Amit Kucheria
Priority:
Undefined
Drafter:
Bobby Batacharia
Direction:
Needs approval
Assignee:
Amit Kucheria
Definition:
Obsolete
Series goal:
Accepted for trunk
Implementation:
Informational Informational
Milestone target:
None
Completed by
Amit Kucheria

Related branches

Sprints

Whiteboard

ARM's workload testsuite makes this obsolete.

(?)

Work Items

Work items:
[amitk] Dummy work item to show up in status.linaro.org: TODO

This blueprint contains Public information 
Everyone can see this information.