LDS - PM Functional testing and energy measurement for interesting workloads

Registered by Mounir Bsaibes

* Automated Functional Test
    - Populate PM QA test suite with more functional tests; track reports
* Energy measurement Lab
    - Setup lab to track interesting work loads and the energy consumed when running those work loads
* Reporting
    - Work with Validation team to get required reporting

ARM: Energy measurement infrastructure

Topology

ARM's infrastructure requires the provisioning of a PC 'host.' The host will co-ordinate activity between target devices and measurement equipment, and collate information for reporting. Both Linux and Windows hosts are supported; it is envisaged that Linaro will prefer to use Linux! Connectivity between host and measurement equipment will typically be via USB: we will need to address host scalability appropriately. Connectivity between host and target will be over Ethernet and/or wifi: we assume both are available in the test lab.

Measurement equipment and configuration

For power measurement, we are using Data Aquisition (DAQ) devices from National Instruments. These are available at a number of price points, supporting a variety of aquisition channels, accuracies and sampling rates. Examples of appropriate DAQ's are: http://sine.ni.com/nips/cds/view/p/lang/en/nid/203224 - USB-6211, or http://sine.ni.com/nips/cds/view/p/lang/en/nid/209074 - USB-6363.

ARM is also prototyping a low-cost energy meter solution. We will endevour to make these available to Linaro during this cycle. However, we recommend Linaro purchase at least one DAQ for initial work. The cost for switching from one to another is low: both devices require boards to be instrumented identically.

The ARM SW Infrastructure supports both the NI DAQ and the ARM hardware using their native interfaces.

We envisage that power may be measured at one or more power supplies for each board. Minimally, each board will be instrumented to measure the whole board's supply. Depending on the facilities of the board, this may be by using (or placing) a small shunt resistor on the board itself, or by placing a small resistor in-line with the power supply. Some development boards are designed with probe points for multiple supplies: for instance, they may allow CPU or GPU power supplies to be measured independently, depending on SoC internals.

The infrastructure is built to be flexible to accomodate various sampling rates and shunt resistor values. ARM will advise Linaro of the sampling rates and resistor values that we recommend.

Licencing and availability

The licencing and distribution mechanism for the ARM measurement SW infrastructure is TBC.
 - Current thinking is ARM-hosted GIT, code supplied under Apache, but this may change.

Availability is TBC
 - We intend to drop early, however this depends on some legal gates

Workloads

Thoughts on workloads are captured in the blueprint https://blueprints.launchpad.net/linaro-pm-wg/+spec/measure-integrate-important-workloads-in-testsuite

Blueprint information

Status:
Complete
Approver:
None
Priority:
Undefined
Drafter:
Olivier Cozette
Direction:
Needs approval
Assignee:
Olivier Cozette
Definition:
New
Series goal:
None
Implementation:
Implemented
Milestone target:
None
Started by
Amit Kucheria
Completed by
Amit Kucheria

Related branches

Sprints

Whiteboard

DISCUSSION
 - goal
    o measure improvement / degradation often
    o measurement on android
 - Design targets
    - host OS
    - Limited external dependencies.
    - Low host/target overhead: Fast
 - Design details
 - Workloads (blueprint)
 - HW integration (blueprint)
 - Quick demo (if there's time)

POWER MEASUREMENT
Linaro's ability to improve power management is contingent on our ability to measure improvement (or degradation) on a regular basis.

ARM's infrastructure requires the provisioning of a PC 'host.' The host will co-ordinate activity between target devices and measurement equipment, and collate information for reporting. Both Linux and Windows hosts are supported; it is envisaged that Linaro will prefer to use Linux! Connectivity between host and measurement equipment will typically be via USB: we will need to address host scalability appropriately. Connectivity between host and target will be over Ethernet and/or wifi: we assume both are available in the test lab.

MEASUREMENT SERVER:
- could be Linux/Windows or both.
- NI-DAQ driver currently tested for Windows. In theory this will port easily to Linux, but we're currently working through installation issues.
- Some space on your file systems to store the logs (1GB is more than enough).
- No USB/serial port debug stuff is needed.
- We have also integrated this infrastructure with the prototype ARM energy meter, under Linux. However, this interface will continue to be refined.

CONNECTION BETWEEN BOARD AND SERVER:
- TCP/IP
- Any Ethernet interface is appropriate, but we note that the knock-on energy impact will need to be evaluated for both wired and 802.11.

MEASUREMENT TARGET:
- Android 2.2 or higher
- You would install an apk named PowerMeasurementClient (the name could change) on Android.
- At least one shunt on the power supply to get the power usage (current/voltage), but something in the board is better.

PACKAGING/DEPENDENCIES
Currently there is no .deb, the code is mostly C++ with a part of java for Android.
The current dependencies are the following:
- Linux host binaries depends on:
 - libc, pthread and libstd++.
 - If you use a NI DAQ: DAQmx software (DAQmx802f0.iso/LabView libraries).
 - No need to install anything else.
- Windows host binaries depends on:
 - MSVCRT/ws2_32/mswsock/NIDAQmx on WIN32 (only requested for NIDAQmx).
 - It needs to install the NI libraries.
- Android board: binaries depends on no extra library.
 - It needs a kernel built with ftrace enable to get ftrace data (optional).
 - It use process/task/mount/window maneager/internet permission (internet is the minimum).
- Linux host build: build depends on:
 - Android NDK
 - Android SDK
 - If you use a NI DAQ: DAQmx software (DAQmx802f0.iso/LabView libraries).
 - libc, pthread, make, bash, grep, sed.

LICENSING/DISTRIBUTION
The licensing and distribution mechanism for the ARM measurement SW infrastructure is TBC.
 - Current thinking is ARM-hosted GIT, code supplied under Apache, but this may change.

Availability is TBC
 - We intend to drop early & often, however this depends on a legal gate for the initial drop.

REPORTING
Average/Min/Max power consumption for a application like web browsing.
Power consumption drawing will be added later:
 - Off line power consumption drawing will be added later, it will export the data in graphics application friendly (like gnuplot or more complex tools).
 - On line power consumption drawing will use python for drawing graphics.

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.