Manually Characterize Android Benchmarks with DS-5

Registered by Zach Pfeffer on 2012-09-04

Why?
We need to characterize the Android benchmarks for hotspots.

Context?
This is part of the larger ARM benchmarking effort.

What gets produced?
A report detailing where each benchmark spends its time, a how-to on reproducing the report including a DS-5 set-up and installation guide and a set of scripts that can run the charaterizations on a local computer.

Where will the work get put?
The wiki for the how-to and the report and android.git.linaro.org for the scripts.

Blueprint information

Status:
Complete
Approver:
Zach Pfeffer
Priority:
High
Drafter:
vishal
Direction:
Approved
Assignee:
vishal
Definition:
Approved
Series goal:
Accepted for 2012q3
Implementation:
Implemented
Milestone target:
milestone icon 12.09
Started by
vishal on 2012-09-07
Completed by
vishal on 2012-09-28

Related branches

Sprints

Whiteboard

Notes:
[2012/9/4 pfefferz] Please put notes here.
[2012/9/17 vishalbhoj] We are currently documenting the characterization in Google doc . It will be published in wiki once it is reviewed. Here is the link to the document:
https://docs.google.com/a/linaro.org/document/d/1YqCUeZe--VWe3dPM5wbIfLnAw7ZIECZEw7H3LxQPaLg/edit
[2012/9/24 pfefferz] Will close out by Wednesday.
[2012/9/28 vishalbhoj] All the benchmarks have been characterized. The data has been shared with ARM for review. Depending on the inputs we will refine the results. Automation of the streamline capture is being captured under this BP :
https://blueprints.launchpad.net/linaro-android/+spec/automate-streamline-with-monkey-runner
[2012/9/28 vishalbhoj] Currently perf seems to be broken in our builds hence we cannot use perf for profiling currently

Meta:
Headline: Android benchmarks have been characterized and the method to reproduce them has been documented.
Acceptance: A report detailing where each benchmark spends its time, a how-to on reproducing the report including a DS-5 set-up and installation guide and a set of scripts that can run the charaterizations on a local computer.

(?)

Work Items

Work items:
Install DS-5 and document: DONE
Create a test methodology and document the methodology to characterize each benchmark: DONE
Manually characterize each benchmark and record and share the data: DONE
Characterize AndeBench : DONE
Characterize Linpack: DONE
Characterize CaffeineMark: DONE
Document AndeBench characterization (put link to doc in notes): DONE
Document CaffeineMark characterization: DONE
Document LinPack characterization: DONE
Characterize Quadrant and Document the results : DONE
Characterize Vellamo and Document the results: DONE
Characterize nBench and Document the results: DONE
Characterize GeekBench2 and Document the results: DONE
Characterize GLBenchmark and Document the results: DONE
Review of the characterization by ARM benchmarking team: DONE
Investigate use of perf for hotspot analysis: DONE

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.