Benchmark Dalvik, Hard-Float OpenJDK for ARM and JAVA SE with SciMark

Registered by Zach Pfeffer on 2012-09-25

Why?
Benchmarking SciMark with the hard-float OpenJDK for ARM and Dalvik on the same platform is a means for approximating the amount of headroom available to optimize in Dalivk.

Context?
This is part of the larger ARM benchmarking effort and to inform a possible Dalvik optimization effort.

What gets produced?
A set of measurements on a Wiki, documentation for producing those measurements and a script to automate the measurements.

 Where will the work get put?
Th scripts will be put in git and the docs on the Wiki or Google Doc.

Blueprint information

Status:
Complete
Approver:
Zach Pfeffer
Priority:
High
Drafter:
vishal
Direction:
Approved
Assignee:
vishal
Definition:
Approved
Series goal:
Accepted for 2012q3
Implementation:
Implemented
Milestone target:
milestone icon 12.10
Started by
vishal on 2012-10-03
Completed by
Zach Pfeffer on 2012-10-22

Related branches

Sprints

Whiteboard

Notes:
[2012/9/25 pfefferz] Please put notes here.
[2012/10/03 vishalbhoj] We can run the scimark2 jar on android after converting it to dex format.
[2012/10/04 vishalbhoj] Dalvik seems to beat OpenJDK comfortably (both in server configuration and client configuration) by 1.5x to 2x better performance by Dalvik and that is because OpenJDK is running in interpreter mode. I tried to use CACAO JVM (which has someJIT support) which crashed on OpenJDK 6 and it doesn't seem to be supported on OpenJDK 7 yet.
[2012/10/22 vishabhoj] Here are the results of the profiling for speed:
From current analysis this is what I see on Panda :

For OpenJDK Interpreter mode :
Composite Score: 36.30357859627802
FFT (1024): 24.552936620511193
SOR (100x100): 76.22448400556964
Monte Carlo : 9.044321200804992
Sparse matmult (N=1000, nz=5000): 33.921325587575936
LU (100x100): 37.77482556692837

On Dalvik with JIT:
Composite Score: 59.69456554120022
FFT (1024): 38.754014007974014
SOR (100x100): 147.5011473414838
Monte Carlo : 11.748750919975837
Sparse matmult (N=1000, nz=5000): 37.118259792851305
LU (100x100): 63.350655643716145

java SE Development Kit 7u6 :
Server Configuration:
Composite Score: 128.53424774750374
FFT (1024): 105.63472204418967
SOR (100x100): 231.49386412669065
Monte Carlo : 79.82023302651025
Sparse matmult (N=1000, nz=5000): 90.59441865315316
LU (100x100): 135.12800088697497

Client Configuration:
Composite Score: 108.99643713221958
FFT (1024): 104.67118856922178
SOR (100x100): 197.1107818625708
Monte Carlo : 29.266840166767814
Sparse matmult (N=1000, nz=5000): 78.42987125282515
LU (100x100): 135.50350380971238

Meta:
Headline: SciMark has been benchmarked against Dalvik and Hard-Float OpenJDK
Acceptance:
1. SciMark has been benchmarked against Dalvik
2. SciMark has been benchmarked against Hard Float OpenJDK
3. Scripts have been written that run the benchmarks
4. Results have been published

(?)

Work Items

Work items:
Run SciMark into Android builds: DONE
Benchmark SciMark against Dalvik for speed: DONE
Benchmark SciMark against Hard Float OpenJDK for ARM on Ubuntu for speed: DONE
Benchmark Scimark against java SE Development Kit 7u6 : DONE
Benchmark SciMark against Hard Float OpenJDK for ARM on Ubuntu for start-up time in long-time-to-warm-up-but-fastest mode (server): DONE
Benchmark SciMark against Hard Float OpenJDK for ARM on Ubuntu for start-up time in quickly-warms-but-tops-out mode (desktop): DONE

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.