Benchmarking OpenGL ES 2.0

Registered by Jesse Barker on 2011-04-20

his session will discuss general approaches and techniques for benchmarking and profiling the use of the OpenGL ES 2.0 API. Particular attention will be given to enhancements and extensions to the glmark2 utility (https://launchpad.net/glmark2), including feedback to library, toolkit and application projects to yield optimizations based upon results.

Blueprint information

Status:
Complete
Approver:
Jesse Barker
Priority:
High
Drafter:
None
Direction:
Approved
Assignee:
Alexandros Frantzis
Definition:
Approved
Series goal:
Proposed for trunk
Implementation:
Informational Informational
Milestone target:
None
Started by
Alexandros Frantzis
Completed by
Alexandros Frantzis

Related branches

Sprints

Whiteboard

Summit session notes:

* glmark2 tries to be windowing system agnostic. Do we need to change that?
 * Can be complicated... see glcompbench.

* glmark2 vs spandex vs glmemperf
 * Spandex offers very focused low-level tests (test exactly one thing)
 * glmark2 tests are somewhat more high-level (test one thing but in the context of a more real-life scenario).
 * glmemperf tests texture upload performance
 * Split tests depending on focus? low-level => spandex, higher-level => glmark2 ?

* Command line interface for test selection etc

* Synthetic tests
 * Shadershttp://summit.ubuntu.com/uds-o/meeting/linaro-graphics-o-glmark2/
  * Targeted
   * Conditionals
   * loops
   * Function calls
   * variable precision
   * uniform upload
  * Common shaders
   * Lighting
   * Effects
 * Data upload
  * VBO vs client arrays
  * MapBuffer vs BufferData
 * Comparative/Informational tests
  * Many ways to perform ops in GL (VBO vs arrays, MapBuffer vs BufferData, different shader strategies)
  * Answers developer question: "What should I use to get the best performance on ARM based hardware"
  * Could lead to rule-of-thumb advice (eg for up to 1000 vertices it is faster to use BufferData on most ARM based gfx)
  * Leverage tests mentioned above by making them parametrical.
  * Do we want such tests?
  * Power efficiency?
  * glFinish instead of eglSwapBuffer

* Real-life GL tests
 * nux
 * Problem: lack of free-as-in-freedom GLES2.0 content
 * Games are a good choice (but missing for GLES2.0)
 * Use traces?
 * "Humble Bundle" games? http://www.humblebundle.com/
 * "Tea Worlds? http://www.teeworlds.com/
 * Panda3D
 * Shader evaluation from http://www.iquilezles.org/apps/shadertoy (WebGL) (contact developer).
 * 3D screen savers (good for test cases and general use).

Tools for debugging and for benchmarking:
  * Spandex : http://www.gitorious.org/spandex/pages/Home
  * ApiTrace : http://zrusin.blogspot.com/2011/04/apitrace.html
  * Tracy : https://gitorious.org/tracy
(the last 2 can be used to generate c files for a specific part, then this can be used for benchmarking)

Benchmarking use cases:
  * Performance
  * Power efficiency

Benchmarking GPU:
How we can compare GPU (triangles are no more meaningful)

Work items (tentative):
  * [ACTION] Talk to validation team about getting test boards wired up to power measurement gear.
  * [ACTION] Implement switch in glmark2 to replace eglSwapBuffer with glFinish.
  * [ACTION] Enumerate new shader cases to add (e.g., additional lighting models, nux shader effects).
  * [ACTION] Make packages for Tracy available on working group PPA.
  * [ACTION] Implement shader test to evaluate behavior of conditionals (branching).
  * [ACTION] Identify list of 3D screen savers to port both for test cases

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.