Browser based validation and benchmarks

Registered by Michael Hope on 2011-10-19

We'd like to use Firefox as a toolchain testsuite and a benchmark. Discuss approach, existing testsuites, and the parts to benchmark.

I would like to check if these workloads are the interesting ones?

    Load and rendering of a good set of web pages
    JavaScript
    Start up of browser
    Shutdown of browser
    Imaging - or are we covered by EEMBC DENBench, ConsumerBench

Input data that can be redistributed?
I would like to hear the views on this and understand how to proceed.

    Can Mozilla share the tp5 pages?
    Is it necessary that the content is redistributed, or can it be internal to the tcwg?

How to proceed with cross-compiling?

   I would like to use a multistrap/xdeb approach, but it seems hard.
   Is it OK to create a chroot with everything installed, zip it up and reuse it for some months?

Configuring the build for different flags and features

    Should we use .mozconfig, myRules.mk, and myConfig?
    If debian package is used for building, can myrules and myconfig still be used?

Which benchmark(s) to choose for browsing?

    ARMBBench
    EEMBC BrowsingBench
    Talos with a set of pages

    Would it be possible to integrate ARMBBench or EEMBC BrowsingBench in the Talos framework?

System level testing

    What are the plans for system level?
    Is there an overlap? Can we benfit, work together...
    When done, should the toolchain browsing benchmark tests be run in LAVA?

Blueprint information

Status:
Not started
Approver:
Michael Hope
Priority:
Medium
Drafter:
Michael Hope
Direction:
Approved
Assignee:
None
Definition:
Drafting
Series goal:
None
Implementation:
Informational Informational
Milestone target:
None

Sprints

Whiteboard

[2013-08-22 matthew-gretton-dann] This is a system level benchmarking activity and not really TCWG targetted at this time.

Agenda:
 * Workloads
 * Input data that can be redistributed
 * Cross-building to speed things up
 * Configuring the build for different flags and features
 * As a browsing benchmark
 * As a JavaScript benchmark
 * System level testing

(?)

Work Items