Implementation of bl-mp-extended test case scenarios

[nkambo Jan 30 2012]

Test case description is available in below Doc:
https://docs.google.com/a/linaro.org/spreadsheet/ccc?key=0Ai9ggMs8bsGJdDh6QTA5NjlHb1ZaU21QVVdWN0F4M0E#gid=1

Test results spreadsheet available from the below location:
https://docs.google.com/a/linaro.org/spreadsheet/ccc?key=0Ai9ggMs8bsGJdER3azBqWGtXNjRwb1UzSnZ4RGg3OUE#gid=5

big.LITTLE MP scheduler test results have been published in dev-linaro list.

work items have been started and initial test execution is completed on Week-4.
most of the test development is completed, now we are focusing on stabilization of build setup and results. this will be done in the next cycle.

[basil-eljuse 25 Feb 2013]
With the introduction to global load balance functionality there are a number of tests which are deprecated from extended suite as against initial specification. Replacement tests are also planned to be added taking into account global load balance capability.
Latest version of test suite shared do have the updated set of tests which have the tests deprecated. The replacement tests considering global load balance functionality is expected in the subsequent release.
Updated the blueprint based on request from nkambo

[nkambo 13 March 2013]
Latest version of test suite contains mpextended test suite.
these tests have been executing and publishing results every week on linaro-dev list.

[nkambo 13 March 2013]
testing mpextended test suite with multiple configurations plan needs to be discussed with bL MP team.

[nkambo 13 March 2013]
IRC Discussion:
<nkambo> amitk, vireshk, do we have any plan to test benchmark with multiple configs like enable/disable RCU, NO_HZ, THUMB2, FTRACE,..etc ?
<nkambo> amitk, vireshk, i would like to take your opinion on this.
<amitk> nkambo: not for regular testing. These will be benchmarked by the developers who will then recommend the optimal configs to platform team
<nkambo> amitk, ok.. only one defconfig for testing.
<amitk> nkambo: right
<amitk> nkambo: though you are welcome to do your experiments and share the results if you strongly feel that some Kconfig option will improve the numbers.
<amitk> nkambo: that kind of analysis is welcome

[nkambo 13 March 2013]
from this blueprint only one item testing with multiple configs is pending. this will be taken care in March cycle.
so status for blueprint changed to beta-available.

[nkambo 31 March 2013]
ARM proposed configuration which needs to be enable/disabled to get best performance/power.
which needs to tested on Linaro Golden board to validate these configs.
https://git.linaro.org/gitweb?p=landing-teams/working/arm/kernel.git;a=blob;f=linaro/configs/vexpress-tuning.conf;h=7bdf2732c927f41b4167cfec62e89bfe5491a200;hb=ee88877401caedd2e339e95d07745676506acb94

[nkambo 24 April 2013]
config fragments are identified as below and workload automation testing performed and results have been shared.
linaro/configs/linaro-base.conf
linaro/configs/android.conf
linaro/configs/big-LITTLE-MP.conf
linaro/configs/vexpress.conf
linaro/configs/vexpress-tuning.conf

Testing workload automation with these configs will be continued.

Meta:
Headline: Implementation of bl-mp-extended test case scenarios
Acceptance:
1. Test cases from extended test suite is able to be executed and arrive at the pass fail verdict.
2. The test cases should be able to execute on various configs listed in the test approach like CPUidle/CPUfreq/CPUhotplug being enabled or disabled.
Roadmap id: BIG_LITTLE_MP

Deliverable :
scheduler test suite sub modules like mpbasic, mpcore, mpexetended and mploadbalance arrived to pass fail verdict.
tests are executing on defined config fragments.

Work items:
Develop extended test cases under scheduler test suite: DONE
extended test case should produce pass fail verdict: DONE
Test extended test cases on big.LITTLLE MP Android image: DONE
Run test cases on various kernel configs like CPUidle/CPUfreq/CPUhotplug: DONE
Test these test cases on TC2 platform with Android image: DONE