Independent Test suite Builder
Lot of Test suite are being developed in Linaro for Android for the members only access. Currently these test suites are compiled as part of android sources making it difficult to build them independently and distribute them to members. The goal with the test-suite-builder is :
1. Build the tests independent of Android source.
2. Should support build with Makefiles and Android NDK.
3. The artifacts should be available to be downloaded and added as an overlay to system image.
4. The test-suite-builder should be able to patch the system tarball with the artifacts and submit them to LAVA for testing.
Blueprint information
- Status:
- Complete
- Approver:
- vishal
- Priority:
- Essential
- Drafter:
- Axel Fagerstedt
- Direction:
- Approved
- Assignee:
- Axel Fagerstedt
- Definition:
- Approved
- Series goal:
- None
- Implementation:
-
Implemented
- Milestone target:
-
13.04
- Started by
- Axel Fagerstedt
- Completed by
- Fathi Boudra
Related branches
Related bugs
Sprints
Whiteboard
[20130422 fagerstedt-axel]
Build now includes prebuilt binaries.
bL-iks and sched tests are currently the same job, need to split them.
[20130415 fagerstedt-axel]
Android build script available on Launchpad branch.
Builds tests and produces test artifacts + test and android combined artifacts.
Next steps:
Create shell script to mirror functionality of sched tests makefile.
Verify in Lava.
Handle feedback from Naresh.
[20130408 fagerstedt-axel]
Local setup which can build both test suites from a unified makefile setup.
Next step will be moving on to android-build script.
[20130421 nkambo] you may include pre-build binaries in test git repo as android-
[20130421 nkambo] Testing update:
https:/
above build tested on my local TC2 and found below observations.
1. This build comes with bL MP image. bL MP scheduler test suite successful executed.
2. bL IKS core test suite can be executed on bL MP kernel, so I have replaced bL MP with BL IKS kernel (which i built). Executed bL IKS core test suite on bL IKS kernel and found ffpmeg binary did not found so all tests related to ffmpeg tests falied. The perf command related test cases failed. due to the fact that the tests use a bL IKS customized perf command.
Summary ..
Total Tests = 30
Tests Passed = 20
Tests Failed = 10 (4 ffpmeg tests + 6 perf test cases)
[2013/04/22 vishalbhoj] We can't provide the perf binary since it needs to be compiled from kernel sources. ffmpeg binary needs to be checked into the test suite for deploying.
[2013/04/24 vishalbhoj] WIll handle the support for restricted build and some more cleanup in the next cycle in a different BP.
Issue identified;
bL-iks tests are currently not buildeble with NDK due to symbols missing in the NDK libc.
possible solutions:
Make a linaro NDK with the symbols and use that
build bL-iks against glibc and link statically?
Meta:
Roadmap id: CARD 484 and CARD485
Headline: Independent test suites can be built on Android builds.
Acceptance:
1. Build the tests independent of Android source.
2. Should support build with Makefiles and Android NDK.
3. The artifacts should be available to be downloaded and added as an overlay to system image.
4. The test-suite-builder should be able to patch the system tarball with the artifacts and submit them to LAVA for testing.
Work Items
Work items:
Decide on spec to provide the info on the output/artifacts of the test suite: DONE
Decide on overlay layout: DONE
Extend android-build to support independent test suites: DONE
Update b.L IKS test suite to be built independently: DONE
update b.L MP test suite to be built independently: DONE
[fboudra] Copy release builds on releases.
