Improvements to lava-dispatcher test reporting
It's often useful to have lava-dispatcher itself run tests and report results. For example, when running tests involving multiple systems, it's natural that the coordinator (lava-dispatcher) would monitor for pass or fail conditions rather than attempt peer to peer communication to verify overall pass/fail.
This blueprint adds the ability for lava-dispatcher to instruct an action to generate test results, including pass/fail of multiple test cases, console logs of the activity during the test, etc.
It also adds the ability to conveniently report the results of these tests. In addition to the normal reporting to the dashboard via xmlrpc, human readable text and JUnit style XML results can be generated.
Blueprint information
- Status:
- Complete
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Jason Hobbs
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Obsolete
- Series goal:
- Accepted for linaro-11.11
- Implementation:
-
Beta Available
- Milestone target:
-
backlog
- Started by
- Jason Hobbs
- Completed by
- Neil Williams
Whiteboard
User Stories:
* As a tester, I want to make assertions about the behavior of actions run in lava-dispatcher, and report the results as normal test results.
* As a tester, I would like the option for lava-dispatcher to report results as a JUnit format XML file for consumption by a continuous integration system.
* As a tester, I want to be able to run tests on the command line using lava-dispatcher, and see the results of the test on the command line in a clearly human readable format.
Work Items
Work items:
* Add a test_action decorator to make Action's behave like test cases: DONE
* Let test_action decorated Actions have pass/fail patterns to match in their serial log: DONE
* Add console logging of test results: DONE
* Add JUnit XML logging of test results: DONE
