Improvements to lava-dispatcher test reporting

Registered by Jason Hobbs on 2011-09-29

It's often useful to have lava-dispatcher itself run tests and report results. For example, when running tests involving multiple systems, it's natural that the coordinator (lava-dispatcher) would monitor for pass or fail conditions rather than attempt peer to peer communication to verify overall pass/fail.

This blueprint adds the ability for lava-dispatcher to instruct an action to generate test results, including pass/fail of multiple test cases, console logs of the activity during the test, etc.

It also adds the ability to conveniently report the results of these tests. In addition to the normal reporting to the dashboard via xmlrpc, human readable text and JUnit style XML results can be generated.

Blueprint information

Status:
Complete
Approver:
None
Priority:
Undefined
Drafter:
Jason Hobbs
Direction:
Needs approval
Assignee:
None
Definition:
Obsolete
Series goal:
Accepted for linaro-11.11
Implementation:
Beta Available
Milestone target:
milestone icon backlog
Started by
Jason Hobbs on 2011-09-29
Completed by
Neil Williams on 2014-07-02

Sprints

Whiteboard

User Stories:
* As a tester, I want to make assertions about the behavior of actions run in lava-dispatcher, and report the results as normal test results.
* As a tester, I would like the option for lava-dispatcher to report results as a JUnit format XML file for consumption by a continuous integration system.
* As a tester, I want to be able to run tests on the command line using lava-dispatcher, and see the results of the test on the command line in a clearly human readable format.

(?)

Work Items

Work items:
* Add a test_action decorator to make Action's behave like test cases: DONE
* Let test_action decorated Actions have pass/fail patterns to match in their serial log: DONE
* Add console logging of test results: DONE
* Add JUnit XML logging of test results: DONE

This blueprint contains Public information 
Everyone can see this information.