Simplification of Checkbox with standalone scripts and test plans
Checkbox is too complicated. It has a plugable architecture that is flexible in some ways at the cost of being difficult to maintain. We want to simplify the architecture so that it is flexible in ways that matter to users while coping more easily changes.
First, we should move the flow of the application from the prompt plugins to the frontend process. This change would make the user interface extensible independently from the backend. As a result, this would make it trivial to run a single test after completing a test run.
Second, we should move the responsibility of the remaining plugins to standalone scripts. The command to call the scripts would be configurable to remain flexible. For example, this could be the command to run tests in random order:
checkbox-runner $CHECKBOX_
| checkbox-transform --blacklist=
| checkbox-random
Last, we should introduce the concept of a test plan to invalidate forking Checkbox with custom tests. A test plan would have the sorted jobs from a whitelist with their scripts, so any version of checkbox with this support would run the same tests the same way.
Blueprint information
- Status:
- Not started
- Approver:
- Ara Pulido
- Priority:
- Undefined
- Drafter:
- Marc Tardif
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- New
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
- None
- Started by
- Completed by
Whiteboard
Definition of done:
Replace the current message store with a single result file: TODO
Develop scripts to generate the submission.xml file and upload it: TODO
Develop scripts to generate a test plan and read it from command line: TODO
Define a clear D-Bus API to replace the current user interface plugins: TODO
Use the D-Bus API to implement a command line interface: TODO
ZK: I think that DBus is not a part of this story. Once we have a plugin-less API we can expose it over dbus when the need arises. IMHO adding dbus as a requirement now will complicate this task. Update: after discussing with cr3 I'd like to rephrase that to: "we should not spend time on stabilizing the current existing dbus api". The rationale for that is that we may be not needing dbus in the end and that marking current dbus interface as internal is sufficient. Originally dbus was introduced to accomodate for lack of pyqt on the CD (which is no longer the case), to support the resulting native C++ Qt GUI client. It is my consistent opinion that adding dbus where it is not _required_ is a bad thing as it largely increases complexity. If we come to a point where we can rewrite the UI in pyqt and use the api directly then we may be able to drop the dbus interface altogether.
