Testing with the ubuntu sdk

Registered by Nicholas Skaggs

As an application developer targetting the ubuntu touch platform, testing plays a key role in your development needs. Let's talk about what tools are availible to you as an application developer and how you can use them to help deliver a well tested application. In addition, let's gather feedback on your needs, and if they are being met.

Blueprint information

Not started
Jono Bacon
Needs approval
Series goal:
Milestone target:

Related branches




Testing needs:
unit testing
Functional / end user testing

Comments on tools
Qt Creator
Ubuntu SDK Emulator
as sdk updates, we can update the module and not be forced to update each individual test
Work in progress, but is being used no

Qml Testing
better suited to unit testing
much faster
no need to start full application or dbus
can easily verify application behaves as you expect
Anything that is self-contained makes sense for qml
should run everywhere qt can run
supports c++ compiled code, etc
qml uses the toolkit directly to automate, while autopilot uses the dispay server, signals, etc
changes to sdk potentially have a more limited impact
emulators less needed
less shared code
more specific tests
unity8 has collection of helper functions
could be upstreamed


We need a way to tell when a test should be written in autopilot, and when it should be written in qml or c.
discuss about this after the training session

We need a suite for the sdk user-facing features. What we currently have is a suite for the emulators and a suite for the showcase.
can we review coverage of sdk?
no tests that are interactive

For the tests written in autopilot, we need a better way to tell if we are testing the installed modules or the modules from source code.
if you have installed modules as well as the source code, can be confusing and caused incorrect results
testrunner could run "testing" version from upstream source
hardcoded paths don't seem to be the answer
clean qmlscene code found in __init__.py

We need to add a base test case to ubuntu-ui-toolkit that's inherited by all the core app suites. That way we will be able to change things in the base test case, instead of changing them in all the apps.

 we need templates/better templates in qtcreator for everything
 autopilot test template
 qml testing tempate?
 not worth implementing?
 autopilot test with emulator template? We need first the base test case.

the ubuntu sdk emulator for autopilot could include verbose logging for each action and call


Work Items

Work items:
[nskaggs] Update the QML testing tutorial: TODO
[nskaggs] IRC training session for QML testing (coordinate as part of showdown?): TODO
[nskaggs] create qml testing video: TODO
[nskaggs] get with ubuntu sdk team (juhapekka?) to review sdk testing coverage; are all components well covered? [juhapekka maybe?]: TODO
[elopio] clean up qmlscene installed vs local checks: TODO
[elopio] write a base test case to be used by all the apps: TODO
[mzanetti] expand base template with simple qml test: TODO
[elopio] propose an annotation for logging emulator actions.: TODO

This blueprint contains Public information 
Everyone can see this information.