Improving QA for seeded server packages

Registered by James Page on 2012-10-09

Rationale:

By improving the basic level of testing that can be applied to all packages in the server supported seeds, we decrease the amount of manual testing effort required during the development release and to support stable release updates.

This reduces the risk of changes impacting basic functionality of supported packages.

Goal:

DEP-8 tests for all supported server packages.

UTAH tests using DEP-8 tests for testing.

Blueprint information

Status:
Not started
Approver:
Dave Walker
Priority:
Medium
Drafter:
Ubuntu Server Team
Direction:
Approved
Assignee:
James Page
Definition:
Approved
Series goal:
Proposed for saucy
Implementation:
Deferred
Milestone target:
milestone icon ubuntu-13.10

Related branches

Whiteboard

User Stories:

Michael is preparing a SRU for haproxy in raring; he's able to confidently test this package prior to upload because it contains a DEP-8 test which means that even though he does not know the tool that well, it passes a basic level of smoke testing.

James is able to have confidence that during the development cycle, the server supported packages continue to function due to automated testing of all packages in the server supported seeds on a daily basis and on upload to the archive.

Risks:

Test coverage of packages does not cover all of the supported seeds.

Test Plans:

Kinda implicit in this BP.

Release Note:

N/A

(?)

Work Items

Work items for ubuntu-13.08:
[yolanda.robla] write dep-8 test for ganglia: DONE
[yolanda.robla] write dep-8 test for iscsitarget: DONE
[yolanda.robla] write dep-8 test for nagios3: DONE
[yolanda.robla] write dep-8 test for open-iscsi: DONE
[yolanda.robla] write dep-8 test for quagga: DONE
[yolanda.robla] write dep-8 test for radvd: DONE
[yolanda.robla] write dep-8 test for tgt: DONE
[james-page] write dep-8 test for openvswitch: DONE
[james-page] Update https://wiki.ubuntu.com/QATeam/RequiredTests where appropriate: DONE
[yolanda.robla] write dep-8 test for clamav: DONE
[yolanda.robla] write dep-8 test for exim4: DONE
[yolanda.robla] write dep-8 test for mailman: DONE
[yolanda.robla] write dep-8 test for memcached: DONE
[yolanda.robla] write dep-8 test for squid3: DONE
[yolanda.robla] write dep-8 test for varnish: DONE
[serge-hallyn] write dep-8 test for lxc: DONE
[yolanda.robla] write dep-8 test for amavisd-new: DONE
[yolanda.robla] write dep-8 test for nut: DONE
[yolanda.robla] write dep-8 test for postfix: DONE
[yolanda.robla] write dep-8 test for slapd/openldap: DONE
[yolanda.robla] write dep-8 test for spamassassin: DONE
[yolanda.robla] write dep-8 test for freeradius: DONE
[yolanda.robla] write dep-8 tests for nova: DONE
[yolanda.robla] write dep-8 tests for keystone: DONE
[yolanda.robla] write dep-8 tests for quantum: DONE
[yolanda.robla] write dep-8 tests for horizon: DONE
[yolanda.robla] write dep-8 tests for glance: DONE
[yolanda.robla] write dep-8 tests for python-novaclient: DONE
[yolanda.robla] write dep-8 tests for python-keystoneclient: DONE
[yolanda.robla] write dep-8 tests for heat: DONE
[yolanda.robla] write dep-8 tests for python-quantumclient: DONE
[yolanda.robla] write dep-8 tests for python-glanceclient: DONE
[yolanda.robla] write dep-8 tests for python-ceilometerclient: DONE
[yolanda.robla] write dep-8 tests for python-cinderclient: DONE
[yolanda.robla] write dep-8 tests for python-swiftclient: DONE
[yolanda.robla] write dep-8 tests for cinder: DONE
[yolanda.robla] write dep-8 tests for swift: DONE
Get all tests automated by QA team: DONE

Work items:
[andreserl] write dep-8 test for heartbeat/corosync/pacemaker: TODO
[andreserl] write dep-8 test for tftpd-hpa: TODO
[gandelman-a] write dep-8 test for haproxy: TODO
[gandelman-a] write dep-8 tests puppet + facter: TODO
[ivoks] write dep-8 test for bacula: TODO
[serge-hallyn] write dep-8 test for kvm: TODO
[pcarrier] write dep-8 test for snmpd: TODO
[zulcss] write dep-8 test for rabbitmq-server: TODO
examine testing status for libvirt: TODO
[yolanda.robla] write dep-8 tests for ceilometer: DONE
[jibel] Report for DEP-8 test failures in Ubuntu Server package set: TODO
[yolanda.robla] improve quantum to test plugins: TODO
[racb] write dep-8 test for vsftpd: TODO
review backuppc for continued seed inclusion: TODO
review likewise-open for continued seed inclusion: TODO
[pcarrier] write dep-8 test for munin: TODO
[james-page] Write some guidelines for test authors so we get things right first time: TODO
[james-page] write dep-8 test for solr-tomcat/jetty: TODO
[matsubara] write dep-8 test for jenkins: TODO

Dependency tree

* Blueprints in grey have been implemented.