Automated Server Testing
Session for discussion about automated testing of Ubuntu Server; automated ISO testing was implemented for Maverick using libvirt/kvm + Hudson (see http://
Blueprint information
- Status:
- Complete
- Approver:
- Robbie Williamson
- Priority:
- Essential
- Drafter:
- James Page
- Direction:
- Approved
- Assignee:
- James Page
- Definition:
- Approved
- Series goal:
- Accepted for natty
- Implementation:
-
Implemented
- Milestone target:
-
ubuntu-11.04-beta-2
- Started by
- James Page
- Completed by
- James Page
Whiteboard
General Update:
* Alpha 2 work items now complete
* Testing packages can be found in ppa:ubuntu-
* -desktop testing using livecd has been added by jibel.
* Packages have been refactored to provide support for this expanded capability.
Content from ideas pool:
* Testing Automation (through Hudson)
* Automated Server ISO Testing (using ubuntu-
* Automated EC2 AMI Testing using existing framework + Hudson
* Automated package testing using puppet, kvm etc... for UEC, Hadoop and any other moderately complex stack.
* Continuous performance non-regression testing
Topics for discussion:
* Automated Server ISO Testing (http://
* Achievements to-date
* Automation of ISO testing in Maverick.
* Control through ec2 hosted Hudson. (http://
* Current challenges:
* Test execution overhead and concurrency.
* Lack of dedicated hardware.
* Current state of project
* Not packaged for Ubuntu so not that accessible
* Codebase needs tidy + full documentation
* Next Steps:
* General tidyup of project and codebase
* Packaging for Ubuntu to ease adoption
* Release 1.0, comms to potential adopters to provide physical resources.
* Automated EC2 AMI Testing using existing framework + Hudson
* Automated package testing using puppet, kvm etc... for UEC, Hadoop and any other moderately complex stack.
* Continuous performance non-regression testing
Work items for natty-alpha-2:
[hggdh2] Way forward on production deployment of ISO testing: DONE
[james-page] Move Server ISO tests to normal PXE + TFTP instead for broader fit with potential test architectures: DONE
[james-page] Server ISO test - review what the iso overlay looks like and refactor as required: DONE
[james-page] Package ubuntu-
Work items for natty-alpha-3:
[james-page] Automate EC2 testing and increase depth of image testing using unittest/subunit: DONE
[james-page] Server ISO test - fix concurrency in ISO download: DONE
[cr3] Output plugin for checkbox to write to couchdb: POSTPONED
[cr3] Checkbox plugin to download tests from couchdb to execute: POSTPONED
[james-page] Move Server ISO tests to checkbox and integrate with guest: POSTPONED
[james-page] proof of concept for complex package testing - use case openldap/mysql: POSTPONED
Work items for ubuntu-
[james-page] Test Jenkins automated ec2 tests: DONE
[james-page] Find a long term home for ec2 automated tests: DONE
Work items for ubuntu-
[james-page] Update ubuntu-
[james-page] Move to exectuing tests via SSH rather than cloud-init: DONE
[james-page] Rework Jenkins jobs to use Matrix Projects: DONE
