Infrastructure for Running Long Tests

Registered by Lars Butler on 2012-03-07

It has become apparent that we would benefit from having testing infrastructure in place for long-running OpenQuake engine jobs. We think that if large jobs are continuously run against the latest oq-engine code we can better test for performance and scalability, as well as catch job configuration corner case issues.

Adding more CI-like infrastructure is one way to solve this problem.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Medium
Drafter:
Lars Butler
Direction:
Needs approval
Assignee:
None
Definition:
Discussion
Series goal:
None
Implementation:
Unknown
Milestone target:
milestone icon 0.7.0

Whiteboard

Use Jenkins (or similar) to execute long-running QA tests on the open-quake engine and related projects.

Some defects are identified only with larger sets then can be conveniently executed in a short time on a development workstation. In order to identify such problems we should execute long running tests periodically/continuously.

It seems likely that a dedicated server will be required in order to prevent the standard Jenkins builds from blocking or failing due to lack of computing resources.

The defect identified 2012-03-22 installing version 0.6.0 on the cluster suggests that a performing tests periodically in a distributed environment would be beneficial in detecting some kinds of problem.

The suggested approach would be to turn up another Linode, in the London datacenter where the existing CI server is located. The 2nd CI configuration would have a full OpenQuake install, but would only act as a worker with the Primary CI server acting as the Control Node. This will emulate our cluster environment.
Total effort to create this solution would be in the range of 3 days effort.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.