use ipython's distributed processing features in xpaxs

Registered by Darren Dale

In the past we have used parallelpython for distributed multiprocessing on an SMP-capable computer and/or over a network. PP had some nice features like automatic discovery of PP servers, and automatic generation of simple statistical reports on jobs. A big problem with this approach is that we had to pass a function to the PP jobserver to be remotely executed, which required that all of the supporting information like modules and data had to be transmitted over the network. The distributed capability of xpaxs currently appears to be bandwidth limited. Perhaps our use of PP could be tweaked in order to improve performance. Another alternative is to use ipython's new support for distributed computing. Ipython-0.9.1 appears to be extremely flexible and supports more varieties of distributed computing. The distributed capabilities in xpaxs were abstracted such that investigating using ipython should not be disruptive. The documentation for ipythons distributed features can be found at http://ipython.scipy.org/doc/rel-0.9.1/html/parallel/index.html

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
Darren Dale
Direction:
Needs approval
Assignee:
XPaXS Development Team
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.