use ipython's distributed processing features in xpaxs
In the past we have used parallelpython for distributed multiprocessing on an SMP-capable computer and/or over a network. PP had some nice features like automatic discovery of PP servers, and automatic generation of simple statistical reports on jobs. A big problem with this approach is that we had to pass a function to the PP jobserver to be remotely executed, which required that all of the supporting information like modules and data had to be transmitted over the network. The distributed capability of xpaxs currently appears to be bandwidth limited. Perhaps our use of PP could be tweaked in order to improve performance. Another alternative is to use ipython's new support for distributed computing. Ipython-0.9.1 appears to be extremely flexible and supports more varieties of distributed computing. The distributed capabilities in xpaxs were abstracted such that investigating using ipython should not be disruptive. The documentation for ipythons distributed features can be found at http://
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Darren Dale
- Direction:
- Needs approval
- Assignee:
- XPaXS Development Team
- Definition:
- New
- Series goal:
- None
- Implementation:
- Unknown
- Milestone target:
- None
- Started by
- Completed by