Detector code cleanup

Registered by Michael Lange

Detector features such as Lagrangian advection and distribution of detectors in parallel are currently all held in the diagnostic_variables module, which is primarily intended to do I/O. In order to enable future extensions of the detector code we need to clean up the module structure and unify the underlying API for handling detector lists. This includes: separating modules for parallel distribution, lagrangian advection and the two implemented algorithms as well as re-writing parts of the detector_tools module. This change should have minimal impact on functionality, but will enable other modules to add and maintain separate detector lists while re-using existing code and improve the overall maintainability of the detector code.

Blueprint information

Not started
Michael Lange
Needs approval
Michael Lange
Series goal:
Milestone target:



Key changes:

 * Removed Bisection method, making the Runge-Kutta Guided Search the default algorithm for
   lagrangian detector advection. Therefore, Fluidity will now complain and exit if it finds lagrangian
   detectors, but no associated timestepping options.

 * In parallel each detector now should always be held on the processor that owns the detector's element, ie.
    they should not reside in a halo. The only exemption to this rule is within the RK-GS sub-timesteps, in order
    to avoid unnecessary sends. However, after the timestep is completed the detector will be send.
    The reason for this is that it will make integration with Zoltan easier and more performance efficient.

 * Detector initialisation: All code paths initialising detectors (single/array/checkpoint) now use the same
   subroutine to create the detector, which in parallel first checks whether we would own the according
   element, and only then allocates the detector. Previously we over-initialised and then deleted detectors.

 * Modularisation: A new module detector_parallel now provides the routine exchange_detectors(), which
    serialises and sends detectors point-to-point during the advection routine. This module also provide
    distribute_detector(), which checks that all local detectors actually reside in a local element, and sends
    them if necessary. This routine is invoked after the lagrangian advection.

 * Modularisation: detector_tools now provides a cleaner interface for detector list handling:
    - remove (no deallocation)
    - delete and delete_all (with deallocation)
    - move and move_all for moving detectors to a different list
    - pack and unpack: Serialisation routines that are used by Zoltan and the lagrangian advection.
      The routines assert that the given buffer is of the right size in the right context, since we need to send
      more information during advection than during the adapt. The reason for this is that we now have a
      single point-of-entry for extending the detectors themselves.

 * detector_linked_list type now also holds meta-information, such as the names of fields to output,
    I/O parameters and parameters for the RK-GS advection (stage_matrix, timestep_weights, etc).
    This effectively allows other modules than diagnostic_variables to use and maintain detector lists,
    since all detector routines outside of diagnostic_variables work on generic detector_linked_list types.

 * Zoltan integration: Changed the way we search for detectors to send with an element. By using a
    buffer detector list per element-to-transfer we only need to traverse each local detector once
    in the search loop, and can thus handle multiple detector lists.

 * Testing: Added the analytical 2D solution for 100 lagrangian detectors to detectors_parallel_adaptive.

 * Testing: Added a test to check from_file detector initialisation to binary_detectors.

 * Schema: Added Forward Euler and RK4 as pre-defined options

 * Note: The code is now capable of handling multiple detector lists. However, detectors defined under
    the /io/detectors option (used in all tests) are all still held in one list. True multi-list behaviour is
    tested in a separate branch, because it requires schema changes and is part of a new feature.


Work Items

This blueprint contains Public information 
Everyone can see this information.


No subscribers.