Task progress

Execution progress for task

Motivations

Currently taskflows provide information only about task state (task finished successfully or failed), but there are many situations when this is not enough. Even in the current cinder code some the of tasks perform operations that take a long time. So the idea is to provide standard mechanism for getting progress of this operations - tasks in a given taskflow. And eventually, the overall progress of the entire taskflow.

Implementation

Progress for task can be implemented by adding the following attributes to the task class:

    start_value (default is 0.0) defines initial value for task progress;
    end_value (default is 1.0) defines termination value for task progress;
    current_value defines current value of progress between start_value and end_value.

These attributes must be stored in the task flow engine, to be available through the taskflow API.

Also required is a callback function that will be used to periodically update progress (update_progress) value associated with an active task. This function gets invoked during the task execution and updates the current task progress.

For example (this would be Cinder example), let's say there is a taskflow that back up a given volume. This flow executes tasks that use volume driver code to perform a sequence of required actions.

In particular, this "backup" task flow will run the following tasks:

One of this tasks can be:

    1. create backup for volume;
    2. download backup file to cinder node;
    3. upload backup file to backup service.

Each of these operations (tasks) can take a long time, and so the suggested mechanism will provide users information on the current progress and an estimate of time it will take for a task to complete.

The update_progress() callback function can be passed as an additional parameter to the corresponding driver code, to keep updating the task's progress value based on exact (driver's) knowledge on how many bytes were already transferred.

JH: What exactly will be persisted when this occurs, will the partial status be persisted (?), one of the key principles/goals of taskflow is to allow for the tasks and engine to be resumed (if say the machine running the tasks loses power) so I wonder how partial status will be resumed (can it be?) or will resumption just restart at the last task and then hope that calling the code running in the task (which is updating the progress from 0.0 -> 1.0) can handle figuring out how to pickup from where it left off. For example the task could be "tie shoes", and it could fail at 0.5, taskflow though would just know that 'tie shoes' failed and could attempt to resume it from there, but it would not know that 50% was finished.

VR: I think in that start_value, end_value and current_value should be persisted. I think in most cases partial status can't be resumed and this functionality not required. So if task failed, when it resumed again current_value must restarted to start_value.

imelnikov: IMO representing progress as float between 0.0 and 1.0 inclusive should be enough. We could also provide API for tasks to update (part of?) task metadata from the same callback so that it could save more application-specific information there, including application-specific progress data (e.g. current byte count and total value for that).

VR: I agree with this, store progress in task metadata is good idea and we can store aditional data to calculate for example transfer rate.

imelnikov: I think that task progress should be equal to 0.0 in all states but RUNNING, SUCCESS, and FAILURE. In RUNNING state, progress should be still 0.0, but can be updated by task. In FAILURE state, last value from RUNNING state is preserved. In SUCCESS state, progress is always 1.0.

VR: Agree.

Gerrit topic: https://review.openstack.org/#q,topic:bp/task-progress,n,z

Addressed by: https://review.openstack.org/46331
    Task progress