Expiration policies for Executions

Registered by Guy Paz

Currently there is no mechanism that removes past execution and that might lead to huge amounts of data being stored on DB / disk.
One simple workflow with a cron-trigger (For example monitor and heal workflow) can create large amounts of data in the DB even more in production environment when we will have dozens - say the least - of cron-trigers and over a time.

Proposed change
In the mistral.conf will be a ‘expiration policy’ that defines what is expired executions (for example for that last x days), what to do with expired executions and how often to check for expired executions.

Policies options:

  Expired criteria:
    - Older than x minutes (phase 1)
      older_than int in minutes

  Cleanup action:
    - Delete executions (phase 1)
      Delete expired executions via API so all TaskExecution and ActionExecution will
      be removed too (all executions that update_at date are older_than)

Only final states (SUCCESS and ERROR) executions will remove (phase 1)
In case that there is a workbook that contain 2 wfs that one call the second so the child wf finished and expired (SUCCESS) , but the parent still running (RUNNING) we will not remove the 'child' since the executions graph are still running. We will remove the all exec-wfs together when the parent will be done and expired (SUCCESS/ERROR)

There are two parameters that define the Expiration Policy task (pahse 1):
"Evaluation interval" - time interval between Call Expiration Policy (in minutes).
"Older than" - which execution expired (every execution that 'update_at' < 'older_than' is expired)

** Note ** that default values are Null/None so there will be no Expiration Policy running.

This is the mistral.conf:


# From mistral.config

# How often will the executions be evaluated (in minutes). For example
# for value 120 the interval will be 2 hours (every 2 hours).
# (integer value)
evaluation_interval = 5

# Evaluate from which time remove executions in minutes. For example
# when older_than = 60, remove all executions that finished a 60
# minutes ago or more. Note that only final state execution will
# remove ( SUCCESS / ERROR ) (integer value)
older_than = 10

Blueprint information

Guy Paz
Needs approval
Guy Paz
Series goal:
Accepted for liberty
Milestone target:
milestone icon 1.0.0
Started by
Nikolay Makhotkin
Completed by
Renat Akhmerov

Related branches




Work Items

This blueprint contains Public information 
Everyone can see this information.


No subscribers.