Support for per-project data retention policy

Registered by Bradley Klein on 2015-04-16

This blueprint will describe an approach for supporting a per-project data retention policy in Monasca. This would support setting a default retention policy which would be enforced on a per-project basis, but could also be customized per-project to allow some projects to keep data for longer or shorter periods of time.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Low
Drafter:
Bradley Klein
Direction:
Needs approval
Assignee:
Bradley Klein
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

This blueprint is primarily focused on the current most broadly supported backend for monasca, influxdb. Influxdb supports retention policies, and Monasca could deliver this functionality via retention policies with the following changes:

1. The Monasca perisister would have a new config option to set the default retention policy duration.
2. Monasca would create a retention policy for a project (presumably using the project/tenant id as the retention policy name) the first time data is written for a given project. Care must be taken with the implementation of this step -- so as to not adversely affect API performance for writes.
3. Monasca API would support changing the project level retention policy (and presumably store in mysql). This would require monasca-admin level privileges.
4. With 1-3 done, Influxdb would automatically prune older data in compliance with the project's retention policy.

Other databases have similar features (time to live...) that could likely also perform the same cleanup on the database server side.

This feature would be entirely config file driven, so current and future deployments that don't want to take advantage of this feature will be unaffected.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.