Support for per-project data retention policy
This blueprint will describe an approach for supporting a per-project data retention policy in Monasca. This would support setting a default retention policy which would be enforced on a per-project basis, but could also be customized per-project to allow some projects to keep data for longer or shorter periods of time.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Low
- Drafter:
- Bradley Klein
- Direction:
- Needs approval
- Assignee:
- Bradley Klein
- Definition:
- New
- Series goal:
- None
- Implementation:
- Unknown
- Milestone target:
- None
- Started by
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
This blueprint is primarily focused on the current most broadly supported backend for monasca, influxdb. Influxdb supports retention policies, and Monasca could deliver this functionality via retention policies with the following changes:
1. The Monasca perisister would have a new config option to set the default retention policy duration.
2. Monasca would create a retention policy for a project (presumably using the project/tenant id as the retention policy name) the first time data is written for a given project. Care must be taken with the implementation of this step -- so as to not adversely affect API performance for writes.
3. Monasca API would support changing the project level retention policy (and presumably store in mysql). This would require monasca-admin level privileges.
4. With 1-3 done, Influxdb would automatically prune older data in compliance with the project's retention policy.
Other databases have similar features (time to live...) that could likely also perform the same cleanup on the database server side.
This feature would be entirely config file driven, so current and future deployments that don't want to take advantage of this feature will be unaffected.