Retain a record of events after stack deletion

Registered by Zane Bitter

Currently when we delete a stack, we remove all information associated with it from the database. This is a very bad idea (and not what AWS do) because it means that there is no record of the stack having ever existed. In particular, it is bad with the new rollback feature, since if a stack fails it will be rolled back by default and all records of *how* it failed destroyed.

When a stack is deleted, we should mark it as deleted in the database (with a timestamp). Deleted stacks should not show up in the stack list or be accessible by name. New stacks whose names conflict with deleted stacks should be allowed. However, access to the deleted stack and its events using the ARN or UUID or (canonical) URL should be maintained.

Blueprint information

Status:
Complete
Approver:
Steven Hardy
Priority:
High
Drafter:
Liang Chen
Direction:
Approved
Assignee:
Liang Chen
Definition:
Approved
Series goal:
Accepted for havana
Implementation:
Implemented
Milestone target:
milestone icon 2013.2
Started by
Liang Chen
Completed by
Liang Chen

Related branches

Whiteboard

Agree, but probably also need to clean them out at some point?
maybe a periodic task that deletes week/month old "deleted" stacks?
Basically how do we *really* delete them, or are you suggesting we keep them forever?

[ZB] I believe AWS deletes them after 90 days. I think a periodic task is the answer, and the minimum age at which they become eligible for deletion should probably be configurable in heat-engine.conf to allow providers to decide, but a default somewhere between 30 and 90 days seems reasonable.

Other places in the docs imply that events (but not other stack info) remain available forever (they're filtered only by stack name, so you just get all the events from any stack by that name). I suspect we're probably better off _not_ implementing that.

Gerrit topic: https://review.openstack.org/#q,topic:bp/event-persistence,n,z

Addressed by: https://review.openstack.org/38816
    DB models and migration scripts for soft delete

Addressed by: https://review.openstack.org/38817
    enabling soft delete for event persistence

Addressed by: https://review.openstack.org/38818
    Provide a way to clean up soft deleted data

Addressed by: https://review.openstack.org/40522
    Provide a way to clean up testing database

Addressed by: https://review.openstack.org/40523
    Refactor some tests for stack soft-delete

Addressed by: https://review.openstack.org/40524
    Remove raw_template_get_all from db api

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.