Nova Database Archiving Strategy

Registered by Joe Gordon

Outcome of the talk at the Grizzly summit was a general agreement that:
* table bloat with deleted records is bad for performance
* some deployers may want to keep deleted records around for various reasons
* some processes may rely on recently-deleted records

There seemed to be several ways to deal with this, some short-term and some long-term.
1. event/cron that moves deleted=1 records to a shadow table
2. on_delete trigger that moves records to a shadow table
3. amqp message to broadcast deleted records

The current plan is to move forward with (1).

Blueprint information

Status:
Complete
Approver:
Vish Ishaya
Priority:
High
Drafter:
aeva black
Direction:
Approved
Assignee:
David Ripton
Definition:
Approved
Series goal:
Accepted for grizzly
Implementation:
Implemented
Milestone target:
milestone icon 2013.1
Started by
Russell Bryant
Completed by
David Ripton

Related branches

Sprints

Whiteboard

Another approach not documented at Grizzly summit, that Vish just reminded me of -- instead of "deleted=1", use "deleted=id", which would allow a UNIQUE index on (some_col, deleted). This would effectively enforce uniqueness for each value of some_col where deleted=0, which is the desired behavior. It is not really an ideal long-term solution IMO, but seems to mitigate some of the issues preventing unique indexes currently.

List of some potential race conditions that might benefit from UNIQUE constraints:
   https://review.openstack.org/#/c/15709

-----------

Adding unique keys is orthogonal to archiving deleted records, and is being moved to blueprint db-unique-keys.

-devananda, 2012-11-15

Gerrit topic: https://review.openstack.org/#q,topic:bp/db-archiving,n,z

Addressed by: https://review.openstack.org/18493
    Allow archiving deleted rows to shadow tables, for performance.

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.