Making algorithm of purging soft deleted records older than <date> unbreakable.
Current tool of "cinder-manage -d db purge <days>" is assuming pristine condition of database data. To be specific, assumption that every row in "volume" table, marked as "deleted" and "deleted_at", has all other tables that are using "volume_id" Foreign Key are also marked is far from reality. In practice when operating Cinder for many years and going through different backends, upgrades, outages. There is a high chance of some tables with FK constraint missing "deleted" or "deleted_at".
Proposed change
Change the logic of the "cinder-manage -d db purge <days>" to be equivalent of:
-------
delete from backups where volume_id IN (select id from volumes where deleted_at < '<date>');
delete from snapshots where volume_id IN (select id from volumes where deleted_at < '<date>');
delete from transfers where volume_id IN (select id from volumes where deleted_at < '<date>');
delete from volume_
delete from volume_
delete from volume_attachment where volume_id IN (select id from volumes where deleted_at < '<date>');
delete from volume_
delete from volume_metadata where volume_id IN (select id from volumes where deleted_at < '<date>');
# and finally last one that will now always work, making it unbreakable :)
delete from volumes where deleted_at < '<date>'
-------
Once this is done, you will never see the following error when you run:
cinder-exec cinder-manage -d db purge 90
2019-10-30 22:33:25.524 1 ERROR cinder.
DBError detected when purging from volumes: (pymysql.
(1451, u' Cannot delete or update a parent row: a foreign key constraint fails
(`cinder`
[SQL: u'DELETE FROM volumes WHERE volumes.deleted_at < %(deleted_at_1)s']
[parameters: {u'deleted_at_1': datetime.
(Background on this error at: http://
DBReferenceError: (pymysql.
(1451, u'Cannot delete or update a parent row: a foreign key constraint fails
(`cinder`
[SQL: u'DELETE FROM volumes WHERE volumes.deleted_at < %(deleted_at_1)s'] [parameters: {u'deleted_at_1': datetime.
(Background on this error at: http://
Purge command failed, check cinder-manage logs for more details.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Marcin Kosobucki
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- New
- Series goal:
- None
- Implementation:
- Unknown
- Milestone target:
- None
- Started by
- Completed by