Add incremental backup support for existing backup API
Cinder backup API does not have support for incremental backup. As the size of the volume grows and the changes to volume between backups tend to be low, performing full backups on the volume may become resource intensive. This proposal suggests following flags to existing backup api to support incremental backup functionality.
Enhance current backup api to include —incr option
cinder backup-create <volumeid> —incr <full backup container>
—incr option specified for incremental backups. To keep the API implementation simple, we only support incremental from the last full backup. In order to keep the deltas manageable size, user can periodically take full backups and then incrementals.
The other option is snapshot based backups. To support snapshot based backups, the backup api is changed as follows:
cinder backup-create <volumeid> —snapshot
This cli takes a snapshot of the volume and performs the backup of the snapshot. The volume can remain online and in-use for the duration of the operation. At the end of the backup, the snapshot is deleted.
cinder backup-create <volumeid> —snapshot —incr <full backup container>
This cli takes a snapshot of the volume and uploads only the changes since last backup to swift. Again at the end of the backup, the snapshot will be deleted.
Blueprint information
- Status:
- Complete
- Approver:
- John Griffith
- Priority:
- Medium
- Drafter:
- Murali Balcha
- Direction:
- Approved
- Assignee:
- Murali Balcha
- Definition:
- Approved
- Series goal:
- Accepted for kilo
- Implementation:
-
Implemented
- Milestone target:
-
2015.1.0
- Started by
- John Griffith
- Completed by
- Mike Perez
Related branches
Related bugs
Sprints
Whiteboard
Gerrit topic: https:/
Addressed by: https:/
Proposal for incremental backup functionality
Gerrit topic: https:/
Addressed by: https:/
Add support to incremental backup in cinder
Addressed by: https:/
Add support to backups in xiv_ds8k driver
<jdg>
There's a number of concerns raised from core team members about the design here and the ability to extend it to incremental in the future (currently it only handles differentials). Personally I think this is "ok" but I do agree that rushing reviews at the end of the cycle is probably not worth the trouble it may cause down the road. I'm going to propose we retarget this to Kilo in tomorrows weekly meeting.
<jdg>
Hi Murali,
This is a great start and you've got some really good work here. At this stage of the release however I think it might be best if we step back and spend some more time going through the details of the design and getting more input on the implementation. I'd like to move this to Kilo, and ideally would like to see it worked on VERY early in the Kilo release. Please let me know if you have any questions.
<mdb>
The current implementation of differential backup expects parent-id argument for differential backup. The differential changes are computed based on the full backup identified by the parent-id. The full backup process is quite simple. Every time a full backup is taken, the backup process divides the volume data into fixed blocks of size CONF.backup_
During restore, if a differential backup needs to be restored, the restore process first restores the full backup identified by the parent-id of the differential backup and overlays it with data from differential backup to complete the restore.
Extending existing differential algorithm to implement incremental backups is straight forward. Differential backups are calculated with respect to full backups where as incremental backups are done with respect to last backup. Incremental backups usually contain a chain of parents where as differential backups usually has a reference to full backup.
For example an incremental backup chain may look like: Full <- incr1 <- incr2 <- incr3
To take another incremental for this existing chain user can specify
Cinder backup-create —parent-id “incr3” volume_id
The backup process will compute new shafile for the current data. It then compares the sha-256 of each block of current data with sha-256 of incr3 data. The data that is identified as changed is uploaded to swift. The backup process is no different than differential backup process.
When restoring an incremental backup, the restore process need to walk all incremental chains until it find the full backup to complete the restore process. This is slight change from differential backup where restoring differential backup only require traversing to full backup.
________________
