Reconcile Cinder Volumes with Actual Storage (DR and B/R)

Problem:
Cinder and cinder drivers do a very good job in managing my volumes as long as my controllers are running, however when there is a need to restore the cinder database then I am left with cinder volumes in a completely out-of-sych state with the actual storage systems. Volumes are in cinder database are no-longer available on the actual storage system, this is a problem for nova, horizon, cloud admins, cloud users, et.c.

Cinder volumes that have been created post restore are left orphaned on the storage systems. Not easy to clean-up basically either dead volumes. Unable to import volumes that have managed by cinder. Volumes attached to instances that no-longer exists.

As a storage Admin, I want to be able to restore my cinder database and synch-up with the actual storage at the time of the cinder restore point.

As a solution, I want be leverage the cinder backend drivers to identify the actual state of the cinder volumes and synch it up with my cinder db. For that to happen, I want the ability to list all volumes and determine their history with cinder. To do that: I need some bread crumbs add to each volume (light weight: volume id, driver guid, snapshot count, instance uuid if attached, and vol typeid).
As a deployer, I will be able to create scripts or tools to use this data and sync-up the actual state of the volumes (delete non-existent volumes, as the user if they want to import volumes, set the correct state of the volume)

Each driver knows how to manage its volumes. We want to make sure that we can address this issue the same approach so end users don't get frustrated with the various approaches.

Background:
In private and hybrid cloud, enterprise customers managing cloud resources depend on backup/restore solutions. OpenStack provides a current view of the resource that it manages, this view is based on the information stored by the OpenStack projects such nova db, cinder db, etc. No cinder volume or nova instance information is stored with the volume on the actual storage system making it very difficult to determine if a storage volume was created by OpenStack on the storage system. Thus, when an admin restores the cloud controller running the nova and cinder services it will not be able to synchronize storage volumes which are managed by OpenStack to the restore point leaving actual volume on the storage system in unmanageable state.

Proposal:

Add a meta-data object and query method to the cinder block storage drivers to help synchronize backend volumes with cinder and nova upon backup/restore of the controller.
The problem can be solved by adding the volume meta-data (volume uuid, name, volume type, instance uuid, host, size, etc.) object which would include a signature (source controller owner). The data must be created when the volume is created, and updated during nova volume-attach/volume-detach to include the nova device mapping info.

The cinder backend drivers already have the access to the storage system; some may also have a query method to list existing volumes. An explicit method will need to be exposed to list all volumes and the meta-data object which is created by cinder.

This allows OpenStack users to implement a backup/restore solution using the data to sync-up the actual state of the cinder volumes with the actual storage resource on the storage system.

Summary:

1. Add an admin method to list all backend volumes
2. Add Identification info on the actual volumes managed by cinder

See etherpad discusion:
         https://etherpad.openstack.org/p/juno-cinder-backup-restore