Glance

Add multifilesystem store to support NFS servers as backend

Registered by Abhishek Kekane

Description:

Currently if you want to configure multiple nfs server as a backend using filesystem store, you cannot mount all disks to a single directory. filesystem store allows administrator to configure only single directory with filesystem_store_datadir parameter in the glance-api.conf.
It is possible to use mhddfs (fuse plugin :https://romanrm.net/mhddfs) which mounts multiple nfs servers to a single directory but it doesn't allow you to evenly store the data on all the disks. Another major drawback is when one of the disk is broken it is very hard to know how many and what images are stored on that disk as glance registry stores location specified in the filesystem_store_datadir parameter.

With multifilesystem, it is possible to over come above issues.

We are proposing to do following things.
1. Add a new store multifilesystem.
2. Add new parameters in the glance-api.conf

Example

3 nfs servers, exporting 1TB of disks
10.2.3.10
10.2.3.11
10.2.3.12

On Glance server,
Administrator will mount complete export tree as below

10.2.3.10:/export/nfs-partition-1 /var/glance/store-1
10.2.3.11:/export/nfs-partition-1 /var/glance/store-2
10.2.3.12:/export/nfs-partition-1 /var/glance/store-3

Edit glance-api.conf

Add new parameter multifilesystem_store_datadir to configure multiple directories.
#multifilesystem_store_datadir = <directory>:<priority number>
multifilesystem_store_datadir = /var/glance/store-1:100,/var/glance/store-2:200,/var/glance/store-3:300

Priority number: Each directory will be given a priority number.
Priority number with less value will be given top priority. If all directories is given same priority number, then the disk with more available space will be selected for storing image.

Blueprint information

Status:
Complete
Approver:
Mark Washenberger
Priority:
Medium
Drafter:
None
Direction:
Approved
Assignee:
Aswad Rangnekar
Definition:
Approved
Series goal:
Accepted for icehouse
Implementation:
Implemented
Milestone target:
milestone icon 2014.1
Started by
Thierry Carrez
Completed by
Mark Washenberger

Related branches

Sprints

Whiteboard

For clarification, this blueprint doesn't include implementation of NFS-related operations to glance. We intend to add multiple directories specification. Note that NFS is a kind of use cases for this blueprint.

Blueprint[1] and implementation[2] were rejected with the objections below:
 - NFS isn't need for Glance because it is equal to local file storage.
 - Glance's storage should not use root privileges for working. So automount cannot be implemented.

Since the scope of this blueprint is just enabling administrator to specify multiple directories in glance-api.conf, the objections above should not be applicable to this blueprint.

[1] https://blueprints.launchpad.net/glance/+spec/glance-nfs-storage-support
[2] https://review.openstack.org/#/c/39080/

<markwash>
This feature looks like a very small and relatively simple addition to the filesystem store to look for an image in multiple directories.

<abhishek-kekane>
Mark, Instead of adding new parameter in glance-api.conf and new store multifilesystem, we are making changes to existing filesystem store.

Gerrit topic: https://review.openstack.org/#q,topic:bp/glance-multifilesystem-store,n,z

Addressed by: https://review.openstack.org/58997
    Add multifilesystem store to support NFS servers as backend

SO SO close to landing in icehouse-2, but the gate is really being quite difficult so let's just land it as soon as we can in icehouse 3

(?)

Work Items

Nearby

This blueprint contains Public information 
Everyone can see this information.