GlusterFS Heketi layout for smart GlusterFS management

Registered by Csaba Henk on 2015-07-21

The Heketi project (https://github.com/heketi/heketi) implements a REST API (https://github.com/heketi/heketi/wiki/API) for smart management of GlusterFS (in particular, to create and extend volumes without knowing of the resources that will constitute the volume / extension).

Our plan is to use Heketi in the GlusterFS drivers to achieve a much more scalable GlusterFS management mechanism.

Utilizing the Heketi client library (https://blueprints.launchpad.net/manila/+spec/heketi-client-lib) we aim to create a layout (a resource management codebase common to glusterfs drivers) that operates through Heketi.

Design considerations:

- Heketi is assumed to be fed by raw resources (nodes and clusters) by the storage admin before putting it to use for Manila.
     - The setup will also require to have the nodes set up with key based ssh
       authorization that allows admin access to them with a key given in layouts' config.
- The association scheme of the Heketi layout will be the same as that of volume layout, ie. a share is backed by a whole volume.
- However, there is no pre-allocated volume pool; share creation is handled by volume creation, share
deletion is handled by volume deletion.
- Share resize can be implemented with quota, as for vol layout (cf. https://blueprints.launchpad.net/manila/+spec/glusterfs-native-extend-shrink-share), up to the volume size; however we don't need to limit extensions by size of the volume: beyond that we can handle share extension by volume expansion (cf. https://github.com/heketi/heketi/wiki/API#volume_expand).
- We will set up cert based auth for the volumes we manage to reliably seal the volumes from unsolicited access. (This is almost a logical necessity: given that Heketi puts us into the storage admin's seat to a high degree, we can't get away with just hinting at various possible ways to block unsolicited / out-of-band access, rather we have to set up the volumes so that we take care of if; and then the only suitably flexible and reliable way to do this is to use cert based auth.)
   - Currently we'll assume that the layout will create and sign the gluster keys (being the single CA used in the setup).
   - The layout will deploy the gluster keys to nodes of the volumes upon volume creation and expansion through ssh.
   - The Manila host will have to be set up with keys and authorized to access the volumes, similarly to how it is with glusterfs-native driver.

Blueprint information

Status:
Started
Approver:
Ben Swartzlander
Priority:
Low
Drafter:
Csaba Henk
Direction:
Approved
Assignee:
Csaba Henk
Definition:
Approved
Series goal:
Accepted for newton
Implementation:
Needs Code Review
Milestone target:
milestone icon newton-1
Started by
Ben Swartzlander on 2015-08-20

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/smart-glusterfs-management-with-heketi,n,z

Addressed by: https://review.openstack.org/279090
    glusterfs: introducing Heketi layout

Addressed by: https://review.openstack.org/277141
    gluster*: clean up volume option querying

Addressed by: https://review.openstack.org/280486
    glusterfs: heketi: Add support for JWT Auth

Addressed by: https://review.openstack.org/282101
    glusterfs-native: use Heketi layout via self-signing

Addressed by: https://review.openstack.org/282069
    glusterfs.common: move the numreduct function to toplevel

Addressed by: https://review.openstack.org/280935
    glusterfs_native: relocate module under glusterfs

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.