GlusterFS Heketi layout for smart GlusterFS management

Registered by Csaba Henk

The Heketi project ( implements a REST API ( for smart management of GlusterFS (in particular, to create and extend volumes without knowing of the resources that will constitute the volume / extension).

Our plan is to use Heketi in the GlusterFS drivers to achieve a much more scalable GlusterFS management mechanism.

Utilizing the Heketi client library ( we aim to create a layout (a resource management codebase common to glusterfs drivers) that operates through Heketi.

Design considerations:

- Heketi is assumed to be fed by raw resources (nodes and clusters) by the storage admin before putting it to use for Manila.
     - The setup will also require to have the nodes set up with key based ssh
       authorization that allows admin access to them with a key given in layouts' config.
- The association scheme of the Heketi layout will be the same as that of volume layout, ie. a share is backed by a whole volume.
- However, there is no pre-allocated volume pool; share creation is handled by volume creation, share
deletion is handled by volume deletion.
- Share resize can be implemented with quota, as for vol layout (cf., up to the volume size; however we don't need to limit extensions by size of the volume: beyond that we can handle share extension by volume expansion (cf.
- We will set up cert based auth for the volumes we manage to reliably seal the volumes from unsolicited access. (This is almost a logical necessity: given that Heketi puts us into the storage admin's seat to a high degree, we can't get away with just hinting at various possible ways to block unsolicited / out-of-band access, rather we have to set up the volumes so that we take care of if; and then the only suitably flexible and reliable way to do this is to use cert based auth.)
   - Currently we'll assume that the layout will create and sign the gluster keys (being the single CA used in the setup).
   - The layout will deploy the gluster keys to nodes of the volumes upon volume creation and expansion through ssh.
   - The Manila host will have to be set up with keys and authorized to access the volumes, similarly to how it is with glusterfs-native driver.

Blueprint information

Ben Swartzlander
Csaba Henk
Csaba Henk
Series goal:
Accepted for newton
Milestone target:
milestone icon newton-1
Started by
Ben Swartzlander
Completed by
Goutham Pacha Ravi

Related branches



Gerrit topic:,topic:bp/smart-glusterfs-management-with-heketi,n,z

Addressed by:
    glusterfs: introducing Heketi layout

Addressed by:
    gluster*: clean up volume option querying

Addressed by:
    glusterfs: heketi: Add support for JWT Auth

Addressed by:
    glusterfs-native: use Heketi layout via self-signing

Addressed by:
    glusterfs.common: move the numreduct function to toplevel

Addressed by:
    glusterfs_native: relocate module under glusterfs


Work Items

This blueprint contains Public information 
Everyone can see this information.