Glance should import stores through configuration

Possibly the following new format for these from config:

--- a/etc/glance-api.conf
+++ b/etc/glance-api.conf
@@ -5,10 +5,17 @@ verbose = True
 # Show debugging output in logs (sets DEBUG log level output)
 debug = False

-# Which backend store should Glance use by default is not specified
+# Which backend scheme should Glance use by default is not specified
 # in a request to add a new image to Glance? Default: 'file'
-# Available choices are 'file', 'swift', and 's3'
-default_store = file
+default_scheme = file
+
+# List of which stores (and schemes) are currently known to glance at
+# startup and the module+class used to construct that store.
+# Default: glance.store.filesystem.Store(filesystem; file)
+known_stores = glance.store.filesystem.Store(filesystem; file)
+ glance.store.http.Store(http; https),
+ glance.store.rbd.Store(rbd),
+ glance.store.s3.Store(s3; s3+http; s3+https)

Brian Waldon said:

It would be great if the initialization of these stores was moved out of the api altogether. The logical place would be the glance.store module. I bring this up because we need the same thing for the v2 API.

Joshua Harlow said:

RIghty o:

Here is a little diff for that so far...

--- a/glance/store/__init__.py
+++ b/glance/store/__init__.py
@@ -17,6 +17,7 @@

 import logging
 import os
+import re
 import sys
 import time

@@ -29,7 +30,7 @@ from glance.store import location
 logger = logging.getLogger('glance.store')

 # Set of known store modules
-REGISTERED_STORE_MODULES = []
+REGISTERED_STORE_CLASSES = []

 # Set of store objects, constructed in create_stores()
 STORES = {}
@@ -128,40 +129,57 @@ class Indexable(object):
         return self.size

-def register_store(store_module, schemes):
+known_stores_opt = cfg.ListOpt('known_stores',
+ default=['glance.store.filesystem.Store(file;filesystem)'])
+
+
+def register_stores(conf):
     """
     Registers a store module and a set of schemes
     for which a particular URI request should be routed.
-
- :param store_module: String representing the store module
- :param schemes: List of strings representing schemes for
- which this store should be used in routing
     """
- try:
- utils.import_class(store_module + '.Store')
- except exception.NotFound:
- raise BackendException('Unable to register store. Could not find '
- 'a class named Store in module %s.'
- % store_module)
- REGISTERED_STORE_MODULES.append(store_module)
- scheme_map = {}
- for scheme in schemes:
- scheme_map[scheme] = store_module
- location.register_scheme_map(scheme_map)
+ conf.register_opt(known_stores_opt)
+ store_count = 0
+ for entry in conf.known_stores:
+ store_mtch = re.match(r"(.*)\(\s*(.*?)\s*\)", entry)
+ if store_mtch:
+ store_cls = store_mtch.group(1).strip()
+ logger.debug("Testing import of store %s", store_cls)
+ try:
+ utils.import_class(store_cls)
+ except exception.NotFound:
+ raise BackendException('Unable to register store. Could not find '
+ 'a class named %s.'
+ % store_cls)
+ schemes = store_mtch.group(2)
+ schemes = schemes.split(";")
+ schemes = [s.strip() for s in schemes if len(s.strip())]
+ if not schemes:
+ raise BackendException('Unable to register store %s. '
+ 'No schemes registered with it.'
+ % store_cls)
+ else:
+ logger.debug("Registering store %s with schemes %s", store_cls, schemes)
+ REGISTERED_STORE_CLASSES.append(store_cls)
+ scheme_map = {}
+ for scheme in schemes:
+ scheme_map[scheme] = store_module
+ location.register_scheme_map(scheme_map)
+ store_count += 1
+ else:
+ raise BackendException('Unable to register store %s. '
+ 'Unknown store entry format.'
+ % entry)
+ return store_count

 def create_stores(conf):
     """
     Construct the store objects with supplied configuration options
     """
- for store_module in REGISTERED_STORE_MODULES:
- try:
- store_class = utils.import_class(store_module + '.Store')
- except exception.NotFound:
- raise BackendException('Unable to create store. Could not find '
- 'a class named Store in module %s.'
- % store_module)
+ for store_class in REGISTERED_STORE_CLASSES:
         STORES[store_module] = store_class(conf)
+ return len(REGISTERED_STORE_CLASSES)

Alot of tests are adjusted due to this though, they seem to be depending on loading the inbuilt stores, so instead now they will need to register the stores they want, instead of depending on the defaults. Similarly a couple other places for that as well...

Gerrit topic: https://review.openstack.org/#q,topic:bp/import-dynamic-stores,n,z

Addressed by: https://review.openstack.org/7258
    Implements blueprint import-dynamic-stores.