Glance should import stores through configuration

Registered by Joshua Harlow on 2012-05-04

Instead of having to list out all the stores in the api/v1/images.py in the import list there we should be able to provide the list of stores in config and then have glance itself dynamically load those stores and register there schemes with its own internal registry.

Blueprint information

Status:
Complete
Approver:
Brian Waldon
Priority:
Medium
Drafter:
Joshua Harlow
Direction:
Approved
Assignee:
Joshua Harlow
Definition:
Approved
Series goal:
Accepted for folsom
Implementation:
Implemented
Milestone target:
milestone icon 2012.2
Started by
Brian Waldon on 2012-05-07
Completed by
Brian Waldon on 2012-05-16

Related branches

Sprints

Whiteboard

Possibly the following new format for these from config:

--- a/etc/glance-api.conf
+++ b/etc/glance-api.conf
@@ -5,10 +5,17 @@ verbose = True
 # Show debugging output in logs (sets DEBUG log level output)
 debug = False

-# Which backend store should Glance use by default is not specified
+# Which backend scheme should Glance use by default is not specified
 # in a request to add a new image to Glance? Default: 'file'
-# Available choices are 'file', 'swift', and 's3'
-default_store = file
+default_scheme = file
+
+# List of which stores (and schemes) are currently known to glance at
+# startup and the module+class used to construct that store.
+# Default: glance.store.filesystem.Store(filesystem; file)
+known_stores = glance.store.filesystem.Store(filesystem; file)
+ glance.store.http.Store(http; https),
+ glance.store.rbd.Store(rbd),
+ glance.store.s3.Store(s3; s3+http; s3+https)

Brian Waldon said:

It would be great if the initialization of these stores was moved out of the api altogether. The logical place would be the glance.store module. I bring this up because we need the same thing for the v2 API.

Joshua Harlow said:

RIghty o:

Here is a little diff for that so far...

--- a/glance/store/__init__.py
+++ b/glance/store/__init__.py
@@ -17,6 +17,7 @@

 import logging
 import os
+import re
 import sys
 import time

@@ -29,7 +30,7 @@ from glance.store import location
 logger = logging.getLogger('glance.store')

 # Set of known store modules
-REGISTERED_STORE_MODULES = []
+REGISTERED_STORE_CLASSES = []

 # Set of store objects, constructed in create_stores()
 STORES = {}
@@ -128,40 +129,57 @@ class Indexable(object):
         return self.size

-def register_store(store_module, schemes):
+known_stores_opt = cfg.ListOpt('known_stores',
+ default=['glance.store.filesystem.Store(file;filesystem)'])
+
+
+def register_stores(conf):
     """
     Registers a store module and a set of schemes
     for which a particular URI request should be routed.
-
- :param store_module: String representing the store module
- :param schemes: List of strings representing schemes for
- which this store should be used in routing
     """
- try:
- utils.import_class(store_module + '.Store')
- except exception.NotFound:
- raise BackendException('Unable to register store. Could not find '
- 'a class named Store in module %s.'
- % store_module)
- REGISTERED_STORE_MODULES.append(store_module)
- scheme_map = {}
- for scheme in schemes:
- scheme_map[scheme] = store_module
- location.register_scheme_map(scheme_map)
+ conf.register_opt(known_stores_opt)
+ store_count = 0
+ for entry in conf.known_stores:
+ store_mtch = re.match(r"(.*)\(\s*(.*?)\s*\)", entry)
+ if store_mtch:
+ store_cls = store_mtch.group(1).strip()
+ logger.debug("Testing import of store %s", store_cls)
+ try:
+ utils.import_class(store_cls)
+ except exception.NotFound:
+ raise BackendException('Unable to register store. Could not find '
+ 'a class named %s.'
+ % store_cls)
+ schemes = store_mtch.group(2)
+ schemes = schemes.split(";")
+ schemes = [s.strip() for s in schemes if len(s.strip())]
+ if not schemes:
+ raise BackendException('Unable to register store %s. '
+ 'No schemes registered with it.'
+ % store_cls)
+ else:
+ logger.debug("Registering store %s with schemes %s", store_cls, schemes)
+ REGISTERED_STORE_CLASSES.append(store_cls)
+ scheme_map = {}
+ for scheme in schemes:
+ scheme_map[scheme] = store_module
+ location.register_scheme_map(scheme_map)
+ store_count += 1
+ else:
+ raise BackendException('Unable to register store %s. '
+ 'Unknown store entry format.'
+ % entry)
+ return store_count

 def create_stores(conf):
     """
     Construct the store objects with supplied configuration options
     """
- for store_module in REGISTERED_STORE_MODULES:
- try:
- store_class = utils.import_class(store_module + '.Store')
- except exception.NotFound:
- raise BackendException('Unable to create store. Could not find '
- 'a class named Store in module %s.'
- % store_module)
+ for store_class in REGISTERED_STORE_CLASSES:
         STORES[store_module] = store_class(conf)
+ return len(REGISTERED_STORE_CLASSES)

Alot of tests are adjusted due to this though, they seem to be depending on loading the inbuilt stores, so instead now they will need to register the stores they want, instead of depending on the defaults. Similarly a couple other places for that as well...

Gerrit topic: https://review.openstack.org/#q,topic:bp/import-dynamic-stores,n,z

Addressed by: https://review.openstack.org/7258
    Implements blueprint import-dynamic-stores.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.