Deployer Configurable Dynamic Property Mappings

Note: Potential Spec

Summary
=======
Allow dynamic property mapping - taking advantage of Glance Metadefs wherever possible.

Problem Statement
===============
Most services in OpenStack have both static API properties that are well known and can be mapped
statically by a plugin. For example, a network has admin_state_up and description properties. These can be statically mapped in the plugin. The data type is particularly useful for ensuring that the correct type of search can be performed against the field (e.g. string vs date range).

However, many services also support additional metadata properties, some of which are well known and some which are not. The term metadata can become very overloaded and confusing. This spec is about the additional metadata that is passed as arbitrary key / value pairs across various artifacts and OpenStack services.

Below are a few examples of the various terms used for metadata across OpenStack services today:

Nova
  flavor extra specs
  server metadata, schedule hints
Cinder
  volume & snapshot image metadata, user metadata
Glance
  properties
Swift
 custom metadata

ElasticSearch supports dynamic_mapping where it makes a best guess on mapping data based on the first time it receives a particular property. However, this can be problematic if it does not guess correctly for various reasons.

Description
=========

At the mitaka summit, a team demonstrated with Swift how valuable it is to be able to define additional metadata mappings. It allowed them to do things like geo spatial search, etc.

This spec has three aspects:

1) Potentially take better advantage of dynamic mapping capabilities: https://www.elastic.co/guide/en/elasticsearch/guide/current/custom-dynamic-mapping.html

2) Allow deployers to specify additional custom mappings for a given resource type(s).

3) Take advantage of the metadata definitions catalog in glance to correctly setup mappings.

Glance has a Metadata Definitions catalog where the dynamic key value pairs that can be applied to various resources in an OpenStack cloud can be defined. These are also known as properties or metadata.

http://docs.openstack.org/developer/glance/metadefs-concepts.html

Searchlight imports the metadefs now. We should enable datamapping to take advantage of this catalog of metadata definition so that we can define the ES mapping of data types.

See here for examples:

https://github.com/openstack/glance/tree/master/etc/metadefs