ElasticSearch Storage Driver Support

Registered by Nachi Ueno on 2013-09-12

This bp implements ElasticSearch Storage driver.

ElasticSearch: http://www.elasticsearch.org/
ElasticSearch has good query language and UI : Kibana3 http://three.kibana.org/

ElasticSearch is also good for storing logs, so we can use ceilometer + Elasticsearch as Log as a Service framework.

Blueprint information

Status:
Complete
Approver:
Eoghan Glynn
Priority:
Medium
Drafter:
gordon chung
Direction:
Approved
Assignee:
gordon chung
Definition:
Approved
Series goal:
Accepted for kilo
Implementation:
Implemented
Milestone target:
milestone icon 2015.1.0
Started by
Eoghan Glynn on 2015-01-06
Completed by
Eoghan Glynn on 2015-02-04

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/elasticsearch-driver,n,z

Addressed by: https://review.openstack.org/46383
    Add ElasticSearch Support for ceilometer

Discussion Point.
[1] We should add elastic search query api for ceilometer? or we should let user kick Elasticsearch api directory?

Note that ElasticSearch has no tenant based authentication, in that case we need to integrate Keystone and ElasticSearch. (or Horizon)

[2] Log (syslog or any application log) should be stored in Ceilometer? (or it should be new OpenStack project? )

DEMO Video http://www.youtube.com/watch?v=8SmA0W0hd4I&feature=youtu.be

How To Test
Checkout ceilometer review
https://review.openstack.org/#/c/46383/

Install ElasticSearch
http://www.elasticsearch.org/guide/reference/setup/

Run Devstack with this patch
https://review.openstack.org/#/c/46384/

Please add this line for your localrc
enable_service ceilometer-acompute ceilometer-acentral ceilometer-collector ceilometer-api
CEILOMETER_BACKEND=elasticsearch

Install Kibana3
http://three.kibana.org/intro.html

Setup ceilometer dashboard
https://gist.github.com/nati/6545382
# please make sure change index default to tenant_id or _all for admin view

Gerrit topic: https://review.openstack.org/#q,topic:elasticsearch-driver,n,z

Addressed by: https://review.openstack.org/126395
    add ElasticSearch driver backend for events

Addressed by: https://review.openstack.org/137131
    add elasticsearch events db

FYI that glance is adding usage of Elastic Search as well

(?)

Work Items