provide centralized service to aggregate and view openstack logs

Registered by Umesh Moghariya

Provide a centralised service to aggregate, process and provide a dashboard + UI to visualize, search and view openstack logs.

Blueprint information

Status:
Complete
Approver:
Michał Jastrzębski
Priority:
High
Drafter:
Umesh Moghariya
Direction:
Approved
Assignee:
Alicja Kwasniewska
Definition:
Approved
Series goal:
Accepted for mitaka
Implementation:
Implemented
Milestone target:
milestone icon mitaka-3
Started by
Alicja Kwasniewska
Completed by
Steven Dake

Related branches

Sprints

Whiteboard

Some discussion around this is needed, specifically what would be running and where. At the completion of the logging-container blueprint we should have an rsyslog server on each host aggregating the logs and writing them to disk the the option to forward them. That means this blueprint should be about what services we want to run and the containers needed to run them. I believe the common stack is Elasticsearch Logstash and Kibana -- SamYaple

I like the blueprint, but needs an assignee before approval -- sdake

Gerrit topic: https://review.openstack.org/#q,topic:bp/central-logging-service,n,z

Addressed by: https://review.openstack.org/252968
    Added ELK containers for centralised logging

Just wanted to throw another option to the discussion: https://jujucharms.com/papertrail/precise/3
Would the logging model being landed support multiple logging providers? How much effort would be involve in integrating with papertrail for example? -- diogogmt

Addressed by: https://review.openstack.org/267714
    Added Elasticearch and its deployment.

Addressed by: https://review.openstack.org/270214
    Added Kibana and its deployment.

Addressed by: https://review.openstack.org/278381
    Added Logstash and its deployment.

Gerrit topic: https://review.openstack.org/#q,topic:heka-es-integration,n,z

Addressed by: https://review.openstack.org/284188
    Make Heka send logs to Elasticsearch

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.