Enhance logging experience with remote logs collecting and filtering
I'd like to raise this blueprint again, Kibana UI is a good option for filtering logs in RT, and Elasticsearch is a good backend as a search-engine for Logstash logging engine to replace(evntually) rsyslog as well.
We should also reuse Openstack Infra experience as well, see http://
Goal for beta (milestone 1) is to support deployment for non clustered Elasticsearch embedded with Logstash and Kibana3 UI (also embedded) resided onto the Fuel master node. The maximum messages throughput provided would be not greater than 2k/sec.
Goal for product (milestone 2) is to provide a Logging as a Service for all Openstack projects as well for its Cloud Apps (Paas) and user instances - exposed via REST API. It should also support horizontally scaled clustered automated deployments on demand. The maximum messages throughput provided should be near to 60k/sec.
Blueprint information
- Status:
- Complete
- Approver:
- Mike Scherbakov
- Priority:
- Low
- Drafter:
- Bogdan Dobrelya
- Direction:
- Needs approval
- Assignee:
- Bogdan Dobrelya
- Definition:
- Superseded
- Series goal:
- None
- Implementation:
-
Informational
- Milestone target:
- None
- Started by
- Completed by
- Bogdan Dobrelya
Related branches
Related bugs
Sprints
Whiteboard
For Fuel, superseded by https:/
Related LogaaS project (https:/
*Beta (milestone 1)*
WIP branches (Fuel 5.0 base)
https:/
* Demo of PoC
https:/
https:/
* Demo of Kibana UI
http://
Gerrit topic: https:/
Addressed by: https:/
Add puppet-logstash module
Addressed by: https:/
Add Logstash, Elasticsearch, Kibana prototype for logging
Addressed by: https:/
Send bootstrap logs to logstash as well
Work Items
Work items:
Design: DONE
PoC: DONE
Stability issues: DONE
Gerrit patches for verbatim copies of puppet-logstash, puppet-
Blueprint conversion to the Logging as a service (LogaaS): INPROGRESS
