Remove the rax bits from the community logging roles

Registered by Kevin Carter

Overview
########

Logging needs to be more generic for a more community focused approach.

Problem Description
-------------------

The logging bits that are in master are focused on only what Rackspace Private Cloud was doing using Kibana, Logstash, and Elasticsearch. The rsyslog role is still scraping through the log entries on a host and spooling shippable items until its eventual failure. Because the repository has removed the Rackspace logging components the Rsyslog role should be broken out into two pieces, client and server.

Proposed Change
---------------

The changed required to make logging work in a more "generic" sense will be three parts.

First:
  The logging components, Kibana, Logstash, and Elasticsearch need to be removed from the default `openstack_environment.yml`. Additionally the rsyslog containers will need to have their build limited to the **log_containers** group. The ``rsyslog`` role and the ``rsyslog-install.yml`` play will be deleted.

Second:
   A new rsyslog_server role will need to be created that will aggregate logs to it. This new role should be a generic log aggregation role. The role will be installed with a new ``rsyslog-install.yml`` play.

Third:
  A new rsyslog_client role will need to be created that will ship logs to multiple locations. By default the logging system "should" ship logs to hosts that have been provisioned with the *rsyslog_server* role. However, this new client role should be generic enough to accommodate most logging requirements, e.g. ELK-stack, splunk, loggly, ect...

Playbook Impact
---------------

* All of the service playbooks will be updated to support the new client logging role.
* The ``rsyslog-install.yml`` play will be rewritten.

Alternatives
------------

N/A - Logging **must** be made generic.

Security Impact
---------------

N/A

Performance Impact
------------------

The proposed changes will run rsyslog in all containers for all OpenStack and host services. This will have a network impact as logs will be shipped via UDP or TCP. That said, we believe that the overall network impact will be negligible.

End User Impact
---------------

This change will allow operators to go to **one** place to view all logs from all hosts/containers. This change should provide deployers and operators the ability to quickly search through logs whenever needed without having to interact directly with the containers.

Deployer Impact
---------------

N/A - There would be no direct deployer impact.

Developer Impact
----------------

This change will allow us to easily ship logs to a host for processing later. This change will modify what containers are built by default when using this system however that is the intention as the Rackspace logging components must be removed.

Dependencies
------------

N/A

Documentation Impact
--------------------

This change will require documentation to be written with regard to where the agregated logs will live.

References
----------

N/A

Blueprint information

Status:
Complete
Approver:
Kevin Carter
Priority:
Essential
Drafter:
Kevin Carter
Direction:
Needs approval
Assignee:
Kevin Carter
Definition:
Approved
Series goal:
None
Implementation:
Implemented
Milestone target:
None
Started by
Kevin Carter
Completed by
Kevin Carter

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/rsyslog-update,n,z

Addressed by: https://review.openstack.org/164470
    Removed all rackspace related logging parts

Addressed by: https://review.openstack.org/164471
    Add new rsyslog server role

Addressed by: https://review.openstack.org/164714
    Adds rsyslog client role and enables it in all plays

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.