Magnum should emit the notifications to Ceilometer to collect the meters

Registered by Zi Lian Ji

The goal of the Ceilometer is to collect the meters in the OpenStack platform. And the goal of the Magnum is to manage the containers in the OpenStack platform.

In order that the OpenStack admin can monitor the status of containers managed by the Magnum, Magnum should has the ability to emit the notifications to the Magnum. The notifications should involve the Magnum service notifications like bay operations and the notifications of the containers status collected by the Magnum.

cAdvisor (Container Advisor) provides container users an understanding of the resource usage and performance characteristics of their running containers. It is a running daemon that collects, aggregates, processes, and exports information about running containers. Specifically, for each container it keeps resource isolation parameters, historical resource usage, histograms of complete historical resource usage and network statistics. This data is exported by container and machine-wide.cadvisor also exposes its raw and processed stats via a versioned remote REST API.

cadvisor link: https://github.com/google/cadvisor

The corresponding tasks about using cadvisor to monitor the containers have started in the last release. So in this blueprint, the following tasks should be completed.

a. Format the Magnum service notifications in order that the ceilometer can consume them easily
b. Format the notifications of the container status
c. Collect the meter of the containers

In order to enhance the performance, the cache mechanism should be implement, but it's a option.

The integration of Magnum and Ceilometer will be helpful to collect the container status and make the admin know them easily.

ceilometer blueprint link: https://blueprints.launchpad.net/ceilometer/+spec/ceilometer-magnum-integration

Blueprint information

Status:
Complete
Approver:
hongbin
Priority:
Not
Drafter:
Zi Lian Ji
Direction:
Needs approval
Assignee:
Zi Lian Ji
Definition:
Obsolete
Series goal:
None
Implementation:
Unknown
Milestone target:
None
Completed by
hongbin

Related branches

Sprints

Whiteboard

I would like to see remarks from tango and hongbin prior to making further decisions about this. --adrian_otto
Temporarily setting the priority level of this blueprint to "Not" pending further discussion. --adrian_otto

(hongbin): Magnum already supports sending notifications to ceilometer [1]. In my point of view, the main change is to deploy a cAdvisor container to each bay node. In general, I think it is OK, because cAdvisor is a popular container monitoring solution. I would suggest to introduce a field in baymodel to enable/disable this feature (Because maybe some users want it, others not).

[1] https://blueprints.launchpad.net/magnum/+spec/magnum-notifications

(tango): This blueprint covers notification for Magnum and the containers.

The blueprint that Hongbin points out already covers Magnum, so it's probably a good idea to link that blueprint as a dependency and to take another close look to see if there is any other operation that should have notification. From the description, currently C/U/D for bay/pod/node are covered. Autoscaling is being discussed at the moment and some initial implementation is already in place, so notification should be considered for this set of operation. We can double check other operations such as container-pause, reboot, start, stop, unpause, etc.

For container metrics, I think the answer is less clear since it's somewhat out of Magnum's scope. It would be helpful to define the concrete use cases first: who would consume the data and for what purpose, and why it would make sense for Magnum to handle the data. Kubernetes, for instance, has its own strategy for managing and using its container metrics, so Magnum has little role here. The admin for a Kubernetes cluster would probably rely on Kubernetes tool and interface. Once we understand the use cases well, then as Hongbin suggested, we should make the container notification optional so the user can adapt to the particular situation.

(hongbin): I think we need a monitoring and measurement solution, and the approach suggested in this BP looks reasonable. I am going to approve this BP if there is no further objections. Any further inputs?

(hongbin): According to the decision in the design summit, we are not going to pursuit this in short term. Here is the summarize of the discussion. 1) We need to use cases from operators for how they monitor the containers. 2) Ceilometer integration might incur scalability problem when there are thousands of containers.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.