Events As a Service - EAAS for Monasca

Registered by Tomasz Trębski on 2016-10-28

Events as a Service

Description:

Events, what they could be used for, how they should be processed were discussed several times over on different occasions.
Following blueprints is meant to provide high level overview of EAAS for monasca which will be used to define architecture, implementation and will drive library/database decisions.

High level use-cases:

* Get a notification if a VM was created or destroyed
* Delete alarms when a VM was removed
* Watch time between events --> Get notified if time was exceeded/event is missing

This blueprint shows the first version of monasca-events. It provides basic functionality to work with events:

* Collect events
* Authenticate
* Persist
* View

[RabbitMQ] -> [Collector/Agent] -> [EventsApi] -> [Kafka] -> [Events Persister] -> [Elasticsearch]

Proposed functionality / components:

Events Agent (Collector):

Part of the service that would automate collecting the events from the OpenStack and sending them over to the API.

* Collect events
* Identify tenant-id of the event
* Events need to stored on behalf of the tenant
* Secure connection is required

Events Api:

* Stateless HTTP Restful Api
  * HTTP Service could base on monasca-log-api (https://github.com/openstack/monasca-log-api)
* Retrieve events
* Retrieve data on behalf of a tenant
* sending events to proper channel bassed on the project-id
* Secure connection is required
* Authentication + Authorization
* Validate retrieved data on size, format, etc.?
  * Is truncation required?
* No event analyzing
* Store data in Kafka (new topic)

Events Persister:

* Consume data from kafka
* persist to elasticsearch
* use tenantid as index to store data (same as monasca-log do)

First steps:

* Analyze OpenStack Events
* Define interfaces and Objects for Events-Api and Events-Persister
* Analyze/Implement Events-Agent, Events-Api and Events-Persister incl. elasticsearch schema
* Devstack implementation

Possible enhancements. Not covered in this blueprint/version:

1. Get notified if a certain event occur (notifications on events "Noe"):

* Send notification if a certain event occur
* Create Event-Alarm in monasca-ui and save this via Events-Api in MariaDB
* Read configuration from MariaDB, collect data from kafka and process stream of events
* Recognize if event occured and create a notification
* Notification will be collected by existing notification-engine

2. Trigger an action if a certain event occur (trigger on events "Toe"):

This enhancement requires "notifications on events (Noe)"

* Certain event occur
* Trigger an action based on the occurred event
* Flexible (configurable) trigger and actions

Example:

* Delete alarms when a vm was destroyed

3. Recognize uncommon behaviour (malfunction on events "Moe"):

Examples:

* Measure time between events (like instance.create.start and instance.create.end). Trigger something (notification, etc.) if instance.create.end event is missing.
* Event sequence check (like start, provision, network, etc.) -> notify if failure

4. Correlation between logs, metrics and events

5. Other ideas:

* saving raw events for the monasca internal applications (low retention time)
* stripping down events from non-essential information a.k.a. normalization
* transforming events according to rules provided by the users identified by the project-id
* aggregating events
* saving events into the configured storage
* executing internal monasca functionality
* triggers (for instance: reacting upon instance deleting, pausing, starting etc)
* metrics (calculating deltas for instance based events, for example: how long deploying an instance took and publishing that variable as a metric)

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
Tomasz Trębski
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Questions without an answer:
1) Should the normalization be part of the transformation phase
2) What library/framework should be chosen for the monasca-events-engine (Spark like for the monasca-transform, or something else)
3) What language, regarding transformation rules, should be supported ? Proposal: Lua vs JS vs Python
4) What exactly would be the purpose of the aggregation phase

Ideas for implemenatation:
1) List of triggers should be configurable (similar to Nova filters)
2) List of metric-actions should be configurable (similar to Nova filters)
3) Transforming/Aggregating/Normalizing should be executed in parallel to the monasca internal actions (triggers, metric-actions)

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.