Support for logs and log operations

Registered by Victor Ion Munteanu on 2014-11-24

Currently, Monasca has no real support for logs. It has some support for log metrics (or will have), meaning statistics coming from logs are sent as metrics.

The goals of this blueprint would be to:
1) Support the sending of logs (entries) and their storage
2) Support for alarm expressions for logs
3) Support for aggregation of logs

In order to achieve 1) several components need to be changed:
a) monasca-agent: add support for log extraction and/or support for logs in monastatsd
b) monasca-api: add support for metric querying and forwarding in case the metric is a log (string)
c) monasca-thresh: add support for the metric aggregation and processing

For 2) there is a need for some basic expressions to be implemented. The support should be for LIKE and REGEX functions on logs. This involves:
a) monasca-common: support expressions with thresholds which are metrics. Extend ANTLR rules for LIKE AND REGEX
b) moansca-api: support the new expressions (parsing, comparison, validation of metrics)
c) monasca-thresh: support for validating expressions against metrics

For 3), monasca-thresh needs to implement log aggregation (in this case it would be CONCAT).

An example of an alarm:
CONCAT(apache.log) LIKE "File does not exist"
or
CONCAT(apache.log) REGEX "\[error\].*"

The changes are quite substantial and one of the easiest ways would be to move from double to string in metrics and alarms expressions. This would require many small rewrites and comparator rewrites.

Blueprint information

Status:
Not started
Approver:
Roland Hochmuth
Priority:
Undefined
Drafter:
Victor Ion Munteanu
Direction:
Needs approval
Assignee:
Victor Ion Munteanu
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.