Support storing the entire event body.
Reference Branch: https:/
In our experiences with Stacktach, we've found it extremely valuable to store the entire event body so that new datapoints and metrics can be rolled up after the fact. This feature would allow the storing of the entire event body in Ceilometer's database. In Stacktach, event bodies are stored as single JSON strings. Due to that it makes building new metrics a very slow and painful process of selecting large subsets of events, parsing the json to a dictionary and the grabbing the data needed. A better solution would allow the indexed searching of these bodies without too much complexity.
This feature would introduce a new api model called EventBody containing two fields, the event, and the event's body as a dictionary. The reference branch above outlines the beginnings of a SQLAlchemy implementation. In SQLAlchemy, EventBodies are made up of EventBodyEntries. Each entry has four fields, the event, a key (UniqueName), index (Text), and value (Text).
Sample Dictionary with it's corresponding EventBodyEntries:
https:/
With that model it would be relatively simple to search for and grab all events where 'payload.
Another thing to note is that EventBodies are not made up of Traits. Traits are values of known types for which we currently see as useful. The EventBodies are made of of values of unknown type and for which we currently don't have a use for. The importance of keeping these values around is so that when a new use is discovered, we can generate that trait for past data.
Blueprint information
- Status:
- Complete
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Andrew Melton
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Obsolete
- Series goal:
- None
- Implementation:
- Unknown
- Milestone target:
- None
- Started by
- Completed by
- gordon chung
Related branches
Related bugs
Sprints
Whiteboard
Seems to me this will add significant amount of data into the database, and not mention slow down the data insertion. Would be nice to think about using some kind of callout based on search/index engine like Lucene.
is this being worked on? i think this could be restarted and wouldn't be that big a change. (at least not for some db. --gordc (19/11/14)
i'm modifying this to store unindexed body... storing full decomposed notification will blow up everything. -- gordc(6/2/15)
i actually implemented the ability to store raw notification in Events. they will be unindexed as mentioned above https:/
Work Items
Dependency tree
* Blueprints in grey have been implemented.