Google Protocol Buffer server for analytics

contrail-collector should be able to ingest Google Protocol Buffer (protobuf) messages from downstream generators, process/normalize the messages and insert them into the analytics database. Self describing protobuf messages as described in https://developers.google.com/protocol-buffers/docs/techniques#self-description provide a mechanism to write a protobuf receiver that does not need a apriori knowledge of the protobuf messages / .proto files of the generators. This is similar in concept to AVRO - http://avro.apache.org, and even the XML protocol used by Sandesh.

contrail-collector will listen on a UDP port and receive self describing protobuf messages, process them and add them to the statistics tables in cassandra. The self describing message format used is:

import "google/protobuf/descriptor.proto";

message SelfDescribingMessage {
  // Timestamp
  required uint64 timestamp = 1;

  // Set of .proto files which define the type.
  required google.protobuf.FileDescriptorSet proto_files = 2;

  // Name of the message type. Must be defined by one of the files in
  // proto_files.
  required string type_name = 3;

  // The message data.
  required bytes message_data = 4;
}

and is specified in the contrail-collector repo at src/analytics/self_describing_message.proto.

contrail-collector will parse the received protobuf messages and insert into the statistics table based on the following convention/rules:
1. All strings will be used as tags/indexes for fields at the same or lower level of the message.
2. All numbers will be used as attributes at the same level of the message.
3. Structures will be inserted as hierarchical statistics with the top level structure/message name as the statistics name followed by the variable name of the structure.
4. The above rules will be applied hierarchically for structures.

For example, for the following proto file definition (specified in the contrail-collector repo at src/analytics/test/test_message.proto):

message TestMessageInner {
    required string tm_inner_name = 1;
    optional int32 tm_inner_status = 2;
    optional int32 tm_inner_counter = 3;
}

message TestMessage {
    required string tm_name = 1;
    optional string tm_status = 2;
    optional int32 tm_counter = 3;
    repeated TestMessageInner tm_inner = 4;
}

the following statistics will be inserted into the statistics table:

1) StatName: TestMessage StatAttr: TestMessage
Tag: TestMessage.tm_name
Tag: TestMessage.tm_status
Attribute: TestMessage.tm_counter
Attribute: TestMessage.tm_name
Attribute: TestMessage.tm_status

2) StatName: TestMessage StatAttr: TestMessage.tm_inner
Tag: TestMessage.tm_inner.tm_inner_name
Tag: TestMessage.tm_name
Tag: TestMessage.tm_status
Attribute: TestMessage.tm_inner.tm_inner_counter
Attribute: TestMessage.tm_inner.tm_inner_name
Attribute: TestMessage.tm_inner.tm_inner_status
Attribute: TestMessage.tm_name
Attribute: TestMessage.tm_status

Example code in the contrail-controller repo at src/analytics/test/protobuf_test.cc illustrates the above.