Integrate with Nova Versioned Notifications

Registered by Travis Tripp on 2016-03-23

Summary
=======
Searchlight should support versioned notifications (and the versioned API).

Problem Statement
===============
Nova has added a concept called versioned notifications. Searchlight doesn't work with them. It also doesn't work with the versioned API at present; some fields (like description) are not present as a result.

Description
=========
Investigate and implement support for working with nova versioned notifications if needed.

http://docs.openstack.org/developer/nova/notifications.html
https://review.openstack.org/#/q/topic:bp/versioned-notification-api
http://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/versioned-notification-api.html

FROM: https://blueprints.launchpad.net/searchlight/+spec/nova-server-versioning-extensions

Required reading: https://dague.net/2015/06/05/the-nova-api-in-kilo-and-beyond-2/

http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/api-microversions.html

We should index almost everything (stripping links is fine). If Nova has data that we can rely on, then it would be bad to force them to make a call to nova api to get that data. The caveat being that we have to be able to have a list of fields that are excluded unless admin is searching.

We need to have a good methodology for handling microversions in nova. This may come down to just allowing a config option for the nova plugin to specify the desired version to index or most likely will require a bit more complexity to handle data mapping (not relying on dynamic mapping) and config based specification of admin only fields.

It is also unclear if we should do anything with extensions. Extensions are deprecated in Nova, but we need to understand what that really means. If we have to support extensions, we might need to support having separate extension files for all these extension fields. The extensions enabled would all be invoked during serialize function and also would be called to add to the mapping. This way deployers could add their extension without modifying source just like they do for nova. BP needed.

    {
      "hostId": "f57bcb1e1ba43de39322dc473d69e180f9b5a554aa0892c7a34a11e3",
      "OS-EXT-STS:task_state": null,
      "OS-EXT-STS:vm_state": "active",
      "OS-EXT-SRV-ATTR:instance_name": "instance-00000001",
      "OS-SRV-USG:launched_at": "2015-08-12T23:49:55.000000",
      "OS-EXT-SRV-ATTR:hypervisor_hostname": "ubuntu",
      "security_groups": [
        {
          "name": "foo sg"
        },
        {
          "name": "a sg"
        },
        {
          "name": "lolz sg"
        }
      ],
      "OS-DCF:diskConfig": "AUTO",
      "os-extended-volumes:volumes_attached": [],
      "accessIPv4": "",
      "accessIPv6": "",
      "progress": 0,
      "config_drive": "True",
    }

We also need to understand how the Nova Versioned Object notifications affect this: https://review.openstack.org/#/c/224755/11/specs/mitaka/approved/versioned-notification-api.rst

Blueprint information

Status:
Not started
Approver:
None
Priority:
Essential
Drafter:
Travis Tripp
Direction:
Approved
Assignee:
None
Definition:
New
Series goal:
Accepted for pike
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

1) Review Versioned Specs:

[1] https://review.openstack.org/#/c/286675/ Versioned notification transformation
[2] https://review.openstack.org/#/c/311194/ versionedobjects: add json schema generation
[3] https://review.openstack.org/#/c/311948/

2) Create Searchlight Spec

Gerrit topic: https://review.openstack.org/#q,topic:bp/nova-versioned-notifications,n,z

Addressed by: https://review.openstack.org/453352
    WIP Enable versioned nova notifications

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.