Generalize a receiver abstraction for Senlin

Registered by Julio Ruano

Currently the only way for Senlin to receive "notifications" from external services is via an action to a webhook. We can further generalize the webhook + action interface to be a receiver abstraction. This will allow for additional implementations beyond just webhook to handle "notifications".

Blueprint information

Status:
Complete
Approver:
Qiming Teng
Priority:
High
Drafter:
Qiming Teng
Direction:
Approved
Assignee:
Qiming Teng
Definition:
Approved
Series goal:
Proposed for mitaka
Implementation:
Implemented
Milestone target:
milestone icon mitaka-2
Started by
Qiming Teng
Completed by
Qiming Teng

Related branches

Sprints

Whiteboard

To enable Senlin actions to be triggered by external systems or services, Senlin needs to provide a receiver abstraction for preparing the "sinks" for the alarms/signals/alerts from those sources. Currently Senlin only provide webhook as an approach to trigger actions. When we are envisioning a future where other types of receiver types, we feel that the webhook API might be a little too restrictive. Further more, once the API is published, it will take a *long* time to deprecate it if we want to.

This blueprint proposes a new generic interface for receiver abstractions. The design is roughly based on the webhook implementation today. It will supersede the webhook API as the generic interface for creating receivers. Webhook itself will become one of the receiver types that Senlin supports.

Use Cases
=========

Considering that the monitoring of cloud infrastructure and/or cloud applications itself is a huge domain where dedicated services can do a better job than Senlin could possibly do, Senlin team has made a decision to stay away from this functionality. Instead, Senlin will support the creation of receivers for different kinds of alarm/event signals.

There are several typical use cases to be supported:

A) An user developed software that is capable of making RESTful API calls from the control plane wants to trigger an action on a Senlin cluster. We assume that this software can authorize itself via keystone identity service and then invoke Senlin APIs directly.
B) An existing cloud metering/monitoring service can be easily extended to make a RESTful API invocation on Senlin API to trigger an action.
C) An existing cloud metering/monitoring service can be easily extended to send a HTTP POST request to a given URI (i.e. a webhook) to trigger an action.
D) A software running inside a Senlin cluster is able to send a HTTP POST request to a proxy which then redirects the request to Senlin API to trigger an action.
E) A software running inside a Senlin cluster is able to send a message via a message queue service from which Senlin can receive the message and starts an action.

The use case A and B are out of scope of this blueprint. We mainly focus on use cases C, D and E. Use case C and D can be supported by webhooks while use case E can be supported by a user-facing message queue service.

Receiver Abstraction
====================

A receiver, when created, will prepare the receiving end of a signal channel. For example, it can produce a webhook URL to which a monitoring service can send a POST request; it can create a queue in a messaging service to which a monitoring service can post messages.

Speaking of triggering an action, we have 4 entities to specify:

 - the object: the target cluster (we use to support cluster, node and policy as objects,
   but it seems too broad).
 - the action: the action to be triggered once a signal is received.
 - the params: optionally, the parameters associated with the action.
 - the user: this will default to the creator of the receiver (subject to access check
   regarding whether he/she has access to the cluster); if the creator is admin, then
   the user will be the owner of the cluster; or the receiver can specify the credentials
   of another user when creating the receiver.

Among many possible receiver types, webhooks are the most restrictive ones because we don't have other places to inject parameters or options. Other types of triggering may provide rooms for additional options.

Creating
--------

When creating a receiver, the requester needs to provide the following information:

  - the name of the receiver
  - the receiver type (only support 'webhook' at the moment)
  - the target object (cluster)
  - the action

Optionally, 'params' and 'credentials' can be provided as well. On successful creation, the following information (among other properties) will be returned:

  - the UUID of the receiver
  - the name of the receiver
  - the type of the receiver
  - the target cluster
  - the action and its parameters (if provided)
  - the customized user credential (if specified)
  - a dictionary named 'info' containing the information about how to trigger the receiver

Different types of receivers will return different contents in the 'info' dictionary, e.g. the url used to trigger a webhook or the information of a queue to which you can post messages.

Listing/Retrieval
~~~~~~~~~~~~~~~~~

Listing and retrieval of receivers will be checked against request's user id and project. In addition to multi-tenancy supported by the project_safe parameter, we will check if the user id is authorized to perform a list or get operation.

Previously, we have tried to encrypt the details of user credentials in database. When user wants to trigger a webhook, he/she has to provide a 'key' to decipher the credentials. This seems an over-design. If a user gets the alarm url of a webhook, there is not secret to him/her. If a user cannot get the alarm url, the chance of security breech is zero. The only value we had that encryption is that the credentials in database won't be leaked to unauthorized person. However, if a user has got direct access to your database, there is no security at all.

So, in the new design, we are not using any encryption. If the data is to be stored in a secure way, we can use barbican. However, that is an implementation change, completely transparent to users.

Triggering
~~~~~~~~~~

The triggering of a receiver is up to users. Users will decide which system/service will be used for monitoring and how alerts will be sent to Senlin.

Gerrit topic: https://review.openstack.org/#q,topic:bp/support-general-receiver-abstraction,n,z

Addressed by: https://review.openstack.org/257001
    DB support for receivers

Addressed by: https://review.openstack.org/257046
    Engine support for receivers

Addressed by: https://review.openstack.org/259747
    RPC support to receivers

Gerrit topic: https://review.openstack.org/#q,topic:receiver-creation,n,z

Addressed by: https://review.openstack.org/259738
    Creation of receivers (webhooks)

Addressed by: https://review.openstack.org/259860
    API support to receivers

Addressed by: https://review.openstack.org/260037
    Remove unused webhook APIs

Addressed by: https://review.openstack.org/260297
    Remove useless webhook RPC calls

Addressed by: https://review.openstack.org/260299
    Remove webhook from service layer

Addressed by: https://review.openstack.org/261621
    Rework webhook middleware

Addressed by: https://review.openstack.org/261644
    Rework webhook tests

Addressed by: https://review.openstack.org/261645
    Remove webhook module

Addressed by: https://review.openstack.org/261650
    Remove DB API for webhooks

Addressed by: https://review.openstack.org/261678
    Remove webhook from DB model

Addressed by: https://review.openstack.org/261714
    Receiver developer documentation update

Addressed by: https://review.openstack.org/261791
    User documentation for receivers

(?)

Work Items

Work items:
DB design: DONE
DB API: DONE
Engine API: DONE
RPC: DONE
REST API: DONE
MIDDLEWARE: DONE
SDK Resource: DONE
Client: DONE
API Documentation: DONE
User Documentation: DONE

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.