Emit event notifications

(boden) -------
We have a requirement to consume AMQP events for most major resources in Keystone.
This includes create/update/delete (CUD) of the following:
* CUD based AMQP events for project/user/domain/role/group/role-grant

As part of this eventing, we will also need support for Qpid based notifications (rather than via Rabbit), but hopefully this is "for free" given the common event publishers.

At the current moment, I don't see a need to include passwords in AMQP events -- so a create user event would not need to include the user's password.

Note that I believe Brant K (on the Keystone team) is available to help with this implementation need be. Therefore if you cannot contain the requirements mentioned above could you please let him know?

Thanks
----------

(dolph) ---------------
My thinking for implementation was to simply emit an ID (or perhaps a full URL? there are some complications there surround API versioning, however) of the resource as the message body, using the topic to describe the event (e.g. identity.users.delete or whatever), and allow consumers to call back to keystone to get details -- it appears you're expecting full resources to be emitted, however? If so, why? -dolph

I also hadn't remotely considered role assignments; again, why??

--------------

(boden) -----------
I was assuming the events would have some high level metadata, but that was just an assumption based on the other events have seen to date... The idea was to get as much data in the event as made sense to save the consumer a trip back to Keystone to query the resource change. However if the events had enough structure to indicate the type of event and the resource type/id (URI or other) that would probably be a good enough start.

(dolph) ---------------
I definitely don't want to expose another complicated API where we have to maintain backwards compatibility etc, so I'd like to keep it as simple as possible.

(boden) -----------
On the role assignments -- we have Cloud managers which "sit atop" openstack and manage metadata about identity artifacts like users/projects/roles. Moreover these cloud managers have the notion of resource ownership and roles which must reflect the grants in Keystone... This metadata must be "in sync" with Keystone to allow us to update the reflected role grant in the cloud manager... Therefore if a role grant changes for some domain/project/user, our manager needs to know aobut this so we can reflect the access in our Cloud manager resources.
------------

(dolph) ---------------
This just sounds like keystone shouldn't own your authn/z data.

(boden) ------------
regarding events for role grants; maybe the example I gave wasn't a good one and isn't obvious without getting into arch details. however consider a few others which fall into the BAMA space:
* As a Cloud provider, I need my billing application to initiate accounting information when a user is granted access to a project.
* As a Cloud provider, I need my auditing application to record/track project membership and role assignment
the above are just a few... the idea here (IMO) is that the eventing provided should be as generic as possible to fulfill a broad spectrum of use cases; as such i think it should cover CUD of most persistent metadata in Keystone, including role grants.

also i didn't mention this before, but we also need CUD events related to service/endpoints. for example:
* As a Cloud provider, I need my management system notified when an endpoint is added/updated/deleted
* As a Cloud provider, I need my management system notified when a service is added/updated/deleted

(dolph) -----

I think the use case of being notified for a user's role assignments changing should simply be a topic like identity.users.role_assignments where the message is simply the user ID. (You could also emit a notification for the subject of the assignment, like identity.projects.role_assignments). I really don't want to make it more complicated than that (complex messages, etc).

Gerrit topic: https://review.openstack.org/#q,topic:bp/notifications,n,z

Addressed by: https://review.openstack.org/40943 (merged)
    Initial notification commit

Addressed by: https://review.openstack.org/41487 (merged)
    Sync notifier module from Oslo

Addressed by: https://review.openstack.org/41970 (merged)
    Implement notifications in Keystone

Addressed by: https://review.openstack.org/42205 (documentation)
    Document usage notifications

Addressed by: https://review.openstack.org/43340 (merged)
    Add project CRUD to assignment_api Manager

Gerrit topic: https://review.openstack.org/#q,topic:keystone_notifications,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bug/967832,n,z

Addressed by: https://review.openstack.org/44446 (documentation)
    add 'project' notifications to docs

Addressed by: https://review.openstack.org/66718
    Remove kwargs from trust_api.create_trust

Addressed by: https://review.openstack.org/66719
    Implement notifications for trusts

(amichon) -----
I really need this use case for my needs
* As a Cloud provider, I need my auditing application to record/track project membership and role assignment

In assignement/core.py I only found delete_grant but no create_grant. I've implemented it (quite simple) but the payload does not contain any info on project_id, role_id, only on user_id that ceilometer (notification consumer) could manage
@dolph you saw something different : a message with only the resource Id, responsibility for the event consumer (resp ceilometer) to request keystone for the resource (project, role) related to ?
I will commit a change soon for create_grant anyway

Gerrit topic: https://review.openstack.org/#q,topic:notifications,n,z

Addressed by: https://review.openstack.org/112204
    Add notifications for role assignment created and deleted events