Provide the user with access to the logs

Registered by Angus Salkeld

If heat is provided by the cloud provider then the log messages will never be visible to the end user.
We log lots of useful log messages that are really useful to an end user esp. when debugging a templete.
We need a way (similar to the current events - perhaps just a different type) for the user to
get access to these logs.

Blueprint information

Status:
Complete
Approver:
None
Priority:
High
Drafter:
None
Direction:
Needs approval
Assignee:
None
Definition:
Obsolete
Series goal:
None
Implementation:
Not started
Milestone target:
None
Completed by
Rico Lin

Related branches

Sprints

Whiteboard

I think we need a separate db table for stack logs. We store the format string and arguments separately (everything stored in English). Then when the user requests the log it gets translated into whatever language is appropriate. This means that you could have say a Japanese and German speaker been able to view the the same stack in their own language.

I think we need a stack logger:
self.log.info('string %(arg1)s bla %(arg2)s', arg1='foo', arg2='fee')

and each will store the relevant contextual information (current action, resource id).

(Angus) I am going to hold off on this, this needs some more thought...

So there are some existing projects that like to store logs (even an openstack one):
http://developer.rackspace.com/blog/introducing-project-meniscus-the-python-event-cloud-logging-service.html

Some issues to think about:
1) we want user level logging of things that happened as a result of user actions and automated actions (scaling/restarting)
2) this should be stored per tenant (accessible by the end user)
3) the logs should be in the user's locale that read the log if possible (if we have the translations)
    - so if user_german creates a stack
    - user_french should be able to read the logs

How I see this happening (whether implementing it internally or not)
1) When ever an action is started on behalf of a user we create a log target
2) Only user level logs are sent to this target (what we want the users to see, not internal stuff)
3) the formatting and arguments are stored separately (structured logging?)
4) we give a url to the user to go and view the logs
5) on request (given the user's locale) we translate and format the logs

Yet another thing to think about: I think nova has/is moving towards using rpc notifications instead of many of the info and above log messages. I am just not sure how we are supposed to get those
back to the user.

Some examples of what I wanted to log (for the user's consumption):
warning: "you provided a parameter that doesn't exist"
warning: "you provided a property that is marked as not implemented"
warning: "a resource with in the stack you asked me to delete does not exist"
info: "progress: resource X is getting created/deleted"
info: "a resource will be replaced and not modified during a update action"
error: "the automated action restart failed"
error: "the auto scale action failed"
info: "an automated action is taking place because ..."

(Angus) I am starting to think that the right approach is to use Marconi.
- it fits in really well with the cfn SNS queue.
- it is also at the same level as Heat (mechanical)
- doesn't require a lot of setup
The way this would work would be:
1) the user sets up a queue and passes that into Heat as a part of the request (or even in the environment)
2) Heat then sends messages to that queue.
3) the user can have a look at the queue at will.

Gerrit topic: https://review.openstack.org/#q,topic:bp/user-visible-logs,n,z

Addressed by: https://review.openstack.org/63230
    WIP: Send events via the notification system

Addressed by: https://review.openstack.org/169591
    Messages for users

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.