Profiling OpenStack from the Developer Dashboard

Registered by Timur Sufiev

Summary
=======
To provide Horizon operators, developers and cloud architects a new tool for profiling OpenStack deployment right from the Horizon Web UI.

Motivation
=========
If you have ever run Horizon Dashboard on top of large OpenStack deployment, you must have asked yourself: Why is Horizon so slow? Or is it OpenStack so slow? Identifying the real performance bottlenecks in an actual OpenStack deployment is the first step before fixing these performance issues. There already exists openstack/osprofiler Oslo library which is being integrated into different OpenStack projects (say, Nova) or is already integrated (say, Cinder). Yet profiler UI is pretty scarce and limited to CLI of osprofiler client.

To make profiling of OpenStack more accessible and thus draw a greater attention to the problems of OpenStack performance, creating a Web UI-based tool is a natural choice. Implementing it right inside Horizon also solves different Horizon performance problems as well. Moreover, integrating the osprofiler library into Horizon enables us to add a new Rally benchmarking job for Horizon, which will tell the community in a quantifiable way whether the commit under question makes Horizon performance better or worse.

Description
=========
1. Integrate osprofiler into Horizon Django middleware classes pipeline
2. Add trace points into openstack_dashboard.api wrappers
3. Implement a new panel in Developer Dashboard to allow for profiling from UI.

UX
===
A new Profiler panel appears within Developer dashboard. When profiling is enabled there, a new dropdown appears in the right side of the header bar, which allows to profile the rendering of the current page. Once it's done, a message box with a link appears which leads to a Profiler panel. There one will be able to see all the traces written on behalf of the current user, including the new one.

Wireframes, Mocks, Videos and UI Markup
---------------------------------------------------------------
N/A

Testing
======
N/A as a developer feature

Outside Dependencies
==================
MongoDB cluster needs to be installed on Devstack host (default configuration), see https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/#install-mongodb-community-edition for instructions. Then, change net:bindIp: key to 0.0.0.0 inside /etc/mongod.conf and invoke `sudo service mongod restart` for the changes to have an effect.

Requirements Update Required
========================
osprofiler and pymongo requirements are added to horizon

Doc Impact
=========
N/A

Blueprint information

Status:
Complete
Approver:
Richard Jones
Priority:
High
Drafter:
Timur Sufiev
Direction:
Approved
Assignee:
Timur Sufiev
Definition:
Approved
Series goal:
Accepted for 11.0.0-ocata
Implementation:
Implemented
Milestone target:
milestone icon ocata-2
Started by
Richard Jones
Completed by
Rob Cresswell

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/openstack-profiler-at-developer-dashboard,n,z

Addressed by: https://review.openstack.org/273085
    [WIP] start playing with middleware

Addressed by: https://review.openstack.org/281976
    Do not cache templates in TEMPLATE_DEBUG mode

Addressed by: https://review.openstack.org/284335
    Add 2 more actions to profiler dropdown

Gerrit topic: https://review.openstack.org/#q,topic:profiler-panel,n,z

Gerrit topic: https://review.openstack.org/#q,topic:osprofiler_api_tracking,n,z

Gerrit topic: https://review.openstack.org/#q,topic:osprofiler_api_tracking-patch10,n,z

Gerrit topic: https://review.openstack.org/#q,topic:osprofiler,n,z

Gerrit topic: https://review.openstack.org/#q,topic:osprofiler-patch40,n,z

Addressed by: https://review.openstack.org/303514
    Add the Profiler panel to the Developer dashboard

Addressed by: https://review.openstack.org/339675
    Wrap api calls with tracing decorators

Gerrit topic: https://review.openstack.org/#q,topic:bug/1595095,n,z

Addressed by: https://review.openstack.org/372400
    Fix getting the images list in Admin->Images

Addressed by: https://review.openstack.org/372408
    [WIP] Test integration tests with recent FF

Gerrit topic: https://review.openstack.org/#q,topic:dev-enabled-files-in-contrib,n,z

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.