Download Reports of Horizon Detailed Views, in dfferent formats

I don't think it's responsibility of Horizon to generate any reports. As a rule, Horizon is the UI for all the OpenStack services, and doesn't add any business logic of its own. That means, that all the same things you can do with Horizon, you should also be able to do using the command-line tools or APIs of the services.

So, if any of those services ever grows an option to generate a report, by all means Horizon should have user interface to use that option conveniently from the UI. But I don't think any of the services have such an option, so I think the right thing to do would be to first implement those options in the corresponding servies, together with API and command-line tools, and then add the UI for that to Horizon.

Alternatively, maybe it's an opportunity to create a new project that would focus on reporting?
-- Radomir Dopieralski, 2014-11-26 16:20 UTC

Hi Radomir,

There are two different aspects here that you are referring to -

1. Reporting - As you said, the individual openstack components should implement this first and then horizon should use it. This can be more of scheduled reports, graphs, comparisons, etc which can be daily/weekly/monthly generated/mailed to a set of users for tracking purpose.

2. Data output in different formats - Horizon is already generating HTML tabulated data through its different components' api. We are just giving the option of downloading this data in different formats -pdf/csv/png so that this can be used by all users at any time, to download the records in the user system location that can later be used for various purposes like helping/keeping them in checking and making decisions, etc.

Hence, here we are not focussing on the Reporting module at all and just rendering the data in formats that can be downloaded on user machine, analysed and monitored by any user at any time (2nd option ).

For example, 'Download CSV Summary' option is already present in the Overview, Resource Usage, etc sections. I am trying to extend this feature to other sections and enhancing it by giving the option of downloading in other formats as well.

--Swati

[mrunge, Dec 17, 2014]
Swati, when marking work items as done, could you please link to review requests?

As someone who implemented PDF report generation in a totally different scope: I'd expect almost instantly feature requests to change generated reports (theme, summarization etc.)
There are quite a log, really sophisticated report generators out there; IMHO we might provide data as CSV, but nothing more.

(Comparable to billing, which is something we as OpenStack project are not trying to implement at all)
Thanks

Gerrit topic: https://review.openstack.org/#q,topic:bp/horizon-reports-in-different-formats,n,z

Addressed by: https://review.openstack.org/142676
    Download Reports of Horizon Detailed Views, in dfferent formats

Hi Matthias,

I have submitted the first patch with partial implementation. As of now, I have implemented the 'Download Reports' section for Instances (Admin) section. After incorporating all review comments and getting go-ahead, I will extend this to other sections as well.

Also, there are certain advantages of getting the reports generated as mentioned in the blueprint description earlier, therefore I think such features should be incorporated and depending upon the extent of customisations required, 'Reports' can later be taken up as a completly seperate entity for Openstack.

Apart from this, I would like to mention 1 limitation and 3 alternatives for the PNG format--
Limitation:
======
Currently, it gets the instance data visible on the current page.
Alternatives:
=======
1. I should get all the instances' data of all pages and zip the file, to allow for download later.
2. Get all the instances data in 1 PNG, depending on the user to zoom-in/zoom-out.
3. Leave it upto the user to get the PNG of the current page.

Thanks,
Swati.

Hi,

I have some doubts.

For the implementation of the Blueprint https://blueprints.launchpad.net/horizon/+spec/horizon-reports-in-different-formats,

I have used some additional packages and recorded these dependencies in requirements.txt file.They are -
1) html5lib>=0.999
2) pisa>=3.0.33
3) pyPdf2>=1.24
4) reportlab>=2.5
5) xhtml2pdf>=0.0.6

However, I was also told to check out https://github.com/openstack/requirements, which has guidelines to add the packages in global-requirements.txt. One of the guidelines says that it should be Python 3 compatible, and only exception cases are allowed. For this blueprint, html5lib and pyPdf2 are not Python 3 compatible. Therefore, is it fine if I go ahead with these, or should I try out other options like 'Django Easy PDF'??
Please guide me with the process, if I am following it correctly by listing the additional packages in global-requirements.txt file, or is there some other approach?

Thanks,
Swati

[2015-02-20 david-lyle] I'm not sure all these packages are really necessary to accomplish the goal. I also think a simple CSV format will cover most use cases. Another tool can complete the conversion.
Since there is disagreement as to the approach, I'm removing from kilo until we can settle on an acceptable path forward.

Hi David,

Currently, I am converting the tabular data directly into CSV, PDF and PNG formats. As you said, using a tool/API to convert into a PNG/PDF file will involve 2 steps each ("Convert to csv file" --> "store in temp location" --> "convert to PDF/PNG" ). This, I believe will be slow as far as the performance is considered.

Still if you wish to adopt this approach, I will use CSV in this manner and compare the performance for you.

As of now, the code is running smoothly for the patches I have submitted and successfully converting into all formats (the problem lies in Python 3 compatibility).

Please provide your valuable inputs for the same.

Thanks and Regards,
Swati

Hi David,

In addition to the discussion we had on mail, I would like to bring this to your notice that, I have figured out some other Python 3 compatible package for PDF reports.

So, I will be shortly submitting the patch for CSV and PDF reports.

Rest, you can then provide your inputs after the submission.

[2015-03-19 robcresswell]
I strongly disagree that Horizon should be handling the logic here. In my opinion, it should be a separate project, and if Keystone shows it is available Horizon can provide the UI. The other advantage here is that deployers/sysadmins can use it via CLI.

Regards,
Rob

Hi Rob,

As far as handling reporting feature in Horizon is considered, I strongly feel we can have it here. The reason being it is helping the project users also to access the instances, images, hypervisors, volumes, etc data in csv and pdf formats. Also, the main fact supporting this is we already have the csv option available for some categories in horizon. You can even cross-check this to confirm that though the admin users can access it via CLI, it is already available in horizon. As suggested, we can also discuss this in the IRC meeting.

Thanks

[2015-03-20 robcresswell]
The primary justification here seems to be "we're already doing CSV, so why not do X other formats". This makes all the functionality tightly coupled with the dashboard, when it really doesn't need to be. It would be good to hear more from deployers, but I still think it makes more sense to run the logic and requirements as a separate project. I agree that being able to easily generate multi-format reports is definitely helpful behaviour, I just don't think this is the way to go about it.

Regards,
Rob

Hi Rob,

I understand your comments. However, I believe that this reporting feature is required as a seperate project or via dashboard. For the next steps, we can discuss it over the next IRC meeting and finalise the future action plan.

Regards,
Swati

Hi All,

I have submitted a new patch with incorporated comments. However, in Kilo release since now the Horizon part has been partly migrated to Angular JS and no supporting document as such is available to help with the part migrated, this blueprint is working with the PDF functionality and CSV generation JS code is partially working.

Dependencies are added to https://review.openstack.org/#/c/163304

Looking forward for the review and suggestion for the future approach -
1)Have another button for CSV generation rather than drop-down
2)Wait for supporting doc for Angualr JS

Will be joining IRC also for this.

Thanks.
Swati

Hi All,

Please let me know if I should proceed with the CSV functionality only for all tabular data getting rendered in all sections or in case I need to work on the PDF part also, I request to kindly review the dependencies added for this on https://review.openstack.org/#/c/163304.

Thanks,
Swati (25-Aug-2015)

----------------
[mrunge, 2015-08-27]
several persons voiced the concern, creating reports is not horizons job. One could even argue, outputting .csv data is not.

I share this concern. You could integrate an external utility to create those reports.

[2016-03-30 robcresswell] This was raised during the Horizon Drivers meeting (https://wiki.openstack.org/wiki/Meetings/HorizonDrivers) We believe it should exist as a plugin or the logic be maintained in a python client, not core Horizon content.

Work items:
Design the UI (for 1 section) : DONE
Implement PDF Output (for 1 section) : DONE
Implement CSV Output (for 1 section) : DONE
Implement PNG Output (for 1 section) : DONE
Testing (for 1 section) : DONE
Mention dependencies (in requirements.txt) : DONE
Submit Patches with one section(Admin->Instances) implemented, for feedback : DONE
Updated code for Kilo release : DONE
Incorporate Review comments (with changes to the requirements and dependencies): INPROGRESS
Implement in all sections for Admin : TODO
Implement in all sections for Project : TODO
Write Test-cases for all sections : TODO
Test the UI and complete functionality : TODO