Fetch resources in parallel

Registered by Ivan Kolodyazhny on 2017-09-19

Summary
=======
Speed up resource fetching using futurist library and GreenThreadPoolExecutor

Motivation
========
On several tabs (e.g. networks, qoutas) we have to call different server APIs to fetch all needed data. We can do it in parallel to mage pages rendering faster.

Description
=========
Futurist library and GreenThreadPoolExecutor shuld be used where it's possible.

Testing
======
Current tests should be enough to verify that nothing is broken.

Outside Dependencies
==================
Futurist library is already including to requirements.txt, we need to add eventlet to get GreenThreadPoolExecutor working

Blueprint information

Status:
Started
Approver:
Ivan Kolodyazhny
Priority:
Undefined
Drafter:
Ivan Kolodyazhny
Direction:
Needs approval
Assignee:
Ivan Kolodyazhny
Definition:
Approved
Series goal:
None
Implementation:
Started
Milestone target:
None
Started by
Ivan Kolodyazhny on 2018-04-04

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bug/1660025,n,z

Addressed by: https://review.openstack.org/426493
    Make API calls in Volumes view parallel

Could you show the fact that how making fetching resources in parallel contributes the speedup? Does futurist used in the proposed review use native threads rather than python greenthread? If so, this changes the way of optimization as it changes the balance of apache configurations like the number of processes or workers. At a first glance, I am not sure at all how this contributes the total performance. -- amotoki

[e0ne] related thread in the ML: http://lists.openstack.org/pipermail/openstack-dev/2017-October/124129.html
[e0ne] some performance testing results: https://docs.google.com/spreadsheets/d/14zDpdkPUfGDR_ZycaGUoT64kHsGi1gL9JOha8n5PVeg/edit?usp=sharing

Gerrit topic: https://review.openstack.org/#q,topic:bp/fetch-resources-in-parallel,n,z

Addressed by: https://review.openstack.org/522556
    Refactor futurist calls

Addressed by: https://review.openstack.org/483948
    Speed up quotas fetching using futurist library

Addressed by: https://review.openstack.org/495911
    Load networks and security groups in parallel

Addressed by: https://review.openstack.org/559962
    Speed up quotas fetching using futurist library

Addressed by: https://review.openstack.org/559971
    Refactor futurist calls

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.