Streamline glance images upload via Horizon
Summary
=======
To upload Glance images directly to Glance, bypassing Horizon's Django views and provide status of upload in UI. There are 2 implementations of this feature in Horizon: for Django [legacy] content and for new Angular content.
Motivation
=========
Piping large Glance images through Horizon incurs the risk of exhausting space on the Controller node, because usually it's more limited the the space of the Storage node used by Glance.
Also from an operator perspective the service of uploading images has different characteristics than serving the dashboard. The dashboard can live and thrive on 1Gbps links possible on the controllers nodes themselves. Glance on the other hand may have to be deployed on service nodes possibly with multiple 10Gbps links to handle the user data, storage etc.
Finally, transfer of gigabyte-sized images takes a significant time and in a modern web it seems a waste of time to upload it in a synchronous way thus blocking all other operations in dashboard. Even if the user could open another tab/window to continue doing some other things in Horizon, that isn't a good pattern. Hence, it's better to upload images asynchronously and provide the progress of upload in the UI, so the user is aware that in order to upload to finish he should not close the page/tab.
Description
=========
[Django] First we need to expunge the 'Image File' field from the list of fields managed by Django, so the contents of file field doesn't even get to Django. Then using some cleverly written JS code, attached to the 'Create Image' form, an asynchronous request directly to Glance API should be initiated as soon as form is considered valid by Django. In order for request to succeed both Horizon and Glance must play nice around CORS headers.
[Angular] No need to do all the magic specific to Django, since we could upload the image directly to Glance service using ng-file-upload library.
UX
===
[Django] A standard spinner modal element which is shown on form submit needs to be enhanced with the progress bar.
[Angular] Create Image modal doesn't hide until the upload is complete, with upload progress meter is being updated constantly and shown instead of the 'Select File...' button.
Wireframes, Mocks, Videos and UI Markup
-------
TBD
Testing
======
Should be covered both with unit and integration tests.
Outside Dependencies
==================
The client-side code relies on CORS being enabled for Glance service (otherwise browser would forbid the PUT request to a location different from the one form content came from). In a Devstack setup you'll need to edit [cors] section of glance-api.conf file, setting `allowed_origin` setting to the full hostname of the web server (say, http://<HOST_IP>
Requirements Update Required
=======
N/A
Doc Impact
=========
[Django] HORIZON_
[Angular] Angular Create Image file upload uses the same setting HORIZON_
Blueprint information
- Status:
- Complete
- Approver:
- Rob Cresswell
- Priority:
- Medium
- Drafter:
- Timur Sufiev
- Direction:
- Approved
- Assignee:
- Timur Sufiev
- Definition:
- Approved
- Series goal:
- Accepted for 10.0.0-newton
- Implementation:
-
Implemented
- Milestone target:
-
newton-3
- Started by
- Rob Cresswell
- Completed by
- Rob Cresswell
Related branches
Related bugs
Sprints
Whiteboard
Noted by david-lyle during team meeting the new glance tasks API may be useful in this scenario https:/
[tsufiev, Sep 23, 2015] According to https:/
Also I've spoken with mfedosin, he confirmed that fixing long-living image operations to not interrupt due to Keystone token expiration is planned for Mitaka, Keystone trusts will be used. And yes, that's a Glance responsibility in that case.
Gerrit topic: https:/
Addressed by: https:/
[WIP] Glance images async upload
[tsufiev, Oct 16, 2015] One more thing to consider here: Glance API expects the image contents being uploaded to consist entirely of raw binary data, while using combination of ajax + FormData() results in having binary data prepended with 'multipart/
So the next idea here is to slice large images into chunks and send each chunk (as a binary string) with a separate ajax request. For this to happen, Glance should support partial uploads - not a big deal, I suppose (could be done with HTTP headers, like Content-Length etc). So I'm going first to discuss this first with Glance folks.
Again, the current limitation as I see it: either you use FormData() and get it encoded as multipart/form-data (but it handles uploading large files for us), or you should read binary content into browser's memory and send it without extra headers. Please correct me if I missed something.
[tsufiev, Nov 24, 2015] Waiting for multipart/form-data parser be implemented by Glance team. Resumable upload was considered more complicated solution (not preferred one).
Gerrit topic: https:/
Gerrit topic: https:/
Addressed by: https:/
Embed support for external data sinks into api.glance
Gerrit topic: https:/
Gerrit topic: https:/
Addressed by: https:/
[NG] Support local file upload in Create Image workflow
Addressed by: https:/
[WIP] See how CORS works for Image Create + upload
Addressed by: https:/
[NG] Enhance Create Image workflow with upload tracking
Gerrit topic: https:/
