Support restart of partial download

Registered by Eoghan Glynn

Downloads of very large images may be interrupted prior to completion, due to dropped connections and such.

Rather than discard the partial image and start again from the zero offset, it could be preferable to restart from the current offset where the storage backage support range-based GETs and then knit the remainder of the image content into the partial result of the first GET.

In order to avoid polluting the API, this logic should be limited to the service->backend leg (as opposed to end-to-end interaction from the client side).

The change should be restricted to the glance backend store gluecode for those
stores that support Ranges (S3, Swift, HTTP).

In the S3 case, the boto.s3.resumable_download_handler already provides restartable logic in a fairly generic way, similar logic could be applied to the Swift and HTTP backends.

UPDATE:

At the J summit, we discussed adding this to both, the http and the store, APIs.

Blueprint information

Status:
Complete
Approver:
None
Priority:
Medium
Drafter:
Eoghan Glynn
Direction:
Approved
Assignee:
Flavio Percoco
Definition:
Discussion
Series goal:
Accepted for juno
Implementation:
Implemented
Milestone target:
milestone icon 2014.2
Started by
Mark Washenberger
Completed by
Mark Washenberger

Related branches

Sprints

Whiteboard

I am interested in this work. I would like it to include uploads as well.
----

I think we should move this blueprint forward.
-- flaper87

Looks like we have some support for this in swift since ae568d7858edeee4d60e52778243dc406651d46d

But it would be nice to support more stores as well. Perhaps the ranged read functionality could be part of the store interface and the retry logic could move up?

markwash wishlist 2014-02-14

Agreed, I think this is very useful but it has to wait 'til we complete the glance.store library. We can support this for some stores already: FS, swift, rdb, (gridFS I think).

-- flaper87 2014-04-10

rosmaita wishlist 2014-04-25

Gerrit topic: https://review.openstack.org/#q,topic:glance.store,n,z

Addressed by: https://review.openstack.org/103068
    API support for random access to images

Gerrit topic: https://review.openstack.org/#q,topic:bp/restartable-image-download,n,z

Addressed by: https://review.openstack.org/119000
    Complete random_access for the filesystem store

Addressed by: https://review.openstack.org/100636
    Adopt glance.store library in Glance

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.