Support restart of partial download

Registered by Eoghan Glynn

Downloads of very large images may be interrupted prior to completion, due to dropped connections and such.

Rather than discard the partial image and start again from the zero offset, it could be preferable to restart from the current offset where the storage backage support range-based GETs and then knit the remainder of the image content into the partial result of the first GET.

In order to avoid polluting the API, this logic should be limited to the service->backend leg (as opposed to end-to-end interaction from the client side).

The change should be restricted to the glance backend store gluecode for those
stores that support Ranges (S3, Swift, HTTP).

In the S3 case, the boto.s3.resumable_download_handler already provides restartable logic in a fairly generic way, similar logic could be applied to the Swift and HTTP backends.


At the J summit, we discussed adding this to both, the http and the store, APIs.

Blueprint information

Eoghan Glynn
Flavio Percoco
Series goal:
Accepted for juno
Milestone target:
milestone icon 2014.2
Started by
Mark Washenberger
Completed by
Mark Washenberger

Related branches



I am interested in this work. I would like it to include uploads as well.

I think we should move this blueprint forward.
-- flaper87

Looks like we have some support for this in swift since ae568d7858edeee4d60e52778243dc406651d46d

But it would be nice to support more stores as well. Perhaps the ranged read functionality could be part of the store interface and the retry logic could move up?

markwash wishlist 2014-02-14

Agreed, I think this is very useful but it has to wait 'til we complete the library. We can support this for some stores already: FS, swift, rdb, (gridFS I think).

-- flaper87 2014-04-10

rosmaita wishlist 2014-04-25

Gerrit topic:,,n,z

Addressed by:
    API support for random access to images

Gerrit topic:,topic:bp/restartable-image-download,n,z

Addressed by:
    Complete random_access for the filesystem store

Addressed by:
    Adopt library in Glance


Work Items

This blueprint contains Public information 
Everyone can see this information.