allow glance to use memcache for image uploads
add a memcache to glance api for uploads to backend store. Currently, the data is written to disk and then streamed onto backend store (for most stores). However, if we can have a cache which avoid disk IO it might help speed up uploads.
Blueprint information
- Status:
- Complete
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Nikhil Komawar
- Direction:
- Needs approval
- Assignee:
- Nikhil Komawar
- Definition:
- Obsolete
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
- None
- Started by
- Completed by
- Nikhil Komawar
Related branches
Related bugs
Sprints
Whiteboard
Can you provide some more detail about how this caching mechanism would work? Which seam in current glance code could it be stitched into?
In addition, I'm concerned that memcache is not the right approach for the bulky data glance is accessing, can you provide some motivation about memcached that shows its advantages over the existing file-based cache? Its also possible that if we look at this problem more generally other solutions might come to mind.
I've marked https:/
markwash more-info 2014-02-17
Comment (nikhil):-
Yeah, I agree that it seems rather tough to integrate this mechanism to be included in the current glance code seamlessly. Also, with the kind of bulky data we get, file based approach sounds better.
However, this BP was an attempt of some performance enhancement we would be able to on top of import and export BP. For some operations like image validation, we could leverage this functionality (until something like zerovm becomes feasibility). The goal is to enable running multiple tasks on the same node without doing bulk operations and helping improve the node capacity IO-wise. (Possibly use the same thing for downloads)
Maybe as we continue to advance with the import work, the optimization use case and an appropriate technology will reveal itself. But for the moment, it seems like memcached is not really going to fit into the picture for bulk image data, so I'm going to mark this as rejected for now.
markwash rejected 2014-03-07
