glance-bittorrent-delivery
This blueprint is to add bittorrent support to Glance for image delivery to host machines. It enables us to optimize the transferring of common images and lessen the load on the Glance API servers by distributing the the transfer of data to all hosts within a zone.
Blueprint information
- Status:
- Complete
- Approver:
- Jay Pipes
- Priority:
- Undefined
- Drafter:
- Antony Messerli
- Direction:
- Approved
- Assignee:
- Antony Messerli
- Definition:
- Obsolete
- Series goal:
- None
- Implementation:
-
Not started
- Milestone target:
- None
- Started by
- Completed by
- Mark Washenberger
Related branches
Related bugs
Sprints
Whiteboard
Glance-api servers pull down image from swift and cache it. They then generate a torrent file based off the cache.
When compute requests image, it would check if an is_torrentable flag is present and if it is, it would pull down a torrent file instead of the image or relay a magnet uri to the client running on the host machine.
The torrent file is loaded into the client and begins to pull down the cache bits into storage on the host utilizing the trackers and the peer to peer seeds that are present within the zone. The rate of download and upload could be throttled to prevent the host from being sapped of resources. Once completed, the bittorrent client could continue to run and act as a seeder for other future builds. At some point, the seed could expire and the bits could be removed from the host.
In the case of XenServer, the image would land directly into the Storage Repository (SR) in order to prevent additional writes caused from copying it from one location and then copying it. Then in order to maintain a cache of the seed on the host, and prevent additional boot up time of the instance, the instance would be snapshotted which would create a read only base copy. We'd then copy the base copy to the cache on the host while the instance boots up. The host would then have a cache of the torrent and could begin to seed future builds. As space fills up, the oldest seeds could be removed and cycled out. The most popular builds would always be seeded across all the hosts and Glance-apis in a zone.
We could also look at promoting images that are popular. For example if an image receives 10 hits within 10 minutes, then we automatically mark it as torrentable and start rolling it out to the hosts.
Hosts would run a client and Glance-api nodes would run the trackers. This would be able to efficiently handle restores of images. There could also be another way to do saves as well.
<markwash>
I think the right way to do this is to have a backend that supports hosting torrents of image files, and expose the location of the image in that backend to the client. Then the client can choose to join the torrent.
</markwash>
