Efficient sparse image transfer

Registered by Yusuke Hayashi on 2015-09-25

This adds option for efficient copy-image-to-volume for sparse image file.

In copy image to volume, image files are written to volume using 'qemu-img convert'.
When the image files include many sparse area, since qemu-img convert writes all data including '0' data, image transfer is inefficient.

This blueprint adds option for image transfer with skip '0' data by following way.
1. Convert image file to raw format file in local temporaly area
2. Write the raw file to volume with 'dd' command with 'conv=sparse' option, which skips sparse data.

Here are data measured in my environment where Linux servers connected directly via 1G iSCSI network and I used RHEL6.7 KVM guest image(https://rhn.redhat.com/rhn/software/channel/downloads/Download.do?cid=16952)

[Conventinal way]
# time qemu-img convert -t none -O raw rhel-guest-image-6.7-20150706.0.x86_64.qcow2 /dev/disk/by-path/ip-xxx.xxx.xxx.xxx:3260-iscsi-iqn.2015-09.com.test:server-lun-1

real 3m31.587s
user 0m8.539s
sys 0m1.475s

==> 3min32sec

[My proposal]
# time qemu-img convert -O raw rhel-guest-image-6.7-20150706.0.x86_64.qcow2 ./tmp.img

real 0m7.046s
user 0m4.732s
sys 0m2.406s

# time dd if=./tmp.img of=/dev/disk/by-path/ip-xxx.xxx.xxx.xxx:3260-iscsi-iqn.2015-09.com.test:server-lun-1 bs=1M conv=fdatasync,sparse
16384+0 records in
16384+0 records out
17179869184 bytes (17 GB) copied, 25.6639 s, 669 MB/s

real 0m25.667s
user 0m2.162s
sys 0m3.877s

==> Total : 33sec

Blueprint information

Status:
Started
Approver:
Sean McGinnis
Priority:
Medium
Drafter:
Yusuke Hayashi
Direction:
Approved
Assignee:
None
Definition:
Approved
Series goal:
None
Implementation:
Started
Milestone target:
None
Started by
Yusuke Hayashi on 2015-11-18

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/efficient-sparse-image-xfer,n,z

Addressed by: https://review.openstack.org/245635
    Add argment for efficient sparse image transfer

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.