Overview -------- This patch series adds image streaming support for QED image files. QMP/HMP commands are added to perform image streaming at runtime. This interface is already supported by libvirt.
The goal is to implement image streaming in a generic way for all image formats that support backing files. In the meantime, I want to share the latest QED-specific patch series. Image streaming populates the file in the background while the guest is running. This makes it possible to start the guest before its image file has been fully provisioned. Example use cases include: * Providing small virtual appliances for download that can be launched immediately but provision themselves in the background. * Reducing guest provisioning time by creating local image files but backing them with shared master images which will be streamed. When image streaming is enabled, the unallocated regions of the image file are populated with the data from the backing file. This occurs in the background and the guest can perform regular I/O in the meantime. Once the entire backing file has been streamed, the image no longer requires a backing file and will drop its reference. Example invocation ------------------ $ # my_fedora.qed is a tiny file initially but will be streamed when the guest starts $ ./qemu-img create -f qed -o backing_file=fedora-14.img my_fedora.qed Formatting 'my_fedora.qed', fmt=qed size=10737418240 backing_file='fedora-14.img' cluster_size=0 table_size=0 $ # run the guest and stream fedora-14.img into my_fedora.qed $ x86_64-softmmu/qemu-system-x86_64 -m 512 -enable-kvm -drive if=virtio,file=my_fedora.qed,cache=none,stream=on Details on changes ------------------ Image streaming introduces a new bdrv_aio_copy_backing() interface. Block drivers that implement this interface support streaming. This function scans for an unallocated cluster and populates it with data from the backing file. The details of populating the image file are actually best implemented as a copy-on-read operation. Copy-on-read means that a read request will populate the image file if it needs to fetch data from the backing file. The copy-on-read feature can be used outside the context of streaming and this patch series therefore introduces the -drive copy-on-read=on option for that purpose. The new block_stream QMP/HMP command can be used to start streaming a block device. QMP events are raised on completion and failure so that polling is not required. Adam Litke <a...@us.ibm.com> has implemented the libvirt APIs for image streaming: http://www.redhat.com/archives/libvir-list/2011-July/msg01570.html Patches 1-6 block: add -drive copy-on-read=on|off qed: replace is_write with flags field qed: extract qed_start_allocating_write() qed: make qed_aio_write_alloc() reusable qed: add support for copy-on-read qed: avoid deadlock on emulated synchronous I/O These patches add copy-on-read support and implement it for QED. Patches 7-13 block: add bdrv_aio_copy_backing() qmp: add block_stream command qmp: add block_job_cancel command qmp: add query-block-jobs command qmp: add block_job_set_speed command block: add -drive stream=on|off qed: intelligent streaming implementation These patches implement image streaming using copy-on-read. Patch 14 trace: trace bdrv_aio_readv/writev error paths Additional trace events to identify I/O errors. Patch 15 tests: add image streaming QMP interface tests A Python script that performs basic QMP tests of image streaming. v2: * Implement latest block_stream QMP/HMP API * Split monitor command patches into separate commits * Add rate-limiting * Remove iteration interface where client drives streaming v1: * -drive copy-on-read=on|off,stream=on|off instead of image header bits * Latest libvirt API compatibility * Workaround and assert for synchronous I/O emulation deadlock Anthony Liguori (3): qed: add support for copy-on-read block: add bdrv_aio_copy_backing() qed: intelligent streaming implementation Stefan Hajnoczi (12): block: add -drive copy-on-read=on|off qed: replace is_write with flags field qed: extract qed_start_allocating_write() qed: make qed_aio_write_alloc() reusable qed: avoid deadlock on emulated synchronous I/O qmp: add block_stream command qmp: add block_job_cancel command qmp: add query-block-jobs command qmp: add block_job_set_speed command block: add -drive stream=on|off trace: trace bdrv_aio_readv/writev error paths tests: add image streaming QMP interface tests block.c | 66 +++++++++- block.h | 6 + block/qed.c | 363 ++++++++++++++++++++++++++++++++++++++++++++++-------- block/qed.h | 8 +- block_int.h | 3 + blockdev.c | 286 +++++++++++++++++++++++++++++++++++++++++++ blockdev.h | 8 ++ hmp-commands.hx | 51 ++++++++- monitor.c | 19 +++ monitor.h | 1 + qemu-config.c | 8 ++ qemu-options.hx | 13 ++- qerror.c | 8 ++ qerror.h | 6 + qmp-commands.hx | 171 ++++++++++++++++++++++++++ test-stream.py | 193 +++++++++++++++++++++++++++++ trace-events | 10 ++- 17 files changed, 1156 insertions(+), 64 deletions(-) create mode 100644 test-stream.py -- 1.7.5.4
