This is a continuation of previous versions of this series. v2's cover letter was the most elaborate, which you can find here (includes benchmarks): http://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg03430.html
See patch 1 for an explanation of why this series exists and what it does. Patch 1 is basically the core of this series, the rest just employs the functions introduced there. In a later patch, we may want to change the meaning of the "constant" overlap checking option to mean the same as "cached", which is everything except for inactive L2 tables. This series does make checking for overlaps with inactive L2 tables at runtime just as cheap as everything else (constant time plus caching), but using these checks means qemu has to read all the snapshot L1 tables when opening a qcow2 file. This does not take long, of course, but it does result in a bit of overhead so I did not want to enable it by default. --- v4 part (RFC is still relevant) --- The new thing about v4 is that there now is a way to limit the total combined size of all data structures used for the new overlap prevention algorithm. Bad news: The limit is disabled by default. This is mainly because for image creation, it is pretty hard to find a sane default (it depends on the desired image size; but the image is not created with that size, but rather with size 0 and then truncated; inferring the default from the image size seems to make sense, but we cannot do this for images that are going to be truncated). RFC: An alternative would be to infer the limit like so: MAX(1M, guest_image_size / 1M). It seems sane to me and I guess it should work even for images with very, very large guest sizes. But I think it will break if you're trying to preallocate like a 1 PB image (which is not really a concern right now, but let's see about it in ten years)... On the other hand, the worst that happens is that the event is generated and the overlap checks won't be performed for some parts of the image during its creation. Doesn't sound too bad to me. What do you think, dears reviewers? v6: - Rebased on master - Patch 1: Fixed check_metadata_overlap() function if there is no metadata list (because overlap checks are switched off) git-backport-diff against v5: Key: [----] : patches are identical [####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/17:[0004] [FC] 'qcow2: Add new overlap check functions' 002/17:[----] [--] 'qcow2: Pull up overlap check option evaluation' 003/17:[----] [--] 'qcow2: Create metadata list' 004/17:[----] [--] 'qcow2/overlaps: Protect image header' 005/17:[----] [--] 'qcow2/overlaps: Protect refcount table' 006/17:[----] [--] 'qcow2/overlaps: Protect refcount blocks' 007/17:[----] [--] 'qcow2/overlaps: Protect active L1 table' 008/17:[----] [--] 'qcow2/overlaps: Protect active L2 tables' 009/17:[----] [--] 'qcow2/overlaps: Protect snapshot table' 010/17:[----] [--] 'qcow2/overlaps: Protect inactive L1 tables' 011/17:[----] [--] 'qcow2/overlaps: Protect inactive L2 tables' 012/17:[----] [--] 'qcow2: Use new metadata overlap check function' 013/17:[----] [--] 'qcow2/overlaps: Add "memory limit reached" event' 014/17:[----] [--] 'qcow2/overlaps: Add memory usage limit' 015/17:[----] [--] 'qcow2: Add overlap structure memory size options' 016/17:[----] [--] 'qapi: Expose new qcow2 overlap check options' 017/17:[----] [--] 'iotests: Test qcow2's overlap check memory limit' Max Reitz (17): qcow2: Add new overlap check functions qcow2: Pull up overlap check option evaluation qcow2: Create metadata list qcow2/overlaps: Protect image header qcow2/overlaps: Protect refcount table qcow2/overlaps: Protect refcount blocks qcow2/overlaps: Protect active L1 table qcow2/overlaps: Protect active L2 tables qcow2/overlaps: Protect snapshot table qcow2/overlaps: Protect inactive L1 tables qcow2/overlaps: Protect inactive L2 tables qcow2: Use new metadata overlap check function qcow2/overlaps: Add "memory limit reached" event qcow2/overlaps: Add memory usage limit qcow2: Add overlap structure memory size options qapi: Expose new qcow2 overlap check options iotests: Test qcow2's overlap check memory limit block/Makefile.objs | 3 +- block/qcow2-cluster.c | 13 + block/qcow2-overlap.c | 614 +++++++++++++++++++++++++++++++++++++++++++++ block/qcow2-refcount.c | 202 ++++++--------- block/qcow2-snapshot.c | 105 +++++++- block/qcow2.c | 162 ++++++++---- block/qcow2.h | 15 ++ docs/qmp/qmp-events.txt | 27 ++ qapi/block-core.json | 31 +++ qapi/event.json | 27 ++ tests/qemu-iotests/060 | 222 ++++++++++++++++ tests/qemu-iotests/060.out | 47 ++++ tests/qemu-iotests/group | 2 +- 13 files changed, 1297 insertions(+), 173 deletions(-) create mode 100644 block/qcow2-overlap.c -- 2.4.6
