v3:
Hi all! After long delay I am starting a new thread about persistent
dirty bitmaps. Not all ideas/requests are done from v2 thread, but I
need to start somewhere. Also, there are a lot of code changes to be
considered before everything is finished. Also, v2 thread is too large
and it is hard to handle it.
Here I'll summarize, what was changed. All ideas/comments, that was not
done from v2 thread I'll add as replays to this cover letter. If I miss
something, please add.
01-02: new patches, necessary preparation
03: docs
- add constraints
- document auto clear bit
- renames
L1 table -> Dirty Bitmap Table
Dirty Bitmaps Table -> Dirty Bitmap Directory
- remove Standard cluster with its zero bit,
struct changes:
extension header:
add 32bit dirty_bitmap_directory_size
it is needed to read the whole Dirty Bitmap Directory.
nb_dirty_bitmaps becomes excessive (it may be calculated by looking
through Dirty Bitmap Directory).
RFC: is it bad? should I drop nb_dirty_bitmaps?
directory entry:
- bitmap size is renamed to 'number of virtual bits'
- flags added. (self/aut/read_only are not reliazed in code for now)
- granularity is now represented by granularity_bits
04: structs and consts
Structures changed, accordingly to spec.
QCowDirtyBitmap now only contains name and offset. Offset is offset in
bytes in Dirty Bitmap Directory, so, bitmap header can be accessed:
cached version: s->dirty_bitmap_directory + offset
in image file: s->dirty_bitmap_directory_offset + offset
== dirty-bitmaps feature ==
initial patch [PATCH 2/8] qcow2: add dirty-bitmaps feature
is splitted out to 5 parts. The first with license notice and
constants is in 04. Then, there are:
05: read Dirty Bitmap Directory
06: load dirty bitmap
07: store dirty bitmap
08: add dirty bitmap extension
The code is significantly changed and reorganized. For your comments:
- fixes about flushes
- g_try_... functions are used. But I left the old g_malloc/g_new for
things with constant or strictly limited size (dirty bitmap header,
with its name, name size < 1024, one cluster, bitmap name (< 1024))
- input validation added.
- dirty bitmap directory is read/written wholly, not by parts.
- load/store through bdrv_dirty_bitmap_[de]serialize_part
- 'delete dirty bitmap' function is removed. This function is not used
in v3. May be this functionality should appear in qemu_img, or with qmp
series.
- qcow2_write_dirty_bitamps function is removed. This functionality is
rewritten, now Bitmap Directory is rewritten from
s->dirty_bitmap_directory, (directory_write()), and not generated from
s->dirty_bitmaps
09: load check
- new functionality, just check existance of dirty bitmap by name
10: store block dirty bitmap
- bdrv_store_dirty_bitmap simplified, as bitmap is simply passed to
qcow2_dirty_bitmap_store
11: load block dirty bitmap
- bdrv_load_dirty_bitmap simplified, as bitmap is created by
qcow2_dirty_bitmap_load. Also, granularity parameter is removed
patch [[PATCH 5/8] qcow2: add qcow2_dirty_bitmap_delete_all] is
removed. This function is not used in v3. May be this functionality
should appear in qemu_img, or with qmp series.
12: autoclear bit
- if it is not set, but bitmap extension is found, it just skipped and
warning printed. Dirty bitmaps are not cleread from the image as in v2
- handling in qcow2-dirty-bitmap.c is changed, the bit is just set when
adding entry to Dirty Bitmap Directory in the image.
13: cmd line
create=on|off flag added. Now, the bitmap is not auto-created.
By default flag is off.
If on, then new bitmap will be created in the image, if the bitmap with
same name is already exists an error will be generated.
If off, then the bitmap will be loaded from the image, if there is no
one an error will be generated.
If create=off and granularity is specified then granularity will be
checked for loaded bitmap and if not match an error will be generated.
also, s/drive/node
also, dirty_bitmap_func():
- add errp parameter and handle this in main()
- open image with BDRV_O_RDWR for loading bitmaps
14-16: new patches
17: test
- add three test cases.
v2:
- rebase on my 'Dirty bitmaps migration' series
- remove 'print dirty bitmap', 'query-dirty-bitmap' and use md5 for
testing like with dirty bitmaps migration
- autoclean features
v1:
The bitmaps are saved into qcow2 file format. It provides both
'internal' and 'external' dirty bitmaps feature:
- for qcow2 drives we can store bitmaps in the same file
- for other formats we can store bitmaps in the separate qcow2 file
QCow2 header is extended by fields 'nb_dirty_bitmaps' and
'dirty_bitmaps_offset' like with snapshots.
Proposed command line syntax is the following:
-dirty-bitmap [option1=val1][,option2=val2]...
Available options are:
name The name for the bitmap (necessary).
file The file to load the bitmap from.
file_id When specified with 'file' option, then this file will
be available through this id for other -dirty-bitmap
options when specified without 'file' option, then it
is a reference to 'file', specified with another
-dirty-bitmap option, and it will be used to load the
bitmap from.
drive The drive to bind the bitmap to. It should be specified
as 'id' suboption of one of -drive options. If nor
'file' neither 'file_id' are specified, then the bitmap
will be loaded from that drive (internal dirty bitmap).
granularity The granularity for the bitmap. Not necessary, the
default value may be used.
enabled on|off. Default is 'on'. Disabled bitmaps are not
changing regardless of writes to corresponding drive.
Examples:
qemu -drive file=a.qcow2,id=disk -dirty-bitmap name=b,drive=disk
qemu -drive file=a.raw,id=disk \
-dirty-bitmap name=b,drive=disk,file=b.qcow2,enabled=off
Vladimir Sementsov-Ogievskiy (8):
spec: add qcow2-dirty-bitmaps specification
qcow2: add dirty-bitmaps feature
block: store persistent dirty bitmaps
block: add bdrv_load_dirty_bitmap
qcow2: add qcow2_dirty_bitmap_delete_all
qcow2: add autoclear bit for dirty bitmaps
qemu: command line option for dirty bitmaps
iotests: test internal persistent dirty bitmap
block.c | 82 +++++++
block/Makefile.objs | 2 +-
block/qcow2-dirty-bitmap.c | 537 ++++++++++++++++++++++++++++++++++++++++++
block/qcow2.c | 69 +++++-
block/qcow2.h | 61 +++++
blockdev.c | 38 +++
docs/specs/qcow2.txt | 66 ++++++
include/block/block.h | 9 +
include/block/block_int.h | 10 +
include/sysemu/blockdev.h | 1 +
include/sysemu/sysemu.h | 1 +
qemu-options.hx | 37 +++
tests/qemu-iotests/118 | 83 +++++++
tests/qemu-iotests/118.out | 5 +
tests/qemu-iotests/group | 1 +
tests/qemu-iotests/iotests.py | 6 +
vl.c | 100 ++++++++
17 files changed, 1105 insertions(+), 3 deletions(-)
create mode 100644 block/qcow2-dirty-bitmap.c
create mode 100755 tests/qemu-iotests/118
create mode 100644 tests/qemu-iotests/118.out
--
1.9.1
