On Mon, Apr 07, 2025 at 02:46:17PM +0530, prashant patil wrote:
> Thanks Eric.
[top-posting makes conversations harder to follow, so on this list we
typically reply inline]
> I have a few questions about the bitmap content shown by 'qemu-img map'.
> From below sample bitmap data:
> 1. Why only some of the extents have start and offset values? And why are
> they the same values?
> 2. What does the start value indicate? Is it logical offset or physical
> offset of data into qcow2?
Normally (when there is no x-dirty-bitmap in play), 'start' denotes
the logical offset being reported on (you'd expect a map to list every
logical offset; so the start of entry N+1 should be the sum of start +
length of entry N), while 'offset' is where that extent begins in the
underlying file. For a raw source, offset and start will be
identical; for other sources, like qcow2, start is obviously logical,
while offset is physical. Entries without 'offset' are places where
the logical contents are compressed, synthesized, or otherwise have no
1:1 correspondence to an offset in the physical file. 'present'
indicates whether the data is synthesized or not; 'offset' is going to
be absent if 'present' is false; although it can also be absent even
when 'present' is true such as in the case of compression.
When it comes to exposing a qcow2 file over NBD, you generally want to
have:
qemu-nbd using '-f qcow2' => raw view => qemu-img using '-f raw'
to expose only the logical contents over the wire. It is also
possible to flip the responsibility:
qemu-nbd using '-f raw' => qcow2 view => qemu-img using '-f qcow2'
to expose the bare-metal qcow2 contents over the wire, but that gets
less testing, in part because if you make the image writable, it tends
to cause problems if the client writing to the qcow2 layer needs to
allocate (since NBD does not have a resize command). Having the
server open the file as qcow2 and only serving raw contents means the
server can resize transparently. What's more, block status commands
over NBD only work when the server is aware of the qcow2 nature of the
file it is serving (if you expose qcow2 bits over the wire, the server
treats the entire file as allocated, and there is is no dirty bitmap
context for the client to read over NBD).
Therefore, 'start' and 'offset' are going to be identical if you are
using qemu-img map to read bitmaps from a server, since bitmaps can
only be read when the wire has the raw view (not the qcow2 view).
But when you add x-dirty-bitmap into the mix, you are asking qemu to
DISREGARD the normal rules of backing file information and instead
report on dirty bitmap information as if it were backing file info.
This means that anywhere the dirty bitmap response differs from a
normal backing file response, the output of qemu-img is reporting
garbage data (for example, the 'present' bit is now bogus, which
explains why the rows where qemu-img claims 'present' is false omit an
'offset'). Only 'start', 'length', and 'data' matter when using
x-dirty-bitmap, with 'data=false' meaning the section was reported
dirty, and 'data=true' meaning the section was reported unchanged.
And if that is confusing, well yeah. Which is why I recommend using
libnbd's nbdinfo --map, where the output is more obvious.
>
> root@be-proxmox1:/# qemu-img map --output=json --image-opts
> "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:bitmap1"
> [{ "start": 0, "length": 196608, "depth": 0, "present": true, "zero":
> false, "data": true, "compressed": false, "offset": 0},
The extent starting at logical offset 0 and lasting 196608 bytes is
unchanged.
> { "start": 196608, "length": 65536, "depth": 0, "present": false, "zero":
> false, "data": false, "compressed": false},
The extent starting at 196608 and lasting 65536 bytes is dirty
(altered since the point in time when the bitmap was created).
> > > qemu-img
> > > map with x-dirty-bitmap image-opts.
> >
> > Here, this works, but feels like a hack, because it is relying on the
> > x-dirty-bitmap feature of qemu. The libnbd project ships with an
> > application 'nbdinfo --map' that can read the same information as
> > 'qemu-img map' but with a much nicer layout. It's not going to
> > necessarily be faster, but because it is a fully-supported feature of
> > libnbd rather than a hack in qemu, it may prove more stable in the
> > long run, and certainly easier to understand.
For a worked example:
# Create an image, dirty two different offsets, with a bitmap created in between
$ qemu-img create -f qcow2 foo.qcow2 3M
Formatting 'foo.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off
compression_type=zlib size=3145728 lazy_refcounts=off refcount_bits=16
$ qemu-io -f qcow2 foo.qcow2 -c 'w 1M 512'
wrote 512/512 bytes at offset 1048576
512 bytes, 1 ops; 00.01 sec (42.906 KiB/sec and 85.8112 ops/sec)
$ qemu-img bitmap -f qcow2 foo.qcow2 --add b1
$ qemu-io -f qcow2 foo.qcow2 -c 'w 2M 512'
wrote 512/512 bytes at offset 2097152
512 bytes, 1 ops; 00.00 sec (107.735 KiB/sec and 215.4704 ops/sec)
# Now, expose the raw bytes of the image over NBD, and inspect with nbdinfo
$ qemu-nbd -f qcow2 foo.qcow2 -A -Bb1 -t
$ nbdinfo --map=qemu:dirty-bitmap:b1 nbd://localhost
0 2097152 0 clean
2097152 65536 1 dirty
2162688 983040 0 clean
$ nbdinfo --map nbd://localhost
0 1048576 3 hole,zero
1048576 4096 0 data
1052672 61440 2 zero
1114112 983040 3 hole,zero
2097152 4096 0 data
2101248 61440 2 zero
2162688 983040 3 hole,zero
Comparing those to qemu-img map --output=json should be instructive.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
