Thank you, Eric, for the thorough information—truly appreciate it.
Just to confirm what I understood, when we are reading a bitmap with
'x-dirty-bitmap' (for powered on vm of course), the 'start' is always a
logical offset no matter whether the record has 'offset' value or not. Is
this correct?
Also, I came across a case wherein we get the entire disk as allocated for
a raw format disk which is present on lvm or lvm-thin storage (the disk has
just a few MB data added, and the vm is in running state). Here is an
example of 1Gb data. Is this expected behaviour?
[{ "start": 0, "length": 1073741824, "depth": 0, "present": true, "zero":
false, "data": true, "compressed": false, "offset": 0}]
Regards,
Prashant
On Mon, Apr 7, 2025 at 8:24 PM Eric Blake <ebl...@redhat.com> wrote:
> On Mon, Apr 07, 2025 at 02:46:17PM +0530, prashant patil wrote:
> > Thanks Eric.
>
> [top-posting makes conversations harder to follow, so on this list we
> typically reply inline]
>
> > I have a few questions about the bitmap content shown by 'qemu-img map'.
> > From below sample bitmap data:
> > 1. Why only some of the extents have start and offset values? And why are
> > they the same values?
> > 2. What does the start value indicate? Is it logical offset or physical
> > offset of data into qcow2?
>
> Normally (when there is no x-dirty-bitmap in play), 'start' denotes
> the logical offset being reported on (you'd expect a map to list every
> logical offset; so the start of entry N+1 should be the sum of start +
> length of entry N), while 'offset' is where that extent begins in the
> underlying file. For a raw source, offset and start will be
> identical; for other sources, like qcow2, start is obviously logical,
> while offset is physical. Entries without 'offset' are places where
> the logical contents are compressed, synthesized, or otherwise have no
> 1:1 correspondence to an offset in the physical file. 'present'
> indicates whether the data is synthesized or not; 'offset' is going to
> be absent if 'present' is false; although it can also be absent even
> when 'present' is true such as in the case of compression.
>
> When it comes to exposing a qcow2 file over NBD, you generally want to
> have:
>
> qemu-nbd using '-f qcow2' => raw view => qemu-img using '-f raw'
>
> to expose only the logical contents over the wire. It is also
> possible to flip the responsibility:
>
> qemu-nbd using '-f raw' => qcow2 view => qemu-img using '-f qcow2'
>
> to expose the bare-metal qcow2 contents over the wire, but that gets
> less testing, in part because if you make the image writable, it tends
> to cause problems if the client writing to the qcow2 layer needs to
> allocate (since NBD does not have a resize command). Having the
> server open the file as qcow2 and only serving raw contents means the
> server can resize transparently. What's more, block status commands
> over NBD only work when the server is aware of the qcow2 nature of the
> file it is serving (if you expose qcow2 bits over the wire, the server
> treats the entire file as allocated, and there is is no dirty bitmap
> context for the client to read over NBD).
>
> Therefore, 'start' and 'offset' are going to be identical if you are
> using qemu-img map to read bitmaps from a server, since bitmaps can
> only be read when the wire has the raw view (not the qcow2 view).
>
> But when you add x-dirty-bitmap into the mix, you are asking qemu to
> DISREGARD the normal rules of backing file information and instead
> report on dirty bitmap information as if it were backing file info.
> This means that anywhere the dirty bitmap response differs from a
> normal backing file response, the output of qemu-img is reporting
> garbage data (for example, the 'present' bit is now bogus, which
> explains why the rows where qemu-img claims 'present' is false omit an
> 'offset'). Only 'start', 'length', and 'data' matter when using
> x-dirty-bitmap, with 'data=false' meaning the section was reported
> dirty, and 'data=true' meaning the section was reported unchanged.
>
> And if that is confusing, well yeah. Which is why I recommend using
> libnbd's nbdinfo --map, where the output is more obvious.
>
> >
> > root@be-proxmox1:/# qemu-img map --output=json --image-opts
> > "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:bitmap1"
> > [{ "start": 0, "length": 196608, "depth": 0, "present": true, "zero":
> > false, "data": true, "compressed": false, "offset": 0},
>
> The extent starting at logical offset 0 and lasting 196608 bytes is
> unchanged.
>
> > { "start": 196608, "length": 65536, "depth": 0, "present": false, "zero":
> > false, "data": false, "compressed": false},
>
> The extent starting at 196608 and lasting 65536 bytes is dirty
> (altered since the point in time when the bitmap was created).
>
> > > > qemu-img
> > > > map with x-dirty-bitmap image-opts.
> > >
> > > Here, this works, but feels like a hack, because it is relying on the
> > > x-dirty-bitmap feature of qemu. The libnbd project ships with an
> > > application 'nbdinfo --map' that can read the same information as
> > > 'qemu-img map' but with a much nicer layout. It's not going to
> > > necessarily be faster, but because it is a fully-supported feature of
> > > libnbd rather than a hack in qemu, it may prove more stable in the
> > > long run, and certainly easier to understand.
>
>
> For a worked example:
>
> # Create an image, dirty two different offsets, with a bitmap created in
> between
> $ qemu-img create -f qcow2 foo.qcow2 3M
> Formatting 'foo.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off
> compression_type=zlib size=3145728 lazy_refcounts=off refcount_bits=16
> $ qemu-io -f qcow2 foo.qcow2 -c 'w 1M 512'
> wrote 512/512 bytes at offset 1048576
> 512 bytes, 1 ops; 00.01 sec (42.906 KiB/sec and 85.8112 ops/sec)
> $ qemu-img bitmap -f qcow2 foo.qcow2 --add b1
> $ qemu-io -f qcow2 foo.qcow2 -c 'w 2M 512'
> wrote 512/512 bytes at offset 2097152
> 512 bytes, 1 ops; 00.00 sec (107.735 KiB/sec and 215.4704 ops/sec)
>
> # Now, expose the raw bytes of the image over NBD, and inspect with nbdinfo
>
> $ qemu-nbd -f qcow2 foo.qcow2 -A -Bb1 -t
> $ nbdinfo --map=qemu:dirty-bitmap:b1 nbd://localhost
> 0 2097152 0 clean
> 2097152 65536 1 dirty
> 2162688 983040 0 clean
> $ nbdinfo --map nbd://localhost
> 0 1048576 3 hole,zero
> 1048576 4096 0 data
> 1052672 61440 2 zero
> 1114112 983040 3 hole,zero
> 2097152 4096 0 data
> 2101248 61440 2 zero
> 2162688 983040 3 hole,zero
>
> Comparing those to qemu-img map --output=json should be instructive.
>
>
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.
> Virtualization: qemu.org | libguestfs.org
>
>
