On Tue, Jun 9, 2020 at 3:31 AM Yi Li <y...@winhong.com> wrote:
>
> Since Ceph version Infernalis (9.2.0) the new fast-diff mechanism
> of RBD allows for querying actual rbd image usage.
>
> Prior to this version there was no easy and fast way to query how
> much allocation a RBD image had inside a Ceph cluster.
>
> To use the fast-diff feature it needs to be enabled per RBD image
> and is only supported by Ceph cluster running version Infernalis
> (9.2.0) or newer.
>
> Without the fast-diff feature enabled qemu-img will report an allocation
> identical to the image capacity.
>
> 'qemu-img info rbd:cepharm/liyi-rbd' might output for example:
>
> image: json:{"driver": "raw", "file": {"pool": "cepharm",
> "image": "liyi-rbd", "driver": "rbd"}}
> file format: raw
> virtual size: 20 GiB (21474836480 bytes)
> disk size: 0 B
> cluster_size: 4194304
>
> Newly created rbds will have the fast-diff feature enabled.
>
> Signed-off-by: Yi Li <y...@winhong.com>
> ---
> block/rbd.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 60 insertions(+)
>
> diff --git a/block/rbd.c b/block/rbd.c
> index 617553b022..f231653f7b 100644
> --- a/block/rbd.c
> +++ b/block/rbd.c
> @@ -1107,6 +1107,65 @@ static int64_t qemu_rbd_getlength(BlockDriverState *bs)
> return info.size;
> }
>
> +#if LIBRBD_VERSION_CODE > 265
> +static int disk_usage_callback(uint64_t offset, size_t len, int exists,
> + void *arg)
> +{
> + uint64_t *used_size = (uint64_t *)(arg);
> + if (exists) {
> + (*used_size) += len;
> + }
> + return 0;
> +}
> +#endif
> +
> +static int64_t qemu_rbd_allocated_file_size(BlockDriverState *bs)
> +{
> + BDRVRBDState *s = bs->opaque;
> + rbd_image_info_t info;
> + int r;
> + uint64_t used_size = 0;
> + uint64_t features = 0;
> +
> + r = rbd_stat(s->image, &info, sizeof(info));
> + if (r < 0) {
> + return r;
> + }
> +
> + r = rbd_get_features(s->image, &features);
> + if (r < 0) {
> + return r;
> + }
You should probably test the flags to ensure that the
RBD_FLAG_FAST_DIFF_INVALID flag is not set [1]. It's potentially very
slow and expensive to calculate the disk usage w/o a fast-diff (on
large images) since it requires iterating over every possible 4MiB
backing data object (by default) to query its actual usage.
> + /*
> + * rbd_diff_iterate2() is available in versions above Ceph 0.94 (Hammer)
> + * It uses a object map inside Ceph which is faster than
> rbd_diff_iterate()
> + * which iterates all objects.
> + * LIBRBD_VERSION_CODE for Ceph 0.94 is 265. In 266 and upwards
> diff_iterate2
> + * is available
> + */
> +#if LIBRBD_VERSION_CODE > 265
> + if (features & RBD_FEATURE_FAST_DIFF) {
> +
> + /*
> + * RBD image fast-diff feature enabled
> + * Querying for actual allocation.
> + */
> + r = rbd_diff_iterate2(s->image, NULL, 0, info.size, 0, 1,
> + &disk_usage_callback,
> + &used_size);
> + if (r < 0) {
> + return r;
> + }
> + } else {
> + used_size = info.size;
> + }
> +#else
> + used_size = info.size;
> +#endif
> + return used_size;
> +}
> +
> static int coroutine_fn qemu_rbd_co_truncate(BlockDriverState *bs,
> int64_t offset,
> bool exact,
> @@ -1316,6 +1375,7 @@ static BlockDriver bdrv_rbd = {
> .bdrv_get_info = qemu_rbd_getinfo,
> .create_opts = &qemu_rbd_create_opts,
> .bdrv_getlength = qemu_rbd_getlength,
> + .bdrv_get_allocated_file_size = qemu_rbd_allocated_file_size,
> .bdrv_co_truncate = qemu_rbd_co_truncate,
> .protocol_name = "rbd",
>
> --
> 2.25.3
>
>
>
>
[1]
https://github.com/libvirt/libvirt/commit/21deeaf02fdf216b08210fc899579736973ca81d#diff-107c5451015e5980c90048ff615becb8
--
Jason
