On 03/20/2014 12:59 AM, Andrei Mikhailovsky wrote:
Hi guys,
I was wondering if this is a bug?
No, it's a "feature".
I've noticed that during volume migration from NFS to RBD primary storage the
volume image is first copied to /tmp and only then to the RBD storage. This
seems silly to me as one would expect a typical volume to be larger than the
host's hard disk. Also, it is a common practice to use tmpfs as /tmp for
performance reasons. Thus, a typical host server will have far smaller /tmp
folder than the size of an average volume. As a result, volume migration would
break after filling the /tmp and could probably cause a bunch of issue for the
KVM host itself as well as any vms running on the server.
Correct. The problem was that RBD images know two formats. Format 1
(old/legacy) and format 2.
In order to perform cloning images should be in RBD format 2.
When running qemu-img convert with a RBD image as a destination qemu-img
will create a RBD image in format 1.
That's due to this piece of code in block/rbd.c in Qemu:
ret = rbd_create(io_ctx, name, bytes, &obj_order);
rbd_create() creates images in format 1. To use format 2 you should use
rbd_create2() or rbd_create3().
With RBD format 1 we can't do snapshotting or cloning, which we require
in ACS.
So I had to do a intermediate step where I first wrote the RAW image
somewhere and afterwards write it to RBD.
After some discussion a config option has been added to Ceph:
OPTION(rbd_default_format, OPT_INT, 1)
This allows me to do this:
qemu-img convert .. -O raw .. rbd:rbd/myimage:rbd_default_format=2
This causes librbd/RBD to create a format 2 image and we can skip the
convert step to /tmp.
This option is available since Ceph Dumpling 0.67.5 and was not
available when ACS 4.2 was written.
I'm going to make changes in master which skip the step with /tmp.
Technically this can be backported to 4.2, but then you would have to
run your own homebrew version of 4.2
It also seems that the /tmp is temporarily used during a template creation .
Same story as above.
My setup:
ACS 4.2.1
Ubuntu 12.04 with KVM
RBD + NFS for Primary storage
NFS for Staging and Secondary storage
Thanks
Andrei
