Do you think I can apply the patch manually to the 4.3 branch? I would love to try it with 4.3, but not too adventitious to upgrade my setup to 4.4 yet ))
Andrei ----- Original Message ----- From: "Wido den Hollander" <w...@widodh.nl> To: dev@cloudstack.apache.org Sent: Monday, 24 March, 2014 12:29:36 PM Subject: Re: ACS and KVM uses /tmp for volumes migration and templates On 03/23/2014 08:01 PM, Andrei Mikhailovsky wrote: > Wido, > > Could you please let me know when you've done this so I could try it out. > Would it be a part of the 4.3 branch or 4.4? > I'll do that. It will go into master which is 4.4 and I'm not sure if this will be backported to 4.3.1 Wido > Thanks > ----- Original Message ----- > > From: "Wido den Hollander" <w...@widodh.nl> > To: dev@cloudstack.apache.org > Sent: Sunday, 23 March, 2014 3:56:44 PM > Subject: Re: ACS and KVM uses /tmp for volumes migration and templates > > > > On 03/21/2014 02:23 PM, Andrei Mikhailovsky wrote: >> >> Wido, >> >> >> i would be happy to try the custom ACS build unless 4.3 comes out soon. It >> has been overdue for sometime now )). Has this feature been addressed in the >> 4.3 release? >> > > No, it hasn't been fixed yet. I have to admit, I forgot about this until > you sent this e-mail to the list. > > I'll fix this in master later this week. > >> >> I can leave with this feature for the time being, but i do see a longer term >> issue when my volumes become large as i've only got about 100gb free space >> on my host servers. >> >> > > I fully agree. While writing this code I was aware of this. See my > comments in the code: > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java;h=5de8bd26ae201187f5db5fd16b7e3ca157cab53a;hb=master#l1087 > > >> From what i can tell by looking at the rbd ls -l info all of my volumes are >> done in Format 2 >> > > Correct, because I by-pass libvirt and Qemu at some places right now. > >> >> Cheers, >> >> >> Andrei >> >> >> >> >> ----- Original Message ----- >> >> From: "Wido den Hollander" <w...@widodh.nl> >> To: dev@cloudstack.apache.org >> Sent: Thursday, 20 March, 2014 9:40:29 AM >> Subject: Re: ACS and KVM uses /tmp for volumes migration and templates >> >> On 03/20/2014 12:59 AM, Andrei Mikhailovsky wrote: >>> Hi guys, >>> >>> I was wondering if this is a bug? >>> >> >> No, it's a "feature". >> >>> I've noticed that during volume migration from NFS to RBD primary storage >>> the volume image is first copied to /tmp and only then to the RBD storage. >>> This seems silly to me as one would expect a typical volume to be larger >>> than the host's hard disk. Also, it is a common practice to use tmpfs as >>> /tmp for performance reasons. Thus, a typical host server will have far >>> smaller /tmp folder than the size of an average volume. As a result, volume >>> migration would break after filling the /tmp and could probably cause a >>> bunch of issue for the KVM host itself as well as any vms running on the >>> server. >>> >> >> Correct. The problem was that RBD images know two formats. Format 1 >> (old/legacy) and format 2. >> >> In order to perform cloning images should be in RBD format 2. >> >> When running qemu-img convert with a RBD image as a destination qemu-img >> will create a RBD image in format 1. >> >> That's due to this piece of code in block/rbd.c in Qemu: >> >> ret = rbd_create(io_ctx, name, bytes, &obj_order); >> >> rbd_create() creates images in format 1. To use format 2 you should use >> rbd_create2() or rbd_create3(). >> >> With RBD format 1 we can't do snapshotting or cloning, which we require >> in ACS. >> >> So I had to do a intermediate step where I first wrote the RAW image >> somewhere and afterwards write it to RBD. >> >> After some discussion a config option has been added to Ceph: >> >> OPTION(rbd_default_format, OPT_INT, 1) >> >> This allows me to do this: >> >> qemu-img convert .. -O raw .. rbd:rbd/myimage:rbd_default_format=2 >> >> This causes librbd/RBD to create a format 2 image and we can skip the >> convert step to /tmp. >> >> This option is available since Ceph Dumpling 0.67.5 and was not >> available when ACS 4.2 was written. >> >> I'm going to make changes in master which skip the step with /tmp. >> >> Technically this can be backported to 4.2, but then you would have to >> run your own homebrew version of 4.2 >> >>> It also seems that the /tmp is temporarily used during a template creation >>> . >>> >> >> Same story as above. >> >>> My setup: >>> >>> ACS 4.2.1 >>> Ubuntu 12.04 with KVM >>> RBD + NFS for Primary storage >>> NFS for Staging and Secondary storage >>> >>> >>> Thanks >>> >>> Andrei >>> >> >> >> > >
