On Thu, 5 Jul 2012 15:00:26 +0100 "Daniel P. Berrange" <berra...@redhat.com> wrote:
> In the Libvirt driver there is special-case code for LXC to deal with > the volume-attach functionality, since there is no "block device > attach" functionality in libvirt for LXC. The code in question was > added in > > commit e40b659d320b3c6894862b87adf1011e31cbf8fc > Author: Chuck Short <chuck.sh...@canonical.com> > Date: Tue Jan 31 20:53:24 2012 -0500 > > Add support for LXC volumes. > > This introduces volume support for LXC containers in Nova. > The way that this works is that when a device is attached to an > LXC container is that, the xml is parsed to find out which device > to connect to the LXC container, binds the device to the LXC > container, and allow the device through cgroups. > > This bug fixes LP: #924601. > > Change-Id: I00b41426ae8354b3cd4212655ecb48319a63aa9b > Signed-off-by: Chuck Short <chuck.sh...@canonical.com> > > First a little background.... > > The way LXC works with Nova, is that the image file assigned to the > instance eg > > /var/lib/nova/instances/instance-0000000e/disk > > is exported via qemu-nbd, and then mounted on the host at > > /var/lib/nova/instances/instance-0000000e/rootfs > > > When libvirt starts the container it uses that directory as the root > filesystem. libvirt will *also* mount a private /dev, /dev/pts, /proc > and /sys for the container. This is all fine > > Now, when using 'nova volume-attach': > > # nova volume-attach 05eb16df-03b8-451b-85c1-b838a8757736 > a5ad1d37-aed0-4bf6-8c6e-c28543cd38ac /dev/sdf > > nova will import an iSCSI LUN from the nova volume service, on the > compute node. The kernel will assign it the next free SCSI drive > letter, in my case '/dev/sdc'. > > The libvirt nova driver will then do a mknod, using the volume name > passed to 'nova volume-attach'. > eg it will do > > mknod /var/lib/nova/instances/instance-0000000e/rootfs/dev/sdf > > this is where it has all gone horribly wrong... > > * The iSCSI LUN is completely randomly allocated, and unrelated to > the block device name the user will give to 'nova volume-attach'. So > there is no association between the /dev/sdf in the container and the > /dev/sdc in the host, and you can't expect the caller of > 'volume-attach' to be able to predict what the next assigned LUN will > be on the host. > > * The /var/lib/nova/instances/instance-0000000e/rootfs/dev/ > directory where nova did the mknod is a completely different > filesystem to the one seen by the container. The /dev in the > container is a tmpfs that is never visible to the host, so a mknod in > the host won't appear to the container. > > AFAIK, there is no way to resolve either of these problems given the > current level kernel support for LXC, which is why libvirt has never > implemented block volume attach itself. > > Thus I'm wondering how this LXC volume-attach code in Nova has ever > worked, or was tested ? My testing of Nova shows no sign of it working > today. Unless someone can demonstrate a flaw in my logic, I'm inclined > to simply revert this whole commit from Nova. > > Regards, > Daniel Hi, It *was* working at one point. Its on my todo list to make sure that it still works properly. Otherwise Ill remove it myself. Regards chuck _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
