In the Libvirt driver there is special-case code for LXC to deal with
the volume-attach functionality, since there is no "block device attach"
functionality in libvirt for LXC. The code in question was added in
commit e40b659d320b3c6894862b87adf1011e31cbf8fc
Author: Chuck Short <chuck.sh...@canonical.com>
Date: Tue Jan 31 20:53:24 2012 -0500
Add support for LXC volumes.
This introduces volume support for LXC containers in Nova.
The way that this works is that when a device is attached to an
LXC container is that, the xml is parsed to find out which device to
connect to the LXC container, binds the device to the LXC container,
and allow the device through cgroups.
This bug fixes LP: #924601.
Change-Id: I00b41426ae8354b3cd4212655ecb48319a63aa9b
Signed-off-by: Chuck Short <chuck.sh...@canonical.com>
First a little background....
The way LXC works with Nova, is that the image file assigned to the instance
eg
/var/lib/nova/instances/instance-0000000e/disk
is exported via qemu-nbd, and then mounted on the host at
/var/lib/nova/instances/instance-0000000e/rootfs
When libvirt starts the container it uses that directory as the root
filesystem. libvirt will *also* mount a private /dev, /dev/pts, /proc
and /sys for the container. This is all fine
Now, when using 'nova volume-attach':
# nova volume-attach 05eb16df-03b8-451b-85c1-b838a8757736
a5ad1d37-aed0-4bf6-8c6e-c28543cd38ac /dev/sdf
nova will import an iSCSI LUN from the nova volume service, on the compute
node. The kernel will assign it the next free SCSI drive letter, in my
case '/dev/sdc'.
The libvirt nova driver will then do a mknod, using the volume name
passed to 'nova volume-attach'.
eg it will do
mknod /var/lib/nova/instances/instance-0000000e/rootfs/dev/sdf
this is where it has all gone horribly wrong...
* The iSCSI LUN is completely randomly allocated, and unrelated to the
block device name the user will give to 'nova volume-attach'. So there
is no association between the /dev/sdf in the container and the
/dev/sdc in the host, and you can't expect the caller of 'volume-attach'
to be able to predict what the next assigned LUN will be on the host.
* The /var/lib/nova/instances/instance-0000000e/rootfs/dev/ directory
where nova did the mknod is a completely different filesystem to
the one seen by the container. The /dev in the container is a tmpfs
that is never visible to the host, so a mknod in the host won't
appear to the container.
AFAIK, there is no way to resolve either of these problems given the
current level kernel support for LXC, which is why libvirt has never
implemented block volume attach itself.
Thus I'm wondering how this LXC volume-attach code in Nova has ever
worked, or was tested ? My testing of Nova shows no sign of it working
today. Unless someone can demonstrate a flaw in my logic, I'm inclined
to simply revert this whole commit from Nova.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
