Taylor, I am adding some IBMers who are also looking into making this work. PowerVC has a solution hacked into place but we are working on getting a more general solution implemented for Mitaka. Hoping we can all work together to get a solution in place.
Will you be in Tokyo? If so, it would be good to have a plan together to discuss there. Gerald and Sam, Can you guys join the OpenStack dev mailing list and continue the discussion? Thanks! Jay On Fri, Aug 21, 2015 at 8:20 PM <taylor.ber...@solnet.co.nz> wrote: > For RBDs it IS as simple as making calls to virsh after an attached volume > is extended. I've done it half a dozen time with no intermediate steps and > it works. I'd love to test it more robustly, obviously, but unfortunately I > got bigger fish to fry with BAU. > > iSCSI might involve more work, I acknowledge that, but there is nothing > wrong with putting the framework in place now and throwing an "unsupported > volume type" error message if we haven't worked out the best method for > doing this for a particular type. > > The way I see it, the only ones that are going to cause us problems are > ones that require the host to suspend the disk prior to operating on it. In > other words if the notification to the host can't be done atomically, that > could definitely cause issues. > > However, all the examples I have seem implemented in OpenStack volumes > thus far (iSCSI, RDB) are either atomic or no notification required (in the > case of RBD). Even Multipath is atomic (granted, it's multiple chained > atomic operations, but still, they won't be left in an irrecoverable > failure state). > > Yes, the page you linked does warn about the issue when there is no path > the device, however I think that if you're trying to resize a volume the > compute node can't connect to, you got bigger problems (that is to say, > throwing an error here is perfectly reasonable). > > Regards, > > Taylor Bertie > Enterprise Support Infrastructure Engineer > > Mobile +64 27 952 3949 > Phone +64 4 462 5030 > Email taylor.ber...@solnet.co.nz > > Solnet Solutions Limited > Level 12, Solnet House > 70 The Terrace, Wellington 6011 > PO Box 397, Wellington 6140 > > www.solnet.co.nz > > > -----"Walter A. Boring IV" <walter.bor...@hp.com> wrote: ----- > To: openstack-dev@lists.openstack.org > From: "Walter A. Boring IV" <walter.bor...@hp.com> > Date: 2015-08-22 7:13 > Subject: Re: [openstack-dev] [nova][cinder] Extending attached disks > > This isn't as simple as making calls to virsh after an attached volume > is extended on the cinder backend, especially when multipath is involved. > You need the host system to understand that the volume has changed size > first, or virsh will really never see it. > > For iSCSI/FC volumes you need to issue a rescan on the bus (iSCSI > session, FC fabric), and then when multipath is involved, it gets quite > a bit more complex. > > This lead to one of the sticking points with doing this at all, is > because when cinder extends the volume, it needs to tell nova that it > has happened, and the nova (or something on the compute node), will have > to issue the correct commands in sequence for it all to work. > > You'll also have to consider multi-attached volumes as well, which adds > yet another wrinkle. > > A good quick source of some of the commands and procedures that are > needed you can see here: > > https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Online_Storage_Reconfiguration_Guide/online-logical-units.html > > > You can see that volumes with multipath requires a lot of hand holding > to be done correctly. It's non trivial. I see this as being very error > prone, and any failure > in the multipath process could lead to big problems :( > > Walt > > Hi everyone, > > > > Apologises for the duplicate send, looks like my mail client doesn't > create very clean HTML messages. Here is the message in plain-text. I'll > make sure to send to the list in plain-text from now on. > > > > In my current pre-production deployment we were looking for a method to > live extend attached volumes to an instance. This was one of the > requirements for deployment. I've worked with libvirt hypervisors before so > it didn't take long to find a workable solution. However I'm not sure how > transferable this will be across deployment models. Our deployment model is > using libvirt for nova and ceph for backend storage. This means obviously > libvirt is using rdb to connect to volumes. > > > > Currently the method I use is: > > > > - Force cinder to run an extend operation. > > - Tell Libvirt that the attached disk has been extended. > > > > It would be worth discussing if this can be ported to upstream such that > the API can handle the leg work, rather than this current manual method. > > > > Detailed instructions. > > You will need: volume-id of volume you want to resize, > hypervisor_hostname and instance_name from instance volume is attached to. > > > > Example: extending volume f9fa66ab-b29a-40f6-b4f4-e9c64a155738 attached > to instance-00000012 on node-6 to 100GB > > > > $ cinder reset-state --state available > f9fa66ab-b29a-40f6-b4f4-e9c64a155738 > > $ cinder extend f9fa66ab-b29a-40f6-b4f4-e9c64a155738 100 > > $ cinder reset-state --state in-use f9fa66ab-b29a-40f6-b4f4-e9c64a155738 > > > > $ssh node-6 > > node-6$ virsh qemu-monitor-command instance-00000012 --hmp "info block" > | grep f9fa66ab-b29a-40f6-b4f4-e9c64a155738 > > drive-virtio-disk1: removable=0 io-status=ok > file=rbd:volumes-slow/volume-f9fa66ab-b29a-40f6-b4f4-e9c64a155738:id=cinder:key=<keyhere>==:auth_supported=cephx\\;none:mon_host=10.1.226.64\\:6789\\;10.1.226.65\\:6789\\;10.1.226.66\\:6789 > ro=0 drv=raw encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0 > > > > This will get you the disk-id, which in this case is drive-virtio-disk1. > > > > node-6$ virsh qemu-monitor-command instance-00000012 --hmp "block_resize > drive-virtio-disk1 100G" > > > > Finally, you need to perform a drive rescan on the actual instance and > resize and extend the file-system. This will be OS specific. > > > > I've tested this a few times and it seems very reliable. > > > > Taylor Bertie > > Enterprise Support Infrastructure Engineer > > > > Mobile +64 27 952 3949 > > Phone +64 4 462 5030 > > Email taylor.ber...@solnet.co.nz > > > > Solnet Solutions Limited > > Level 12, Solnet House > > 70 The Terrace, Wellington 6011 > > PO Box 397, Wellington 6140 > > > > www.solnet.co.nz > > > > Attention: > > This email may contain information intended for the sole use of > > the original recipient. Please respect this when sharing or > > disclosing this email's contents with any third party. If you > > believe you have received this email in error, please delete it > > and notify the sender or postmas...@solnetsolutions.co.nz as > > soon as possible. The content of this email does not necessarily > > reflect the views of Solnet Solutions Ltd. > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > Attention: > This email may contain information intended for the sole use of > the original recipient. Please respect this when sharing or > disclosing this email's contents with any third party. If you > believe you have received this email in error, please delete it > and notify the sender or postmas...@solnetsolutions.co.nz as > soon as possible. The content of this email does not necessarily > reflect the views of Solnet Solutions Ltd. > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
