On 02/09/2016 02:04 PM, Ildikó Váncsa wrote:
Hi Walt,
Thanks for starting this thread. It is a good summary of the issue and the
proposal also looks feasible to me.
I have a quick, hopefully not too wild idea based on the earlier discussions we
had. We were considering earlier to store the target identifier together with
the other items of the attachment info. The problem with this idea is that when
we call initialize_connection from Nova, Cinder does not get the relevant
information, like instance_id, to be able to do this. This means we cannot do
that using the functionality we have today.
My idea here is to extend the Cinder API so that Nova can send the missing
information after a successful attach. Nova should have all the information
including the 'target', which means that it could update the attachment
information through the new Cinder API.
I think we need to do is to allow the connector to be passed at
os-attach time. Then cinder can save it in the attachment's table entry.
We will also need a new cinder API to allow that attachment to be
updated during live migration, or the connector for the attachment will
get stale and incorrect.
Walt
It would mean that when we request for the volume info from Cinder at detach
time the 'attachments' list would contain all the required information for each
attachments the volume has. If we don't have the 'target' information because
of any reason we can still use the approach described below as fallback. This
approach could even be used in case of live migration I think.
The Cinder API extension would need to be added with a new microversion to
avoid problems with older Cinder versions talking to new Nova.
The advantage of this direction is that we can reduce the round trips to Cinder
at detach time. The round trip after a successful attach should not have an
impact on the normal operation as if that fails the only issue we have is we
need to use the fall back method to be able to detach properly. This would
still affect only multiattached volumes, where we have more than one
attachments on the same host. By having the information stored in Cinder as
well we can also avoid removing a target when there are still active
attachments connected to it.
What do you think?
Thanks,
Ildikó
-----Original Message-----
From: Walter A. Boring IV [mailto:walter.bor...@hpe.com]
Sent: February 09, 2016 20:50
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Nova][Cinder] Multi-attach, determining when to call
os-brick's connector.disconnect_volume
Hey folks,
One of the challenges we have faced with the ability to attach a single
volume to multiple instances, is how to correctly detach that
volume. The issue is a bit complex, but I'll try and explain the problem, and
then describe one approach to solving one part of the
detach puzzle.
Problem:
When a volume is attached to multiple instances on the same host.
There are 2 scenarios here.
1) Some Cinder drivers export a new target for every attachment on a
compute host. This means that you will get a new unique
volume path on a host, which is then handed off to the VM instance.
2) Other Cinder drivers export a single target for all instances on a
compute host. This means that every instance on a single host, will
reuse the same host volume path.
When a user issues a request to detach a volume, the workflow boils down to
first calling os-brick's connector.disconnect_volume
before calling Cinder's terminate_connection and detach. disconnect_volume's
job is to remove the local volume from the host OS
and close any sessions.
There is no problem under scenario 1. Each disconnect_volume only affects the
attached volume in question and doesn't affect any
other VM using that same volume, because they are using a different path that
has shown up on the host. It's a different target
exported from the Cinder backend/array.
The problem comes under scenario 2, where that single volume is shared for
every instance on the same compute host. Nova needs
to be careful and not call disconnect_volume if it's a shared volume, otherwise
the first disconnect_volume call will nuke every
instance's access to that volume.
Proposed solution:
Nova needs to determine if the volume that's being detached is a shared or
non shared volume. Here is one way to determine that.
Every Cinder volume has a list of it's attachments. In those attachments
it contains the instance_uuid that the volume is attached to.
I presume Nova can find which of the volume attachments are on the same host.
Then Nova can call Cinder's initialize_connection for
each of those attachments to get the target's connection_info dictionary.
This connection_info dictionary describes how to connect to the target on the
cinder backend. If the target is shared, then each of the
connection_info dicts for each attachment on that host will be identical. Then
Nova would know that it's a shared target, and then
only call os-brick's disconnect_volume, if it's the last attachment on that
host. I think at most 2 calls to cinder's initialize_connection
would suffice to determine if the volume is a shared target. This would only
need to be done if the volume is multi-attach capable and
if there are more than 1 attachments on the same host, where the detach is
happening.
Walt
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
