RE: Storage Subsystem 2.0 plugin docs

Vladimir Popovski Thu, 28 Mar 2013 12:27:07 -0700

Hi Mike,

Say hi to John from me. He is an amazing guy and most likely I will meet
him during the OS summit.

We don’t have any hypervisor-specific code for manipulating with storage
objects. There was no such a need for the OpenStack drives. Inside of our
systems, we usually provide a direct access to the HW (PCI Passthrough) for
our VMs and they skip the HV level all together (and access network-related
storage resources “directly”).

I hope we will not need to invent it for the CS driver/plugin.

Regarding datastores per volume – I hope there will be no limitations in
terms of number of datastores per server and per cluster/zone. Potentially,
this approach will create thousands/millions of datastores and we must be
ready for it.

Regards,

-Vladimir

*From:* Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
*Sent:* Wednesday, March 27, 2013 1:18 PM
*To:* Vladimir Popovski
*Cc:* Edison Su; cloudstack-...@incubator.apache.org
*Subject:* Re: Storage Subsystem 2.0 plugin docs

Hi Vladimir,

I figured you were away and would get back to us when you could. :)

I was talking to John Griffith. It sounds like you worked with him on
OpenStack.

You are correct that I am interested in your Option 1: Mapping a single VM
or a single data disk to a single volume on our SAN (via a single Datastore
for VMware, a single Storage Repository for XenServer, etc.).

I have code written that creates a XenServer Storage Repository and I am
testing code I finished writing last night to create a VMware Datastore.

We should decide where this will be placed in CloudStack because, as you
and others have commented, the storage plug-ins themselves should not have
to maintain this code. For example, when HyperV support comes out in CS
later, we don't want to have to update all the storage plug-ins to support
it.

You don't happen to have code lying around that creates the necessary
storage objects for KVM and/or Oracle VM, do you? :)

If not, we should start in on creating such logic at some point relatively
soon.

I'll send out my XenServer and VMware code to you and Edison when I finish
my testing (probably tomorrow).

Talk to you later!

On Wed, Mar 27, 2013 at 1:22 PM, Vladimir Popovski <
vladi...@zadarastorage.com> wrote:

Hi All,

I was away for couple of days, so sorry for the delay.

I’m completely with Mike & John (& others) on separating storage plugins
from hypervisor-related functions. If every plugin will need to implement
hypervisor-related code, it will be a big mess.

Our use-case is very similar to Mike’s – we would like to be able to
provision volumes with different QoS characteristics directly to VMs,
rather than adding them into “shared” datastores. It might be achieved in
two ways:

- either to create separate data stores per each volume (storage
LUN), and from there to create volumes & assign to instances.

- or to assign devices recognized by iSCSI Initiators directly to
instances (I’m not sure if it will be possible in VMware without datastores)

It looks like Mike started to work on the 1st approach. In this case, for
every volume there will be a separate datastore. If this is the preferred
direction for all storage plugins, then the hypervisor-specific logic of
datastore creation and creating/assigning volumes from the datastore will
be fairly common for all storage plug-ins. At the same time, the storage
plugin should have the control over the datastore management. It will
depend on the plugin, if dedicated or shared datastores should be created.

For the 2nd option (skipping the datastore layer) there might be plenty of
common code as well.

So, my questions are:

- do you think it is beneficial to support both options for the CS
(or are we good with potentially huge number of datastores)?

- are we all agree that HV-dependent storage code should be
generic and appropriate interfaces to be exposed?

Regards,

-Vladimir

*From:* Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
*Sent:* Wednesday, March 27, 2013 10:21 AM

*To:* Edison Su
*Cc:* cloudstack-...@incubator.apache.org; Vladimir Popovski
*Subject:* Re: Storage Subsystem 2.0 plugin docs

Sounds good, Edison

Last night I finished up code that uses the VI Java API to create a VMware
Datastore.

I want to test it a bit more before I have you look at it.

Today there is a Citrix CloudPlatform demo I'm participating in to handle
part of the SolidFire section of the demo, so I might not have time to do
my Datastore testing, but I should be done with it tomorrow.

Talk to you later!

On Wed, Mar 27, 2013 at 11:15 AM, Edison Su <edison...@citrix.com> wrote:

For vmware, current cloudstack doesn’t create a vmware datastore through
vmware’s API, admin needs to create the datastore in Vcenter at first, then
add it back into cloudstack. I am not familiar with how to create a VMware
datastore through Vmware’s API, but regarding to add a new host into a
cluster, the current framework lets storage provider adding a listener
which can listen on adding host event.

*From:* Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
*Sent:* Tuesday, March 26, 2013 6:45 PM

*To:* Edison Su
*Cc:* cloudstack-...@incubator.apache.org; Vladimir Popovski
*Subject:* Re: Storage Subsystem 2.0 plugin docs

Great - thanks, Edison!

I can take a look at that code.

I've almost gotten the VMware code written.

It's a little more involved than the XenServer code because you have to add
static IQNs for discovery to each host in a VMware cluster (this is somehow
handled behind the scenes, I suppose, with XenServer) before you can create
a Datastore based on your iSCSI target.

One thing I was wondering, though, is when you add a new host to this
VMware cluster. It will need to "inherit" the list of IQNs to discover. I
image this is the case today. Do you know anything about that? I might
just try it out and see if that works today.

On Tue, Mar 26, 2013 at 5:18 PM, Edison Su <edison...@citrix.com> wrote:

Thanks!

FYI, there are some code at both xen and kvm hypervisor resource code to
deal with storage pool creation.

For example, in CitrixResourceBase-> getNfsSR or getIscsiSR to create a nfs
SR or ISCSI SR. In LibvirtStorageAdaptor, which can create storage pool in
libvirt.

*From:* Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
*Sent:* Tuesday, March 26, 2013 1:52 PM
*To:* Edison Su
*Cc:* cloudstack-...@incubator.apache.org; Vladimir Popovski

*Subject:* Re: Storage Subsystem 2.0 plugin docs

Hi Edison,

Sounds good.

I already have code to create a XenServer Storage Repository (and
optionally use CHAP credentials) and I'm working right now on creating a
vSphere Datastore.

When I have this working and in a nicer state, I can check in with you to
review it and provide comments.

Once those two hypervisors are handled, I'll move on to KVM and OVM.

Thanks!

On Tue, Mar 26, 2013 at 2:33 PM, Edison Su <edison...@citrix.com> wrote:

Yes, code is welcome!!! Currently Only the interface at the management
server side is defined. At the hypervisor resource side, we may need some
kind of utility library or another plugin framework, as John proposed few
months ago.

*From:* Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
*Sent:* Monday, March 25, 2013 2:37 PM
*To:* Edison Su; cloudstack-...@incubator.apache.org; Vladimir Popovski

*Subject:* Re: Storage Subsystem 2.0 plugin docs

Hey Edison,

So...if you think my understanding is correct (please check out the e-mail
below), then I have a question.

Do we really want to have the storage plug-ins taking on the responsibility
of talking to the hypervisors to hook up the storage that they just created?

I'm a bit familiar with how OpenStack does this and it seems that it only
has its storage plug-ins create a volume (LUN, whatever) and then the
framework handles the process of dealing with the hypervisor in question to
hook up the storage.

It seems like otherwise we'd need to create a utility for all storage
plug-ins to share otherwise they'd be duplicating efforts in talking to
hypervisors.

What do you think?

On Thu, Mar 21, 2013 at 7:52 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

Hi Edison,

I believe I understand the requirements for the plug-in better now.

It sounds like the flow will be as such:

* The user executes a Compute or Disk Offering that is tied via a storage
tag to a Primary Storage that is associated with a plug-in.

* The storage framework will ask the plug-in to create a volume. The
plug-in will create a volume and hook the volume up to the appropriate
hypervisor. For VMware, this means the plug-in will create a Datastore.
For XenServer, this means the plug-in will create a Storage Repository.
(So on and so forth for other hypervisors.)

* The VM or data disk is then deployed to the hypervisor.

Does that sound correct, Edison?

Thanks!

On Thu, Mar 21, 2013 at 5:44 PM, Edison Su <edison...@citrix.com> wrote:

*From:* Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
*Sent:* Thursday, March 21, 2013 4:18 PM
*To:* Edison Su
*Subject:* Re: Storage Subsystem 2.0 plugin docs

Hi Edison,

I wanted to dive into these comments a bit more:

[Edison] plugin’s driver->createasync will be called when mgt server want
to create a volume on the storage. In the driver’s implementation, it can
directly call storage box’s api, or send a command to hypervisor host, then
call storage box’s api to create an iscsi.

Then create a datastore(for vmware), SR(for xenserver), or storage pool(for
KVM) on hypervisor host, based on the iscsi iqn.

If the volume is created from a template(for root disk), need to find a way
to import that template(which is nfs based currently, it will be just a
plain http url the future) into the root disk.

The part about creating a datastore or a storage repository...is that
something the plug-in will be responsible for doing or will the storage
framework cover that piece? I'm thinking the storage framework will since
all sorts of plug-ins would seem to need that ability (to have their
storage hooked up to a datastore or a storage repository).

[Edison] It’s a specific requirement for per volume per LUN case, and
specific for certain hypervisors(seems KVM doesn’t need to create a storage
pool when using iscsi LUN), so the storage framework will not deal with it
right now.

Thanks for your time, Edison! :)

On Thu, Mar 21, 2013 at 4:45 PM, Edison Su <edison...@citrix.com> wrote:

Feedback/comments are appreciated, need to know your input from storage
vendor point of view.

*From:* Vladimir Popovski [mailto:vladi...@zadarastorage.com]
*Sent:* Thursday, March 21, 2013 11:52 AM
*To:* Edison Su; cloudstack

*Cc:* mike.tutkow...@solidfire.com
*Subject:* RE: Storage Subsystem 2.0 plugin docs

Hi Edison,

Thank you for the reply. We will check it out.

Regards,

-Vladimir

*From:* Edison Su [mailto:edison...@citrix.com]
*Sent:* Thursday, March 21, 2013 11:36 AM
*To:* 'Vladimir Popovski'; cloudstack
*Cc:* mike.tutkow...@solidfire.com
*Subject:* RE: Storage Subsystem 2.0 plugin docs

*From:* Vladimir Popovski
[mailto:vladi...@zadarastorage.com<vladi...@zadarastorage.com>]

*Sent:* Wednesday, March 20, 2013 9:05 AM
*To:* cloudstack
*Cc:* mike.tutkow...@solidfire.com; Edison Su
*Subject:* Storage Subsystem 2.0 plugin docs

Hi All,

Thank you for a great work on CloudStack! We are interested in integrating
CS with our storage system and started to look at your documentation and
storage-related code. I see that Mike from SolidFire started working on
something similar some time ago and Edison even created an empty plugin for
it (in Nov’12?).

We have couple of questions related to that:

- Is there any documentation about plugins (except of
https://cwiki.apache.org/CLOUDSTACK/storage-subsystem-20.html)

[Edison] There are not much docs about the plugins other than the above
link. See below.

- Are there any exemplary plugins for primary & secondary
datastores? Was the SolidFire plugin ever finished?

[Edison] yesterday, I checked in some code to separate existing cloudstack
storage code into a standalone maven project, called:
cloud-plugin-storage-volume-default, which can give you an example how a
storage plugin will look like.

- How to activate a new plugin and use it (at least through
CLIs/APIs)

[Edison] First, put a bean configuration in client/tomcatconf/
componentContext.xml.in for your plugin provider class, like:

</bean>

Second, when adding a data store into cloudstack, with an extra parameter
in createstoragepoolcmd: provider=your-provider-name,
liststorageproviderscmd can list all the registered providers in mgt server.

- How to integrate it with the UI

There is no UI part of example code for storage yet, the idea is to use
pluggable UI(
https://cwiki.apache.org/confluence/display/CLOUDSTACK/UI+Plugin+Tutorial),
for each storage provider may need a separate UI to add a storage. For
example, in adding primary storage ui, there will be a drop down list, show
all the registered providers, if user selects one of the drop down list,
then UI will pop up a diagram, based on providers’ pluggable ui, then user
can type whatever information needed for a storage(e.g. nfs server, nfs
path, if its nfs). At the end, UI will call createstoragepoolcmd to
register a storage into cloudstack.

Thanks,

-Vladimir

-------

Vladimir Popovski

VP, Cloud Operations

Zadara Storage
(949) 677-2095

vladi...@zadarastorage.com

www.zadarastorage.com

--
*Mike Tutkowski*