I understand what you're saying now, Marcus.
I wasn't sure if the Libvirt iSCSI Storage Pool was still an option
(looking into that still), but I see what you mean: If it is, we don't need
a new adaptor; otherwise, we do.
If Libivirt's iSCSI Storage Pool does work, I could update the current
adaptor, if need be, to make use of it.
On Mon, Sep 16, 2013 at 12:24 PM, Marcus Sorensen <shadow...@gmail.com>wrote:
> Well, you'd use neither of the two pool types, because you are not letting
> libvirt handle the pool, you are doing it with your own pool and adaptor
> class. Libvirt will be unaware of everything but the disk XML you attach to
> a vm. You'd only use those if libvirts functions were advantageous, I.e. if
> it already did everything you want. Since neither of those seem to provide
> both iscsi and the 1:1 mapping you want that's why we are talking about
> your own pool/adaptor.
>
> You can log into the target via your implementation of getPhysicalDisk as
> you mention in AttachVolumeCommand, or log in during your implementation of
> createStoragePool and simply rescan for luns in getPhysicalDisk. Presumably
> in most cases the host will be logged in already and new luns have been
> created in the meantime.
> On Sep 16, 2013 12:09 PM, "Mike Tutkowski" <mike.tutkow...@solidfire.com>
> wrote:
>
>> Hey Marcus,
>>
>> Thanks for that clarification.
>>
>> Sorry if this is a redundant question:
>>
>> When the AttachVolumeCommand comes in, it sounds like we thought the best
>> approach would be for me to discover and log in to the iSCSI target using
>> iscsiadm.
>>
>> This will create a new device: /dev/sdX.
>>
>> We would then pass this new device into the VM (passing XML into the
>> appropriate Libvirt API).
>>
>> If this is an accurate understanding, can you tell me: Do you think we
>> should be using a Disk Storage Pool or an iSCSI Storage Pool?
>>
>> I believe I recall you leaning toward a Disk Storage Pool because we will
>> have already discovered the iSCSI target and, as such, will already have a
>> device to pass into the VM.
>>
>> It seems like either way would work.
>>
>> Maybe I need to study Libvirt's iSCSI Storage Pools more to understand if
>> they would do the work of discovering the iSCSI target for me (and maybe
>> avoid me having to use iscsiadm).
>>
>> Thanks for the clarification! :)
>>
>>
>> On Mon, Sep 16, 2013 at 11:08 AM, Marcus Sorensen <shadow...@gmail.com>wrote:
>>
>>> It will still register the pool. You still have a primary storage
>>> pool that you registered, whether it's local, cluster or zone wide.
>>> NFS is optionally zone wide as well (I'm assuming customers can launch
>>> your storage only cluster-wide if they choose for resource
>>> partitioning), but it registers the pool in Libvirt prior to use.
>>>
>>> Here's a better explanation of what I meant. AttachVolumeCommand gets
>>> both pool and volume info. It first looks up the pool:
>>>
>>> KVMStoragePool primary = _storagePoolMgr.getStoragePool(
>>> cmd.getPooltype(),
>>> cmd.getPoolUuid());
>>>
>>> Then it looks up the disk from that pool:
>>>
>>> KVMPhysicalDisk disk = primary.getPhysicalDisk(cmd.getVolumePath());
>>>
>>> Most of the commands only pass volume info like this (getVolumePath
>>> generally means the uuid of the volume), since it looks up the pool
>>> separately. If you don't save the pool info in a map in your custom
>>> class when createStoragePool is called, then getStoragePool won't be
>>> able to find it. This is a simple thing in your implementation of
>>> createStoragePool, just thought I'd mention it because it is key. Just
>>> create a map of pool uuid and pool object and save them so they're
>>> available across all implementations of that class.
>>>
>>> On Mon, Sep 16, 2013 at 10:43 AM, Mike Tutkowski
>>> <mike.tutkow...@solidfire.com> wrote:
>>> > Thanks, Marcus
>>> >
>>> > About this:
>>> >
>>> > "When the agent connects to the
>>> > management server, it registers all pools in the cluster with the
>>> > agent."
>>> >
>>> > So, my plug-in allows you to create zone-wide primary storage. This
>>> just
>>> > means that any cluster can use the SAN (the SAN was registered as
>>> primary
>>> > storage as opposed to a preallocated volume from the SAN). Once you
>>> create a
>>> > primary storage based on this plug-in, the storage framework will
>>> invoke the
>>> > plug-in, as needed, to create and delete volumes on the SAN. For
>>> example,
>>> > you could have one SolidFire primary storage (zone wide) and currently
>>> have
>>> > 100 volumes created on the SAN to support it.
>>> >
>>> > In this case, what will the management server be registering with the
>>> agent
>>> > in ModifyStoragePool? If only the storage pool (primary storage) is
>>> passed
>>> > in, that will be too vague as it does not contain information on what
>>> > volumes have been created for the agent.
>>> >
>>> > Thanks
>>> >
>>> >
>>> > On Sun, Sep 15, 2013 at 11:53 PM, Marcus Sorensen <shadow...@gmail.com
>>> >
>>> > wrote:
>>> >>
>>> >> Yes, see my previous email from the 13th. You can create your own
>>> >> KVMStoragePool class, and StorageAdaptor class, like the libvirt ones
>>> >> have. The previous email outlines how to add your own StorageAdaptor
>>> >> alongside LibvirtStorageAdaptor to take over all of the calls
>>> >> (createStoragePool, getStoragePool, etc). As mentioned,
>>> >> getPhysicalDisk I believe will be the one you use to actually attach a
>>> >> lun.
>>> >>
>>> >> Ignore CreateStoragePoolCommand. When the agent connects to the
>>> >> management server, it registers all pools in the cluster with the
>>> >> agent. It will call ModifyStoragePoolCommand, passing your storage
>>> >> pool object (with all of the settings for your SAN). This in turn
>>> >> calls _storagePoolMgr.createStoragePool, which will route through
>>> >> KVMStoragePoolManager to your storage adapter that you've registered.
>>> >> The last argument to createStoragePool is the pool type, which is used
>>> >> to select a StorageAdaptor.
>>> >>
>>> >> From then on, most calls will only pass the volume info, and the
>>> >> volume will have the uuid of the storage pool. For this reason, your
>>> >> adaptor class needs to have a static Map variable that contains pool
>>> >> uuid and pool object. Whenever they call createStoragePool on your
>>> >> adaptor you add that pool to the map so that subsequent volume calls
>>> >> can look up the pool details for the volume by pool uuid. With the
>>> >> Libvirt adaptor, libvirt keeps track of that for you.
>>> >>
>>> >> When createStoragePool is called, you can log into the iscsi target
>>> >> (or make sure you are already logged in, as it can be called over
>>> >> again at any time), and when attach volume commands are fired off, you
>>> >> can attach individual LUNs that are asked for, or rescan (say that the
>>> >> plugin created a new ACL just prior to calling attach), or whatever is
>>> >> necessary.
>>> >>
>>> >> KVM is a bit more work, but you can do anything you want. Actually, I
>>> >> think you can call host scripts with Xen, but having the agent there
>>> >> that runs your own code gives you the flexibility to do whatever.
>>> >>
>>> >> On Sun, Sep 15, 2013 at 10:44 PM, Mike Tutkowski
>>> >> <mike.tutkow...@solidfire.com> wrote:
>>> >> > I see right now LibvirtComputingResource.java has the following
>>> method
>>> >> > that
>>> >> > I might be able to leverage (it's probably not called at present and
>>> >> > would
>>> >> > need to be implemented in my case to discover my iSCSI target and
>>> log in
>>> >> > to
>>> >> > it):
>>> >> >
>>> >> > protected Answer execute(CreateStoragePoolCommand cmd) {
>>> >> >
>>> >> > return new Answer(cmd, true, "success");
>>> >> >
>>> >> > }
>>> >> >
>>> >> > I would probably be able to call the KVMStorageManager to have it
>>> use my
>>> >> > StorageAdaptor to do what's necessary here.
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > On Sun, Sep 15, 2013 at 10:37 PM, Mike Tutkowski
>>> >> > <mike.tutkow...@solidfire.com> wrote:
>>> >> >>
>>> >> >> Hey Marcus,
>>> >> >>
>>> >> >> When I implemented support in the XenServer and VMware plug-ins for
>>> >> >> "managed" storage, I started at the execute(AttachVolumeCommand)
>>> >> >> methods in
>>> >> >> both plug-ins.
>>> >> >>
>>> >> >> The code there was changed to check the AttachVolumeCommand
>>> instance
>>> >> >> for a
>>> >> >> "managed" property.
>>> >> >>
>>> >> >> If managed was false, the normal attach/detach logic would just
>>> run and
>>> >> >> the volume would be attached or detached.
>>> >> >>
>>> >> >> If managed was true, new 4.2 logic would run to create (let's talk
>>> >> >> XenServer here) a new SR and a new VDI inside of that SR (or to
>>> >> >> reattach an
>>> >> >> existing VDI inside an existing SR, if this wasn't the first time
>>> the
>>> >> >> volume
>>> >> >> was attached). If managed was true and we were detaching the
>>> volume,
>>> >> >> the SR
>>> >> >> would be detached from the XenServer hosts.
>>> >> >>
>>> >> >> I am currently walking through the execute(AttachVolumeCommand) in
>>> >> >> LibvirtComputingResource.java.
>>> >> >>
>>> >> >> I see how the XML is constructed to describe whether a disk should
>>> be
>>> >> >> attached or detached. I also see how we call in to get a
>>> StorageAdapter
>>> >> >> (and
>>> >> >> how I will likely need to write a new one of these).
>>> >> >>
>>> >> >> So, talking in XenServer terminology again, I was wondering if you
>>> >> >> think
>>> >> >> the approach we took in 4.2 with creating and deleting SRs in the
>>> >> >> execute(AttachVolumeCommand) method would work here or if there is
>>> some
>>> >> >> other way I should be looking at this for KVM?
>>> >> >>
>>> >> >> As it is right now for KVM, storage has to be set up ahead of time.
>>> >> >> Assuming this is the case, there probably isn't currently a place
>>> I can
>>> >> >> easily inject my logic to discover and log in to iSCSI targets.
>>> This is
>>> >> >> why
>>> >> >> we did it as needed in the execute(AttachVolumeCommand) for
>>> XenServer
>>> >> >> and
>>> >> >> VMware, but I wanted to see if you have an alternative way that
>>> might
>>> >> >> be
>>> >> >> better for KVM.
>>> >> >>
>>> >> >> One possible way to do this would be to modify VolumeManagerImpl
>>> (or
>>> >> >> whatever its equivalent is in 4.3) before it issues an
>>> attach-volume
>>> >> >> command
>>> >> >> to KVM to check to see if the volume is to be attached to managed
>>> >> >> storage.
>>> >> >> If it is, then (before calling the attach-volume command in KVM)
>>> call
>>> >> >> the
>>> >> >> create-storage-pool command in KVM (or whatever it might be
>>> called).
>>> >> >>
>>> >> >> Just wanted to get some of your thoughts on this.
>>> >> >>
>>> >> >> Thanks!
>>> >> >>
>>> >> >>
>>> >> >> On Sat, Sep 14, 2013 at 12:07 AM, Mike Tutkowski
>>> >> >> <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>
>>> >> >>> Yeah, I remember that StorageProcessor stuff being put in the
>>> codebase
>>> >> >>> and having to merge my code into it in 4.2.
>>> >> >>>
>>> >> >>> Thanks for all the details, Marcus! :)
>>> >> >>>
>>> >> >>> I can start digging into what you were talking about now.
>>> >> >>>
>>> >> >>>
>>> >> >>> On Sat, Sep 14, 2013 at 12:02 AM, Marcus Sorensen
>>> >> >>> <shadow...@gmail.com>
>>> >> >>> wrote:
>>> >> >>>>
>>> >> >>>> Looks like things might be slightly different now in 4.2, with
>>> >> >>>> KVMStorageProcessor.java in the mix.This looks more or less like
>>> some
>>> >> >>>> of the commands were ripped out verbatim from
>>> >> >>>> LibvirtComputingResource
>>> >> >>>> and placed here, so in general what I've said is probably still
>>> true,
>>> >> >>>> just that the location of things like AttachVolumeCommand might
>>> be
>>> >> >>>> different, in this file rather than
>>> LibvirtComputingResource.java.
>>> >> >>>>
>>> >> >>>> On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen
>>> >> >>>> <shadow...@gmail.com>
>>> >> >>>> wrote:
>>> >> >>>> > Ok, KVM will be close to that, of course, because only the
>>> >> >>>> > hypervisor
>>> >> >>>> > classes differ, the rest is all mgmt server. Creating a volume
>>> is
>>> >> >>>> > just
>>> >> >>>> > a db entry until it's deployed for the first time.
>>> >> >>>> > AttachVolumeCommand
>>> >> >>>> > on the agent side (LibvirtStorageAdaptor.java is analogous to
>>> >> >>>> > CitrixResourceBase.java) will do the iscsiadm commands (via a
>>> KVM
>>> >> >>>> > StorageAdaptor) to log in the host to the target and then you
>>> have
>>> >> >>>> > a
>>> >> >>>> > block device. Maybe libvirt will do that for you, but my quick
>>> >> >>>> > read
>>> >> >>>> > made it sound like the iscsi libvirt pool type is actually a
>>> pool,
>>> >> >>>> > not
>>> >> >>>> > a lun or volume, so you'll need to figure out if that works or
>>> if
>>> >> >>>> > you'll have to use iscsiadm commands.
>>> >> >>>> >
>>> >> >>>> > If you're NOT going to use LibvirtStorageAdaptor (because
>>> Libvirt
>>> >> >>>> > doesn't really manage your pool the way you want), you're
>>> going to
>>> >> >>>> > have to create a version of KVMStoragePool class and a
>>> >> >>>> > StorageAdaptor
>>> >> >>>> > class (see LibvirtStoragePool.java and
>>> LibvirtStorageAdaptor.java),
>>> >> >>>> > implementing all of the methods, then in KVMStorageManager.java
>>> >> >>>> > there's a "_storageMapper" map. This is used to select the
>>> correct
>>> >> >>>> > adaptor, you can see in this file that every call first pulls
>>> the
>>> >> >>>> > correct adaptor out of this map via getStorageAdaptor. So you
>>> can
>>> >> >>>> > see
>>> >> >>>> > a comment in this file that says "add other storage adaptors
>>> here",
>>> >> >>>> > where it puts to this map, this is where you'd register your
>>> >> >>>> > adaptor.
>>> >> >>>> >
>>> >> >>>> > So, referencing StorageAdaptor.java, createStoragePool accepts
>>> all
>>> >> >>>> > of
>>> >> >>>> > the pool data (host, port, name, path) which would be used to
>>> log
>>> >> >>>> > the
>>> >> >>>> > host into the initiator. I *believe* the method getPhysicalDisk
>>> >> >>>> > will
>>> >> >>>> > need to do the work of attaching the lun. AttachVolumeCommand
>>> >> >>>> > calls
>>> >> >>>> > this and then creates the XML diskdef and attaches it to the
>>> VM.
>>> >> >>>> > Now,
>>> >> >>>> > one thing you need to know is that createStoragePool is called
>>> >> >>>> > often,
>>> >> >>>> > sometimes just to make sure the pool is there. You may want to
>>> >> >>>> > create
>>> >> >>>> > a map in your adaptor class and keep track of pools that have
>>> been
>>> >> >>>> > created, LibvirtStorageAdaptor doesn't have to do this because
>>> it
>>> >> >>>> > asks
>>> >> >>>> > libvirt about which storage pools exist. There are also calls
>>> to
>>> >> >>>> > refresh the pool stats, and all of the other calls can be seen
>>> in
>>> >> >>>> > the
>>> >> >>>> > StorageAdaptor as well. There's a createPhysical disk, clone,
>>> etc,
>>> >> >>>> > but
>>> >> >>>> > it's probably a hold-over from 4.1, as I have the vague idea
>>> that
>>> >> >>>> > volumes are created on the mgmt server via the plugin now, so
>>> >> >>>> > whatever
>>> >> >>>> > doesn't apply can just be stubbed out (or optionally
>>> >> >>>> > extended/reimplemented here, if you don't mind the hosts
>>> talking to
>>> >> >>>> > the san api).
>>> >> >>>> >
>>> >> >>>> > There is a difference between attaching new volumes and
>>> launching a
>>> >> >>>> > VM
>>> >> >>>> > with existing volumes. In the latter case, the VM definition
>>> that
>>> >> >>>> > was
>>> >> >>>> > passed to the KVM agent includes the disks, (StartCommand).
>>> >> >>>> >
>>> >> >>>> > I'd be interested in how your pool is defined for Xen, I
>>> imagine it
>>> >> >>>> > would need to be kept the same. Is it just a definition to the
>>> SAN
>>> >> >>>> > (ip address or some such, port number) and perhaps a volume
>>> pool
>>> >> >>>> > name?
>>> >> >>>> >
>>> >> >>>> >> If there is a way for me to update the ACL list on the SAN to
>>> have
>>> >> >>>> >> only a
>>> >> >>>> >> single KVM host have access to the volume, that would be
>>> ideal.
>>> >> >>>> >
>>> >> >>>> > That depends on your SAN API. I was under the impression that
>>> the
>>> >> >>>> > storage plugin framework allowed for acls, or for you to do
>>> >> >>>> > whatever
>>> >> >>>> > you want for create/attach/delete/snapshot, etc. You'd just
>>> call
>>> >> >>>> > your
>>> >> >>>> > SAN API with the host info for the ACLs prior to when the disk
>>> is
>>> >> >>>> > attached (or the VM is started). I'd have to look more at the
>>> >> >>>> > framework to know the details, in 4.1 I would do this in
>>> >> >>>> > getPhysicalDisk just prior to connecting up the LUN.
>>> >> >>>> >
>>> >> >>>> >
>>> >> >>>> > On Fri, Sep 13, 2013 at 10:27 PM, Mike Tutkowski
>>> >> >>>> > <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >> OK, yeah, the ACL part will be interesting. That is a bit
>>> >> >>>> >> different
>>> >> >>>> >> from how
>>> >> >>>> >> it works with XenServer and VMware.
>>> >> >>>> >>
>>> >> >>>> >> Just to give you an idea how it works in 4.2 with XenServer:
>>> >> >>>> >>
>>> >> >>>> >> * The user creates a CS volume (this is just recorded in the
>>> >> >>>> >> cloud.volumes
>>> >> >>>> >> table).
>>> >> >>>> >>
>>> >> >>>> >> * The user attaches the volume as a disk to a VM for the first
>>> >> >>>> >> time
>>> >> >>>> >> (if the
>>> >> >>>> >> storage allocator picks the SolidFire plug-in, the storage
>>> >> >>>> >> framework
>>> >> >>>> >> invokes
>>> >> >>>> >> a method on the plug-in that creates a volume on the
>>> SAN...info
>>> >> >>>> >> like
>>> >> >>>> >> the IQN
>>> >> >>>> >> of the SAN volume is recorded in the DB).
>>> >> >>>> >>
>>> >> >>>> >> * CitrixResourceBase's execute(AttachVolumeCommand) is
>>> executed.
>>> >> >>>> >> It
>>> >> >>>> >> determines based on a flag passed in that the storage in
>>> question
>>> >> >>>> >> is
>>> >> >>>> >> "CloudStack-managed" storage (as opposed to "traditional"
>>> >> >>>> >> preallocated
>>> >> >>>> >> storage). This tells it to discover the iSCSI target. Once
>>> >> >>>> >> discovered
>>> >> >>>> >> it
>>> >> >>>> >> determines if the iSCSI target already contains a storage
>>> >> >>>> >> repository
>>> >> >>>> >> (it
>>> >> >>>> >> would if this were a re-attach situation). If it does contain
>>> an
>>> >> >>>> >> SR
>>> >> >>>> >> already,
>>> >> >>>> >> then there should already be one VDI, as well. If there is no
>>> SR,
>>> >> >>>> >> an
>>> >> >>>> >> SR is
>>> >> >>>> >> created and a single VDI is created within it (that takes up
>>> about
>>> >> >>>> >> as
>>> >> >>>> >> much
>>> >> >>>> >> space as was requested for the CloudStack volume).
>>> >> >>>> >>
>>> >> >>>> >> * The normal attach-volume logic continues (it depends on the
>>> >> >>>> >> existence of
>>> >> >>>> >> an SR and a VDI).
>>> >> >>>> >>
>>> >> >>>> >> The VMware case is essentially the same (mainly just
>>> substitute
>>> >> >>>> >> datastore
>>> >> >>>> >> for SR and VMDK for VDI).
>>> >> >>>> >>
>>> >> >>>> >> In both cases, all hosts in the cluster have discovered the
>>> iSCSI
>>> >> >>>> >> target,
>>> >> >>>> >> but only the host that is currently running the VM that is
>>> using
>>> >> >>>> >> the
>>> >> >>>> >> VDI (or
>>> >> >>>> >> VMKD) is actually using the disk.
>>> >> >>>> >>
>>> >> >>>> >> Live Migration should be OK because the hypervisors
>>> communicate
>>> >> >>>> >> with
>>> >> >>>> >> whatever metadata they have on the SR (or datastore).
>>> >> >>>> >>
>>> >> >>>> >> I see what you're saying with KVM, though.
>>> >> >>>> >>
>>> >> >>>> >> In that case, the hosts are clustered only in CloudStack's
>>> eyes.
>>> >> >>>> >> CS
>>> >> >>>> >> controls
>>> >> >>>> >> Live Migration. You don't really need a clustered filesystem
>>> on
>>> >> >>>> >> the
>>> >> >>>> >> LUN. The
>>> >> >>>> >> LUN could be handed over raw to the VM using it.
>>> >> >>>> >>
>>> >> >>>> >> If there is a way for me to update the ACL list on the SAN to
>>> have
>>> >> >>>> >> only a
>>> >> >>>> >> single KVM host have access to the volume, that would be
>>> ideal.
>>> >> >>>> >>
>>> >> >>>> >> Also, I agree I'll need to use iscsiadm to discover and log
>>> in to
>>> >> >>>> >> the
>>> >> >>>> >> iSCSI
>>> >> >>>> >> target. I'll also need to take the resultant new device and
>>> pass
>>> >> >>>> >> it
>>> >> >>>> >> into the
>>> >> >>>> >> VM.
>>> >> >>>> >>
>>> >> >>>> >> Does this sound reasonable? Please call me out on anything I
>>> seem
>>> >> >>>> >> incorrect
>>> >> >>>> >> about. :)
>>> >> >>>> >>
>>> >> >>>> >> Thanks for all the thought on this, Marcus!
>>> >> >>>> >>
>>> >> >>>> >>
>>> >> >>>> >> On Fri, Sep 13, 2013 at 8:25 PM, Marcus Sorensen
>>> >> >>>> >> <shadow...@gmail.com>
>>> >> >>>> >> wrote:
>>> >> >>>> >>>
>>> >> >>>> >>> Perfect. You'll have a domain def ( the VM), a disk def, and
>>> the
>>> >> >>>> >>> attach
>>> >> >>>> >>> the disk def to the vm. You may need to do your own
>>> >> >>>> >>> StorageAdaptor
>>> >> >>>> >>> and run
>>> >> >>>> >>> iscsiadm commands to accomplish that, depending on how the
>>> >> >>>> >>> libvirt
>>> >> >>>> >>> iscsi
>>> >> >>>> >>> works. My impression is that a 1:1:1 pool/lun/volume isn't
>>> how it
>>> >> >>>> >>> works on
>>> >> >>>> >>> xen at the momen., nor is it ideal.
>>> >> >>>> >>>
>>> >> >>>> >>> Your plugin will handle acls as far as which host can see
>>> which
>>> >> >>>> >>> luns
>>> >> >>>> >>> as
>>> >> >>>> >>> well, I remember discussing that months ago, so that a disk
>>> won't
>>> >> >>>> >>> be
>>> >> >>>> >>> connected until the hypervisor has exclusive access, so it
>>> will
>>> >> >>>> >>> be
>>> >> >>>> >>> safe and
>>> >> >>>> >>> fence the disk from rogue nodes that cloudstack loses
>>> >> >>>> >>> connectivity
>>> >> >>>> >>> with. It
>>> >> >>>> >>> should revoke access to everything but the target host...
>>> Except
>>> >> >>>> >>> for
>>> >> >>>> >>> during
>>> >> >>>> >>> migration but we can discuss that later, there's a migration
>>> prep
>>> >> >>>> >>> process
>>> >> >>>> >>> where the new host can be added to the acls, and the old
>>> host can
>>> >> >>>> >>> be
>>> >> >>>> >>> removed
>>> >> >>>> >>> post migration.
>>> >> >>>> >>>
>>> >> >>>> >>> On Sep 13, 2013 8:16 PM, "Mike Tutkowski"
>>> >> >>>> >>> <mike.tutkow...@solidfire.com>
>>> >> >>>> >>> wrote:
>>> >> >>>> >>>>
>>> >> >>>> >>>> Yeah, that would be ideal.
>>> >> >>>> >>>>
>>> >> >>>> >>>> So, I would still need to discover the iSCSI target, log in
>>> to
>>> >> >>>> >>>> it,
>>> >> >>>> >>>> then
>>> >> >>>> >>>> figure out what /dev/sdX was created as a result (and leave
>>> it
>>> >> >>>> >>>> as
>>> >> >>>> >>>> is - do
>>> >> >>>> >>>> not format it with any file system...clustered or not). I
>>> would
>>> >> >>>> >>>> pass that
>>> >> >>>> >>>> device into the VM.
>>> >> >>>> >>>>
>>> >> >>>> >>>> Kind of accurate?
>>> >> >>>> >>>>
>>> >> >>>> >>>>
>>> >> >>>> >>>> On Fri, Sep 13, 2013 at 8:07 PM, Marcus Sorensen
>>> >> >>>> >>>> <shadow...@gmail.com>
>>> >> >>>> >>>> wrote:
>>> >> >>>> >>>>>
>>> >> >>>> >>>>> Look in LibvirtVMDef.java (I think) for the disk
>>> definitions.
>>> >> >>>> >>>>> There are
>>> >> >>>> >>>>> ones that work for block devices rather than files. You can
>>> >> >>>> >>>>> piggy
>>> >> >>>> >>>>> back off
>>> >> >>>> >>>>> of the existing disk definitions and attach it to the vm
>>> as a
>>> >> >>>> >>>>> block device.
>>> >> >>>> >>>>> The definition is an XML string per libvirt XML format.
>>> You may
>>> >> >>>> >>>>> want to use
>>> >> >>>> >>>>> an alternate path to the disk rather than just /dev/sdx
>>> like I
>>> >> >>>> >>>>> mentioned,
>>> >> >>>> >>>>> there are by-id paths to the block devices, as well as
>>> other
>>> >> >>>> >>>>> ones
>>> >> >>>> >>>>> that will
>>> >> >>>> >>>>> be consistent and easier for management, not sure how
>>> familiar
>>> >> >>>> >>>>> you
>>> >> >>>> >>>>> are with
>>> >> >>>> >>>>> device naming on Linux.
>>> >> >>>> >>>>>
>>> >> >>>> >>>>> On Sep 13, 2013 8:00 PM, "Marcus Sorensen"
>>> >> >>>> >>>>> <shadow...@gmail.com>
>>> >> >>>> >>>>> wrote:
>>> >> >>>> >>>>>>
>>> >> >>>> >>>>>> No, as that would rely on virtualized network/iscsi
>>> initiator
>>> >> >>>> >>>>>> inside
>>> >> >>>> >>>>>> the vm, which also sucks. I mean attach /dev/sdx (your
>>> lun on
>>> >> >>>> >>>>>> hypervisor) as
>>> >> >>>> >>>>>> a disk to the VM, rather than attaching some image file
>>> that
>>> >> >>>> >>>>>> resides on a
>>> >> >>>> >>>>>> filesystem, mounted on the host, living on a target.
>>> >> >>>> >>>>>>
>>> >> >>>> >>>>>> Actually, if you plan on the storage supporting live
>>> migration
>>> >> >>>> >>>>>> I
>>> >> >>>> >>>>>> think
>>> >> >>>> >>>>>> this is the only way. You can't put a filesystem on it and
>>> >> >>>> >>>>>> mount
>>> >> >>>> >>>>>> it in two
>>> >> >>>> >>>>>> places to facilitate migration unless its a clustered
>>> >> >>>> >>>>>> filesystem,
>>> >> >>>> >>>>>> in which
>>> >> >>>> >>>>>> case you're back to shared mount point.
>>> >> >>>> >>>>>>
>>> >> >>>> >>>>>> As far as I'm aware, the xenserver SR style is basically
>>> LVM
>>> >> >>>> >>>>>> with
>>> >> >>>> >>>>>> a xen
>>> >> >>>> >>>>>> specific cluster management, a custom CLVM. They don't
>>> use a
>>> >> >>>> >>>>>> filesystem
>>> >> >>>> >>>>>> either.
>>> >> >>>> >>>>>>
>>> >> >>>> >>>>>> On Sep 13, 2013 7:44 PM, "Mike Tutkowski"
>>> >> >>>> >>>>>> <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>
>>> >> >>>> >>>>>>> When you say, "wire up the lun directly to the vm," do
>>> you
>>> >> >>>> >>>>>>> mean
>>> >> >>>> >>>>>>> circumventing the hypervisor? I didn't think we could do
>>> that
>>> >> >>>> >>>>>>> in
>>> >> >>>> >>>>>>> CS.
>>> >> >>>> >>>>>>> OpenStack, on the other hand, always circumvents the
>>> >> >>>> >>>>>>> hypervisor,
>>> >> >>>> >>>>>>> as far as I
>>> >> >>>> >>>>>>> know.
>>> >> >>>> >>>>>>>
>>> >> >>>> >>>>>>>
>>> >> >>>> >>>>>>> On Fri, Sep 13, 2013 at 7:40 PM, Marcus Sorensen
>>> >> >>>> >>>>>>> <shadow...@gmail.com>
>>> >> >>>> >>>>>>> wrote:
>>> >> >>>> >>>>>>>>
>>> >> >>>> >>>>>>>> Better to wire up the lun directly to the vm unless
>>> there is
>>> >> >>>> >>>>>>>> a
>>> >> >>>> >>>>>>>> good
>>> >> >>>> >>>>>>>> reason not to.
>>> >> >>>> >>>>>>>>
>>> >> >>>> >>>>>>>> On Sep 13, 2013 7:40 PM, "Marcus Sorensen"
>>> >> >>>> >>>>>>>> <shadow...@gmail.com>
>>> >> >>>> >>>>>>>> wrote:
>>> >> >>>> >>>>>>>>>
>>> >> >>>> >>>>>>>>> You could do that, but as mentioned I think its a
>>> mistake
>>> >> >>>> >>>>>>>>> to
>>> >> >>>> >>>>>>>>> go to
>>> >> >>>> >>>>>>>>> the trouble of creating a 1:1 mapping of CS volumes to
>>> luns
>>> >> >>>> >>>>>>>>> and then putting
>>> >> >>>> >>>>>>>>> a filesystem on it, mounting it, and then putting a
>>> QCOW2
>>> >> >>>> >>>>>>>>> or
>>> >> >>>> >>>>>>>>> even RAW disk
>>> >> >>>> >>>>>>>>> image on that filesystem. You'll lose a lot of iops
>>> along
>>> >> >>>> >>>>>>>>> the
>>> >> >>>> >>>>>>>>> way, and have
>>> >> >>>> >>>>>>>>> more overhead with the filesystem and its journaling,
>>> etc.
>>> >> >>>> >>>>>>>>>
>>> >> >>>> >>>>>>>>> On Sep 13, 2013 7:33 PM, "Mike Tutkowski"
>>> >> >>>> >>>>>>>>> <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>>>>
>>> >> >>>> >>>>>>>>>> Ah, OK, I didn't know that was such new ground in KVM
>>> with
>>> >> >>>> >>>>>>>>>> CS.
>>> >> >>>> >>>>>>>>>>
>>> >> >>>> >>>>>>>>>> So, the way people use our SAN with KVM and CS today
>>> is by
>>> >> >>>> >>>>>>>>>> selecting SharedMountPoint and specifying the
>>> location of
>>> >> >>>> >>>>>>>>>> the
>>> >> >>>> >>>>>>>>>> share.
>>> >> >>>> >>>>>>>>>>
>>> >> >>>> >>>>>>>>>> They can set up their share using Open iSCSI by
>>> >> >>>> >>>>>>>>>> discovering
>>> >> >>>> >>>>>>>>>> their
>>> >> >>>> >>>>>>>>>> iSCSI target, logging in to it, then mounting it
>>> somewhere
>>> >> >>>> >>>>>>>>>> on
>>> >> >>>> >>>>>>>>>> their file
>>> >> >>>> >>>>>>>>>> system.
>>> >> >>>> >>>>>>>>>>
>>> >> >>>> >>>>>>>>>> Would it make sense for me to just do that discovery,
>>> >> >>>> >>>>>>>>>> logging
>>> >> >>>> >>>>>>>>>> in,
>>> >> >>>> >>>>>>>>>> and mounting behind the scenes for them and letting
>>> the
>>> >> >>>> >>>>>>>>>> current code manage
>>> >> >>>> >>>>>>>>>> the rest as it currently does?
>>> >> >>>> >>>>>>>>>>
>>> >> >>>> >>>>>>>>>>
>>> >> >>>> >>>>>>>>>> On Fri, Sep 13, 2013 at 7:27 PM, Marcus Sorensen
>>> >> >>>> >>>>>>>>>> <shadow...@gmail.com> wrote:
>>> >> >>>> >>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>> Oh, hypervisor snapshots are a bit different. I need
>>> to
>>> >> >>>> >>>>>>>>>>> catch up
>>> >> >>>> >>>>>>>>>>> on the work done in KVM, but this is basically just
>>> disk
>>> >> >>>> >>>>>>>>>>> snapshots + memory
>>> >> >>>> >>>>>>>>>>> dump. I still think disk snapshots would preferably
>>> be
>>> >> >>>> >>>>>>>>>>> handled by the SAN,
>>> >> >>>> >>>>>>>>>>> and then memory dumps can go to secondary storage or
>>> >> >>>> >>>>>>>>>>> something else. This is
>>> >> >>>> >>>>>>>>>>> relatively new ground with CS and KVM, so we will
>>> want to
>>> >> >>>> >>>>>>>>>>> see how others are
>>> >> >>>> >>>>>>>>>>> planning theirs.
>>> >> >>>> >>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>> On Sep 13, 2013 7:20 PM, "Marcus Sorensen"
>>> >> >>>> >>>>>>>>>>> <shadow...@gmail.com>
>>> >> >>>> >>>>>>>>>>> wrote:
>>> >> >>>> >>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>> Let me back up and say I don't think you'd use a vdi
>>> >> >>>> >>>>>>>>>>>> style
>>> >> >>>> >>>>>>>>>>>> on an
>>> >> >>>> >>>>>>>>>>>> iscsi lun. I think you'd want to treat it as a RAW
>>> >> >>>> >>>>>>>>>>>> format.
>>> >> >>>> >>>>>>>>>>>> Otherwise you're
>>> >> >>>> >>>>>>>>>>>> putting a filesystem on your lun, mounting it,
>>> creating
>>> >> >>>> >>>>>>>>>>>> a
>>> >> >>>> >>>>>>>>>>>> QCOW2 disk image,
>>> >> >>>> >>>>>>>>>>>> and that seems unnecessary and a performance killer.
>>> >> >>>> >>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>> So probably attaching the raw iscsi lun as a disk
>>> to the
>>> >> >>>> >>>>>>>>>>>> VM, and
>>> >> >>>> >>>>>>>>>>>> handling snapshots on the San side via the storage
>>> >> >>>> >>>>>>>>>>>> plugin
>>> >> >>>> >>>>>>>>>>>> is best. My
>>> >> >>>> >>>>>>>>>>>> impression from the storage plugin refactor was that
>>> >> >>>> >>>>>>>>>>>> there
>>> >> >>>> >>>>>>>>>>>> was a snapshot
>>> >> >>>> >>>>>>>>>>>> service that would allow the San to handle
>>> snapshots.
>>> >> >>>> >>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>> On Sep 13, 2013 7:15 PM, "Marcus Sorensen"
>>> >> >>>> >>>>>>>>>>>> <shadow...@gmail.com>
>>> >> >>>> >>>>>>>>>>>> wrote:
>>> >> >>>> >>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>> Ideally volume snapshots can be handled by the SAN
>>> back
>>> >> >>>> >>>>>>>>>>>>> end, if
>>> >> >>>> >>>>>>>>>>>>> the SAN supports it. The cloudstack mgmt server
>>> could
>>> >> >>>> >>>>>>>>>>>>> call
>>> >> >>>> >>>>>>>>>>>>> your plugin for
>>> >> >>>> >>>>>>>>>>>>> volume snapshot and it would be hypervisor
>>> agnostic. As
>>> >> >>>> >>>>>>>>>>>>> far as space, that
>>> >> >>>> >>>>>>>>>>>>> would depend on how your SAN handles it. With
>>> ours, we
>>> >> >>>> >>>>>>>>>>>>> carve out luns from a
>>> >> >>>> >>>>>>>>>>>>> pool, and the snapshot spave comes from the pool
>>> and is
>>> >> >>>> >>>>>>>>>>>>> independent of the
>>> >> >>>> >>>>>>>>>>>>> LUN size the host sees.
>>> >> >>>> >>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>> On Sep 13, 2013 7:10 PM, "Mike Tutkowski"
>>> >> >>>> >>>>>>>>>>>>> <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> Hey Marcus,
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> I wonder if the iSCSI storage pool type for
>>> libvirt
>>> >> >>>> >>>>>>>>>>>>>> won't
>>> >> >>>> >>>>>>>>>>>>>> work
>>> >> >>>> >>>>>>>>>>>>>> when you take into consideration hypervisor
>>> snapshots?
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> On XenServer, when you take a hypervisor
>>> snapshot, the
>>> >> >>>> >>>>>>>>>>>>>> VDI for
>>> >> >>>> >>>>>>>>>>>>>> the snapshot is placed on the same storage
>>> repository
>>> >> >>>> >>>>>>>>>>>>>> as
>>> >> >>>> >>>>>>>>>>>>>> the volume is on.
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> Same idea for VMware, I believe.
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> So, what would happen in my case (let's say for
>>> >> >>>> >>>>>>>>>>>>>> XenServer
>>> >> >>>> >>>>>>>>>>>>>> and
>>> >> >>>> >>>>>>>>>>>>>> VMware for 4.3 because I don't support hypervisor
>>> >> >>>> >>>>>>>>>>>>>> snapshots in 4.2) is I'd
>>> >> >>>> >>>>>>>>>>>>>> make an iSCSI target that is larger than what the
>>> user
>>> >> >>>> >>>>>>>>>>>>>> requested for the
>>> >> >>>> >>>>>>>>>>>>>> CloudStack volume (which is fine because our SAN
>>> >> >>>> >>>>>>>>>>>>>> thinly
>>> >> >>>> >>>>>>>>>>>>>> provisions volumes,
>>> >> >>>> >>>>>>>>>>>>>> so the space is not actually used unless it needs
>>> to
>>> >> >>>> >>>>>>>>>>>>>> be).
>>> >> >>>> >>>>>>>>>>>>>> The CloudStack
>>> >> >>>> >>>>>>>>>>>>>> volume would be the only "object" on the SAN
>>> volume
>>> >> >>>> >>>>>>>>>>>>>> until
>>> >> >>>> >>>>>>>>>>>>>> a hypervisor
>>> >> >>>> >>>>>>>>>>>>>> snapshot is taken. This snapshot would also
>>> reside on
>>> >> >>>> >>>>>>>>>>>>>> the
>>> >> >>>> >>>>>>>>>>>>>> SAN volume.
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> If this is also how KVM behaves and there is no
>>> >> >>>> >>>>>>>>>>>>>> creation
>>> >> >>>> >>>>>>>>>>>>>> of
>>> >> >>>> >>>>>>>>>>>>>> LUNs within an iSCSI target from libvirt (which,
>>> even
>>> >> >>>> >>>>>>>>>>>>>> if
>>> >> >>>> >>>>>>>>>>>>>> there were support
>>> >> >>>> >>>>>>>>>>>>>> for this, our SAN currently only allows one LUN
>>> per
>>> >> >>>> >>>>>>>>>>>>>> iSCSI
>>> >> >>>> >>>>>>>>>>>>>> target), then I
>>> >> >>>> >>>>>>>>>>>>>> don't see how using this model will work.
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> Perhaps I will have to go enhance the current way
>>> this
>>> >> >>>> >>>>>>>>>>>>>> works
>>> >> >>>> >>>>>>>>>>>>>> with DIR?
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> What do you think?
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> Thanks
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>> On Fri, Sep 13, 2013 at 6:28 PM, Mike Tutkowski
>>> >> >>>> >>>>>>>>>>>>>> <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>>> That appears to be the way it's used for iSCSI
>>> access
>>> >> >>>> >>>>>>>>>>>>>>> today.
>>> >> >>>> >>>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>>> I suppose I could go that route, too, but I
>>> might as
>>> >> >>>> >>>>>>>>>>>>>>> well
>>> >> >>>> >>>>>>>>>>>>>>> leverage what libvirt has for iSCSI instead.
>>> >> >>>> >>>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>>> On Fri, Sep 13, 2013 at 6:26 PM, Marcus Sorensen
>>> >> >>>> >>>>>>>>>>>>>>> <shadow...@gmail.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> To your question about SharedMountPoint, I
>>> believe
>>> >> >>>> >>>>>>>>>>>>>>>> it
>>> >> >>>> >>>>>>>>>>>>>>>> just
>>> >> >>>> >>>>>>>>>>>>>>>> acts like a
>>> >> >>>> >>>>>>>>>>>>>>>> 'DIR' storage type or something similar to
>>> that. The
>>> >> >>>> >>>>>>>>>>>>>>>> end-user
>>> >> >>>> >>>>>>>>>>>>>>>> is
>>> >> >>>> >>>>>>>>>>>>>>>> responsible for mounting a file system that all
>>> KVM
>>> >> >>>> >>>>>>>>>>>>>>>> hosts can
>>> >> >>>> >>>>>>>>>>>>>>>> access,
>>> >> >>>> >>>>>>>>>>>>>>>> and CloudStack is oblivious to what is
>>> providing the
>>> >> >>>> >>>>>>>>>>>>>>>> storage.
>>> >> >>>> >>>>>>>>>>>>>>>> It could
>>> >> >>>> >>>>>>>>>>>>>>>> be NFS, or OCFS2, or some other clustered
>>> >> >>>> >>>>>>>>>>>>>>>> filesystem,
>>> >> >>>> >>>>>>>>>>>>>>>> cloudstack just
>>> >> >>>> >>>>>>>>>>>>>>>> knows that the provided directory path has VM
>>> >> >>>> >>>>>>>>>>>>>>>> images.
>>> >> >>>> >>>>>>>>>>>>>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> On Fri, Sep 13, 2013 at 6:23 PM, Marcus Sorensen
>>> >> >>>> >>>>>>>>>>>>>>>> <shadow...@gmail.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> > Oh yes, you can use NFS, LVM, and iSCSI all
>>> at the
>>> >> >>>> >>>>>>>>>>>>>>>> > same
>>> >> >>>> >>>>>>>>>>>>>>>> > time.
>>> >> >>>> >>>>>>>>>>>>>>>> > Multiples, in fact.
>>> >> >>>> >>>>>>>>>>>>>>>> >
>>> >> >>>> >>>>>>>>>>>>>>>> > On Fri, Sep 13, 2013 at 6:19 PM, Mike
>>> Tutkowski
>>> >> >>>> >>>>>>>>>>>>>>>> > <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> >> Looks like you can have multiple storage
>>> pools:
>>> >> >>>> >>>>>>>>>>>>>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >> mtutkowski@ubuntu:~$ virsh pool-list
>>> >> >>>> >>>>>>>>>>>>>>>> >> Name State Autostart
>>> >> >>>> >>>>>>>>>>>>>>>> >> -----------------------------------------
>>> >> >>>> >>>>>>>>>>>>>>>> >> default active yes
>>> >> >>>> >>>>>>>>>>>>>>>> >> iSCSI active no
>>> >> >>>> >>>>>>>>>>>>>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >> On Fri, Sep 13, 2013 at 6:12 PM, Mike
>>> Tutkowski
>>> >> >>>> >>>>>>>>>>>>>>>> >> <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>> Reading through the docs you pointed out.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>> I see what you're saying now.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>> You can create an iSCSI (libvirt) storage
>>> pool
>>> >> >>>> >>>>>>>>>>>>>>>> >>> based on
>>> >> >>>> >>>>>>>>>>>>>>>> >>> an iSCSI target.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>> In my case, the iSCSI target would only
>>> have one
>>> >> >>>> >>>>>>>>>>>>>>>> >>> LUN, so
>>> >> >>>> >>>>>>>>>>>>>>>> >>> there would only
>>> >> >>>> >>>>>>>>>>>>>>>> >>> be one iSCSI (libvirt) storage volume in the
>>> >> >>>> >>>>>>>>>>>>>>>> >>> (libvirt)
>>> >> >>>> >>>>>>>>>>>>>>>> >>> storage pool.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>> As you say, my plug-in creates and destroys
>>> >> >>>> >>>>>>>>>>>>>>>> >>> iSCSI
>>> >> >>>> >>>>>>>>>>>>>>>> >>> targets/LUNs on the
>>> >> >>>> >>>>>>>>>>>>>>>> >>> SolidFire SAN, so it is not a problem that
>>> >> >>>> >>>>>>>>>>>>>>>> >>> libvirt
>>> >> >>>> >>>>>>>>>>>>>>>> >>> does
>>> >> >>>> >>>>>>>>>>>>>>>> >>> not support
>>> >> >>>> >>>>>>>>>>>>>>>> >>> creating/deleting iSCSI targets/LUNs.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>> It looks like I need to test this a bit to
>>> see
>>> >> >>>> >>>>>>>>>>>>>>>> >>> if
>>> >> >>>> >>>>>>>>>>>>>>>> >>> libvirt
>>> >> >>>> >>>>>>>>>>>>>>>> >>> supports
>>> >> >>>> >>>>>>>>>>>>>>>> >>> multiple iSCSI storage pools (as you
>>> mentioned,
>>> >> >>>> >>>>>>>>>>>>>>>> >>> since
>>> >> >>>> >>>>>>>>>>>>>>>> >>> each one of its
>>> >> >>>> >>>>>>>>>>>>>>>> >>> storage pools would map to one of my iSCSI
>>> >> >>>> >>>>>>>>>>>>>>>> >>> targets/LUNs).
>>> >> >>>> >>>>>>>>>>>>>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>> On Fri, Sep 13, 2013 at 5:58 PM, Mike
>>> Tutkowski
>>> >> >>>> >>>>>>>>>>>>>>>> >>> <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> LibvirtStoragePoolDef has this type:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> public enum poolType {
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> ISCSI("iscsi"), NETFS("netfs"),
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> LOGICAL("logical"), DIR("dir"),
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> RBD("rbd");
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> String _poolType;
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> poolType(String poolType) {
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> _poolType = poolType;
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> }
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> @Override
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> public String toString() {
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> return _poolType;
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> }
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> }
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> It doesn't look like the iSCSI type is
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> currently
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> being
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> used, but I'm
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> understanding more what you were getting
>>> at.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> Can you tell me for today (say, 4.2), when
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> someone
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> selects the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> SharedMountPoint option and uses it with
>>> iSCSI,
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> is
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> that
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> the "netfs" option
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> above or is that just for NFS?
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> Thanks!
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> On Fri, Sep 13, 2013 at 5:50 PM, Marcus
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> Sorensen
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> <shadow...@gmail.com>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> Take a look at this:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> http://libvirt.org/storage.html#StorageBackendISCSI
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> "Volumes must be pre-allocated on the
>>> iSCSI
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> server, and
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> cannot be
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> created via the libvirt APIs.", which I
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> believe
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> your
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> plugin will take
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> care of. Libvirt just does the work of
>>> logging
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> in
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> and
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> hooking it up to
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> the VM (I believe the Xen api does that
>>> work
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> in
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> the Xen
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> stuff).
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> What I'm not sure about is whether this
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> provides
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> a 1:1
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> mapping, or if
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> it just allows you to register 1 iscsi
>>> device
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> as
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> a
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> pool. You may need
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> to write some test code or read up a bit
>>> more
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> about
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> this. Let us know.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> If it doesn't, you may just have to write
>>> your
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> own
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> storage adaptor
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> rather than changing
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> LibvirtStorageAdaptor.java.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> We
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> can cross that
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> bridge when we get there.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> As far as interfacing with libvirt, see
>>> the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> java
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> bindings doc.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> http://libvirt.org/sources/java/javadoc/
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> Normally,
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> you'll see a
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> connection object be made, then calls
>>> made to
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> that
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> 'conn' object. You
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> can look at the LibvirtStorageAdaptor to
>>> see
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> how
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> that
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> is done for
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> other pool types, and maybe write some
>>> test
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> java
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> code
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> to see if you
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> can interface with libvirt and register
>>> iscsi
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> storage
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> pools before you
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> get started.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> On Fri, Sep 13, 2013 at 5:31 PM, Mike
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> Tutkowski
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > So, Marcus, I need to investigate
>>> libvirt
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > more,
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > but
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > you figure it
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > supports
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > connecting to/disconnecting from iSCSI
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > targets,
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > right?
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > On Fri, Sep 13, 2013 at 5:29 PM, Mike
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > Tutkowski
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> > <mike.tutkow...@solidfire.com> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> OK, thanks, Marcus
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> I am currently looking through some of
>>> the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> classes
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> you pointed out
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> last
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> week or so.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> On Fri, Sep 13, 2013 at 5:26 PM, Marcus
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> Sorensen
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> <shadow...@gmail.com>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> Yes, my guess is that you will need
>>> the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> iscsi
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> initiator utilities
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> installed. There should be standard
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> packages
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> for
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> any distro. Then
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> you'd call
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> an agent storage adaptor to do the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> initiator
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> login.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> See the info I
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> sent
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> previously about
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> LibvirtStorageAdaptor.java
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> and
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> libvirt iscsi
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> storage type
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> to see if that fits your need.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> On Sep 13, 2013 4:55 PM, "Mike
>>> Tutkowski"
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> <mike.tutkow...@solidfire.com>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>> wrote:
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Hi,
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> As you may remember, during the 4.2
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> release
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> I
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> developed a SolidFire
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> (storage) plug-in for CloudStack.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> This plug-in was invoked by the
>>> storage
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> framework
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> at the necessary
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> times
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> so that I could dynamically create
>>> and
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> delete
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> volumes on the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> SolidFire SAN
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> (among other activities).
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> This is necessary so I can establish
>>> a
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> 1:1
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> mapping
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> between a
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> CloudStack
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> volume and a SolidFire volume for
>>> QoS.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> In the past, CloudStack always
>>> expected
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> admin
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> to create large
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> volumes ahead of time and those
>>> volumes
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> would
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> likely house many
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> root and
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> data disks (which is not QoS
>>> friendly).
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> To make this 1:1 mapping scheme
>>> work, I
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> needed to
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> modify logic in
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> XenServer and VMware plug-ins so they
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> could
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> create/delete storage
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> repositories/datastores as needed.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> For 4.3 I want to make this happen
>>> with
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> KVM.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> I'm coming up to speed with how this
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> might
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> work on
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> KVM, but I'm
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> still
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> pretty new to KVM.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Does anyone familiar with KVM know
>>> how I
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> will need
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> to interact with
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> iSCSI target? For example, will I
>>> have to
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> expect
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Open iSCSI will be
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> installed on the KVM host and use it
>>> for
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> this to
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> work?
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Thanks for any suggestions,
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Mike
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> --
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Mike Tutkowski
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Senior CloudStack Developer,
>>> SolidFire
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Inc.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> e: mike.tutkow...@solidfire.com
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> o: 303.746.7302
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> Advancing the way the world uses the
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>>> cloud™
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >>
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> --
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> Mike Tutkowski
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> Senior CloudStack Developer, SolidFire
>>> Inc.
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> e: mike.tutkow...@solidfire.com
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> o: 303.746.7302
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >> Advancing the way the world uses the
>>> cloud™
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >
>>> >> >>>> >>>>>>>>>>>>>>>> >>>>> >
>>> >> >>>> >>>>>>>>>>>>>>>>
>>>
>> ...
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
