Are we not disconnecting when we stop the vm? There's a method for it, we
should be. disconnectPhysicalDiskViaVmSpec
On Oct 23, 2013 1:28 PM, "Mike Tutkowski" <mike.tutkow...@solidfire.com>
wrote:
> I see one problem for us now, Marcus.
>
> * You have a running VM that you attach a volume to.
> * You stop the VM.
> * You detach the volume.
> * You start up the VM.
>
> The VM will not be connected to the volume (which is good), but the
> hypervisor will still be connected to the volume.
>
> It would be great if we actually sent a command to the last host ID of the
> stopped VM when detaching a volume (to have the hypervisor disconnect from
> the volume).
>
> What do you think about that?
>
>
> On Wed, Oct 23, 2013 at 1:15 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> OK, whatever way you prefer then, Marcus (createVdb first or second).
>>
>> If I leave createVdb first and return 0, it does seem to work.
>>
>>
>> On Wed, Oct 23, 2013 at 11:13 AM, Marcus Sorensen <shadow...@gmail.com>wrote:
>>
>>> I think we could flip-flop these two lines if necessary:
>>>
>>> createVbd(conn, vmSpec, vmName, vm);
>>>
>>> _storagePoolMgr.connectPhysicalDisksViaVmSpec(vmSpec);
>>>
>>> I haven't actually tried it though. But in general I don't see the
>>> Libvirt DiskDef using size at all, which is what createVbd does
>>> (creates XML definitions for disks to attach to the VM definition). It
>>> just takes the device at it's native advertised size when it actually
>>> goes to use it.
>>>
>>> On Wed, Oct 23, 2013 at 10:52 AM, Mike Tutkowski
>>> <mike.tutkow...@solidfire.com> wrote:
>>> > Little problem that I wanted to get your take on, Marcus.
>>> >
>>> > When a VM is being started, we call createVdb before calling
>>> > connectPhysicalDisksViaVmSpec.
>>> >
>>> > The problem is that createVdb calls getPhysicalDisk and my volume has
>>> not
>>> > yet been connected because connectPhysicalDisksViaVmSpec has not yet
>>> been
>>> > called.
>>> >
>>> > When I try to read up the size of the disk to populate a PhysicalDisk,
>>> I get
>>> > an error, of course, because the path does not yet exist.
>>> >
>>> > I could populate a 0 for the size of the physical disk and then the
>>> next
>>> > time getPhysicalDisk is called, it should be filled in with a proper
>>> size.
>>> >
>>> > Do you see a problem with that approach?
>>> >
>>> >
>>> > On Tue, Oct 22, 2013 at 6:40 PM, Marcus Sorensen <shadow...@gmail.com>
>>> > wrote:
>>> >>
>>> >> That's right. All should be well.
>>> >>
>>> >> On Oct 22, 2013 6:03 PM, "Mike Tutkowski" <
>>> mike.tutkow...@solidfire.com>
>>> >> wrote:
>>> >>>
>>> >>> Looks like we disconnect physical disks when the VM is stopped.
>>> >>>
>>> >>> I didn't see that before.
>>> >>>
>>> >>> I suppose that means the disks are physically disconnected when the
>>> VM is
>>> >>> stopped, but the CloudStack DB still has the VM associated with the
>>> disks
>>> >>> for the next time the VM may be started up (unless someone does a
>>> disconnect
>>> >>> while the VM is in the Stopped State).
>>> >>>
>>> >>>
>>> >>> On Tue, Oct 22, 2013 at 4:19 PM, Mike Tutkowski
>>> >>> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>
>>> >>>> Hey Marcus,
>>> >>>>
>>> >>>> Quick question for you related to attaching/detaching volumes when
>>> the
>>> >>>> VM is in the Stopped State.
>>> >>>>
>>> >>>> If I detach a volume from a VM that is in the Stopped State, the DB
>>> >>>> seems to get updated, but I don't see a command going to the KVM
>>> hypervisor
>>> >>>> that leads to the removal of the iSCSI target.
>>> >>>>
>>> >>>> It seems the iSCSI target is only removed the next time the VM is
>>> >>>> started.
>>> >>>>
>>> >>>> Do you know if this is true?
>>> >>>>
>>> >>>> If it is, I'm concerned that the volume could be attached to
>>> another VM
>>> >>>> before the Stopped VM is re-started and when the Stopped VM gets
>>> restarted
>>> >>>> that it would disconnect the iSCSI volume from underneath the VM
>>> that now
>>> >>>> has the volume attached.
>>> >>>>
>>> >>>> I still want to perform some tests on this, but am first trying to
>>> get a
>>> >>>> VM to start after I've attached a volume to it when it was in the
>>> Stopped
>>> >>>> State.
>>> >>>>
>>> >>>> Thanks,
>>> >>>> Mike
>>> >>>>
>>> >>>>
>>> >>>> On Mon, Oct 21, 2013 at 10:57 PM, Mike Tutkowski
>>> >>>> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>
>>> >>>>> Thanks for that info, Marcus.
>>> >>>>>
>>> >>>>> By the way, I wanted to see if I could attach my volume to a VM in
>>> the
>>> >>>>> Stopped State.
>>> >>>>>
>>> >>>>> The attach logic didn't trigger any exceptions; however, when I
>>> started
>>> >>>>> the VM, I received an Insufficient Capacity exception.
>>> >>>>>
>>> >>>>> If I detach the volume and then start the VM, the VM starts just
>>> fine.
>>> >>>>>
>>> >>>>> I noticed a problem here (in StoragePoolHostDaoImpl):
>>> >>>>>
>>> >>>>> @Override
>>> >>>>>
>>> >>>>> public StoragePoolHostVO findByPoolHost(long poolId, long
>>> hostId) {
>>> >>>>>
>>> >>>>> SearchCriteria<StoragePoolHostVO> sc =
>>> PoolHostSearch.create();
>>> >>>>>
>>> >>>>> sc.setParameters("pool_id", poolId);
>>> >>>>>
>>> >>>>> sc.setParameters("host_id", hostId);
>>> >>>>>
>>> >>>>> return findOneIncludingRemovedBy(sc);
>>> >>>>>
>>> >>>>> }
>>> >>>>>
>>> >>>>> The findOneIncludingRemovedBy method returns null (the poolId is my
>>> >>>>> storage pool's ID and the hostId is the expected host ID).
>>> >>>>>
>>> >>>>> I'm not sure what this method is trying to do.
>>> >>>>>
>>> >>>>> I looked in the storage_pool_host_ref table (is that the correct
>>> >>>>> table?) and it only has one row, which maps the local storage pool
>>> of the
>>> >>>>> KVM host to the KVM host (which explains why no match is found for
>>> my
>>> >>>>> situation).
>>> >>>>>
>>> >>>>> Do you understand what this logic is trying to do?
>>> >>>>>
>>> >>>>> Thanks!
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> On Mon, Oct 21, 2013 at 8:08 PM, Marcus Sorensen <
>>> shadow...@gmail.com>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>> Do you have the capability to clone the root disk? Normally the
>>> >>>>>> template is installed to primary, and then cloned for each root
>>> disk.
>>> >>>>>> In some cases (such as CLVM), this isn't efficient and so the
>>> template
>>> >>>>>> is copied fresh to populate each root disk.
>>> >>>>>>
>>> >>>>>> I'm actually not 100% sure how this works in the new code. It
>>> used to
>>> >>>>>> be handled by copyPhysicalDisk in the storage adaptor, called by
>>> >>>>>> copyTemplateToPrimaryStorage, which runs on the agent. It would
>>> pass
>>> >>>>>> template/secondary storage info, and the destination
>>> volume/primary
>>> >>>>>> storage info, and copyPhysicalDisk would do the work of
>>> installing the
>>> >>>>>> image to the destination. Then subsequent root disks would be
>>> cloned
>>> >>>>>> in CreateCommand by calling createDiskFromTemplate.
>>> >>>>>>
>>> >>>>>> In master it looks like this was moved to KVMStorageProcessor
>>> >>>>>> 'cloneVolumeFromBaseTemplate', although I think this just takes
>>> over
>>> >>>>>> as default, and there's something in your storage driver that
>>> should
>>> >>>>>> be capable of cloning templates on the mgmt server side. I'm less
>>> sure
>>> >>>>>> about how the template gets to primary storage in the first
>>> place, I
>>> >>>>>> assume copyTemplateToPrimaryStorage in KVMStorageProcessor calling
>>> >>>>>> copyPhysicalDisk in your adaptor. It's a bit tough for me to tell
>>> >>>>>> since our earlier storage adaptor did everything on the host it
>>> mostly
>>> >>>>>> just worked with the default stuff.
>>> >>>>>>
>>> >>>>>> On Mon, Oct 21, 2013 at 4:49 PM, Mike Tutkowski
>>> >>>>>> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> > Hey Marcus,
>>> >>>>>> >
>>> >>>>>> > So...now that this works well for data disks, I was wondering
>>> what
>>> >>>>>> > might be
>>> >>>>>> > involved in getting this process to work for root disks.
>>> >>>>>> >
>>> >>>>>> > Can you point me in the right direction as far as what gets
>>> invoked
>>> >>>>>> > when a
>>> >>>>>> > VM is being created on KVM (so that its root disk can be
>>> created and
>>> >>>>>> > the
>>> >>>>>> > necessary template laid down or ISO installed)?
>>> >>>>>> >
>>> >>>>>> > Thanks!
>>> >>>>>> >
>>> >>>>>> >
>>> >>>>>> > On Mon, Oct 21, 2013 at 1:14 PM, Mike Tutkowski
>>> >>>>>> > <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> >>
>>> >>>>>> >> Hey Marcus,
>>> >>>>>> >>
>>> >>>>>> >> Just wanted to let you know the branch of mine that has your
>>> code
>>> >>>>>> >> and mine
>>> >>>>>> >> appears to work well with regards to attaching a data disk to a
>>> >>>>>> >> running VM:
>>> >>>>>> >>
>>> >>>>>> >> fdisk -l from hypervisor:
>>> >>>>>> >>
>>> >>>>>> >> http://i.imgur.com/NkP5fo0.png
>>> >>>>>> >>
>>> >>>>>> >> fdisk -l from within VM:
>>> >>>>>> >>
>>> >>>>>> >> http://i.imgur.com/8YwiiC7.png
>>> >>>>>> >>
>>> >>>>>> >> I plan to do more testing on this over the coming days.
>>> >>>>>> >>
>>> >>>>>> >> If all goes well, perhaps we can check this code in by the end
>>> of
>>> >>>>>> >> the
>>> >>>>>> >> week?
>>> >>>>>> >>
>>> >>>>>> >> Talk to you later,
>>> >>>>>> >> Mike
>>> >>>>>> >>
>>> >>>>>> >>
>>> >>>>>> >> On Sun, Oct 20, 2013 at 10:23 PM, Mike Tutkowski
>>> >>>>>> >> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> >>>
>>> >>>>>> >>> Don't ask me, but it works now (I've been having this trouble
>>> >>>>>> >>> quite a
>>> >>>>>> >>> while today).
>>> >>>>>> >>>
>>> >>>>>> >>> I guess the trick is to send you an e-mail. :)
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>> On Sun, Oct 20, 2013 at 10:05 PM, Marcus Sorensen
>>> >>>>>> >>> <shadow...@gmail.com>
>>> >>>>>> >>> wrote:
>>> >>>>>> >>>>
>>> >>>>>> >>>> Did you create a service offering that uses local storage,
>>> or add
>>> >>>>>> >>>> a
>>> >>>>>> >>>> shared primary storage? By default there is no storage that
>>> >>>>>> >>>> matches the
>>> >>>>>> >>>> built in offerings.
>>> >>>>>> >>>>
>>> >>>>>> >>>> On Oct 20, 2013 9:39 PM, "Mike Tutkowski"
>>> >>>>>> >>>> <mike.tutkow...@solidfire.com>
>>> >>>>>> >>>> wrote:
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> Hey Marcus,
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> So, I went back to the branch of mine that has your code and
>>> >>>>>> >>>>> mine and
>>> >>>>>> >>>>> was able to create a new CloudStack install from scratch
>>> with it
>>> >>>>>> >>>>> (once
>>> >>>>>> >>>>> again, after manually deleting what was in
>>> >>>>>> >>>>> /var/lib/libvirt/images to the
>>> >>>>>> >>>>> get system VMs to start).
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> Anyways, my system VMs are running now and I tried to kick
>>> off a
>>> >>>>>> >>>>> VM
>>> >>>>>> >>>>> using the CentOS 6.3 image you provided me a while back.
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> The virtual router has a Status of Running; however, my VM
>>> fails
>>> >>>>>> >>>>> to
>>> >>>>>> >>>>> start (with the generic message of Insufficient Capacity).
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> I've not seen this exception before (related to the VR). Do
>>> you
>>> >>>>>> >>>>> have
>>> >>>>>> >>>>> any insight into this?:
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> com.cloud.exception.ResourceUnavailableException: Resource
>>> >>>>>> >>>>> [Pod:1] is
>>> >>>>>> >>>>> unreachable: Unable to apply userdata and password entry on
>>> >>>>>> >>>>> router
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3793)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyUserData(VirtualNetworkApplianceManagerImpl.java:3017)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.network.element.VirtualRouterElement.addPasswordAndUserdata(VirtualRouterElement.java:933)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElement(NetworkOrchestrator.java:1172)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(NetworkOrchestrator.java:1288)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1224)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:826)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:508)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3338)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2919)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2905)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:421)
>>> >>>>>> >>>>> at
>>> com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$1.runInContext(AsyncJobManagerImpl.java:532)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> >>>>>> >>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> >>>>>> >>>>> at java.lang.Thread.run(Thread.java:724)
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> Thanks!
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>> --
>>> >>>>>> >>> Mike Tutkowski
>>> >>>>>> >>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> >>> e: mike.tutkow...@solidfire.com
>>> >>>>>> >>> o: 303.746.7302
>>> >>>>>> >>> Advancing the way the world uses the cloud™
>>> >>>>>> >>
>>> >>>>>> >>
>>> >>>>>> >>
>>> >>>>>> >>
>>> >>>>>> >> --
>>> >>>>>> >> Mike Tutkowski
>>> >>>>>> >> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> >> e: mike.tutkow...@solidfire.com
>>> >>>>>> >> o: 303.746.7302
>>> >>>>>> >> Advancing the way the world uses the cloud™
>>> >>>>>> >
>>> >>>>>> >
>>> >>>>>> >
>>> >>>>>> >
>>> >>>>>> > --
>>> >>>>>> > Mike Tutkowski
>>> >>>>>> > Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> > e: mike.tutkow...@solidfire.com
>>> >>>>>> > o: 303.746.7302
>>> >>>>>> > Advancing the way the world uses the cloud™
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> Mike Tutkowski
>>> >>>>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>> e: mike.tutkow...@solidfire.com
>>> >>>>> o: 303.746.7302
>>> >>>>> Advancing the way the world uses the cloud™
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> Mike Tutkowski
>>> >>>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>> e: mike.tutkow...@solidfire.com
>>> >>>> o: 303.746.7302
>>> >>>> Advancing the way the world uses the cloud™
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Mike Tutkowski
>>> >>> Senior CloudStack Developer, SolidFire Inc.
>>> >>> e: mike.tutkow...@solidfire.com
>>> >>> o: 303.746.7302
>>> >>> Advancing the way the world uses the cloud™
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Mike Tutkowski
>>> > Senior CloudStack Developer, SolidFire Inc.
>>> > e: mike.tutkow...@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the cloud™
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
