First step is for me to get this working for KVM, though. :) Once I do that, I can perhaps make modifications to the storage framework and hypervisor plug-ins to refactor the logic and such.
On Sun, Sep 22, 2013 at 12:09 AM, Mike Tutkowski < mike.tutkow...@solidfire.com> wrote: > Same would work for KVM. > > If CreateCommand and DestroyCommand were called at the appropriate times > by the storage framework, I could move my connect and disconnect logic out > of the attach/detach logic. > > > On Sun, Sep 22, 2013 at 12:08 AM, Mike Tutkowski < > mike.tutkow...@solidfire.com> wrote: > >> Conversely, if the storage framework called the DestroyCommand for >> managed storage after the DetachCommand, then I could have had my remove >> SR/datastore logic placed in the DestroyCommand handling rather than in the >> DetachCommand handling. >> >> >> On Sun, Sep 22, 2013 at 12:06 AM, Mike Tutkowski < >> mike.tutkow...@solidfire.com> wrote: >> >>> Edison's plug-in calls the CreateCommand. Mine does not. >>> >>> The initial approach that was discussed during 4.2 was for me to modify >>> the attach/detach logic only in the XenServer and VMware hypervisor >>> plug-ins. >>> >>> Now that I think about it more, though, I kind of would have liked to >>> have the storage framework send a CreateCommand to the hypervisor before >>> sending the AttachCommand if the storage in question was managed. >>> >>> Then I could have created my SR/datastore in the CreateCommand and the >>> AttachCommand would have had the SR/datastore that it was always expecting >>> (and I wouldn't have had to create the SR/datastore in the AttachCommand). >>> >>> >>> On Sat, Sep 21, 2013 at 11:56 PM, Marcus Sorensen >>> <shadow...@gmail.com>wrote: >>> >>>> Yeah, I think it probably is as well, but I figured you'd be in a >>>> better position to tell. >>>> >>>> I see that copyAsync is unsupported in your current 4.2 driver, does >>>> that mean that there's no template support? Or is it some other call >>>> that does templating now? I'm still getting up to speed on all of the >>>> 4.2 changes. I was just looking at CreateCommand in >>>> LibvirtComputingResource, since that's the only place >>>> createPhysicalDisk is called, and it occurred to me that CreateCommand >>>> might be skipped altogether when utilizing storage plugins. >>>> >>>> On Sat, Sep 21, 2013 at 11:38 PM, Mike Tutkowski >>>> <mike.tutkow...@solidfire.com> wrote: >>>> > That's an interesting comment, Marcus. >>>> > >>>> > It was my intent that it should work with any CloudStack "managed" >>>> storage >>>> > that uses an iSCSI target. Even though I'm using CHAP, I wrote the >>>> code so >>>> > CHAP didn't have to be used. >>>> > >>>> > As I'm doing my testing, I can try to think about whether it is >>>> generic >>>> > enough to keep those names or not. >>>> > >>>> > My expectation is that it is generic enough. >>>> > >>>> > >>>> > On Sat, Sep 21, 2013 at 11:32 PM, Marcus Sorensen < >>>> shadow...@gmail.com>wrote: >>>> > >>>> >> I added a comment to your diff. In general I think it looks good, >>>> >> though I obviously can't vouch for whether or not it will work. One >>>> >> thing I do have reservations about is the adaptor/pool naming. If you >>>> >> think the code is generic enough that it will work for anyone who >>>> does >>>> >> an iscsi LUN-per-volume plugin, then it's OK, but if there's anything >>>> >> about it that's specific to YOUR iscsi target or how it likes to be >>>> >> treated then I'd say that they should be named something less generic >>>> >> than iScsiAdmStorage. >>>> >> >>>> >> On Sat, Sep 21, 2013 at 7:23 PM, Mike Tutkowski >>>> >> <mike.tutkow...@solidfire.com> wrote: >>>> >> > Great - thanks! >>>> >> > >>>> >> > Just to give you an overview of what my code does (for when you >>>> get a >>>> >> > chance to review it): >>>> >> > >>>> >> > SolidFireHostListener is registered in >>>> SolidfirePrimaryDataStoreProvider. >>>> >> > Its hostConnect method is invoked when a host connects with the CS >>>> MS. If >>>> >> > the host is running KVM, the listener sends a >>>> ModifyStoragePoolCommand to >>>> >> > the host. This logic was based off of DefaultHostListener. >>>> >> > >>>> >> > The handling of ModifyStoragePoolCommand is unchanged. It invokes >>>> >> > createStoragePool on the KVMStoragePoolManager. The >>>> KVMStoragePoolManager >>>> >> > asks for an adaptor and finds my new one: iScsiAdmStorageAdaptor >>>> (which >>>> >> was >>>> >> > registered in the constructor for KVMStoragePoolManager under the >>>> key of >>>> >> > StoragePoolType.Iscsi.toString()). >>>> >> > >>>> >> > iScsiAdmStorageAdaptor.createStoragePool just makes an instance of >>>> >> > iScsiAdmStoragePool, adds it to a map, and returns the pointer to >>>> the >>>> >> > iScsiAdmStoragePool object. The key of the map is the UUID of the >>>> storage >>>> >> > pool. >>>> >> > >>>> >> > When a volume is attached, createPhysicalDisk is invoked for >>>> managed >>>> >> > storage rather than getPhysicalDisk. createPhysicalDisk uses >>>> iscsiadm to >>>> >> > establish the iSCSI connection to the volume on the SAN and a >>>> >> > KVMPhysicalDisk is returned to be used in the attach logic that >>>> follows. >>>> >> > >>>> >> > When a volume is detached, getPhysicalDisk is invoked with the IQN >>>> of the >>>> >> > volume if the storage pool in question is managed storage. >>>> Otherwise, the >>>> >> > normal vol.getPath() is used. >>>> iScsiAdmStorageAdaptor.getPhysicalDisk just >>>> >> > returns a new instance of KVMPhysicalDisk to be used in the detach >>>> logic. >>>> >> > >>>> >> > Once the volume has been detached, >>>> iScsiAdmStoragePool.deletePhysicalDisk >>>> >> > is invoked if the storage pool is managed. deletePhysicalDisk >>>> removes the >>>> >> > iSCSI connection to the volume using iscsiadm. >>>> >> > >>>> >> > >>>> >> > On Sat, Sep 21, 2013 at 5:46 PM, Marcus Sorensen < >>>> shadow...@gmail.com >>>> >> >wrote: >>>> >> > >>>> >> >> Its the log4j properties file in /etc/cloudstack/agent change all >>>> INFO >>>> >> to >>>> >> >> DEBUG. I imagine the agent just isn't starting, you can tail the >>>> log >>>> >> when >>>> >> >> you try to start the service, or maybe it will spit something out >>>> into >>>> >> one >>>> >> >> of the other files in /var/log/cloudstack/agent >>>> >> >> On Sep 21, 2013 5:19 PM, "Mike Tutkowski" < >>>> mike.tutkow...@solidfire.com >>>> >> > >>>> >> >> wrote: >>>> >> >> >>>> >> >> > This is how I've been trying to query for the status of the >>>> service (I >>>> >> >> > assume it could be started this way, as well, by changing >>>> "status" to >>>> >> >> > "start" or "restart"?): >>>> >> >> > >>>> >> >> > mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo /usr/sbin/service >>>> >> >> > cloudstack-agent status >>>> >> >> > >>>> >> >> > I get this back: >>>> >> >> > >>>> >> >> > Failed to execute: * could not access PID file for >>>> cloudstack-agent >>>> >> >> > >>>> >> >> > I've made a bunch of code changes recently, though, so I think >>>> I'm >>>> >> going >>>> >> >> to >>>> >> >> > rebuild and redeploy everything. >>>> >> >> > >>>> >> >> > The debug info sounds helpful. Where can I set enable.debug? >>>> >> >> > >>>> >> >> > Thanks, Marcus! >>>> >> >> > >>>> >> >> > >>>> >> >> > On Sat, Sep 21, 2013 at 4:11 PM, Marcus Sorensen < >>>> shadow...@gmail.com >>>> >> >> > >wrote: >>>> >> >> > >>>> >> >> > > OK, will check it out in the next few days. As mentioned, you >>>> can >>>> >> set >>>> >> >> up >>>> >> >> > > your Ubuntu vm as the management server as well if all else >>>> fails. >>>> >> If >>>> >> >> > you >>>> >> >> > > can get to the mgmt server on 8250 from the KVM host, then >>>> you need >>>> >> to >>>> >> >> > > enable.debug on the agent. It won't run without complaining >>>> loudly >>>> >> if >>>> >> >> it >>>> >> >> > > can't get to the mgmt server, and I didn't see that in your >>>> agent >>>> >> log, >>>> >> >> so >>>> >> >> > > perhaps its not running. I assume you know how to stop/start >>>> the >>>> >> agent >>>> >> >> on >>>> >> >> > > KVM via 'service cloud stacks agent'. >>>> >> >> > > On Sep 21, 2013 3:02 PM, "Mike Tutkowski" < >>>> >> >> mike.tutkow...@solidfire.com> >>>> >> >> > > wrote: >>>> >> >> > > >>>> >> >> > > > Hey Marcus, >>>> >> >> > > > >>>> >> >> > > > I haven't yet been able to test my new code, but I thought >>>> you >>>> >> would >>>> >> >> > be a >>>> >> >> > > > good person to ask to review it: >>>> >> >> > > > >>>> >> >> > > > >>>> >> >> > > > >>>> >> >> > > >>>> >> >> > >>>> >> >> >>>> >> >>>> https://github.com/mike-tutkowski/incubator-cloudstack/commit/ea74b312a8a36801994500407fd54f0cdda55e37 >>>> >> >> > > > >>>> >> >> > > > All it is supposed to do is attach and detach a data disk >>>> (that >>>> >> has >>>> >> >> > > > guaranteed IOPS) with KVM as the hypervisor. The data disk >>>> >> happens to >>>> >> >> > be >>>> >> >> > > > from SolidFire-backed storage - where we have a 1:1 mapping >>>> >> between a >>>> >> >> > > > CloudStack volume and a data disk. >>>> >> >> > > > >>>> >> >> > > > There is no support for hypervisor snapshots or stuff like >>>> that >>>> >> >> > (likely a >>>> >> >> > > > future release)...just attaching and detaching a data disk >>>> in 4.3. >>>> >> >> > > > >>>> >> >> > > > Thanks! >>>> >> >> > > > >>>> >> >> > > > >>>> >> >> > > > On Fri, Sep 20, 2013 at 11:05 PM, Mike Tutkowski < >>>> >> >> > > > mike.tutkow...@solidfire.com> wrote: >>>> >> >> > > > >>>> >> >> > > > > When I re-deployed the DEBs, I didn't remove >>>> cloudstack-agent >>>> >> >> first. >>>> >> >> > > > Would >>>> >> >> > > > > that be a problem? I just did a sudo apt-get install >>>> >> >> > cloudstack-agent. >>>> >> >> > > > > >>>> >> >> > > > > >>>> >> >> > > > > On Fri, Sep 20, 2013 at 11:03 PM, Mike Tutkowski < >>>> >> >> > > > > mike.tutkow...@solidfire.com> wrote: >>>> >> >> > > > > >>>> >> >> > > > >> I get the same error running the command manually: >>>> >> >> > > > >> >>>> >> >> > > > >> mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo >>>> >> /usr/sbin/service >>>> >> >> > > > >> cloudstack-agent status >>>> >> >> > > > >> * could not access PID file for cloudstack-agent >>>> >> >> > > > >> >>>> >> >> > > > >> >>>> >> >> > > > >> On Fri, Sep 20, 2013 at 11:00 PM, Mike Tutkowski < >>>> >> >> > > > >> mike.tutkow...@solidfire.com> wrote: >>>> >> >> > > > >> >>>> >> >> > > > >>> agent.log looks OK to me: >>>> >> >> > > > >>> >>>> >> >> > > > >>> 2013-09-20 19:35:39,010 INFO [cloud.agent.AgentShell] >>>> >> >> (main:null) >>>> >> >> > > > Agent >>>> >> >> > > > >>> started >>>> >> >> > > > >>> 2013-09-20 19:35:39,011 INFO [cloud.agent.AgentShell] >>>> >> >> (main:null) >>>> >> >> > > > >>> Implementation Version is 4.3.0-SNAPSHOT >>>> >> >> > > > >>> 2013-09-20 19:35:39,015 INFO [cloud.agent.AgentShell] >>>> >> >> (main:null) >>>> >> >> > > > >>> agent.properties found at >>>> >> /etc/cloudstack/agent/agent.properties >>>> >> >> > > > >>> 2013-09-20 19:35:39,023 INFO [cloud.agent.AgentShell] >>>> >> >> (main:null) >>>> >> >> > > > >>> Defaulting to using properties file for storage >>>> >> >> > > > >>> 2013-09-20 19:35:39,024 INFO [cloud.agent.AgentShell] >>>> >> >> (main:null) >>>> >> >> > > > >>> Defaulting to the constant time backoff algorithm >>>> >> >> > > > >>> 2013-09-20 19:35:39,029 INFO [cloud.utils.LogUtils] >>>> >> (main:null) >>>> >> >> > > log4j >>>> >> >> > > > >>> configuration found at >>>> /etc/cloudstack/agent/log4j-cloud.xml >>>> >> >> > > > >>> 2013-09-20 19:35:39,163 INFO [cloud.agent.Agent] >>>> (main:null) >>>> >> id >>>> >> >> > is 3 >>>> >> >> > > > >>> 2013-09-20 19:35:39,197 INFO >>>> >> >> > > > >>> [resource.virtualnetwork.VirtualRoutingResource] >>>> (main:null) >>>> >> >> > > > >>> VirtualRoutingResource _scriptDir to use: >>>> >> >> scripts/network/domr/kvm >>>> >> >> > > > >>> >>>> >> >> > > > >>> However, I wasn't aware that setup.log was important. >>>> This >>>> >> seems >>>> >> >> to >>>> >> >> > > be >>>> >> >> > > > a >>>> >> >> > > > >>> problem, but I'm not sure what it might indicate: >>>> >> >> > > > >>> >>>> >> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service >>>> cloudstack-agent >>>> >> status >>>> >> >> > > > >>> DEBUG:root:Failed to execute: * could not access PID >>>> file for >>>> >> >> > > > >>> cloudstack-agent >>>> >> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service >>>> cloudstack-agent >>>> >> start >>>> >> >> > > > >>> >>>> >> >> > > > >>> >>>> >> >> > > > >>> On Fri, Sep 20, 2013 at 10:53 PM, Marcus Sorensen < >>>> >> >> > > shadow...@gmail.com >>>> >> >> > > > >wrote: >>>> >> >> > > > >>> >>>> >> >> > > > >>>> Sorry, I saw that in the log, I thought it was the >>>> agent log >>>> >> for >>>> >> >> > > some >>>> >> >> > > > >>>> reason. Is the agent started? That might be the place >>>> to >>>> >> look. >>>> >> >> > There >>>> >> >> > > > is >>>> >> >> > > > >>>> an >>>> >> >> > > > >>>> agent log for the agent and one for the setup when it >>>> adds >>>> >> the >>>> >> >> > host, >>>> >> >> > > > >>>> both >>>> >> >> > > > >>>> in /var/log >>>> >> >> > > > >>>> On Sep 20, 2013 10:42 PM, "Mike Tutkowski" < >>>> >> >> > > > >>>> mike.tutkow...@solidfire.com> >>>> >> >> > > > >>>> wrote: >>>> >> >> > > > >>>> >>>> >> >> > > > >>>> > Is it saying that the MS is at the IP address or the >>>> KVM >>>> >> host? >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > The KVM host is at 192.168.233.10. >>>> >> >> > > > >>>> > The MS host is at 192.168.233.1. >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > I see this for my host Global Settings parameter: >>>> >> >> > > > >>>> > hostThe ip address of management server192.168.233.1 >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > /etc/cloudstack/agent/agent.properties has a >>>> >> >> host=192.168.233.1 >>>> >> >> > > > value. >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > On Fri, Sep 20, 2013 at 10:21 PM, Marcus Sorensen < >>>> >> >> > > > >>>> shadow...@gmail.com >>>> >> >> > > > >>>> > >wrote: >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > > The log says your mgmt server is 192.168.233.10? >>>> But you >>>> >> >> tried >>>> >> >> > > to >>>> >> >> > > > >>>> telnet >>>> >> >> > > > >>>> > to >>>> >> >> > > > >>>> > > 192.168.233.1? It might be enough to change that in >>>> >> >> > > > >>>> > > /etc/cloudstack/agent/agent.properties, but you >>>> may want >>>> >> to >>>> >> >> > edit >>>> >> >> > > > the >>>> >> >> > > > >>>> > config >>>> >> >> > > > >>>> > > as well to tell it the real ms IP. >>>> >> >> > > > >>>> > > On Sep 20, 2013 10:12 PM, "Mike Tutkowski" < >>>> >> >> > > > >>>> mike.tutkow...@solidfire.com >>>> >> >> > > > >>>> > > >>>> >> >> > > > >>>> > > wrote: >>>> >> >> > > > >>>> > > >>>> >> >> > > > >>>> > > > Here's what my /etc/network/interfaces file looks >>>> >> like, if >>>> >> >> > > that >>>> >> >> > > > >>>> is of >>>> >> >> > > > >>>> > > > interest (the 192.168.233.0 network is the NAT >>>> network >>>> >> >> > VMware >>>> >> >> > > > >>>> Fusion >>>> >> >> > > > >>>> > set >>>> >> >> > > > >>>> > > > up): >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > auto lo >>>> >> >> > > > >>>> > > > iface lo inet loopback >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > auto eth0 >>>> >> >> > > > >>>> > > > iface eth0 inet manual >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > auto cloudbr0 >>>> >> >> > > > >>>> > > > iface cloudbr0 inet static >>>> >> >> > > > >>>> > > > address 192.168.233.10 >>>> >> >> > > > >>>> > > > netmask 255.255.255.0 >>>> >> >> > > > >>>> > > > network 192.168.233.0 >>>> >> >> > > > >>>> > > > broadcast 192.168.233.255 >>>> >> >> > > > >>>> > > > dns-nameservers 8.8.8.8 >>>> >> >> > > > >>>> > > > bridge_ports eth0 >>>> >> >> > > > >>>> > > > bridge_fd 5 >>>> >> >> > > > >>>> > > > bridge_stp off >>>> >> >> > > > >>>> > > > bridge_maxwait 1 >>>> >> >> > > > >>>> > > > post-up route add default gw 192.168.233.2 >>>> metric 1 >>>> >> >> > > > >>>> > > > pre-down route del default gw 192.168.233.2 >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > On Fri, Sep 20, 2013 at 10:08 PM, Mike Tutkowski >>>> < >>>> >> >> > > > >>>> > > > mike.tutkow...@solidfire.com> wrote: >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > > You appear to be correct. This is from the MS >>>> log >>>> >> >> (below). >>>> >> >> > > > >>>> Discovery >>>> >> >> > > > >>>> > > > timed >>>> >> >> > > > >>>> > > > > out. >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > > I'm not sure why this would be. My network >>>> settings >>>> >> >> > > shouldn't >>>> >> >> > > > >>>> have >>>> >> >> > > > >>>> > > > changed >>>> >> >> > > > >>>> > > > > since the last time I tried this. >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > > I am able to ping the KVM host from the MS >>>> host and >>>> >> vice >>>> >> >> > > > versa. >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > > I'm even able to manually kick off a VM on the >>>> KVM >>>> >> host >>>> >> >> > and >>>> >> >> > > > >>>> ping from >>>> >> >> > > > >>>> > > it >>>> >> >> > > > >>>> > > > > to the MS host. >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > > I am using NAT from my Mac OS X host (also >>>> running >>>> >> the >>>> >> >> CS >>>> >> >> > > MS) >>>> >> >> > > > >>>> to the >>>> >> >> > > > >>>> > VM >>>> >> >> > > > >>>> > > > > running KVM (VMware Fusion). >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,141 DEBUG >>>> >> >> > > > >>>> [c.c.h.k.d.LibvirtServerDiscoverer] >>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48) >>>> Timeout, >>>> >> to >>>> >> >> > wait >>>> >> >> > > > for >>>> >> >> > > > >>>> the >>>> >> >> > > > >>>> > > host >>>> >> >> > > > >>>> > > > > connecting to mgt svr, assuming it is failed >>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,144 WARN >>>> >> >> [c.c.r.ResourceManagerImpl] >>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48) >>>> Unable to >>>> >> >> find >>>> >> >> > > the >>>> >> >> > > > >>>> server >>>> >> >> > > > >>>> > > > > resources at http://192.168.233.10 >>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,145 INFO >>>> >> >> > [c.c.u.e.CSExceptionErrorCode] >>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48) >>>> Could not >>>> >> >> find >>>> >> >> > > > >>>> exception: >>>> >> >> > > > >>>> > > > > com.cloud.exception.DiscoveryException in >>>> error code >>>> >> >> list >>>> >> >> > > for >>>> >> >> > > > >>>> > > exceptions >>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,147 WARN >>>> >> [o.a.c.a.c.a.h.AddHostCmd] >>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48) >>>> Exception: >>>> >> >> > > > >>>> > > > > com.cloud.exception.DiscoveryException: Unable >>>> to add >>>> >> >> the >>>> >> >> > > host >>>> >> >> > > > >>>> > > > > at >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> >>>> >> >> > > > >>>> >> >> > > >>>> >> >> > >>>> >> >> >>>> >> >>>> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:778) >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > > I do seem to be able to telnet in from my KVM >>>> host to >>>> >> >> the >>>> >> >> > MS >>>> >> >> > > > >>>> host's >>>> >> >> > > > >>>> > > 8250 >>>> >> >> > > > >>>> > > > > port: >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > > mtutkowski@ubuntu:~$ telnet 192.168.233.1 8250 >>>> >> >> > > > >>>> > > > > Trying 192.168.233.1... >>>> >> >> > > > >>>> > > > > Connected to 192.168.233.1. >>>> >> >> > > > >>>> > > > > Escape character is '^]'. >>>> >> >> > > > >>>> > > > > >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > > -- >>>> >> >> > > > >>>> > > > *Mike Tutkowski* >>>> >> >> > > > >>>> > > > *Senior CloudStack Developer, SolidFire Inc.* >>>> >> >> > > > >>>> > > > e: mike.tutkow...@solidfire.com >>>> >> >> > > > >>>> > > > o: 303.746.7302 >>>> >> >> > > > >>>> > > > Advancing the way the world uses the >>>> >> >> > > > >>>> > > > cloud< >>>> >> http://solidfire.com/solution/overview/?video=play> >>>> >> >> > > > >>>> > > > *™* >>>> >> >> > > > >>>> > > > >>>> >> >> > > > >>>> > > >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> > -- >>>> >> >> > > > >>>> > *Mike Tutkowski* >>>> >> >> > > > >>>> > *Senior CloudStack Developer, SolidFire Inc.* >>>> >> >> > > > >>>> > e: mike.tutkow...@solidfire.com >>>> >> >> > > > >>>> > o: 303.746.7302 >>>> >> >> > > > >>>> > Advancing the way the world uses the >>>> >> >> > > > >>>> > cloud< >>>> http://solidfire.com/solution/overview/?video=play> >>>> >> >> > > > >>>> > *™* >>>> >> >> > > > >>>> > >>>> >> >> > > > >>>> >>>> >> >> > > > >>> >>>> >> >> > > > >>> >>>> >> >> > > > >>> >>>> >> >> > > > >>> -- >>>> >> >> > > > >>> *Mike Tutkowski* >>>> >> >> > > > >>> *Senior CloudStack Developer, SolidFire Inc.* >>>> >> >> > > > >>> e: mike.tutkow...@solidfire.com >>>> >> >> > > > >>> o: 303.746.7302 >>>> >> >> > > > >>> Advancing the way the world uses the cloud< >>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> >>>> >> >> > > > >>> *™* >>>> >> >> > > > >>> >>>> >> >> > > > >> >>>> >> >> > > > >> >>>> >> >> > > > >> >>>> >> >> > > > >> -- >>>> >> >> > > > >> *Mike Tutkowski* >>>> >> >> > > > >> *Senior CloudStack Developer, SolidFire Inc.* >>>> >> >> > > > >> e: mike.tutkow...@solidfire.com >>>> >> >> > > > >> o: 303.746.7302 >>>> >> >> > > > >> Advancing the way the world uses the cloud< >>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> >>>> >> >> > > > >> *™* >>>> >> >> > > > >> >>>> >> >> > > > > >>>> >> >> > > > > >>>> >> >> > > > > >>>> >> >> > > > > -- >>>> >> >> > > > > *Mike Tutkowski* >>>> >> >> > > > > *Senior CloudStack Developer, SolidFire Inc.* >>>> >> >> > > > > e: mike.tutkow...@solidfire.com >>>> >> >> > > > > o: 303.746.7302 >>>> >> >> > > > > Advancing the way the world uses the cloud< >>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> >>>> >> >> > > > > *™* >>>> >> >> > > > > >>>> >> >> > > > >>>> >> >> > > > >>>> >> >> > > > >>>> >> >> > > > -- >>>> >> >> > > > *Mike Tutkowski* >>>> >> >> > > > *Senior CloudStack Developer, SolidFire Inc.* >>>> >> >> > > > e: mike.tutkow...@solidfire.com >>>> >> >> > > > o: 303.746.7302 >>>> >> >> > > > Advancing the way the world uses the >>>> >> >> > > > cloud<http://solidfire.com/solution/overview/?video=play> >>>> >> >> > > > *™* >>>> >> >> > > > >>>> >> >> > > >>>> >> >> > >>>> >> >> > >>>> >> >> > >>>> >> >> > -- >>>> >> >> > *Mike Tutkowski* >>>> >> >> > *Senior CloudStack Developer, SolidFire Inc.* >>>> >> >> > e: mike.tutkow...@solidfire.com >>>> >> >> > o: 303.746.7302 >>>> >> >> > Advancing the way the world uses the >>>> >> >> > cloud<http://solidfire.com/solution/overview/?video=play> >>>> >> >> > *™* >>>> >> >> > >>>> >> >> >>>> >> > >>>> >> > >>>> >> > >>>> >> > -- >>>> >> > *Mike Tutkowski* >>>> >> > *Senior CloudStack Developer, SolidFire Inc.* >>>> >> > e: mike.tutkow...@solidfire.com >>>> >> > o: 303.746.7302 >>>> >> > Advancing the way the world uses the >>>> >> > cloud<http://solidfire.com/solution/overview/?video=play> >>>> >> > *™* >>>> >> >>>> > >>>> > >>>> > >>>> > -- >>>> > *Mike Tutkowski* >>>> > *Senior CloudStack Developer, SolidFire Inc.* >>>> > e: mike.tutkow...@solidfire.com >>>> > o: 303.746.7302 >>>> > Advancing the way the world uses the >>>> > cloud<http://solidfire.com/solution/overview/?video=play> >>>> > *™* >>>> >>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkow...@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the >>> cloud<http://solidfire.com/solution/overview/?video=play> >>> *™* >>> >> >> >> >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkow...@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the >> cloud<http://solidfire.com/solution/overview/?video=play> >> *™* >> > > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *™* > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *™*
