Mike, John The Disk I/O Throttling works like:
(1) For User VM, (1.1) root disks: service offering -> default value in global configuration (1.2) data disks: disk offering -> default value in global configuration (2) For System VM root disks: service offering -Wei 2013/6/12 John Burwell <jburw...@basho.com> > Mike, > > That is one possibility. The other possibility is that hypervisor is > going to throttle I/O on all disks attached. Therefore, we need answers to > the following questions: > > 1. Is I/O throttling applied to the root disk or all disks > attached to the VM? > 2. If I/O throttling is applied to all disks, how is the > throttling distributed amongst the disks if only one read/write value is > defined? > > Thanks, > -John > > On Jun 12, 2013, at 5:13 PM, Mike Tutkowski <mike.tutkow...@solidfire.com> > wrote: > > > Hey John, > > > > Perhaps I don't fully understand how Wei's feature works. > > > > I guess I thought if you choose Hypervisor QoS, you do so on Compute > > Offerings (root disks) and/or Disk Offerings (data disks). > > > > From my thinking, you're root disk could be under Hypervisor QoS, but > your > > data disk could be under Storage QoS. > > > > Is that incorrect? > > > > Thanks > > > > > > On Wed, Jun 12, 2013 at 3:10 PM, John Burwell <jburw...@basho.com> > wrote: > > > >> Mike, > >> > >> As I understand these two patches, the throttled I?O settings are > applied > >> from the hypervisor side, and possibly defined in a compute offering > where > >> provisioned IOPS are defined on the storage side through disk > offerings. I > >> don't see how the management server could enforce this mutual exclusion > >> between provisioned IOPS and throttled I/O until a compute and disk > >> offering were selected. These selections come together when we deploy a > >> VM. Based on these assumptions, I would expect to see enforcement of > this > >> rule in UI at the time of VM creation/definition and on the server > side, as > >> part of the VM creation. It feels like any attempt to enforce this rule > >> when defining offering would be premature, and unnecessarily limit > >> flexibility. Are my assumptions and understanding correct? > >> > >> Thanks, > >> -John > >> > >> On Jun 12, 2013, at 5:04 PM, Mike Tutkowski < > mike.tutkow...@solidfire.com> > >> wrote: > >> > >>> Hi John, > >>> > >>> So, maybe I'm wrong about this, but what I was thinking is that we'd > >> build > >>> two radio buttons into the Add Disk Offering dialog (we can ignore > >> Compute > >>> Offerings for 4.2 since my feature doesn't yet support them). > >>> > >>> Let's just called these radio buttons 1) Hypervisor QoS and 2) Storage > >> QoS > >>> for the purpose of this e-mail. > >>> > >>> By default, neither radio button is selected and no QoS fields are > >> visible. > >>> > >>> If the user selects the Storage QoS radio button, then the Custom IOPS > >>> checkbox and the Min and Max text fields are made visible. > >>> > >>> If the user changes his mind and selects the Hypervisor QoS radio > button, > >>> then the Custom IOPS checkbox and the Min and Max text fields disappear > >> and > >>> are replaced by the two Hypervisor QoS text fields. > >>> > >>> This way, the user can choose neither QoS option or one of them or the > >>> other, but never both. > >>> > >>> On the API side, I was thinking of having logic in place when a request > >>> comes in to create a Disk Offering to confirm these fields are the way > we > >>> want them. > >>> > >>> Once we know the Disk Offering is in order, a user can create a data > disk > >>> from it. Since we checked the validity of the Disk Offering when it was > >>> created, the VM should never be asked to use Hypervisor QoS when > Storage > >>> QoS in being utilized. > >>> > >>> Does that make sense or did I miss something? > >>> > >>> Thanks > >>> > >>> > >>> On Wed, Jun 12, 2013 at 2:54 PM, John Burwell <jburw...@basho.com> > >> wrote: > >>> > >>>> Mike, > >>>> > >>>> Looking through the code, I am trying to understand how > >>>> CreateDiskOfferingCmd would have the context to identify the conflict. > >>>> Naively, it seems to me that this rule would need to be enforced when > a > >>>> virtual machine is being deployed. Looking through the code, it seems > >> like > >>>> we should add a private validateStorageQoS method to > >>>> com.cloud.vm.UserVmManagerImpl to check this condition and throws a > >>>> ResourceAllocationException when the QoS definitions are inconsistent. > >> We > >>>> would then add calls to it from each of the VM creation methods in the > >>>> service. Do this type of approach sound reasonable? > >>>> > >>>> Thanks, > >>>> -John > >>>> > >>>> On Jun 12, 2013, at 4:30 PM, Mike Tutkowski < > >> mike.tutkow...@solidfire.com> > >>>> wrote: > >>>> > >>>>> Hi John, > >>>>> > >>>>> So, here's what I was planning to do. Of course feel free to correct > me > >>>> on > >>>>> this approach. > >>>>> > >>>>> I think it's OK if Wei merges his code into master and then I can > draw > >>>> from > >>>>> the main repo and merge master into mine locally. > >>>>> > >>>>> 1) Once I get Wei's code and merge, I plan to add a little GUI code > to > >>>> make > >>>>> it user friendly (toggle between these features on the Add Disk > >> Offering > >>>>> window). > >>>>> > >>>>> 2) I plan to write validation logic for the create-disk-offering API > >>>>> command which throws an exception if the rules are not followed (this > >>>>> should never be triggered from the GUI since the GUI will have > controls > >>>> in > >>>>> place to toggle between the one feature and the other). > >>>>> > >>>>> I'm not sure about documentation. I haven't had much experience with > it > >>>> on > >>>>> CloudStack projects yet. > >>>>> > >>>>> Thanks! > >>>>> > >>>>> > >>>>> On Wed, Jun 12, 2013 at 2:21 PM, John Burwell <jburw...@basho.com> > >>>> wrote: > >>>>> > >>>>>> Mike, > >>>>>> > >>>>>> Yes, these server-side rails need to be defined and implemented > before > >>>>>> either patch can be merged. From my perspective, I would like to > see > >>>> the > >>>>>> rule implemented in the hypervisor as part of the validation of the > >>>> virtual > >>>>>> machine definition. We also need to make sure that this mutual > >>>> exclusion > >>>>>> is documented. Do we usually include this type of documentation > with > >>>>>> patches of this nature? > >>>>>> > >>>>>> Thanks, > >>>>>> -John > >>>>>> > >>>>>> On Jun 12, 2013, at 2:18 PM, Mike Tutkowski < > >>>> mike.tutkow...@solidfire.com> > >>>>>> wrote: > >>>>>> > >>>>>>> Currently they are not yet implemented. > >>>>>>> > >>>>>>> We have to make sure they are implemented in the GUI from a > usability > >>>>>>> standpoint, but the API must check for consistency and throw an > >>>> exception > >>>>>>> if necessary. > >>>>>>> > >>>>>>> > >>>>>>> On Wed, Jun 12, 2013 at 11:03 AM, John Burwell <jburw...@basho.com > > > >>>>>> wrote: > >>>>>>> > >>>>>>>> Mike, > >>>>>>>> > >>>>>>>> Are the checks only implemented in the UI? > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> -John > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Jun 12, 2013, at 1:02 PM, Mike Tutkowski > >>>>>>>> <mike.tutkow...@solidfire.com> wrote: > >>>>>>>> > >>>>>>>>> Hi John, > >>>>>>>>> > >>>>>>>>> Wei and I have discussed making the two features mutually > >> exclusive. > >>>> We > >>>>>>>>> agree with you that only one should be active at a time. We plan > to > >>>>>>>>> implement in the GUI a mechanism (maybe radio buttons) to turn > his > >>>>>>>> feature > >>>>>>>>> on and mine off and vice versa. > >>>>>>>>> > >>>>>>>>> I was thinking if I wait until he checks in his code, then I > update > >>>> and > >>>>>>>>> merge that I will be the person resolving merge conflicts in the > >>>>>>>> JavaScript > >>>>>>>>> code (there shouldn't be a problem in the Java code) as opposed > to > >>>>>>>> putting > >>>>>>>>> that work on someone else. > >>>>>>>>> > >>>>>>>>> Let me know what you think. > >>>>>>>>> > >>>>>>>>> Oh, was going to ask you what "FS" stands for here. > >>>>>>>>> > >>>>>>>>> Thanks! > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Wed, Jun 12, 2013 at 10:56 AM, John Burwell < > jburw...@basho.com > >>> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Mike, > >>>>>>>>>> > >>>>>>>>>> How have Wei and you resolved the issue of conflicting QoS > >>>> mechanisms > >>>>>>>>>> between the Hypervisor and Storage layers? Have the affected > FSs > >>>> been > >>>>>>>>>> updated with that decision? > >>>>>>>>>> > >>>>>>>>>> In terms of merge timing, can you describe the dependencies > >> between > >>>>>> the > >>>>>>>>>> patches? > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> -John > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Jun 12, 2013, at 12:43 PM, Mike Tutkowski > >>>>>>>>>> <mike.tutkow...@solidfire.com> wrote: > >>>>>>>>>> > >>>>>>>>>>> No problem, John. > >>>>>>>>>>> > >>>>>>>>>>> I still want to re-review it by myself before coming up with a > >> new > >>>>>>>> patch > >>>>>>>>>>> file. > >>>>>>>>>>> > >>>>>>>>>>> Also, maybe I should first wait for Wei's changes to be checked > >> in > >>>>>> and > >>>>>>>>>>> merge those into mine before generating a new patch file? > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On Wed, Jun 12, 2013 at 10:40 AM, John Burwell < > >> jburw...@basho.com > >>>>> > >>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Mike, > >>>>>>>>>>>> > >>>>>>>>>>>> I just realized that I forgot to publish my review. I am > >> offline > >>>>>> ATM, > >>>>>>>>>>>> but I will publish it in the next couple of hours. > >>>>>>>>>>>> > >>>>>>>>>>>> Do you plan to update your the patch in Review Board? > >>>>>>>>>>>> > >>>>>>>>>>>> Sorry for the oversight, > >>>>>>>>>>>> -John > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Jun 12, 2013, at 2:26 AM, Mike Tutkowski > >>>>>>>>>>>> <mike.tutkow...@solidfire.com> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hi Edison, John, and Wei (and whoever else is reading this :) > >> ), > >>>>>>>>>>>>> > >>>>>>>>>>>>> Just an FYI that I believe I have implemented all the areas > we > >>>>>> wanted > >>>>>>>>>>>>> addressed. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I plan to review the code again tomorrow morning or > afternoon, > >>>> then > >>>>>>>>>> send > >>>>>>>>>>>> in > >>>>>>>>>>>>> another patch. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks for all the work on this everyone! > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:29 PM, Mike Tutkowski < > >>>>>>>>>>>>> mike.tutkow...@solidfire.com> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Sure, that sounds good. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 12:11 PM, Wei ZHOU < > >>>> ustcweiz...@gmail.com > >>>>>>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hi Mike, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> It looks the two feature do not have many conflicts in Java > >>>> code, > >>>>>>>>>>>> except > >>>>>>>>>>>>>>> the cloudstack UI. > >>>>>>>>>>>>>>> If you do not mind, I will merge disk_io_throttling branch > >> into > >>>>>>>>>> master > >>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>> week, so that you can develop based on it. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> -Wei > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 2013/6/11 Mike Tutkowski <mike.tutkow...@solidfire.com> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hey John, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> The SolidFire patch does not depend on the object_store > >>>> branch, > >>>>>>>> but > >>>>>>>>>> - > >>>>>>>>>>>> as > >>>>>>>>>>>>>>>> Edison mentioned - it might be easier if we merge the > >>>> SolidFire > >>>>>>>>>> branch > >>>>>>>>>>>>>>> into > >>>>>>>>>>>>>>>> the object_store branch before object_store goes into > >> master. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I'm not sure how the disk_io_throttling fits into this > merge > >>>>>>>>>> strategy. > >>>>>>>>>>>>>>>> Perhaps Wei can chime in on that. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Tue, Jun 11, 2013 at 11:07 AM, John Burwell < > >>>>>>>> jburw...@basho.com> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Mike, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> We have a delicate merge dance to perform. The > >>>>>>>> disk_io_throttling, > >>>>>>>>>>>>>>>>> solidfire, and object_store appear to have a number of > >>>>>>>> overlapping > >>>>>>>>>>>>>>>>> elements. I understand the dependencies between the > >> patches > >>>> to > >>>>>>>> be > >>>>>>>>>> as > >>>>>>>>>>>>>>>>> follows: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> object_store <- solidfire -> disk_io_throttling > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Am I correct that the device management aspects of > >> SolidFire > >>>>>> are > >>>>>>>>>>>>>>> additive > >>>>>>>>>>>>>>>>> to the object_store branch or there are circular > dependency > >>>>>>>> between > >>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> branches? Once we understand the dependency graph, we > can > >>>>>>>>>> determine > >>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> best approach to land the changes in master. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>> -John > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On Jun 10, 2013, at 11:10 PM, Mike Tutkowski < > >>>>>>>>>>>>>>>> mike.tutkow...@solidfire.com> > >>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Also, if we are good with Edison merging my code into > his > >>>>>> branch > >>>>>>>>>>>>>>> before > >>>>>>>>>>>>>>>>>> going into master, I am good with that. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> We can remove the StoragePoolType.Dynamic code after his > >>>> merge > >>>>>>>> and > >>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>> can > >>>>>>>>>>>>>>>>>> deal with Burst IOPS then, as well. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 9:08 PM, Mike Tutkowski < > >>>>>>>>>>>>>>>>>> mike.tutkow...@solidfire.com> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Let me make sure I follow where we're going here: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> 1) There should be NO references to hypervisor code in > >> the > >>>>>>>>>> storage > >>>>>>>>>>>>>>>>>>> plug-ins code (this includes the default storage > plug-in, > >>>>>> which > >>>>>>>>>>>>>>>>> currently > >>>>>>>>>>>>>>>>>>> sends several commands to the hypervisor in use > (although > >>>> it > >>>>>>>> does > >>>>>>>>>>>>>>> not > >>>>>>>>>>>>>>>>> know > >>>>>>>>>>>>>>>>>>> which hypervisor (XenServer, ESX(i), etc.) is actually > in > >>>>>> use)) > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> 2) managed=true or managed=false can be placed in the > url > >>>>>> field > >>>>>>>>>> (if > >>>>>>>>>>>>>>>> not > >>>>>>>>>>>>>>>>>>> present, we default to false). This info is stored in > the > >>>>>>>>>>>>>>>>>>> storage_pool_details table. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> 3) When the "attach" command is sent to the hypervisor > in > >>>>>>>>>>>>>>> question, we > >>>>>>>>>>>>>>>>>>> pass the managed property along (this takes the place > of > >>>> the > >>>>>>>>>>>>>>>>>>> StoragePoolType.Dynamic check). > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> 4) execute(AttachVolumeCommand) in the hypervisor > checks > >>>> for > >>>>>>>> the > >>>>>>>>>>>>>>>> managed > >>>>>>>>>>>>>>>>>>> property. If true for an attach, the necessary > hypervisor > >>>>>> data > >>>>>>>>>>>>>>>>> structure is > >>>>>>>>>>>>>>>>>>> created and the rest of the attach command executes to > >>>> attach > >>>>>>>> the > >>>>>>>>>>>>>>>>> volume. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> 5) When execute(AttachVolumeCommand) is invoked to > >> detach a > >>>>>>>>>> volume, > >>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> same check is made. If managed, the hypervisor data > >>>> structure > >>>>>>>> is > >>>>>>>>>>>>>>>>> removed. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> 6) I do not see an clear way to support Burst IOPS in > 4.2 > >>>>>>>> unless > >>>>>>>>>>>>>>> it is > >>>>>>>>>>>>>>>>>>> stored in the volumes and disk_offerings table. If we > >> have > >>>>>> some > >>>>>>>>>>>>>>> idea, > >>>>>>>>>>>>>>>>>>> that'd be cool. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Thanks! > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2013 at 8:58 PM, Mike Tutkowski < > >>>>>>>>>>>>>>>>>>> mike.tutkow...@solidfire.com> wrote: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> "+1 -- Burst IOPS can be implemented while avoiding > >>>>>>>>>> implementation > >>>>>>>>>>>>>>>>>>>> attributes. I always wondered about the details > field. > >> I > >>>>>>>> think > >>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>> should > >>>>>>>>>>>>>>>>>>>> beef up the description in the documentation regarding > >> the > >>>>>>>>>>>>>>> expected > >>>>>>>>>>>>>>>>> format > >>>>>>>>>>>>>>>>>>>> of the field. In 4.1, I noticed that the details are > >> not > >>>>>>>>>>>>>>> returned on > >>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>> createStoratePool updateStoragePool, or > listStoragePool > >>>>>>>>>> response. > >>>>>>>>>>>>>>>> Why > >>>>>>>>>>>>>>>>>>>> don't we return it? It seems like it would be useful > >> for > >>>>>>>>>> clients > >>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>> able to inspect the contents of the details field." > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Not sure how this would work storing Burst IOPS here. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Burst IOPS need to be variable on a Disk > >> Offering-by-Disk > >>>>>>>>>> Offering > >>>>>>>>>>>>>>>>>>>> basis. For each Disk Offering created, you have to be > >> able > >>>>>> to > >>>>>>>>>>>>>>>> associate > >>>>>>>>>>>>>>>>>>>> unique Burst IOPS. There is a disk_offering_details > >> table. > >>>>>>>> Maybe > >>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>> could > >>>>>>>>>>>>>>>>>>>> go there? > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> I'm also not sure how you would accept the Burst IOPS > in > >>>> the > >>>>>>>> GUI > >>>>>>>>>>>>>>> if > >>>>>>>>>>>>>>>>> it's > >>>>>>>>>>>>>>>>>>>> not stored like the Min and Max fields are in the DB. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>> *Mike Tutkowski* > >>>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>>>>>>>>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>>>>>>>>>>>>>> o: 303.746.7302 > >>>>>>>>>>>>>>>>>>> Advancing the way the world uses the cloud< > >>>>>>>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play> > >>>>>>>>>>>>>>>>>>> *™* > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>> *Mike Tutkowski* > >>>>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>>>>>>>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>>>>>>>>>>>>> o: 303.746.7302 > >>>>>>>>>>>>>>>>>> Advancing the way the world uses the > >>>>>>>>>>>>>>>>>> cloud< > http://solidfire.com/solution/overview/?video=play> > >>>>>>>>>>>>>>>>>> *™* > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>> *Mike Tutkowski* > >>>>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>>>>>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>>>>>>>>>>> o: 303.746.7302 > >>>>>>>>>>>>>>>> Advancing the way the world uses the > >>>>>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >>>>>>>>>>>>>>>> *™* > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>>> *Mike Tutkowski* > >>>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>>>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>>>>>>>>> o: 303.746.7302 > >>>>>>>>>>>>>> Advancing the way the world uses the cloud< > >>>>>>>>>>>> http://solidfire.com/solution/overview/?video=play> > >>>>>>>>>>>>>> *™* > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> *Mike Tutkowski* > >>>>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>>>>>>>> o: 303.746.7302 > >>>>>>>>>>>>> Advancing the way the world uses the > >>>>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >>>>>>>>>>>>> *™* > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>> *Mike Tutkowski* > >>>>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>>>>>> o: 303.746.7302 > >>>>>>>>>>> Advancing the way the world uses the > >>>>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >>>>>>>>>>> *™* > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> *Mike Tutkowski* > >>>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>>>> o: 303.746.7302 > >>>>>>>>> Advancing the way the world uses the > >>>>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >>>>>>>>> *™* > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> *Mike Tutkowski* > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>> o: 303.746.7302 > >>>>>>> Advancing the way the world uses the > >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >>>>>>> *™* > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> *Mike Tutkowski* > >>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>> e: mike.tutkow...@solidfire.com > >>>>> o: 303.746.7302 > >>>>> Advancing the way the world uses the > >>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >>>>> *™* > >>>> > >>>> > >>> > >>> > >>> -- > >>> *Mike Tutkowski* > >>> *Senior CloudStack Developer, SolidFire Inc.* > >>> e: mike.tutkow...@solidfire.com > >>> o: 303.746.7302 > >>> Advancing the way the world uses the > >>> cloud<http://solidfire.com/solution/overview/?video=play> > >>> *™* > >> > >> > > > > > > -- > > *Mike Tutkowski* > > *Senior CloudStack Developer, SolidFire Inc.* > > e: mike.tutkow...@solidfire.com > > o: 303.746.7302 > > Advancing the way the world uses the > > cloud<http://solidfire.com/solution/overview/?video=play> > > *™* > >
