Mike, Can you explain why the the storage driver is hypervisor specific?
Thanks, -John On Jun 3, 2013, at 1:21 PM, Mike Tutkowski <mike.tutkow...@solidfire.com> wrote: > Yes, ultimately I would like to support all hypervisors that CloudStack > supports. I think I'm just out of time for 4.2 to get KVM in. > > Right now this plug-in supports XenServer. Depending on what we do with > regards to 4.2 feature freeze, I have it working for VMware in my sandbox, > as well. > > Also, just to be clear, this is all in regards to Disk Offerings. I plan to > support Compute Offerings post 4.2. > > > On Mon, Jun 3, 2013 at 11:14 AM, Kelcey Jamison Damage <kel...@bbits.ca>wrote: > >> Is there any plan on supporting KVM in the patch cycle post 4.2? >> >> ----- Original Message ----- >> From: "Mike Tutkowski" <mike.tutkow...@solidfire.com> >> To: dev@cloudstack.apache.org >> Sent: Monday, June 3, 2013 10:12:32 AM >> Subject: Re: [MERGE] disk_io_throttling to MASTER >> >> I agree on merging Wei's feature first, then mine. >> >> If his feature is for KVM only, then it is a non issue as I don't support >> KVM in 4.2. >> >> >> On Mon, Jun 3, 2013 at 8:55 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote: >> >>> John, >>> >>> For the billing, as no one works on billing now, users need to calculate >>> the billing by themselves. They can get the service_offering and >>> disk_offering of a VMs and volumes for calculation. Of course it is >> better >>> to tell user the exact limitation value of individual volume, and network >>> rate limitation for nics as well. I can work on it later. Do you think it >>> is a part of I/O throttling? >>> >>> Sorry my misunstand the second the question. >>> >>> Agree with what you said about the two features. >>> >>> -Wei >>> >>> >>> 2013/6/3 John Burwell <jburw...@basho.com> >>> >>>> Wei, >>>> >>>> >>>> On Jun 3, 2013, at 2:13 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote: >>>> >>>>> Hi John, Mike >>>>> >>>>> I hope Mike's aswer helps you. I am trying to adding more. >>>>> >>>>> (1) I think billing should depend on IO statistics rather than IOPS >>>>> limitation. Please review disk_io_stat if you have time. >> disk_io_stat >>>> can >>>>> get the IO statistics including bytes/iops read/write for an >> individual >>>>> virtual machine. >>>> >>>> Going by the AWS model, customers are billed more for volumes with >>>> provisioned IOPS, as well as, for those operations ( >>>> http://aws.amazon.com/ebs/). I would imagine our users would like the >>>> option to employ similar cost models. Could an operator implement >> such a >>>> billing model in the current patch? >>>> >>>>> >>>>> (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS >>>>> limitation for a running virtual machine through command line. >> However, >>>>> CloudStack does not support changing the parameters of a created >>> offering >>>>> (computer offering or disk offering). >>>> >>>> I meant at the Java interface level. I apologize for being unclear. >> Can >>>> we more generalize allocation algorithms with a set of interfaces that >>>> describe the service guarantees provided by a resource? >>>> >>>>> >>>>> (3) It is a good question. Maybe it is better to commit Mike's patch >>>> after >>>>> disk_io_throttling as Mike needs to consider the limitation in >>> hypervisor >>>>> type, I think. >>>> >>>> I will expand on my thoughts in a later response to Mike regarding the >>>> touch points between these two features. I think that >> disk_io_throttling >>>> will need to be merged before SolidFire, but I think we need closer >>>> coordination between the branches (possibly have have solidfire track >>>> disk_io_throttling) to coordinate on this issue. >>>> >>>>> >>>>> - Wei >>>>> >>>>> >>>>> 2013/6/3 John Burwell <jburw...@basho.com> >>>>> >>>>>> Mike, >>>>>> >>>>>> The things I want to understand are the following: >>>>>> >>>>>> 1) Is there value in capturing IOPS policies be captured in a >> common >>>>>> data model (e.g. for billing/usage purposes, expressing offerings). >>>>>> 2) Should there be a common interface model for reasoning about >> IOP >>>>>> provisioning at runtime? >>>>>> 3) How are conflicting provisioned IOPS configurations between a >>>>>> hypervisor and storage device reconciled? In particular, a scenario >>>> where >>>>>> is lead to believe (and billed) for more IOPS configured for a VM >>> than a >>>>>> storage device has been configured to deliver. Another scenario >>> could a >>>>>> consistent configuration between a VM and a storage device at >> creation >>>>>> time, but a later modification to storage device introduces logical >>>>>> inconsistency. >>>>>> >>>>>> Thanks, >>>>>> -John >>>>>> >>>>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski < >>>> mike.tutkow...@solidfire.com> >>>>>> wrote: >>>>>> >>>>>> Hi John, >>>>>> >>>>>> I believe Wei's feature deals with controlling the max number of >> IOPS >>>> from >>>>>> the hypervisor side. >>>>>> >>>>>> My feature is focused on controlling IOPS from the storage system >>> side. >>>>>> >>>>>> I hope that helps. :) >>>>>> >>>>>> >>>>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jburw...@basho.com> >>>> wrote: >>>>>> >>>>>>> Wei, >>>>>>> >>>>>>> My opinion is that no features should be merged until all >> functional >>>>>>> issues have been resolved and it is ready to turn over to test. >>> Until >>>>>> the >>>>>>> total Ops vs discrete read/write ops issue is addressed and >>> re-reviewed >>>>>> by >>>>>>> Wido, I don't think this criteria has been satisfied. >>>>>>> >>>>>>> Also, how does this work intersect/compliment the SolidFire patch ( >>>>>>> https://reviews.apache.org/r/11479/)? As I understand it that >> work >>> is >>>>>>> also involves provisioned IOPS. I would like to ensure we don't >>> have a >>>>>>> scenario where provisioned IOPS in KVM and SolidFire are >>> unnecessarily >>>>>>> incompatible. >>>>>>> >>>>>>> Thanks, >>>>>>> -John >>>>>>> >>>>>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <ustcweiz...@gmail.com> >> wrote: >>>>>>> >>>>>>> Wido, >>>>>>> >>>>>>> >>>>>>> Sure. I will change it next week. >>>>>>> >>>>>>> >>>>>>> -Wei >>>>>>> >>>>>>> >>>>>>> >>>>>>> 2013/6/1 Wido den Hollander <w...@widodh.nl> >>>>>>> >>>>>>> >>>>>>> Hi Wei, >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote: >>>>>>> >>>>>>> >>>>>>> Wido, >>>>>>> >>>>>>> >>>>>>> Exactly. I have pushed the features into master. >>>>>>> >>>>>>> >>>>>>> If anyone object thems for technical reason till Monday, I will >>> revert >>>>>>> >>>>>>> them. >>>>>>> >>>>>>> >>>>>>> For the sake of clarity I just want to mention again that we should >>>>>> change >>>>>>> >>>>>>> the total IOps to R/W IOps asap so that we never release a version >>> with >>>>>>> >>>>>>> only total IOps. >>>>>>> >>>>>>> >>>>>>> You laid the groundwork for the I/O throttling and that's great! We >>>>>> should >>>>>>> >>>>>>> however prevent that we create legacy from day #1. >>>>>>> >>>>>>> >>>>>>> Wido >>>>>>> >>>>>>> >>>>>>> -Wei >>>>>>> >>>>>>> >>>>>>> >>>>>>> 2013/5/31 Wido den Hollander <w...@widodh.nl> >>>>>>> >>>>>>> >>>>>>> On 05/31/2013 03:59 PM, John Burwell wrote: >>>>>>> >>>>>>> >>>>>>> Wido, >>>>>>> >>>>>>> >>>>>>> +1 -- this enhancement must to discretely support read and write >>> IOPS. >>>>>>> >>>>>>> I >>>>>>> >>>>>>> don't see how it could be fixed later because I don't see how we >>>>>>> >>>>>>> correctly >>>>>>> >>>>>>> split total IOPS into read and write. Therefore, we would be stuck >>>>>>> >>>>>>> with a >>>>>>> >>>>>>> total unless/until we decided to break backwards compatibility. >>>>>>> >>>>>>> >>>>>>> >>>>>>> What Wei meant was merging it into master now so that it will go in >>> the >>>>>>> >>>>>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that >>> 4.2 >>>>>>> >>>>>>> will be released with Read and Write instead of Total IOps. >>>>>>> >>>>>>> >>>>>>> This is to make the May 31st feature freeze date. But if the window >>>> moves >>>>>>> >>>>>>> (see other threads) then it won't be necessary to do that. >>>>>>> >>>>>>> >>>>>>> Wido >>>>>>> >>>>>>> >>>>>>> >>>>>>> I also completely agree that there is no association between >> network >>>>>>> >>>>>>> and >>>>>>> >>>>>>> >>>>>>> disk I/O. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> -John >>>>>>> >>>>>>> >>>>>>> On May 31, 2013, at 9:51 AM, Wido den Hollander <w...@widodh.nl> >>>> wrote: >>>>>>> >>>>>>> >>>>>>> Hi Wei, >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote: >>>>>>> >>>>>>> >>>>>>> Hi Wido, >>>>>>> >>>>>>> >>>>>>> Thanks. Good question. >>>>>>> >>>>>>> >>>>>>> I thought about at the beginning. Finally I decided to ignore the >>>>>>> >>>>>>> difference of read and write mainly because the network throttling >>> did >>>>>>> >>>>>>> not >>>>>>> >>>>>>> care the difference of sent and received bytes as well. >>>>>>> >>>>>>> That reasoning seems odd. Networking and disk I/O completely >>> different. >>>>>>> >>>>>>> >>>>>>> Disk I/O is much more expensive in most situations then network >>>>>>> >>>>>>> bandwith. >>>>>>> >>>>>>> >>>>>>> Implementing it will be some copy-paste work. It could be >>>>>>> >>>>>>> implemented in >>>>>>> >>>>>>> >>>>>>> few days. For the deadline of feature freeze, I will implement it >>>>>>> >>>>>>> after >>>>>>> >>>>>>> that , if needed. >>>>>>> >>>>>>> >>>>>>> >>>>>>> It think it's a feature we can't miss. But if it goes into the 4.2 >>>>>>> >>>>>>> window we have to make sure we don't release with only total IOps >> and >>>>>>> >>>>>>> fix >>>>>>> >>>>>>> it in 4.3, that will confuse users. >>>>>>> >>>>>>> >>>>>>> Wido >>>>>>> >>>>>>> >>>>>>> -Wei >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> 2013/5/31 Wido den Hollander <w...@widodh.nl> >>>>>>> >>>>>>> >>>>>>> Hi Wei, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 05/30/2013 06:03 PM, Wei ZHOU wrote: >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> I would like to merge disk_io_throttling branch into master. >>>>>>> >>>>>>> If nobody object, I will merge into master in 48 hours. >>>>>>> >>>>>>> The purpose is : >>>>>>> >>>>>>> >>>>>>> Virtual machines are running on the same storage device (local >>>>>>> >>>>>>> storage or >>>>>>> >>>>>>> share strage). Because of the rate limitation of device (such as >>>>>>> >>>>>>> iops), if >>>>>>> >>>>>>> one VM has large disk operation, it may affect the disk performance >>>>>>> >>>>>>> of >>>>>>> >>>>>>> other VMs running on the same storage device. >>>>>>> >>>>>>> It is neccesary to set the maximum rate and limit the disk I/O of >>>>>>> >>>>>>> VMs. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Looking at the code I see you make no difference between Read and >>>>>>> >>>>>>> Write >>>>>>> >>>>>>> IOps. >>>>>>> >>>>>>> >>>>>>> Qemu and libvirt support setting both a different rate for Read and >>>>>>> >>>>>>> Write >>>>>>> >>>>>>> IOps which could benefit a lot of users. >>>>>>> >>>>>>> >>>>>>> It's also strange, in the polling side you collect both the Read >> and >>>>>>> >>>>>>> Write >>>>>>> >>>>>>> IOps, but on the throttling side you only go for a global value. >>>>>>> >>>>>>> >>>>>>> Write IOps are usually much more expensive then Read IOps, so it >>>>>>> >>>>>>> seems >>>>>>> >>>>>>> like a valid use-case where that an admin would set a lower value >> for >>>>>>> >>>>>>> write >>>>>>> >>>>>>> IOps vs Read IOps. >>>>>>> >>>>>>> >>>>>>> Since this only supports KVM at this point I think it would be of >>>>>>> >>>>>>> great >>>>>>> >>>>>>> value to at least have the mechanism in place to support both, >>>>>>> >>>>>>> implementing >>>>>>> >>>>>>> this later would be a lot of work. >>>>>>> >>>>>>> >>>>>>> If a hypervisor doesn't support setting different values for read >> and >>>>>>> >>>>>>> write you can always sum both up and set that as the total limit. >>>>>>> >>>>>>> >>>>>>> Can you explain why you implemented it this way? >>>>>>> >>>>>>> >>>>>>> Wido >>>>>>> >>>>>>> >>>>>>> The feature includes: >>>>>>> >>>>>>> >>>>>>> >>>>>>> (1) set the maximum rate of VMs (in disk_offering, and global >>>>>>> >>>>>>> configuration) >>>>>>> >>>>>>> (2) change the maximum rate of VMs >>>>>>> >>>>>>> (3) limit the disk rate (total bps and iops) >>>>>>> >>>>>>> JIRA ticket: https://issues.apache.org/**** >>>>>>> >>>>>>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/**** >>>>>>> >>>>>>> jira/browse/CLOUDSTACK-1192< >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192> >>>>>>> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192< >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192> >>>>>>> >>>>>>> <** >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192< >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1192> >>>>>>> >>>>>>> >>>>>>> >>>>>>> FS (I will update later) : >>>>>>> >>>>>>> >>>>>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******< >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****> >>>>>>> >>>>>>> < >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**< >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**> >>>>>>> >>>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/**** >> < >>>>>>> http://cwiki.apache.org/confluence/**> >>>>>>> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.** >>>>>>> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling >> < >>>>>>> >>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Merge check list :- >>>>>>> >>>>>>> >>>>>>> * Did you check the branch's RAT execution success? >>>>>>> >>>>>>> Yes >>>>>>> >>>>>>> >>>>>>> * Are there new dependencies introduced? >>>>>>> >>>>>>> No >>>>>>> >>>>>>> >>>>>>> * What automated testing (unit and integration) is included in the >>>>>>> >>>>>>> new >>>>>>> >>>>>>> feature? >>>>>>> >>>>>>> Unit tests are added. >>>>>>> >>>>>>> >>>>>>> * What testing has been done to check for potential regressions? >>>>>>> >>>>>>> (1) set the bytes rate and IOPS rate on CloudStack UI. >>>>>>> >>>>>>> (2) VM operations, including >>>>>>> >>>>>>> deploy, stop, start, reboot, destroy, expunge. migrate, restore >>>>>>> >>>>>>> (3) Volume operations, including >>>>>>> >>>>>>> Attach, Detach >>>>>>> >>>>>>> >>>>>>> To review the code, you can try >>>>>>> >>>>>>> git diff c30057635d04a2396f84c588127d7e******be42e503a7 >>>>>>> >>>>>>> f2e5591b710d04cc86815044f5823e******73a4a58944 >>>>>>> >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> Wei >>>>>>> >>>>>>> >>>>>>> [1] >>>>>>> >>>>>>> >>>>>>> >> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******< >>>>>> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****> >>>>>>> >>>>>>> < >>>>>>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**< >>>>>> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**> >>>>>>> >>>>>>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/**** >> < >>>>>>> http://cwiki.apache.org/confluence/**> >>>>>>> >>>>>>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.** >>>>>>> >>>>>>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling >> < >>>>>>> >>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> [2] refs/heads/disk_io_throttling >>>>>>> >>>>>>> [3] >>>>>>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301< >>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301> >>>>>>> >>>>>>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301< >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301> >>>>>>> >>>>>>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301< >>>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301> >>>>>>> >>>>>>> <** >>>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301< >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-1301> >>>>>>> >>>>>>> >>>>>>> >>>>>>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071< >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071> >>>>>>> >>>>>>> **< >>>>>>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071< >>>>>> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071> >>>>>>> >>>>>>> <** >>>>>>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071< >>>>>> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071> >>>>>>> >>>>>>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071< >>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071> >>>>>>> >>>>>>> >>>>>>> >>>>>>> (**CLOUDSTACK-1301 >>>>>>> >>>>>>> - VM Disk I/O Throttling) >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Mike Tutkowski* >>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>> e: mike.tutkow...@solidfire.com >>>>>> o: 303.746.7302 >>>>>> Advancing the way the world uses the >>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>>> *™* >>>>>> >>>> >>>> >>> >> >> >> >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkow...@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the >> cloud<http://solidfire.com/solution/overview/?video=play> >> *™* >> > > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *™*
