But why "policy" is being discussed on "quota" thread? On Wed, Oct 15, 2014 at 11:55 AM, Valeriy Ponomaryov < vponomar...@mirantis.com> wrote:
> Manila project does use "policy" common code from incubator. > > Our "small" wrapper for it: > https://github.com/openstack/manila/blob/8203c51081680a7a9dba30ae02d7c43d6e18a124/manila/policy.py > > > On Wed, Oct 15, 2014 at 2:41 AM, Salvatore Orlando <sorla...@nicira.com> > wrote: > >> Doug, >> >> I totally agree with your findings on the policy module. >> Neutron already has some "customizations" there and we already have a few >> contributors working on syncing it back with oslo-incubator during the Kilo >> release cycle. >> >> However, my query was about the quota module. >> From what I gather it seems not a lot of projects use it: >> >> $ find . -name openstack-common.conf | xargs grep quota >> $ >> >> Salvatore >> >> On 15 October 2014 00:34, Doug Hellmann <d...@doughellmann.com> wrote: >> >>> >>> On Oct 14, 2014, at 12:31 PM, Salvatore Orlando <sorla...@nicira.com> >>> wrote: >>> >>> Hi Doug, >>> >>> do you know if the existing quota oslo-incubator module has already some >>> active consumers? >>> In the meanwhile I've pushed a spec to neutron-specs for improving quota >>> management there [1] >>> >>> >>> It looks like a lot of projects are syncing the module: >>> >>> $ grep policy */openstack-common.conf >>> >>> barbican/openstack-common.conf:modules=gettextutils,jsonutils,log,local,timeutils,importutils,policy >>> ceilometer/openstack-common.conf:module=policy >>> cinder/openstack-common.conf:module=policy >>> designate/openstack-common.conf:module=policy >>> gantt/openstack-common.conf:module=policy >>> glance/openstack-common.conf:module=policy >>> heat/openstack-common.conf:module=policy >>> horizon/openstack-common.conf:module=policy >>> ironic/openstack-common.conf:module=policy >>> keystone/openstack-common.conf:module=policy >>> manila/openstack-common.conf:module=policy >>> neutron/openstack-common.conf:module=policy >>> nova/openstack-common.conf:module=policy >>> trove/openstack-common.conf:module=policy >>> tuskar/openstack-common.conf:module=policy >>> >>> I’m not sure how many are actively using it, but I wouldn’t expect them >>> to copy it in if they weren’t using it at all. >>> >>> >>> Now, I can either work on the oslo-incubator module and leverage it in >>> Neutron, or develop the quota module in Neutron, and move it to >>> oslo-incubator once we validate it with Neutron. The latter approach seems >>> easier from a workflow perspective - as it avoid the intermediate steps of >>> moving code from oslo-incubator to neutron. On the other hand it will delay >>> adoption in oslo-incubator. >>> >>> >>> The policy module is up for graduation this cycle. It may end up in its >>> own library, to allow us to build a review team for the code more easily >>> than if we put it in with some of the other semi-related modules like the >>> server code. We’re still working that out [1], and if you expect to make a >>> lot of incompatible changes we should delay graduation to make that simpler. >>> >>> Either way, since we have so many consumers, I think it would be easier >>> to have the work happen in Oslo somewhere so we can ensure those changes >>> are useful to and usable by all of the existing consumers. >>> >>> Doug >>> >>> [1] https://etherpad.openstack.org/p/kilo-oslo-library-proposals >>> >>> >>> What's your opinion? >>> >>> Regards, >>> Salvatore >>> >>> [1] https://review.openstack.org/#/c/128318/ >>> >>> On 8 October 2014 18:52, Doug Hellmann <d...@doughellmann.com> wrote: >>> >>>> >>>> On Oct 8, 2014, at 7:03 AM, Davanum Srinivas <dava...@gmail.com> wrote: >>>> >>>> > Salvatore, Joe, >>>> > >>>> > We do have this at the moment: >>>> > >>>> > >>>> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/quota.py >>>> > >>>> > — dims >>>> >>>> If someone wants to drive creating a useful library during kilo, please >>>> consider adding the topic to the etherpad we’re using to plan summit >>>> sessions and then come participate in the Oslo meeting this Friday 16:00 >>>> UTC. >>>> >>>> https://etherpad.openstack.org/p/kilo-oslo-summit-topics >>>> >>>> Doug >>>> >>>> > >>>> > On Wed, Oct 8, 2014 at 2:29 AM, Salvatore Orlando < >>>> sorla...@nicira.com> wrote: >>>> >> >>>> >> On 8 October 2014 04:13, Joe Gordon <joe.gord...@gmail.com> wrote: >>>> >>> >>>> >>> >>>> >>> On Fri, Oct 3, 2014 at 10:47 AM, Morgan Fainberg >>>> >>> <morgan.fainb...@gmail.com> wrote: >>>> >>>> >>>> >>>> Keeping the enforcement local (same way policy works today) helps >>>> limit >>>> >>>> the fragility, big +1 there. >>>> >>>> >>>> >>>> I also agree with Vish, we need a uniform way to talk about quota >>>> >>>> enforcement similar to how we have a uniform policy language / >>>> enforcement >>>> >>>> model (yes I know it's not perfect, but it's far closer to uniform >>>> than >>>> >>>> quota management is). >>>> >>> >>>> >>> >>>> >>> It sounds like maybe we should have an oslo library for quotas? >>>> Somewhere >>>> >>> where we can share the code,but keep the operations local to each >>>> service. >>>> >> >>>> >> >>>> >> This is what I had in mind as well. A simple library for quota >>>> enforcement >>>> >> which can be used regardless of where and how you do it, which might >>>> depend >>>> >> on the application business logic, the WSGI framework in use, or >>>> other >>>> >> factors. >>>> >> >>>> >>> >>>> >>>> >>>> >>>> >>>> >>>> If there is still interest of placing quota in keystone, let's >>>> talk about >>>> >>>> how that will work and what will be needed from Keystone . The >>>> previous >>>> >>>> attempt didn't get much traction and stalled out early in >>>> implementation. If >>>> >>>> we want to revisit this lets make sure we have the resources >>>> needed and >>>> >>>> spec(s) in progress / info on etherpads (similar to how the >>>> multitenancy >>>> >>>> stuff was handled at the last summit) as early as possible. >>>> >>> >>>> >>> >>>> >>> Why not centralize quota management via the python-openstackclient, >>>> what >>>> >>> is the benefit of getting keystone involved? >>>> >> >>>> >> >>>> >> Providing this through the openstack client in my opinion has the >>>> >> disadvantage that users which either use the REST API direct or >>>> write their >>>> >> own clients won't leverage it. I don't think it's a reasonable >>>> assumption >>>> >> that everybody will use python-openstackclient, is it? >>>> >> >>>> >> Said that, storing quotas in keystone poses a further challenge to >>>> the >>>> >> scalability of the system, which we shall perhaps address by using >>>> >> appropriate caching strategies and leveraging keystone >>>> notifications. Until >>>> >> we get that, I think that the openstack client will be the best way >>>> of >>>> >> getting a unified quota management experience. >>>> >> >>>> >> Salvatore >>>> >> >>>> >> >>>> >>>> >>>> >>>> Cheers, >>>> >>>> Morgan >>>> >>>> >>>> >>>> Sent via mobile >>>> >>>> >>>> >>>> >>>> >>>> On Friday, October 3, 2014, Salvatore Orlando <sorla...@nicira.com >>>> > >>>> >>>> wrote: >>>> >>>>> >>>> >>>>> Thanks Vish, >>>> >>>>> >>>> >>>>> this seems a very reasonable first step as well - and since most >>>> >>>>> projects would be enforcing quotas in the same way, the shared >>>> library would >>>> >>>>> be the logical next step. >>>> >>>>> After all this is quite the same thing we do with authZ. >>>> >>>>> >>>> >>>>> Duncan is expressing valid concerns which in my opinion can be >>>> addressed >>>> >>>>> with an appropriate design - and a decent implementation. >>>> >>>>> >>>> >>>>> Salvatore >>>> >>>>> >>>> >>>>> On 3 October 2014 18:25, Vishvananda Ishaya < >>>> vishvana...@gmail.com> >>>> >>>>> wrote: >>>> >>>>>> >>>> >>>>>> The proposal in the past was to keep quota enforcement local, >>>> but to >>>> >>>>>> put the resource limits into keystone. This seems like an >>>> obvious first >>>> >>>>>> step to me. Then a shared library for enforcing quotas with >>>> decent >>>> >>>>>> performance should be next. The quota calls in nova are extremely >>>> >>>>>> inefficient right now and it will only get worse when we try to >>>> add >>>> >>>>>> hierarchical projects and quotas. >>>> >>>>>> >>>> >>>>>> Vish >>>> >>>>>> >>>> >>>>>> On Oct 3, 2014, at 7:53 AM, Duncan Thomas < >>>> duncan.tho...@gmail.com> >>>> >>>>>> wrote: >>>> >>>>>> >>>> >>>>>>> Taking quota out of the service / adding remote calls for quota >>>> >>>>>>> management is going to make things fragile - you've somehow got >>>> to >>>> >>>>>>> deal with the cases where your quota manager is slow, goes away, >>>> >>>>>>> hiccups, drops connections etc. You'll also need some way of >>>> >>>>>>> reconciling actual usage against quota usage periodically, to >>>> detect >>>> >>>>>>> problems. >>>> >>>>>>> >>>> >>>>>>> On 3 October 2014 15:03, Salvatore Orlando <sorla...@nicira.com >>>> > >>>> >>>>>>> wrote: >>>> >>>>>>>> Hi, >>>> >>>>>>>> >>>> >>>>>>>> Quota management is currently one of those things where every >>>> >>>>>>>> openstack >>>> >>>>>>>> project does its own thing. While quotas are obviously managed >>>> in a >>>> >>>>>>>> similar >>>> >>>>>>>> way for each project, there are subtle differences which >>>> ultimately >>>> >>>>>>>> result >>>> >>>>>>>> in lack of usability. >>>> >>>>>>>> >>>> >>>>>>>> I recall that in the past there have been several calls for >>>> unifying >>>> >>>>>>>> quota >>>> >>>>>>>> management. The blueprint [1] for instance, hints at the >>>> possibility >>>> >>>>>>>> of >>>> >>>>>>>> storing quotas in keystone. >>>> >>>>>>>> On the other hand, the blazar project [2, 3] seems to aim at >>>> solving >>>> >>>>>>>> this >>>> >>>>>>>> problem for good enabling resource reservation and therefore >>>> >>>>>>>> potentially >>>> >>>>>>>> freeing openstack projects from managing and enforcing quotas. >>>> >>>>>>>> >>>> >>>>>>>> While Blazar is definetely a good thing to have, I'm not >>>> entirely >>>> >>>>>>>> sure we >>>> >>>>>>>> want to make it a "required" component for every deployment. >>>> Perhaps >>>> >>>>>>>> single >>>> >>>>>>>> projects should still be able to enforce quota. On the other >>>> hand, >>>> >>>>>>>> at least >>>> >>>>>>>> on paper, the idea of making Keystone "THE" endpoint for >>>> managing >>>> >>>>>>>> quotas, >>>> >>>>>>>> and then letting the various project enforce them, sounds >>>> promising >>>> >>>>>>>> - is >>>> >>>>>>>> there any reason for which this blueprint is stalled to the >>>> point >>>> >>>>>>>> that it >>>> >>>>>>>> seems forgotten now? >>>> >>>>>>>> >>>> >>>>>>>> I'm coming to the mailing list with these random questions >>>> about >>>> >>>>>>>> quota >>>> >>>>>>>> management, for two reasons: >>>> >>>>>>>> 1) despite developing and using openstack on a daily basis I'm >>>> still >>>> >>>>>>>> confused by quotas >>>> >>>>>>>> 2) I've found a race condition in neutron quotas and the fix >>>> is not >>>> >>>>>>>> trivial. >>>> >>>>>>>> So, rather than start coding right away, it might probably >>>> make more >>>> >>>>>>>> sense >>>> >>>>>>>> to ask the community if there is already a known better >>>> approach to >>>> >>>>>>>> quota >>>> >>>>>>>> management - and obviously enforcement. >>>> >>>>>>>> >>>> >>>>>>>> Thanks in advance, >>>> >>>>>>>> Salvatore >>>> >>>>>>>> >>>> >>>>>>>> [1] >>>> https://blueprints.launchpad.net/keystone/+spec/service-metadata >>>> >>>>>>>> [2] https://wiki.openstack.org/wiki/Blazar >>>> >>>>>>>> [3] >>>> https://review.openstack.org/#/q/project:stackforge/blazar,n,z >>>> >>>>>>>> >>>> >>>>>>>> _______________________________________________ >>>> >>>>>>>> OpenStack-dev mailing list >>>> >>>>>>>> OpenStack-dev@lists.openstack.org >>>> >>>>>>>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> -- >>>> >>>>>>> Duncan Thomas >>>> >>>>>>> >>>> >>>>>>> _______________________________________________ >>>> >>>>>>> OpenStack-dev mailing list >>>> >>>>>>> OpenStack-dev@lists.openstack.org >>>> >>>>>>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> _______________________________________________ >>>> >>>>>> OpenStack-dev mailing list >>>> >>>>>> OpenStack-dev@lists.openstack.org >>>> >>>>>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>>>> >>>> >>>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> >>>> OpenStack-dev mailing list >>>> >>>> OpenStack-dev@lists.openstack.org >>>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>>> >>> >>>> >>> >>>> >>> _______________________________________________ >>>> >>> OpenStack-dev mailing list >>>> >>> OpenStack-dev@lists.openstack.org >>>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> OpenStack-dev mailing list >>>> >> OpenStack-dev@lists.openstack.org >>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >> >>>> > >>>> > >>>> > >>>> > -- >>>> > Davanum Srinivas :: https://twitter.com/dims >>>> > >>>> > _______________________________________________ >>>> > OpenStack-dev mailing list >>>> > OpenStack-dev@lists.openstack.org >>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > -- > Kind Regards > Valeriy Ponomaryov > www.mirantis.com > vponomar...@mirantis.com > -- Kind Regards Valeriy Ponomaryov www.mirantis.com vponomar...@mirantis.com
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
