Hi Simon, I have found the problem. A nice optimization was added a while ago - https://review.openstack.org/#/c/33720/. This ensures that a filter will only be run once per request. The problem is that if a instance is unable to be run on a scheduled host then a new host will need to be selected. The scheduling after a failed attempt would not invoke the anti affinity scheduling, which may lead to an invalid host being selected. I am going to post a patch soon. Thanks Gary
On 9/6/13 3:18 PM, "Gary Kotton" <gkot...@vmware.com> wrote: >Hi, >Sorry for the delayed response (it is new years my side of the world and >have some family obligations). >Would it be possible that you please provide the nova configuration file >(I would like to see if you have the group anti affinity filter in your >filter list), and if this exists to at least see a trace that the filter >has been invoked. >I have tested this with the patches that I mentioned below and it works. I >will invest some time on this on Sunday to make sure that it is all >working with the latest code. >Thanks >Gary > >On 9/6/13 10:31 AM, "Simon Pasquier" <simon.pasqu...@bull.net> wrote: > >>Gary (or others), did you have some time to look at my issue? >>FYI, I opened a bug [1] on Launchpad. I'll update it with the outcome of >>this discussion. >>Cheers, >>Simon >> >>[1] https://bugs.launchpad.net/nova/+bug/1218878 >> >>Le 03/09/2013 15:54, Simon Pasquier a écrit : >>> I've done a wrong copy&paste, see correction inline. >>> >>> Le 03/09/2013 12:34, Simon Pasquier a écrit : >>>> Hello, >>>> >>>> Thanks for the reply. >>>> >>>> First of all, do you agree that the current documentation for these >>>> filters is inaccurate? >>>> >>>> My test environment has 2 compute nodes: compute1 and compute3. First, >>>>I >>>> launch 1 instance (not being tied to any group) on each node: >>>> $ nova boot --flavor m1.tiny --image cirros-0.3.1-x86_64-uec >>>>--key-name >>>> local --availability-zone nova:compute1 vm-compute1-nogroup >>>> $ nova boot --flavor m1.tiny --image cirros-0.3.1-x86_64-uec >>>>--key-name >>>> local --availability-zone nova:compute3 vm-compute3-nogroup >>>> >>>> So far so good, everything's active: >>>> $ nova list >>>> >>>>+--------------------------------------+---------------------+--------+ >>>>- >>>>-----------+-------------+------------------+ >>>> >>>> >>>> | ID | Name | Status >>>>| >>>> Task State | Power State | Networks | >>>> >>>>+--------------------------------------+---------------------+--------+ >>>>- >>>>-----------+-------------+------------------+ >>>> >>>> >>>> | 3a465024-85e7-4e80-99a9-ccef3a4f41d5 | vm-compute1-nogroup | ACTIVE >>>>| >>>> None | Running | private=10.0.0.3 | >>>> | c838e0c4-3b4f-4030-b2a2-b21305c0f3ea | vm-compute3-nogroup | ACTIVE >>>>| >>>> None | Running | private=10.0.0.4 | >>>> >>>>+--------------------------------------+---------------------+--------+ >>>>- >>>>-----------+-------------+------------------+ >>>> >>>> >>>> >>>> Then I try to launch one instance in group 'foo' but it fails: >>>> $ nova boot --flavor m1.tiny --image cirros-0.3.1-x86_64-uec >>>>--key-name >>>> local --availability-zone nova:compute3 vm-compute3-nogroup >>> >>> The command is: >>> >>> $ nova boot --flavor m1.tiny --image cirros-0.3.1-x86_64-uec --key-name >>> local --hint group=foo vm1-foo >>> >>>> $ nova list >>>> >>>>+--------------------------------------+---------------------+--------+ >>>>- >>>>-----------+-------------+------------------+ >>>> >>>> >>>> | ID | Name | Status >>>>| >>>> Task State | Power State | Networks | >>>> >>>>+--------------------------------------+---------------------+--------+ >>>>- >>>>-----------+-------------+------------------+ >>>> >>>> >>>> | 3a465024-85e7-4e80-99a9-ccef3a4f41d5 | vm-compute1-nogroup | ACTIVE >>>>| >>>> None | Running | private=10.0.0.3 | >>>> | c838e0c4-3b4f-4030-b2a2-b21305c0f3ea | vm-compute3-nogroup | ACTIVE >>>>| >>>> None | Running | private=10.0.0.4 | >>>> | 743fa564-f38f-4f44-9913-d8adcae955a0 | vm1-foo | ERROR >>>>| >>>> None | NOSTATE | | >>>> >>>>+--------------------------------------+---------------------+--------+ >>>>- >>>>-----------+-------------+------------------+ >>>> >>>> >>>> >>>> I've pasted the scheduler logs [1] and my nova.conf file [2]. As you >>>> will see, the log message is there but it looks like group_hosts() [3] >>>> is returning all my hosts instead of only the ones that run instances >>>> from the group. >>>> >>>> Concerning GroupAffinityFilter, I understood that it couldn't work >>>> simultaneously with GroupAntiAffinityFilter but since I missed the >>>> multiple schedulers, I couldn't figure out how it would be useful. So >>>>I >>>> got it now. >>>> >>>> Best regards, >>>> >>>> Simon >>>> >>>> [1] http://paste.openstack.org/show/45672/ >>>> [2] http://paste.openstack.org/show/45671/ >>>> [3] >>>> >>>>https://github.com/openstack/nova/blob/master/nova/scheduler/driver.py# >>>>L >>>>137 >>>> >>>> >>>> Le 03/09/2013 10:49, Gary Kotton a écrit : >>>>> Hi, >>>>> Hopefully I will be able to address your questions. First lets start >>>>> with >>>>> the group anti-affinity. This was added towards the end of the >>>>>Grizzly >>>>> release cycle as a scheduling hint. At the last summit we sat and >>>>>agreed >>>>> on a more formal approach to deal with this and we proposed and >>>>> developed >>>>> >>>>>https://blueprints.launchpad.net/openstack/?searchtext=instance-group- >>>>>a >>>>>pi-e >>>>> >>>>> >>>>> xtension (https://wiki.openstack.org/wiki/GroupApiExtension). >>>>> At the moment the following are still in review and I hope that we >>>>>will >>>>> make the feature freeze deadline: >>>>> Api support: >>>>> https://review.openstack.org/#/c/30028/ >>>>> >>>>> Scheduler support: >>>>> https://review.openstack.org/#/c/33956/ >>>>> >>>>> Client support: >>>>> https://review.openstack.org/#/c/32904/ >>>>> >>>>> In order to make use of the above you need to add >>>>> GroupAntiAffinityFilter >>>>> to the filters that will be active (this is not one of the default >>>>> filters). When you deploy the first instance of a group you need to >>>>> specify that it is part of the group. This information is used for >>>>> additional VM's that are being deployed. >>>>> >>>>> Can you please provide some extra details so that I can help you >>>>>debug >>>>> the >>>>> issues that you have encountered (I did not encounter the problems >>>>>that >>>>> you have described): >>>>> 1. Please provide the commands that you used with the deploying of >>>>>the >>>>> instance >>>>> 2. Please provide the nova configuration file >>>>> 3. Can you please look at the debug traces and see if you see the log >>>>> message on line 97 >>>>> >>>>>(https://review.openstack.org/#/c/21070/8/nova/scheduler/filters/affin >>>>>i >>>>>ty_f >>>>> >>>>> >>>>> ilter.py) >>>>> >>>>> Now regarding the AffinityFilter. At this stage this does not work >>>>>with >>>>> the AntiAffinity filter. We were banking on this being used with the >>>>> multiple scheduler policies (https://review.openstack.org/#/c/37407/) >>>>> >>>>> Thanks >>>>> Gary >>>>> >>>>> >>>>> >>>>> On 9/3/13 10:16 AM, "Simon Pasquier" <simon.pasqu...@bull.net> wrote: >>>>> >>>>>> Reposting to openstack-dev as I got no answer on the general mailing >>>>>> list. >>>>>> >>>>>> >>>>>> -------- Message original -------- >>>>>> Sujet: [Openstack] Confused about GroupAntiAffinityFilter and >>>>>> GroupAffinityFilter >>>>>> Date : Mon, 2 Sep 2013 11:19:58 +0200 >>>>>> De : Simon Pasquier <simon.pasqu...@bull.net> >>>>>> Organisation : Bulll SAS >>>>>> Pour : <openst...@lists.openstack.org> >>>>>> >>>>>> Hello, >>>>>> >>>>>> I tried to play with GroupAntiAffinityFilter and GroupAffinityFilter >>>>>> filters but it looks like the documentation is misleading [1]. >>>>>>Looking >>>>>> more precisely at the commits that introduced these filters [2][3], >>>>>>my >>>>>> assumption is that to use these filters, one would boot a first >>>>>> instance >>>>>> with '--hint group=foo' and the scheduler would update the >>>>>> instance_system_metadata table with {key:'group',value:'foo}. Then >>>>>>when >>>>>> starting other instances with the same hint option, the scheduler >>>>>>would >>>>>> filter the candidate hosts by querying the instance_system_metadata >>>>>> table. >>>>>> >>>>>> Still this doesn't work for me. In my tests with >>>>>> GroupAntiAffinityFilter, I have 3 compute nodes, each running one >>>>>> instance not in any group. Then when I launch a VM specifying a >>>>>>group >>>>>> hint, the scheduler fails to find a valid host because >>>>>> GroupAntiAffinityFilter filter returns 0 host. >>>>>> >>>>>> Could someone provide some guidance on how to use this filter? >>>>>> >>>>>> Regards, >>>>>> >>>>>> [1] >>>>>> >>>>>>http://docs.openstack.org/trunk/openstack-compute/admin/content/sched >>>>>>u >>>>>>ler- >>>>>> >>>>>> >>>>>> filters.html#groupaffinityfilter >>>>>> [2] https://review.openstack.org/#/c/21070/ >>>>>> [3] https://review.openstack.org/#/c/35788/ >>>>>> >>>>>> -- >>>>>> Simon Pasquier >>>>>> Software Engineer >>>>>> Bull, Architect of an Open World >>>>>> Phone: + 33 4 76 29 71 49 >>>>>> http://www.bull.com >>>>>> >>>>>> _______________________________________________ >>>>>> Mailing list: >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>>> Post to : openst...@lists.openstack.org >>>>>> Unsubscribe : >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> OpenStack-dev mailing list >>>>>> OpenStack-dev@lists.openstack.org >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>> >>>>> _______________________________________________ >>>>> OpenStack-dev mailing list >>>>> OpenStack-dev@lists.openstack.org >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>> >>>> >>> >>> >> >> >>-- >>Simon Pasquier >>Software Engineer >>Bull, Architect of an Open World >>Phone: + 33 4 76 29 71 49 >>http://www.bull.com >> >>_______________________________________________ >>OpenStack-dev mailing list >>OpenStack-dev@lists.openstack.org >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >_______________________________________________ >OpenStack-dev mailing list >OpenStack-dev@lists.openstack.org >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
