On Oct 31, 2012, at 5:57 PM, Vishvananda Ishaya <vishvana...@gmail.com> wrote:
> > Looking at the code it appears that the relevent info is being sent down to > the compute node. That said I can't seem to repro your issue with even just > the ram filter. I can't get it to overallocate on one node unless I > specifically change the ratio, regardless of fill-first or spread-first. > > Vish > I was finally able to reproduce this. It appears that you are scheduling the ~300 instances very rapidly one at a time. If you schedule the instances in one batch (using min_count and max_count) the scheduler schedules them all in a single loop, taking the previous allocations into account. Otherwise it has to wait for a new state report from the compute node for it to update the used resources. Fortunately, I think we can also create a fix for the many requests case by either: a) keeping the host states in memory and using the in memory version (multiple schedulers would have to use something like a shared memcache) or b) updating the host state in the database when we launch the instance so the next request gets the new data. I think that b) still has a pretty large window where there can be a race condition whereas the potential race in a) is almost impossible so I attempted a version of a) My patch here seems to fix the issue in the one scheduler case: https://github.com/vishvananda/nova/commit/2eaf796e60bd35319fe6add6dd04359546a21682 If you could give that a try on your scheduler node and see if it fixes it that would be awesome. Also, it would be fery helpful if you can report a bug for me to reference in my merge proposal. I will see what I can do to write a few tests and have a potential fix for multiple schedulers. Vish _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
