Indeed - thanks, Michael. The last time we looked the Oslo bug was in master, but hadn't been packaged for other projects. That was a few days ago, though. Anyway, if anyone else finds that their cells services suddenly start acting like they have nothing to do, I'd recommend looking at these patches.
Thanks! Matt On 2/12/15 2:53 PM, "Michael Still" <mi...@stillhq.com> wrote: >I just want to note that both of those fixes look to be approved now. > >Cheers, >Michael > >On Fri, Feb 13, 2015 at 6:14 AM, Matt Van Winkle <mvanw...@rackspace.com> >wrote: >> Hey folks, >> Apologies if any of this has been discussed on the list already. I've >>tried >> to check everything ahead of time. >> >> We recently had two bugs combine to hit us in some of our regions as we >> rolled out some new code. The result of them was rabbit servers not >>accept >> connections and/or crashing with OOM errors. I wanted to pass them >>along >> as I know from the Large Deployments Team, there are more and more folks >> using cells to manage larger regions. Here are the specific bugs: >> >> Cells doesn't properly track RabbitMQ connection pools: >> https://review.openstack.org/#/c/152667/ >> >> Oslo messaging bgt in version 1.5.1 that leaks channels : >> Upstream bug: https://bugs.launchpad.net/oslo.messaging/+bug/1406629 >> Upstream fix: >> >>https://review.openstack.org/#/c/145232/9/oslo_messaging/_drivers/impl_ra >>bbit.py >> >> >> We are deploying patches for both in our problem areas now and the rest >>of >> the fleet in the immediate future, but this gave us quite a run for our >> money last week. I wanted to share in case anyone else is chasing these >> issues and/or might after an upcoming code update. >> >> Thanks! >> Matt >> >> _______________________________________________ >> OpenStack-operators mailing list >> OpenStack-operators@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators >> > > > >-- >Rackspace Australia _______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
