We settled on 1251920. https://review.openstack.org/57509 is the fix for that bug.
Note that Oslo was fixed on Jun 28th, nova hasn't synced since then. If we were using oslo as a library we would have had the fix as soon as olso did a release. These are the references to strong_store - and thus broken in nova trunk (and if any references exist in H, in H too): ./nova/network/neutronv2/__init__.py:58: if not hasattr(local.strong_store, 'neutron_client'): ./nova/network/neutronv2/__init__.py:59: local.strong_store.neutron_client = _get_client(token=None) ./nova/network/neutronv2/__init__.py:60: return local.strong_store.neutron_client ./nova/openstack/common/rpc/__init__.py:102: if ((hasattr(local.strong_store, 'locks_held') ./nova/openstack/common/rpc/__init__.py:103: and local.strong_store.locks_held)): ./nova/openstack/common/rpc/__init__.py:108: {'locks': local.strong_store.locks_held, ./nova/openstack/common/local.py:47:strong_store = threading.local() ./nova/openstack/common/lockutils.py:173: if not hasattr(local.strong_store, 'locks_held'): ./nova/openstack/common/lockutils.py:174: local.strong_store.locks_held = [] ./nova/openstack/common/lockutils.py:175: local.strong_store.locks_held.append(name) ./nova/openstack/common/lockutils.py:217: local.strong_store.locks_held.remove(name) ./nova/tests/network/test_neutronv2.py:1837: local.strong_store.neutron_client = None So we can expect lockutils to be broken, and rpc to be broken. Clearly they are being impacted more subtly than the neutron client usage. -Rob On 21 November 2013 07:44, Robert Collins <robe...@robertcollins.net> wrote: > Which of these bugs would be appropriate to use for the fix to > strong_store - it affects lockutils and rpc, both of which are going > to create havoc :) > > -Rob > > On 21 November 2013 07:19, Salvatore Orlando <sorla...@nicira.com> wrote: >> I've noticed that >> https://github.com/openstack/nova/commit/85332012dede96fa6729026c2a90594ea0502ac5 >> stores the network client in local.strong_store which is a reference to >> corolocal.local (the class, not the instance). >> >> In Russell's example instead the code accesses local.store which is an >> instance of WeakLocal (inheriting from corolocal.local). >> >> Perhaps then Roman's findings apply to the issue being observed on the gate. >> >> Regards, >> Salvatore >> >> >> On 20 November 2013 18:32, Russell Bryant <rbry...@redhat.com> wrote: >>> >>> On 11/20/2013 12:21 PM, Alex Gaynor wrote: >>> > Nope, you're totally right, corolocal.local is a class, whose instances >>> > are the actual coroutine local storage. >>> >>> But I don't think his example is what is being used. >>> >>> Here is an example using the openstack.common.local module, which is >>> what nova uses for this. This produces the expected output. >>> >>> http://paste.openstack.org/show/53687/ >>> >>> >>> https://git.openstack.org/cgit/openstack/nova/tree/nova/openstack/common/local.py >>> >>> For reference, original example from OP: >>> http://paste.openstack.org/show/53686/ >>> >>> -- >>> Russell Bryant >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > > -- > Robert Collins <rbtcoll...@hp.com> > Distinguished Technologist > HP Converged Cloud -- Robert Collins <rbtcoll...@hp.com> Distinguished Technologist HP Converged Cloud _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev