This bug was fixed in the package manila - 1:10.0.0-0ubuntu1 --------------- manila (1:10.0.0-0ubuntu1) groovy; urgency=medium
[ Chris MacNaughton ] * New upstream release for OpenStack Ussuri (LP: #1877642). * d/p/monkey-patch-original-current-thread.patch: Cherry-picked from https://review.opendev.org/#/c/728079/. This fixes service failures with Python 3.8 (LP: #1863021) * d/watch: Scope to 10.x series. * d/watch: Update to point at tarballs.opendev.org. [ Corey Bryant ] * d/control: Align (Build-)Depends with upstream. -- Chris MacNaughton <chris.macnaugh...@canonical.com> Thu, 14 May 2020 13:36:55 +0000 ** Changed in: manila (Ubuntu Groovy) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: Fix Released Status in Designate: In Progress Status in Glance: Fix Released Status in OpenStack Shared File Systems Service (Manila): In Progress Status in masakari: In Progress Status in Mistral: Fix Released Status in Murano: Fix Released Status in BaGPipe: Fix Released Status in networking-hyperv: Fix Released Status in networking-l2gw: Fix Released Status in Mellanox backend integration with Neutron (networking-mlnx): In Progress Status in networking-sfc: Fix Released Status in neutron: Fix Released Status in OpenStack Compute (nova): Fix Released Status in oslo.service: Fix Released Status in senlin: Fix Released Status in OpenStack Object Storage (swift): In Progress Status in OpenStack DBaaS (Trove): In Progress Status in watcher: Fix Released Status in barbican package in Ubuntu: Fix Released Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Fix Released Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Fix Released Status in ironic-inspector package in Ubuntu: Fix Released Status in magnum package in Ubuntu: Triaged Status in manila package in Ubuntu: Fix Released Status in masakari package in Ubuntu: Triaged Status in mistral package in Ubuntu: Triaged Status in murano package in Ubuntu: Triaged Status in murano-agent package in Ubuntu: Triaged Status in networking-bagpipe package in Ubuntu: Fix Released Status in networking-hyperv package in Ubuntu: Fix Released Status in networking-l2gw package in Ubuntu: Triaged Status in networking-mlnx package in Ubuntu: Triaged Status in networking-sfc package in Ubuntu: Triaged Status in neutron package in Ubuntu: Fix Released Status in neutron-dynamic-routing package in Ubuntu: Triaged Status in nova package in Ubuntu: Fix Released Status in openstack-trove package in Ubuntu: Triaged Status in python-os-ken package in Ubuntu: Triaged Status in python-oslo.service package in Ubuntu: Triaged Status in sahara package in Ubuntu: Triaged Status in senlin package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in watcher package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in magnum source package in Focal: Triaged Status in manila source package in Focal: Triaged Status in masakari source package in Focal: Triaged Status in mistral source package in Focal: Triaged Status in murano source package in Focal: Triaged Status in murano-agent source package in Focal: Triaged Status in networking-bagpipe source package in Focal: Triaged Status in networking-hyperv source package in Focal: Triaged Status in networking-l2gw source package in Focal: Triaged Status in networking-mlnx source package in Focal: Triaged Status in networking-sfc source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in neutron-dynamic-routing source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in openstack-trove source package in Focal: Triaged Status in python-os-ken source package in Focal: Triaged Status in python-oslo.service source package in Focal: Triaged Status in sahara source package in Focal: Triaged Status in senlin source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in watcher source package in Focal: Triaged Status in barbican source package in Groovy: Fix Released Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Fix Released Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Fix Released Status in ironic-inspector source package in Groovy: Fix Released Status in magnum source package in Groovy: Triaged Status in manila source package in Groovy: Fix Released Status in masakari source package in Groovy: Triaged Status in mistral source package in Groovy: Triaged Status in murano source package in Groovy: Triaged Status in murano-agent source package in Groovy: Triaged Status in networking-bagpipe source package in Groovy: Fix Released Status in networking-hyperv source package in Groovy: Fix Released Status in networking-l2gw source package in Groovy: Triaged Status in networking-mlnx source package in Groovy: Triaged Status in networking-sfc source package in Groovy: Triaged Status in neutron source package in Groovy: Fix Released Status in neutron-dynamic-routing source package in Groovy: Triaged Status in nova source package in Groovy: Fix Released Status in openstack-trove source package in Groovy: Triaged Status in python-os-ken source package in Groovy: Triaged Status in python-oslo.service source package in Groovy: Triaged Status in sahara source package in Groovy: Triaged Status in senlin source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Status in watcher source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: <function _after_fork at 0x7f1100d075e0> Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: <function _after_fork at 0x7f1100d075e0> Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() + import __original_module_threading + import threading + __original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] Tempest regression on a deployed openstack should pass successfully. Individual services can be tested as follows but with the scope of this issue tempest regression testing will be the goal for verifying this SRU. 'systemctl status <service>' should not show the failure reported in [Impact]. [Regression Potential] Should be low. The fix provided is a well-known solution to the problem. Openstack deployment + tempest regression testing will be performed to help reduce regression potential. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1863021/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp