Reviewed: https://review.opendev.org/677221 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4b6c2246c09eccbb193b4a59a8dfee9e24f97bed Submitter: Zuul Branch: master
commit 4b6c2246c09eccbb193b4a59a8dfee9e24f97bed Author: Rodolfo Alonso Hernandez <[email protected]> Date: Mon Aug 19 13:09:20 2019 +0000 Force "network_id" in "subnet" DB registers The "subnet" OVO does not allow to have an empty (None) "network_id" but the DB "subnet" table allows to have this parameter empty (NULL) in the database. In order to avoid any problem like the one described in the bug, this patch ensures the database "subnet" register does always have a "network_id" value and if a "network" register is being deleted, all related "subnet" registers are checked first. Change-Id: Iad210f0585b4201fdb87187b44a9b42267b58db4 Closes-Bug: #1839658 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1839658 Title: "subnet" register in the DB can have network_id=NULL Status in neutron: Fix Released Bug description: (A bit more readable version of the following text is found at http://paste.openstack.org/show/756523/) When deleting a cluster with several networks and subnets, the following error was found when executing the shell command 'openstack subnet list': 019-08-06 08:26:10.000 28 INFO neutron.wsgi [-] 172.17.1.15 "OPTIONS / HTTP/1.0" status: 200 len: 248 time: 0.0012720 2019-08-06 08:26:10.803 28 DEBUG neutron.wsgi [-] (28) accepted ('172.17.1.15', 41484) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:883 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation [req-d757636f-b963-46c1-989e-75dc8fb8406c 404ce30e61e64ce7bc67db81b312c24e dcb34bf98dea46928294ff6762d9c167 - default default] GET failed.: AttributeError: 'Subnet' object has no attribute '_obj_network_id' 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last): 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/pecan/core.py", line 683, in __call__ 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation self.invoke_controller(controller, args, kwargs, state) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/pecan/core.py", line 574, in invoke_controller 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation result = controller(*args, **kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron_lib/db/api.py", line 140, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation setattr(e, '_RETRY_EXCEEDED', True) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation six.reraise(self.type_, self.value, self.tb) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron_lib/db/api.py", line 136, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 154, in wrapper 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation ectxt.value = e.inner_exc 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation six.reraise(self.type_, self.value, self.tb) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 142, in wrapper 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation LOG.debug("Retry wrapper got retriable exception: %s", e) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation six.reraise(self.type_, self.value, self.tb) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return f(*dup_args, **dup_kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron/pecan_wsgi/controllers/utils.py", line 76, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 131, in index 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return self.get(*args, **kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 141, in get 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation **query_params)} 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron/db/api.py", line 123, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return method(*args, **kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron_lib/db/api.py", line 140, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation setattr(e, '_RETRY_EXCEEDED', True) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation six.reraise(self.type_, self.value, self.tb) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron_lib/db/api.py", line 136, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 154, in wrapper 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation ectxt.value = e.inner_exc 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation six.reraise(self.type_, self.value, self.tb) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 142, in wrapper 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation LOG.debug("Retry wrapper got retriable exception: %s", e) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation six.reraise(self.type_, self.value, self.tb) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return f(*dup_args, **dup_kwargs) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron/db/db_base_plugin_v2.py", line 1075, in get_subnets 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation for subnet_object in subnet_objs 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/neutron/db/db_base_plugin_common.py", line 134, in _make_subnet_dict 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation 'network_id': subnet['network_id'], 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 774, in __getitem__ 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return getattr(self, name) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 68, in getter 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation return getattr(self, attrname) 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation AttributeError: 'Subnet' object has no attribute '_obj_network_id' 2019-08-06 08:26:11.520 28 ERROR neutron.pecan_wsgi.hooks.translation 2019-08-06 08:26:11.523 28 INFO neutron.wsgi [req-d757636f-b963-46c1-989e-75dc8fb8406c 404ce30e61e64ce7bc67db81b312c24e dcb34bf98dea46928294ff6762d9c167 - default default] 172.17.1.15 "GET /v2.0/subnets HTTP/1.1" status: 500 len: 344 time: 0.7188911 2019-08-06 08:26:12.002 34 DEBUG neutron.wsgi [-] (34) accepted ('172.17.1.15', 41734) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:883 2019-08-06 08:26:12.004 34 INFO neutron.wsgi [-] 172.17.1.15 "OPTIONS / HTTP/1.0" status: 200 len: 248 time: 0.0012681 Which means that some subnets where left behind without an associated network. In this case ns/openshift-kube-controller-manager-subnet: MariaDB [ovs_neutron]> select * from subnets; +----------------------------------+--------------------------------------+---------------------------------------------+--------------------------------------+------------+----------------+-------------+-------------+--------------+-------------------+--------------------------------------+------------------+------------+ | project_id | id | name | network_id | ip_version | cidr | gateway_ip | enable_dhcp | ipv6_ra_mode | ipv6_address_mode | subnetpool_id | standard_attr_id | segment_id | +----------------------------------+--------------------------------------+---------------------------------------------+--------------------------------------+------------+----------------+-------------+-------------+--------------+-------------------+--------------------------------------+------------------+------------+ | 755fea49ab0643d0b731073e87e739e0 | 26953b3b-fa6c-46a0-9e60-6a7d5cadf580 | lb-mgmt-subnet | 4bfa167b-fcf9-4ad8-b5f3-99c6141b20a4 | 4 | 172.24.0.0/16 | 172.24.0.1 | 1 | NULL | NULL | NULL | 8 | NULL | | dcb34bf98dea46928294ff6762d9c167 | 5ea30bc8-6eb1-4494-96e6-b37d9a70fb09 | ns/openshift-kube-controller-manager-subnet | NULL | 4 | 10.128.34.0/24 | 10.128.34.1 | 0 | NULL | NULL | cc48743c-e44c-48ee-b29c-5ceb53f60fbd | 9493 | NULL | | 755fea49ab0643d0b731073e87e739e0 | 8f4a9baf-ad79-4730-9449-671706f24d7a | external_subnet | 004f76a6-3867-46f4-8718-5498634945a3 | 4 | 10.0.0.0/24 | 10.0.0.1 | 0 | NULL | NULL | NULL | 27 | NULL | +----------------------------------+--------------------------------------+---------------------------------------------+--------------------------------------+------------+----------------+-------------+-------------+--------------+-------------------+--------------------------------------+------------------+------------+ This is only happening rarely, not 100% of the time. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1839658/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
