[Yahoo-eng-team] [Bug 1905538] [NEW] Some OVS bridges may lack OpenFlow10 protocol
Public bug reported: After commit https://review.opendev.org/c/openstack/neutron/+/371455 OVSAgentBridge.setup_controllers() no longer sets OpenFlow10 protocol for the bridge, instead it was moved to ovs_lib.OVSBridge.create(). However some (custom) OVS bridges could be created by nova/os-vif when plugging VM interface. For such bridges neutron does not call create(), only setup_controllers() - as a result such bridges support only OpenFlow13 and ovs-ofctl command fails: 2020-11-24T20:18:38Z|1|vconn|WARN|unix:/var/run/openvswitch/br01711489f-fe.24081.mgmt: version negotiation failed (we support version 0x01, peer supports version 0x04) ovs-ofctl: br01711489f-fe: failed to connect to socket (Broken pipe) Fix: return setting of OpenFlow10 (along with OpenFlow13) to setup_controllers(). It's doesn't hurt even if bridge already has OpenFlow10 in supported protocols. ** Affects: neutron Importance: Low Assignee: Oleg Bondarev (obondarev) Status: New ** Tags: ovs ovs-lib -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1905538 Title: Some OVS bridges may lack OpenFlow10 protocol Status in neutron: New Bug description: After commit https://review.opendev.org/c/openstack/neutron/+/371455 OVSAgentBridge.setup_controllers() no longer sets OpenFlow10 protocol for the bridge, instead it was moved to ovs_lib.OVSBridge.create(). However some (custom) OVS bridges could be created by nova/os-vif when plugging VM interface. For such bridges neutron does not call create(), only setup_controllers() - as a result such bridges support only OpenFlow13 and ovs-ofctl command fails: 2020-11-24T20:18:38Z|1|vconn|WARN|unix:/var/run/openvswitch/br01711489f-fe.24081.mgmt: version negotiation failed (we support version 0x01, peer supports version 0x04) ovs-ofctl: br01711489f-fe: failed to connect to socket (Broken pipe) Fix: return setting of OpenFlow10 (along with OpenFlow13) to setup_controllers(). It's doesn't hurt even if bridge already has OpenFlow10 in supported protocols. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1905538/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1905551] [NEW] functional: test_gateway_chassis_rebalance fails
Public bug reported: The test failure doesn't report much: ft1.10: neutron.tests.functional.services.ovn_l3.test_plugin.TestRouter.test_gateway_chassis_rebalancetesttools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func return f(self, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/services/ovn_l3/test_plugin.py", line 496, in test_gateway_chassis_rebalance self.assertTrue(self.cr_lrp_pb_event.wait(logical_port)) File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/unittest2/case.py", line 702, in assertTrue raise self.failureException(msg) AssertionError: False is not true Observed here: https://ea5c37c06ce6e77863cd-be2db655edae902b1f8d9628c9b7e990.ssl.cf1.rackcdn.com/753847/18/gate/neutron-functional-with-uwsgi/4fa2ab3/testr_results.html ** Affects: neutron Importance: Undecided Status: New ** Tags: functional-tests gate-failure -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1905551 Title: functional: test_gateway_chassis_rebalance fails Status in neutron: New Bug description: The test failure doesn't report much: ft1.10: neutron.tests.functional.services.ovn_l3.test_plugin.TestRouter.test_gateway_chassis_rebalancetesttools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func return f(self, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/services/ovn_l3/test_plugin.py", line 496, in test_gateway_chassis_rebalance self.assertTrue(self.cr_lrp_pb_event.wait(logical_port)) File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional/lib/python3.8/site-packages/unittest2/case.py", line 702, in assertTrue raise self.failureException(msg) AssertionError: False is not true Observed here: https://ea5c37c06ce6e77863cd-be2db655edae902b1f8d9628c9b7e990.ssl.cf1.rackcdn.com/753847/18/gate/neutron-functional-with-uwsgi/4fa2ab3/testr_results.html To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1905551/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1905552] [NEW] neutron-fwaas netlink conntrack driver would catch error while conntrack rules protocol is 'unknown'
Public bug reported: 2020-11-25 11:07:32.606 127 DEBUG oslo_concurrency.lockutils [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Releasing semaphore "iptables-qrouter-9e18395d-961d-46b3-a0e9-4c6a94c32baf" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:228 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Failed to update firewall: daedc38a-04ee-4818-b7a6-3d8311d7fc30: KeyError: 'unknown' 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 Traceback (most recent call last): 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/iptables_fwaas_v2.py", line 144, in update_firewall_group 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 apply_list, self.pre_firewall, firewall) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/iptables_fwaas_v2.py", line 327, in _remove_conntrack_updated_firewall 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 ipt_mgr.namespace) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/netlink_conntrack.py", line 41, in delete_entries 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 entries = nl_lib.list_entries(namespace) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 207, in _wrap 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 return self.channel.remote_call(name, args, kwargs) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 202, in remote_call 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 raise exc_type(*result[2]) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 KeyError: 'unknown' 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 This error appears when configured the neutron-fwaas v2 with netlink_conntrack driver in fwaas_agent.ini vim /etc/kolla/neutron-l3-agent/fwaas_driver.ini [fwaas] enabled = True agent_version = v2 driver = iptables_v2 conntrack_driver = netlink_conntrack And the conntrack list has 'unknown' rules, example below: unknown 2 597 src=169.254.192.2 dst=224.0.0.22 [UNREPLIED] src=224.0.0.22 dst=169.254.192.2 mark=0 use=1 unknown 112 598 src=169.254.192.2 dst=224.0.0.18 [UNREPLIED] src=224.0.0.18 dst=169.254.192.2 mark=0 use=1 This may interrupt conntrack refresh when firewall rules update. ** Affects: neutron Importance: Undecided Assignee: Zhang Jian (jasonzhangj) Status: New ** Changed in: neutron Assignee: (unassigned) => Zhang Jian (q5536487) ** Changed in: neutron Assignee: Zhang Jian (jasonzhangj) => (unassigned) ** Changed in: neutron Assignee: (unassigned) => Zhang Jian (jasonzhangj) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1905552 Title: neutron-fwaas netlink conntrack driver would catch error while conntrack rules protocol is 'unknown' Status in neutron: New Bug description: 2020-11-25 11:07:32.606 127 DEBUG oslo_concurrency.lockutils [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Releasing semaphore "iptables-qrouter-9e18395d-961d-46b3-a0e9-4c6a94c32baf" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:228 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 [req-ab14782d-80b1-4
[Yahoo-eng-team] [Bug 1905552] Re: neutron-fwaas netlink conntrack driver would catch error while conntrack rules protocol is 'unknown'
Neutron-fwaas development is stopped: https://review.opendev.org/c/openstack/governance/+/735828/ ** Changed in: neutron Status: New => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1905552 Title: neutron-fwaas netlink conntrack driver would catch error while conntrack rules protocol is 'unknown' Status in neutron: Won't Fix Bug description: 2020-11-25 11:07:32.606 127 DEBUG oslo_concurrency.lockutils [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Releasing semaphore "iptables-qrouter-9e18395d-961d-46b3-a0e9-4c6a94c32baf" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:228 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Failed to update firewall: daedc38a-04ee-4818-b7a6-3d8311d7fc30: KeyError: 'unknown' 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 Traceback (most recent call last): 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/iptables_fwaas_v2.py", line 144, in update_firewall_group 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 apply_list, self.pre_firewall, firewall) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/iptables_fwaas_v2.py", line 327, in _remove_conntrack_updated_firewall 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 ipt_mgr.namespace) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/netlink_conntrack.py", line 41, in delete_entries 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 entries = nl_lib.list_entries(namespace) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 207, in _wrap 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 return self.channel.remote_call(name, args, kwargs) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 202, in remote_call 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 raise exc_type(*result[2]) 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 KeyError: 'unknown' 2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 This error appears when configured the neutron-fwaas v2 with netlink_conntrack driver in fwaas_agent.ini vim /etc/kolla/neutron-l3-agent/fwaas_driver.ini [fwaas] enabled = True agent_version = v2 driver = iptables_v2 conntrack_driver = netlink_conntrack And the conntrack list has 'unknown' rules, example below: unknown 2 597 src=169.254.192.2 dst=224.0.0.22 [UNREPLIED] src=224.0.0.22 dst=169.254.192.2 mark=0 use=1 unknown 112 598 src=169.254.192.2 dst=224.0.0.18 [UNREPLIED] src=224.0.0.18 dst=169.254.192.2 mark=0 use=1 This may interrupt conntrack refresh when firewall rules update. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1905552/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1905568] [NEW] Sanity checks missing port_name while adding tunnel port
Public bug reported: Functions ovs_vxlan_supported and ovs_vxlan_supported from neutron.cmd.sanity.checks module are creating tunnel port by using neutron.agent.common.ovs_lib.OVSBridge.add_tunnel_port() method but they are not passing port name as a first argument. That argument is mandatory so should be passed there. ** Affects: neutron Importance: Medium Assignee: Slawek Kaplonski (slaweq) Status: Confirmed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1905568 Title: Sanity checks missing port_name while adding tunnel port Status in neutron: Confirmed Bug description: Functions ovs_vxlan_supported and ovs_vxlan_supported from neutron.cmd.sanity.checks module are creating tunnel port by using neutron.agent.common.ovs_lib.OVSBridge.add_tunnel_port() method but they are not passing port name as a first argument. That argument is mandatory so should be passed there. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1905568/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1903531] Re: Update of neutron-server breaks compatibility to previous neutron-agent version
** Also affects: neutron (Ubuntu) Importance: Undecided Status: New ** No longer affects: neutron (Ubuntu) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1903531 Title: Update of neutron-server breaks compatibility to previous neutron- agent version Status in neutron: Confirmed Status in OpenStack Security Advisory: Won't Fix Bug description: During the update of Neutron from 15.1 to 15.3 (Train) server and agent are not compatible (RPC format) anymore. I was able to narrow down the issue to commit https://opendev.org/openstack/neutron/commit/00298fe6e84cd7610b39af50e9517885a182f47c or patchset https://review.opendev.org/#/c/744133/ With this commit, fixing the security issue in bug https://bugs.launchpad.net/neutron/+bug/1867119 was addressed and a change to the RPC data structure now containing also the MAC address was required consequently back-ported to Train (https://bugs.launchpad.net/neutron/+bug/1867119/comments/6). This change breaks compatibility as the loop in method "_sanitize_addresses" (neutron/agent/linux/ipset_manager.py", line 47) does not handle this new format properly. --- 15.1 --- for ip in addresses: ip = netaddr.IPNetwork(ip) if ip.prefixlen == 0: --- /15.1 --- --- 15.3 --- for ip, _mac in addresses: ip = netaddr.IPNetwork(ip) if ip.prefixlen == 0: --- /15.3 --- As neutron server in version 15.1 delivers just a list of addresses, i,e. "['192.168.100.57', '192.168.100.160']", while after the update to 15.3 it sends a list of lists with each list element containing an IP as well a MAC address, i.e. "[['192.168.100.57', None], ['192.168.100.160', None]])" the code on the agent side has no way of handling both cases. According to https://docs.openstack.org/neutron/latest/contributor/internals/upgrade.html which states: " Those requirements are achieved in Neutron by: If the Neutron backend makes use of Neutron agents, the Neutron server have backwards compatibility code to deal with older messaging payloads. isolating a single service that accesses database (neutron- server). To simplify the matter, it’s always assumed that the order of service upgrades is as following: first, all neutron-servers are upgraded. then, if applicable, neutron agents are upgraded. This approach allows us to avoid backwards compatibility code on agent side and is in line with other OpenStack projects that support rolling upgrades (specifically, nova). " an upgraded neutron-server should still work with the previous neutron-agent. I took the liberty to flag this as "security vulnerability" as security groups will likely not be applied / managed properly when running mixed between 15.1 and 15.3 which might be a common case in larger clusters. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1903531/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1905587] [NEW] SGs shared via RBAC for an instance are not handled in Horizon
Public bug reported: It appears we can't edit security groups for an instance shared via rbac. Sharing SGs via RBAC is done via: `openstack network rbac create --target-project $project --action access_as_shared --type security_group ...` We can see the shared SGs in the cli, when doing `openstack server show ...` However I can't see those SGs on the instance in Horizon. Also, when adding another SG in Horizon the one shared via RBAC is removed from the instance One thing we noticed that might possibly be related is that in https://github.com/openstack/horizon/blob/stable/stein/openstack_dashboard/api/neutron.py#L370 the SGs seem to be taken only from the project Versions: OpenStack Stein python3-django-horizon 3:15.2.0-0ubuntu1~cloud0 ** Affects: horizon Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1905587 Title: SGs shared via RBAC for an instance are not handled in Horizon Status in OpenStack Dashboard (Horizon): New Bug description: It appears we can't edit security groups for an instance shared via rbac. Sharing SGs via RBAC is done via: `openstack network rbac create --target-project $project --action access_as_shared --type security_group ...` We can see the shared SGs in the cli, when doing `openstack server show ...` However I can't see those SGs on the instance in Horizon. Also, when adding another SG in Horizon the one shared via RBAC is removed from the instance One thing we noticed that might possibly be related is that in https://github.com/openstack/horizon/blob/stable/stein/openstack_dashboard/api/neutron.py#L370 the SGs seem to be taken only from the project Versions: OpenStack Stein python3-django-horizon 3:15.2.0-0ubuntu1~cloud0 To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1905587/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1905611] [NEW] OVN.ovsdb_probe_interval takes effect only after initial database dump
Public bug reported: ml2/ovn allows the configuration of the OVSDB session probe_interval, and defaults it to 60 seconds, overriding the python-ovs default of 5 seconds. It does this after the ovsdbapp Connection object is created and started, which means it is also after python-ovs sends a monitor request that pulls the initial copy of the database into memory. On a busy system with a large database, this can cause a failure since the 5 second default may not be sufficient to complete this request, causing a failure and infinite reconnection attempts with the error: ERROR ovsdbapp.backend.ovs_idl.vlog [-] tcp:10.240.200.63:6642: no response to inactivity probe after 5.01 seconds, disconnecting" 3 years ago, functionality was added to python-ovs to pass the probe_interval value to the Idl so that it can be set immediately on the jsonrpc connection before connection, so we should use that. ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1905611 Title: OVN.ovsdb_probe_interval takes effect only after initial database dump Status in neutron: New Bug description: ml2/ovn allows the configuration of the OVSDB session probe_interval, and defaults it to 60 seconds, overriding the python-ovs default of 5 seconds. It does this after the ovsdbapp Connection object is created and started, which means it is also after python-ovs sends a monitor request that pulls the initial copy of the database into memory. On a busy system with a large database, this can cause a failure since the 5 second default may not be sufficient to complete this request, causing a failure and infinite reconnection attempts with the error: ERROR ovsdbapp.backend.ovs_idl.vlog [-] tcp:10.240.200.63:6642: no response to inactivity probe after 5.01 seconds, disconnecting" 3 years ago, functionality was added to python-ovs to pass the probe_interval value to the Idl so that it can be set immediately on the jsonrpc connection before connection, so we should use that. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1905611/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1905672] [NEW] Non existing property protection file raises 500 Internal server error
Public bug reported: Non existing property protection file raises 500 Internal server error If admin/operator specifies non existing property protection file in glance-api.conf then create/update image call raises 500 Internal server error. Steps to reproduce: 1. Enable property protection in glance-api.conf and provide non existing file [Default] property_protection_file = non_existing_file.yaml property_protection_rule_format = policies 2. Restart glance-api service 3. Create image by specifiying additional property glance image-create-via-import --disk-format qcow2 --container-format bare --name conversion_test --import-method web-download --uri https://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img --property xyz=xyz Expected Result: API should return 400 Bad request Actual result: Returns 500 Internal server error Glance API Logs: Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.property_utils [None req-43d209a4-3143-4855-b843-2137a9cbb22b admin admin] Couldn't find property protection file /etc/glance/property.yaml: 'NoneType' object is not iterable. Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi [None req-43d209a4-3143-4855-b843-2137a9cbb22b admin admin] Caught error: Invalid configuration in property protection file.: glance.common.exception.InvalidPropertyProtectionConfiguration: Invalid configuration in property protection file. Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi Traceback (most recent call last): Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/property_utils.py", line 119, in _load_rules Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi CONFIG.read(conf_file) Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/usr/lib/python3.6/configparser.py", line 694, in read Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi for filename in filenames: Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi TypeError: 'NoneType' object is not iterable Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi During handling of the above exception, another exception occurred: Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi Traceback (most recent call last): Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/wsgi.py", line 1348, in __call__ Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi request, **action_args) Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/wsgi.py", line 1391, in dispatch Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi return method(*args, **kwargs) Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/utils.py", line 416, in wrapped Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi return func(self, req, *args, **kwargs) Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/api/v2/images.py", line 74, in create Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi image_factory = self.gateway.get_image_factory(req.context) Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/gateway.py", line 50, in get_image_factory Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi property_rules = property_utils.PropertyRules(self.policy) Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/property_utils.py", line 114, in __init__ Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi self._load_rules() Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi File "/opt/stack/glance/glance/common/property_utils.py", line 125, in _load_rules Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi raise InvalidPropProtectConf() Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi glance.common.exception.InvalidPropertyProtectionConfiguration: Invalid configuration in property protection file. Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: ERROR glance.common.wsgi Nov 26 06:49:54 akekane-wallaby-dev glance-api[15444]: INFO eventlet.wsgi.server [None req
