Public bug reported:
kolla upgrade neutron
rpc_loop 1:
Physical bridge br-ex was just re-created,
set bridges_recreated=True
set need_clean_stale_flow=True
process_network_ports argument provisioning_needed is True but throw exception,
not call provision_local_vlan and flag need_clean_stale_flow keep True
rpc_loop 2:
set bridges_recreated=False
process_network_ports argument provisioning_needed is False, and not call
provision_local_vlan
Cleaning stale flows and set need_clean_stale_flow = False
flow lost
fllowing is detailed log:
457:2020-05-29 11:12:37.070 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Agent rpc_loop -
iteration:0 started rpc_loop
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2214
1912:2020-05-29 11:13:04.994 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Processing port:
a31f01a5-371a-4b44-8374-d87272d322ce treat_devices_added_or_updated
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1692
1914:2020-05-29 11:13:04.995 7 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Assigning 43 as local vlan
for net-id=c514bab4-139d-4eb0-8da2-3ab89938cbb2
11362:2020-05-29 11:17:20.688 7 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Cleaning stale br-ex flows
11393:2020-05-29 11:17:21.831 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:0 completed. Processed ports statistics: {'regular': {'updated': 0,
'added': 120, 'removed': 0}}. Elapsed:284.761 loop_count_and_wait
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2003
11396:2020-05-29 11:17:21.832 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:1 started rpc_loop
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2214
11399:2020-05-29 11:17:21.840 7 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Physical bridge br-ex was
just re-created.
12771:2020-05-29 11:19:15.353 7 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Error while processing VIF
ports: MessagingTimeout: Timed out waiting for a reply to message ID
df2deeed7f614e67abd42b088e0258e0
12811:2020-05-29 11:19:15.358 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:1 completed. Processed ports statistics: {'regular': {'updated': 0,
'added': 0, 'removed': 0}}. Elapsed:113.526 loop_count_and_wait
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2003
12813:2020-05-29 11:19:15.359 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:2 started rpc_loop
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2214
14372:2020-05-29 11:21:03.018 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Processing port:
a31f01a5-371a-4b44-8374-d87272d322ce treat_devices_added_or_updated
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1692
20013:2020-05-29 11:23:46.786 7 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Cleaning stale br-ex flows
20037:2020-05-29 11:23:47.709 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:2 completed. Processed ports statistics: {'regular': {'updated': 58,
'added': 120, 'removed': 0}}. Elapsed:272.350 loop_count_and_wait
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2003
20039:2020-05-29 11:23:47.710 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:3 started rpc_loop
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2214
20307:2020-05-29 11:23:49.963 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Processing port:
a31f01a5-371a-4b44-8374-d87272d322ce treat_devices_added_or_updated
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1692
21831:2020-05-29 11:24:27.000 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:3 completed. Processed ports statistics: {'regular': {'updated': 44,
'added': 44, 'removed': 0}}. Elapsed:39.290 loop_count_and_wait
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2003
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1881556
Title:
[ovs-agent]exception cause stale flow cleaned but new flow not
installed
Status in neutron:
New
Bug description:
kolla upgrade neutron
rpc_loop 1:
Physical bridge br-ex was just re-created,
set bridges_recreated=True
set need_clean_stale_flow=True
process_network_ports argument provisioning_needed is True but throw
exception, not call provision_local_vlan and flag need_clean_stale_flow keep
True
rpc_loop 2:
set bridges_recreated=False
process_network_ports argument provisioning_needed is False, and not call
provision_local_vlan
Cleaning stale flows and set need_clean_stale_flow = False
flow lost
fllowing is detailed log:
457:2020-05-29 11:12:37.070 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Agent rpc_loop -
iteration:0 started rpc_loop
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2214
1912:2020-05-29 11:13:04.994 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Processing port:
a31f01a5-371a-4b44-8374-d87272d322ce treat_devices_added_or_updated
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1692
1914:2020-05-29 11:13:04.995 7 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Assigning 43 as local vlan
for net-id=c514bab4-139d-4eb0-8da2-3ab89938cbb2
11362:2020-05-29 11:17:20.688 7 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Cleaning stale br-ex flows
11393:2020-05-29 11:17:21.831 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:0 completed. Processed ports statistics: {'regular': {'updated': 0,
'added': 120, 'removed': 0}}. Elapsed:284.761 loop_count_and_wait
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2003
11396:2020-05-29 11:17:21.832 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:1 started rpc_loop
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2214
11399:2020-05-29 11:17:21.840 7 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Physical bridge br-ex was
just re-created.
12771:2020-05-29 11:19:15.353 7 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Error while processing VIF
ports: MessagingTimeout: Timed out waiting for a reply to message ID
df2deeed7f614e67abd42b088e0258e0
12811:2020-05-29 11:19:15.358 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:1 completed. Processed ports statistics: {'regular': {'updated': 0,
'added': 0, 'removed': 0}}. Elapsed:113.526 loop_count_and_wait
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2003
12813:2020-05-29 11:19:15.359 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:2 started rpc_loop
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2214
14372:2020-05-29 11:21:03.018 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Processing port:
a31f01a5-371a-4b44-8374-d87272d322ce treat_devices_added_or_updated
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1692
20013:2020-05-29 11:23:46.786 7 INFO
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Cleaning stale br-ex flows
20037:2020-05-29 11:23:47.709 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:2 completed. Processed ports statistics: {'regular': {'updated': 58,
'added': 120, 'removed': 0}}. Elapsed:272.350 loop_count_and_wait
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2003
20039:2020-05-29 11:23:47.710 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:3 started rpc_loop
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2214
20307:2020-05-29 11:23:49.963 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -] Processing port:
a31f01a5-371a-4b44-8374-d87272d322ce treat_devices_added_or_updated
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1692
21831:2020-05-29 11:24:27.000 7 DEBUG
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-cc051aec-d6e6-4278-8944-13835ee1c47b - - - - -]Agent rpc_loop -
iteration:3 completed. Processed ports statistics: {'regular': {'updated': 44,
'added': 44, 'removed': 0}}. Elapsed:39.290 loop_count_and_wait
/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2003
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1881556/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
