** Also affects: neutron/kilo
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1538387
Title:
fdb_chg_ip_tun throwing exception because fdb_entries not in correct
format
Status in neutron:
Fix Released
Status in neutron kilo series:
New
Bug description:
I've been trying to track down failures in the DVR multinode job. I'm
now tripping over this one.
For context I've been focusing on a single change, but if you see a
failure in the gate-tempest-dsvm-neutron-dvr-multinode-full job you'll
probably be able to find similar info. This is the change:
http://logs.openstack.org/77/177777/4/check/gate-tempest-dsvm-neutron-
dvr-multinode-full/5abca7b/logs/
The screen-q-agt log shows a traceback here:
http://logs.openstack.org/77/177777/4/check/gate-tempest-dsvm-neutron-
dvr-multinode-
full/5abca7b/logs/screen-q-agt.txt.gz#_2016-01-18_10_11_29_715
<snip>
2016-01-18 10:11:29.724 12932 ERROR oslo_messaging.rpc.dispatcher File
"/opt/stack/new/neutron/neutron/plugins/ml2/drivers/l2pop/rpc_manager/l2population_rpc.py",
line 312, in fdb_chg_ip_tun
2016-01-18 10:11:29.724 12932 ERROR oslo_messaging.rpc.dispatcher
mac_ip.mac_address,
2016-01-18 10:11:29.724 12932 ERROR oslo_messaging.rpc.dispatcher
AttributeError: 'list' object has no attribute 'mac_address'
2016-01-18 10:11:29.724 12932 ERROR oslo_messaging.rpc.dispatcher
The info passed to fdb_chg_ip_tun() should have a "PortInfo"
namedtuple as data, but from the line before we can see it doesn't:
DEBUG neutron.plugins.ml2.drivers.l2pop.rpc_manager.l2population_rpc
[req-671e8634-c753-4002-acfd-68515dd44f29 None None]
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent.OVSNeutronAgent
method fdb_chg_ip_tun called with arguments (<neutron.context.Context
object at 0x7fa2ee6988d0>,
<neutron.plugins.ml2.drivers.openvswitch.agent.openflow.ovs_ofctl.br_tun.DeferredOVSTunnelBridge
object at 0x7fa2ee79abd0>, {u'c4ae0757-e3e5-419c-b2ba-4d7817964237':
{u'10.209.192.28': {u'before': [[u'fa:16:3e:8d:2e:48',
u'2003:0:0:1::1']]}}}, '10.208.193.94', {u'7ca0dcf2-fb63-4959-92ee-
cc757da8d120':
So from this it's clear that _unmarshall_fdb_entries() either hasn't
been called, or didn't do anything.
Looking over in screen-q-svc.log for the info before the RPC call
finds:
DEBUG neutron.plugins.ml2.drivers.l2pop.rpc [req-
f32790a5-0160-47b9-89b4-763b9c23bc08 tempest-
TestGettingAddress-2071048693 tempest-TestGettingAddress-1817548879]
Fanout notify l2population agents at q-agent-notifier the message
update_fdb_entries with {'chg_ip': {u'c4ae0757-e3e5-419c-b2ba-
4d7817964237': {u'10.208.193.94': {'before':
[PortInfo(mac_address=u'fa:16:3e:8d:2e:48',
ip_address=u'2003:0:0:1::1')]}}}} _notification_fanout
/opt/stack/new/neutron/neutron/plugins/ml2/drivers/l2pop/rpc.py:47
This is the message right before _marshall_fdb_entries() was called to
convert the PortInfo into [<mac>, <ip>] pairs, and from the above it
looks like it did.
I'm just starting to look at this now, but maybe someone more familiar
with l2pop has a guess at what's broken.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1538387/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
