I'm having the similar issue, only happens after live migrated instance
to other host. In dashboard, it shows the port with floating IP is
detached and status n/a, under instance tab, it shows the floating IP is
associated with instance.
nova compute showed successfully added bridge and veth, l3-agent showed
successfully added namespace but failed doing arping.
George Zhao
On 08/11/2016 07:18 AM, Jonathan Mills wrote:
Hi all,
I’m running Mitaka on CentOS 7.2 with Neutron in dvr_snat mode.
# uname -msr
Linux 3.10.0-327.22.2.el7.x86_64 x86_64
I’m using vlans, not vxlans, but I don’t think that matters either way.
So basically, I have one NIC “eth2” which is in vlan trunk mode, and on
my switch side, I have every neutron-defined vlan trunked there.
Whether it’s a tenant network vlan, or an external vlan for floating
IPs, it all comes back to that same NIC.
So here’s a compute node “node1”. It has a successfully booted VM,
which has fixed IP 10.97.8.103 and floating IP 10.96.8.107. As seen
from the compute node:
# ip netns
fip-cbe55dc5-c4e4-4ec0-aa52-b4713f1279ee
qrouter-efc60192-97ad-49ef-bab7-cda42ca6bc29
snat-efc60192-97ad-49ef-bab7-cda42ca6bc29
# ip netns exec fip-cbe55dc5-c4e4-4ec0-aa52-b4713f1279ee ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: fpr-efc60192-9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UP qlen 1000
link/ether 32:06:67:df:53:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 169.254.109.47/31 <http://169.254.109.47/31> scope global
fpr-efc60192-9
valid_lft forever preferred_lft forever
inet6 fe80::3006:67ff:fedf:53c6/64 scope link
valid_lft forever preferred_lft forever
19: fg-152dc56a-c1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UNKNOWN
link/ether fa:16:3e:40:9f:5b brd ff:ff:ff:ff:ff:ff
inet 10.96.8.101/23 <http://10.96.8.101/23> brd 10.96.9.255 scope
global fg-152dc56a-c1
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe40:9f5b/64 scope link
valid_lft forever preferred_lft forever
# ip netns exec qrouter-efc60192-97ad-49ef-bab7-cda42ca6bc29 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: rfp-efc60192-9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UP qlen 1000
link/ether 72:49:e7:78:48:5d brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 169.254.109.46/31 <http://169.254.109.46/31> scope global
rfp-efc60192-9
valid_lft forever preferred_lft forever
inet 10.96.8.107/32 <http://10.96.8.107/32> brd 10.96.8.107 scope
global rfp-efc60192-9
valid_lft forever preferred_lft forever
inet6 fe80::7049:e7ff:fe78:485d/64 scope link
valid_lft forever preferred_lft forever
17: qr-ffc302ba-82: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UNKNOWN
link/ether fa:16:3e:8d:7c:62 brd ff:ff:ff:ff:ff:ff
inet 10.97.8.1/23 <http://10.97.8.1/23> brd 10.97.9.255 scope global
qr-ffc302ba-82
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe8d:7c62/64 scope link
valid_lft forever preferred_lft forever
So you can see that I have both the ‘fpr’ and ‘rfp’ namespaces, which
is a good indicator I didn’t totally flub the dvr_snat neutron config.
From within either namespace, I can ping the floating IP 10.96.8.107,
which makes sense. However, for the floating IP to be useful, it would
need to be generally reachable by any other system in its designated
vlan, and that is not the case. In my real-world use case, I would be
running the vlan of this floating IP network back over to my bastion
host, to allow users to ssh into their VMs via the floating IP. I can’t
reach the floating IPs though from anywhere outside the namespace on the
compute node.
One more clue, in the l3-agent log on the compute node in question:
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib [-] Failed
sending gratuitous ARP to 10.96.8.107 on fg-152dc56a-c1 in namespace
fip-cbe55dc5-c4e4-4ec0-aa52-b4713f1279ee
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib Traceback
(most recent call last):
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib File
"/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line
1040, in _arping
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib
ip_wrapper.netns.execute(arping_cmd, check_exit_code=True)
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib File
"/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line
927, in execute
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib
log_fail_as_error=log_fail_as_error, **kwargs)
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib File
"/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line
140, in execute
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib raise
RuntimeError(msg)
2016-08-03 11:14:09.665 6041 ERROR neutron.agent.linux.ip_lib
RuntimeError: Exit code: 2; Stdin: ; Stdout: ; Stderr: bind: Cannot
assign requested address
After a little Googling, I think I may be seeing the same behavior as
this user:
https://bugs.centos.org/view.php?id=11238
I’m reaching out to see if anyone else has witnessed this, or has any
sage advice for me.
Jonathan
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
