Hi, I manually deployed OpenStack with one controller and several computes, which is working well for months. Recently I get one issue after server power off and on again due to high temperate warning.
Issue it neutron-linuxbridge-agent service on compute nodes failed to get messages from rabbitmq on controller nodes. The error log is like follows: 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent [-] Failed reporting state! 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last): 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 130, in _report_state 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent True) 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 88, in report_state 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent return method(context, 'report_state', **kwargs) 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 169, in call 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 97, in _send 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, retry=retry) 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 464, in send 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent retry=retry) 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 453, in _send 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent result = self._waiter.wait(msg_id, timeout) 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 336, in wait 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent message = self.waiters.get(msg_id, timeout=timeout) 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 238, in get 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent 'to message ID %s' % msg_id) 2017-08-01 11:13:45.435 44071 ERROR neutron.plugins.ml2.drivers.agent._common_agent MessagingTimeout: Timed out waiting for a reply to message ID 6c54bbf0b01d49eaa8babfdd5f9ba80 This is what I configured in /etc/neutron/neutron.conf for rabbitmq connection. * transport_url = rabbit://openstack:<password>@controller And from controller I see the user is valid: root@controller:~# rabbitmqctl list_users Listing users ... guest [administrator] openstack [administrator] root@controller:~# rabbitmqctl list_user_permissions openstack Listing permissions for user "openstack" ... / .* .* .* And from rabbitmq management GUI, I see there are connections established between controller and compute. What could cause this problem then? And how can I get further? Thanks. Best Regards Xu Rongjie (Max)
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
