Hi, We have the following setup: Folsom version 1 controller running all apis and also nova-compute 1 compute node only running nova-compute and nova-network
Problem is that when we reboot the controller the compute node reconnects successfully according to the nova-compute and nova-network logs, but it won't accept any messages from rabbitmq. Seems to be that the consumer didn't actually reconnect. I noticed that in impl_kombu.py:Connection._connect on line 457 the variable self.consumers is an empty list. I'm not sure if this is normal. When launching an instance the compute node never receives the message from rabbitmq. If we use durable queues the message will stay in the queue until nova-compute is manually restarted. And if we don't use durable queues the queue for the compute node doesn't get declared/created at all. Another side note is that if we only stop rabbitmq server for 5 minutes and later start it again. Everything works fine. So we only have this problem when executing the command: $ reboot on the controller. The logs look like it's normal, but in fact it's not see nova-compute.log: 2013-08-29 15:38:49 27818 INFO nova.openstack.common.rpc.common [-] Reconnecting to AMQP server on 172.16.200.1:5672 2013-08-29 15:38:49 27818 ERROR nova.openstack.common.rpc.common [-] AMQP server on 172.16.200.1:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. 2013-08-29 15:39:03 27818 ERROR nova.service [-] Recovered model server connection! 2013-08-29 15:39:03 27818 DEBUG nova.service [-] heart_beat rpc send... report_state /usr/lib/python2.7/dist-packages/nova/service.py:558 2013-08-29 15:39:13 27818 DEBUG nova.service [-] heart_beat rpc send... report_state /usr/lib/python2.7/dist-packages/nova/service.py:558 2013-08-29 15:39:19 27818 INFO nova.openstack.common.rpc.common [-] Reconnecting to AMQP server on 172.16.200.1:5672 2013-08-29 15:39:19 27818 INFO nova.openstack.common.rpc.common [-] Connected to AMQP server on 172.16.200.1:5672 Anybody any suggestions on where to look or any hot fixes that can be applied? Regards, Sam Stoelinga
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
