Javier, The stack trace you provided indicates that the allocate network call from Compute (to nova-network) timed out. I would suggest checking the nova-network logs, that might give some insight. Also, IMO, the create instance request is being served by Compute means the RabbitMQ is working. You might want to ensure that nova-network has also connected to the same AMPQ server (as Compute is).
Regards, Unmesh G. On Tue, Feb 26, 2013 at 2:16 PM, Javier Alvarez <javier.alva...@bsc.es>wrote: > Hello JuanFra, > > How can I check that? I have rabbitmq-server running on the controller > node and I have set rabbit_host property in the .conf files. Also, if I > restart nova-compute service, I get the following message: > > 2013-02-26 09:43:52 INFO nova.rpc.common [-] Connected to AMQP server on > 172.16.8.1:5672 > > Is there any other test I could do to verify everything is properly > working? > > Thanks, > > Javi > > > On 25/02/13 20:09, JuanFra Rodriguez Cardoso wrote: > > Hi: > > Have you ensured your messaging queue is working correctly? > > Regards, > JuanFra > > > 2013/2/25 Javier Alvarez <javier.alva...@bsc.es> > >> Hello, >> >> I'm trying to setup OpenStack Essex on Debian and I'm getting an error >> when creating new VMs. The problem arises when trying to allocate the >> network of the VM, but I'm not sure why it's happening, I'm using >> nova-network, and that's the log output: >> >> 2013-02-25 18:20:28 AUDIT nova.compute.manager >> [req-4d211944-bd5c-4e88-a8df-c4a9167676e2 36a7e60b4d134307b61e06949d33735e >> 9558f53959a04cc992c3f8b6d91bfb9f] [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Starting instance... >> 2013-02-25 18:20:40 INFO nova.virt.libvirt.connection [-] Compute_service >> record updated for bscgrid21 >> 2013-02-25 18:20:40 INFO nova.compute.manager [-] Updating host status >> 2013-02-25 18:21:28 ERROR nova.rpc.common >> [req-4d211944-bd5c-4e88-a8df-c4a9167676e2 36a7e60b4d134307b61e06949d33735e >> 9558f53959a04cc992c3f8b6d91bfb9f] Timed out waiting for RPC response: timed >> out >> 2013-02-25 18:21:28 TRACE nova.rpc.common Traceback (most recent call >> last): >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 490, in >> ensure >> 2013-02-25 18:21:28 TRACE nova.rpc.common return method(*args, >> **kwargs) >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 567, in >> _consume >> 2013-02-25 18:21:28 TRACE nova.rpc.common return >> self.connection.drain_events(timeout=timeout) >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 167, in >> drain_events >> 2013-02-25 18:21:28 TRACE nova.rpc.common return >> self.transport.drain_events(self.connection, **kwargs) >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py", line 262, in >> drain_events >> 2013-02-25 18:21:28 TRACE nova.rpc.common return >> connection.drain_events(**kwargs) >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py", line 94, in >> drain_events >> 2013-02-25 18:21:28 TRACE nova.rpc.common return >> self.wait_multi(self.channels.values(), timeout=timeout) >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py", line 100, in >> wait_multi >> 2013-02-25 18:21:28 TRACE nova.rpc.common chanmap.keys(), >> allowed_methods, timeout=timeout) >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py", line 159, in >> _wait_multiple >> 2013-02-25 18:21:28 TRACE nova.rpc.common channel, method_sig, args, >> content = read_timeout(timeout) >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py", line 132, in >> read_timeout >> 2013-02-25 18:21:28 TRACE nova.rpc.common return >> self.method_reader.read_method() >> 2013-02-25 18:21:28 TRACE nova.rpc.common File >> "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/method_framing.py", >> line 221, in read_method >> 2013-02-25 18:21:28 TRACE nova.rpc.common raise m >> 2013-02-25 18:21:28 TRACE nova.rpc.common timeout: timed out >> 2013-02-25 18:21:28 TRACE nova.rpc.common >> 2013-02-25 18:21:28 ERROR nova.compute.manager >> [req-4d211944-bd5c-4e88-a8df-c4a9167676e2 36a7e60b4d134307b61e06949d33735e >> 9558f53959a04cc992c3f8b6d91bfb9f] [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Instance failed network setup >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Traceback (most recent call last): >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 571, in >> _allocate_network >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] requested_networks=requested_networks) >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 178, in >> allocate_for_instance >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] 'args': args}) >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/rpc/__init__.py", line 68, in call >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] return _get_impl().call(context, >> topic, msg, timeout) >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 674, in call >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] return rpc_amqp.call(context, >> topic, msg, timeout, Connection.pool) >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 343, in call >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] rv = list(rv) >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 304, in __iter__ >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] self.done() >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] self.gen.next() >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 301, in __iter__ >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] self._iterator.next() >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 572, in >> iterconsume >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] yield >> self.ensure(_error_callback, _consume) >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 503, in >> ensure >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] error_callback(e) >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 553, in >> _error_callback >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] raise rpc_common.Timeout() >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Timeout: Timeout while waiting on RPC >> response. >> 2013-02-25 18:21:28 TRACE nova.compute.manager [instance: >> 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] >> 2013-02-25 18:21:28 ERROR nova.rpc.amqp >> [req-4d211944-bd5c-4e88-a8df-c4a9167676e2 36a7e60b4d134307b61e06949d33735e >> 9558f53959a04cc992c3f8b6d91bfb9f] Exception during message handling >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp Traceback (most recent call last): >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 253, in >> _process_data >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp rval = >> node_func(context=ctxt, **node_args) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp return f(*args, **kw) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 183, in >> decorated_function >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp sys.exc_info()) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next() >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 177, in >> decorated_function >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp return function(self, >> context, instance_uuid, *args, **kwargs) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 663, in >> run_instance >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp do_run_instance() >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/utils.py", line 945, in inner >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp retval = f(*args, **kwargs) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 662, in >> do_run_instance >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp self._run_instance(context, >> instance_uuid, **kwargs) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 463, in >> _run_instance >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp >> self._set_instance_error_state(context, instance_uuid) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next() >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 436, in >> _run_instance >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp requested_networks) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 571, in >> _allocate_network >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp >> requested_networks=requested_networks) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 178, in >> allocate_for_instance >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp 'args': args}) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/__init__.py", line 68, in call >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp return >> _get_impl().call(context, topic, msg, timeout) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 674, in call >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp return rpc_amqp.call(context, >> topic, msg, timeout, Connection.pool) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 343, in call >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp rv = list(rv) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 304, in __iter__ >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp self.done() >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp self.gen.next() >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 301, in __iter__ >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp self._iterator.next() >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 572, in >> iterconsume >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp yield >> self.ensure(_error_callback, _consume) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 503, in >> ensure >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp error_callback(e) >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp File >> "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 553, in >> _error_callback >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp raise rpc_common.Timeout() >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp Timeout: Timeout while waiting on >> RPC response. >> 2013-02-25 18:21:28 TRACE nova.rpc.amqp >> >> Thanks in advance, >> >> Javi >> >> -- >> Javier Álvarez Cid-Fuentes >> Grid Computing and Clusters Group >> Barcelona Supercomputing Center (BSC-CNS) >> Tel. (+34) 93 413 72 46 >> >> >> WARNING / LEGAL TEXT: This message is intended only for the use of the >> individual or entity to which it is addressed and may contain >> information which is privileged, confidential, proprietary, or exempt >> from disclosure under applicable law. If you are not the intended >> recipient or the person responsible for delivering the message to the >> intended recipient, you are strictly prohibited from disclosing, >> distributing, copying, or in any way using this message. If you have >> received this communication in error, please notify the sender and >> destroy and delete any copies you may have received. >> >> http://www.bsc.es/disclaimer >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp >> > > > > -- > Javier Álvarez Cid-Fuentes > Grid Computing and Clusters Group > Barcelona Supercomputing Center (BSC-CNS) > Tel. (+34) 93 413 72 46 > > > > WARNING / LEGAL TEXT: This message is intended only for the use of the > individual or entity to which it is addressed and may contain information > which is privileged, confidential, proprietary, or exempt from disclosure > under applicable law. If you are not the intended recipient or the person > responsible for delivering the message to the intended recipient, you are > strictly prohibited from disclosing, distributing, copying, or in any way > using this message. If you have received this communication in error, > please notify the sender and destroy and delete any copies you may have > received. > > http://www.bsc.es/disclaimer <http://www.bsc.es/disclaimer.htm> > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > >
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
