trace of error log when nova-compute service was tried to re-started
sudo cat /var/log/nova/nova-compute.log 2013-04-16 06:36:46 CRITICAL nova [-] Unexpected error while running command. Command: sudo nova-rootwrap ip link add link eth2 name vlan100 type vlan id 100 Exit code: 1 Stdout: '' Stderr: 'Cannot find device "eth2"\n' 2013-04-16 06:36:46 TRACE nova Traceback (most recent call last): 2013-04-16 06:36:46 TRACE nova File "/usr/bin/nova-network", line 49, in <module> 2013-04-16 06:36:46 TRACE nova service.wait() 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 413, in wait 2013-04-16 06:36:46 TRACE nova _launcher.wait() 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 131, in wait 2013-04-16 06:36:46 TRACE nova service.wait() 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 166, in wait 2013-04-16 06:36:46 TRACE nova return self._exit_event.wait() 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait 2013-04-16 06:36:46 TRACE nova return hubs.get_hub().switch() 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 177, in switch 2013-04-16 06:36:46 TRACE nova return self.greenlet.switch() 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in main 2013-04-16 06:36:46 TRACE nova result = function(*args, **kwargs) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 101, in run_server 2013-04-16 06:36:46 TRACE nova server.start() 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 162, in start 2013-04-16 06:36:46 TRACE nova self.manager.init_host() 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 1766, in init_host 2013-04-16 06:36:46 TRACE nova NetworkManager.init_host(self) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 758, in init_host 2013-04-16 06:36:46 TRACE nova self._setup_network_on_host(ctxt, network) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 1845, in _setup_network_on_host 2013-04-16 06:36:46 TRACE nova self.l3driver.initialize_gateway(network) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/l3.py", line 98, in initialize_gateway 2013-04-16 06:36:46 TRACE nova gateway=(network_ref['gateway'] is not None)) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 896, in plug 2013-04-16 06:36:46 TRACE nova return _get_interface_driver().plug(network, mac_address, gateway) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 935, in plug 2013-04-16 06:36:46 TRACE nova mac_address) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 958, in ensure_vlan_bridge 2013-04-16 06:36:46 TRACE nova bridge_interface, mac_address) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 987, in inner 2013-04-16 06:36:46 TRACE nova retval = f(*args, **kwargs) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 971, in ensure_vlan 2013-04-16 06:36:46 TRACE nova 'id', vlan_num, run_as_root=True) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 813, in _execute 2013-04-16 06:36:46 TRACE nova return utils.execute(*cmd, **kwargs) 2013-04-16 06:36:46 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 246, in execute 2013-04-16 06:36:46 TRACE nova cmd=' '.join(cmd)) 2013-04-16 06:36:46 TRACE nova ProcessExecutionError: Unexpected error while running command. 2013-04-16 06:36:46 TRACE nova Command: sudo nova-rootwrap ip link add link eth2 name vlan100 type vlan id 100 2013-04-16 06:36:46 TRACE nova Exit code: 1 2013-04-16 06:36:46 TRACE nova Stdout: '' 2013-04-16 06:36:46 TRACE nova* Stderr: 'Cannot find device "eth2"\n'* 2013-04-16 06:36:46 TRACE nova On Tue, Apr 16, 2013 at 3:43 PM, Wangpan <hzwang...@corp.netease.com> wrote: > ** > you may need to check the log of nova-network, may be exist at > /var/log/nova/nova-network.log, look into it and check if any exception > there. > if nothing there, you may run 'nova-network' as nova user, and check the > output on shell. > 2013-04-16 > ------------------------------ > Wangpan > ------------------------------ > *发件人：*Deepak A.P > *发送时间：*2013-04-16 18:01 > *主题：*Re: Re: Re: Re: [Openstack] Fwd: Re: [openStack] instance status > *收件人：*"Wangpan"<hzwang...@corp.netease.com> > *抄送：*"openstack-operat...@lists.openstack.org"< > openstack-operat...@lists.openstack.org>,"OpenStack Mailing List"< > openstack@lists.launchpad.net> > > ok will go ahead and delete the existing stuck instance and re-create a > new instance with tiny flavor. > > Am getting another issue , however how many times i restart the nova > services , the 'nova-network' service is not starting as below > > openstack@openStack:~$ ls /etc/init/nova-* | cut -d '/' -f4 | cut -d '.' > -f1 | while read S; do sudo status $S; done > nova-api start/running, process 923 > nova-cert start/running, process 32736 > nova-compute start/running, process 32752 > nova-network *stop/waiting* > nova-objectstore start/running, process 29539 > nova-scheduler start/running, process 316 > > > i tried stopping and re-starting the 'nova-newtwork' individually , but > still the status shows '*stop/waiting' *for 'nova-compute' what could > be the problem ? > * > * > > > > > On Tue, Apr 16, 2013 at 3:23 PM, Wangpan <hzwang...@corp.netease.com>wrote: > >> ** >> delete the building state instances and re-create them may be better. >> >> 2013-04-16 >> ------------------------------ >> Wangpan >> ------------------------------ >> *发件人：*Deepak A.P >> *发送时间：*2013-04-16 17:47 >> *主题：*Re: Re: Re: [Openstack] Fwd: Re: [openStack] instance status >> *收件人：*"Wangpan"<hzwang...@corp.netease.com> >> *抄送：*"Aaron Rosen"<aro...@nicira.com>," >> openstack-operat...@lists.openstack.org"< >> openstack-operat...@lists.openstack.org>,"OpenStack Mailing List"< >> openstack@lists.launchpad.net> >> >> yes i re-started all the services >> >> openstack@openStack:~$ ls /etc/init/nova-* | cut -d '/' -f4 | cut -d '.' >> -f1 | while read S; do sudo start $S; done >> start: Job is *already running:* nova-api >> nova-cert *start/running*, process 32736 >> nova-compute *start/running*, process 32752 >> nova-network *start/running,* process 32767 >> start: *Job is already running*: nova-objectstore >> nova-scheduler* start/running*, process 316 >> >> >> Should i create a new instance or reload , reboot the existing instance ? >> >> >> >> >> >> >> On Tue, Apr 16, 2013 at 2:41 PM, Wangpan <hzwang...@corp.netease.com>wrote: >> >>> ** >>> nova-cert nova-compute nova-network nova-scheduler are all stopped, >>> >>> nova-api start/running, process 923 ---this is running >>> nova-cert stop/waiting ---this is stop >>> nova-compute stop/waiting ---this is stop >>> nova-network stop/waiting ---this is stop >>> nova-objectstore start/running, process 29539 ---this is running >>> nova-scheduler stop/waiting ---this is stop >>> >>> 2013-04-16 >>> ------------------------------ >>> Wangpan >>> ------------------------------ >>> *发件人：*Deepak A.P >>> *发送时间：*2013-04-16 16:58 >>> *主题：*Re: Re: [Openstack] Fwd: Re: [openStack] instance status >>> *收件人：*"Wangpan"<hzwang...@corp.netease.com> >>> *抄送：*"Aaron Rosen"<aro...@nicira.com>," >>> openstack-operat...@lists.openstack.org"< >>> openstack-operat...@lists.openstack.org>,"OpenStack Mailing List"< >>> openstack@lists.launchpad.net> >>> >>> All the nova service are running as u can see it below >>> >>> openstack@openStack:~$ ls /etc/init/nova-* | cut -d '/' -f4 | cut -d >>> '.' -f1 | while read S; do sudo status $S; done >>> [sudo] password for openstack: >>> nova-api start/running, process 923 >>> nova-cert stop/waiting >>> nova-compute stop/waiting >>> nova-network stop/waiting >>> nova-objectstore start/running, process 29539 >>> nova-scheduler stop/waiting >>> >>> >>> and the glance service >>> >>> openstack@openStack:~$ ls /etc/init/glance-* | cut -d '/' -f4 | cut -d >>> '.' -f1 | while read S; do sudo status $S; done >>> glance-api start/running, process 882 >>> glance-registry start/running, process 879 >>> >>> >>> On Tue, Apr 16, 2013 at 2:22 PM, Wangpan <hzwang...@corp.netease.com>wrote: >>> >>>> ** >>>> Looks like all of your nova sevices are stopped runing, and what the >>>> result of 'service nova-compute status'? >>>> >>>> 2013-04-16 >>>> ------------------------------ >>>> Wangpan >>>> ------------------------------ >>>> *发件人：*Deepak A.P >>>> *发送时间：*2013-04-16 14:36 >>>> *主题：*Re: [Openstack] Fwd: Re: [openStack] instance status >>>> *收件人：*"Aaron Rosen"<aro...@nicira.com> >>>> *抄送：*"openstack-operat...@lists.openstack.org"< >>>> openstack-operat...@lists.openstack.org>,"OpenStack Mailing List"< >>>> openstack@lists.launchpad.net> >>>> >>>> openstack@openStack:~$ sudo service ntp status >>>> * NTP server is running >>>> >>>> >>>> On Fri, Apr 12, 2013 at 11:22 AM, Aaron Rosen <aro...@nicira.com>wrote: >>>> >>>>> Do you have NTP configured? If the nodes running nova-compute have >>>>> clocks that differ from each other the status shows XXX . (Not sure why >>>>> it's done this way though). >>>>> >>>>> Aaron >>>>> >>>>> >>>>> On Thu, Apr 11, 2013 at 10:44 PM, Deepak A.P < >>>>> swift007.dee...@gmail.com> wrote: >>>>> >>>>>> >>>>>> >>>>>> ---------- Forwarded message ---------- >>>>>> From: Deepak A.P <swift007.dee...@gmail.com> >>>>>> Date: Tue, Apr 9, 2013 at 2:25 PM >>>>>> Subject: Re: Re: [Openstack] [openStack] instance status >>>>>> To: Wangpan <hzwang...@corp.netease.com> >>>>>> >>>>>> >>>>>> Hello, >>>>>> >>>>>> The output of 'nova-manage service list' command is >>>>>> >>>>>> >>>>>> >>>>>> * Binary Host Zone >>>>>> Status State >>>>>> Updated_At* >>>>>> >>>>>> nova-compute openStack nova >>>>>> Enabled XXX 2012-04-03 >>>>>> >>>>>> nova-network openStack nova >>>>>> Enabled XXX 2012-04-03 >>>>>> >>>>>> nova-scheduler openStack nova >>>>>> Enabled XXX 2012-04-05 >>>>>> >>>>>> nova-cert openStack nova >>>>>> Enabled XXX 2012-04-05 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Apr 9, 2013 at 12:21 PM, Wangpan >>>>>> <hzwang...@corp.netease.com>wrote: >>>>>> >>>>>>> ** >>>>>>> What the result of running 'nova-manage service list'， 'sudo' may be >>>>>>> needed by non-root user. >>>>>>> >>>>>>> 2013-04-09 >>>>>>> ------------------------------ >>>>>>> Wangpan >>>>>>> ------------------------------ >>>>>>> *发件人：*Deepak A.P >>>>>>> *发送时间：*2013-04-09 13:54 >>>>>>> *主题：*Re: [Openstack] [openStack] instance status >>>>>>> *收件人：*"sachin tripathi"<sachinku...@gmail.com> >>>>>>> *抄送：*"OpenStack Mailing List"<openstack@lists.launchpad.net> >>>>>>> >>>>>>> Below is output which i got on running '*nova console-log*' >>>>>>> >>>>>>> nova console-log *d46530e2-.....* >>>>>>> >>>>>>> Error: The server has either erred or either incapable of performing >>>>>>> the requested operation. HTTP(500) (Request-ID: req-be41c539.......) >>>>>>> >>>>>>> i tried the other nova commands like 'nova reboot' , ' nova reload' >>>>>>> but got the similar error, >>>>>>> >>>>>>> and logs are not created under '/var/log/nova/' >>>>>>> >>>>>>> stuck with the above issue >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Apr 9, 2013 at 10:32 AM, sachin tripathi < >>>>>>> sachinku...@gmail.com> wrote: >>>>>>> >>>>>>>> Hello, >>>>>>>> You can check the console logs >>>>>>>> nova console-log <instance_id> >>>>>>>> >>>>>>>> On the api node, you can find /var/log/nova/nova-*.log, >>>>>>>> >>>>>>>> And from "nova show <instance_id> " get the hypervisor hostname >>>>>>>> and check the hypervisor compute-log too. >>>>>>>> >>>>>>>> Hope, this will give some info to start troubleshoot. >>>>>>>> >>>>>>>> +Sachin >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Apr 9, 2013 at 10:03 AM, Deepak A.P < >>>>>>>> swift007.dee...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> After launching the instance i waited for long time , but >>>>>>>>> still the status of the instance shows 'BUILD' , the flavor of the >>>>>>>>> image >>>>>>>>> is 'm1.small' . Is there any work around to know what's happening >>>>>>>>> with the >>>>>>>>> >>>>>>>>> instance ?. Are there any logs where i can check the >>>>>>>>> instance launch status ? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Apr 6, 2013 at 12:01 AM, Lloyd Dewolf < >>>>>>>>> lloydost...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> On Fri, Apr 5, 2013 at 4:18 AM, Deepak A.P < >>>>>>>>>> swift007.dee...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi , >>>>>>>>>>> >>>>>>>>>>> i have a list of instances created using the below command >>>>>>>>>>> >>>>>>>>>>> nova boot myInstance --image >>>>>>>>>>> 0e2f43a8-e614-48ff-92bd-be0c68da19f4 >>>>>>>>>>> >>>>>>>>>>> --flavor 2 --key_name openstack >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> i ran the below command to check the status of instances >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> nova list >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> all the instances show status as "*BUILD*" , how to se the status >>>>>>>>>>> of the image to >>>>>>>>>>> >>>>>>>>>>> "ACTIVE " , i tried rebooting the instance am getting the below >>>>>>>>>>> error >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Once the instance finishes building then it will be in the >>>>>>>>>> "active" state. Depending on the image, flavor and configuration >>>>>>>>>> starting >>>>>>>>>> an instance can take a long time. I would suggest first trying with >>>>>>>>>> a small >>>>>>>>>> image like Cirrus and using a "tiny" flavor. >>>>>>>>>> >>>>>>>>>> http://docs.openstack.org/trunk/openstack-compute/admin/content/starting-images.html >>>>>>>>>> >>>>>>>>>> Hope that helps, >>>>>>>>>> -- >>>>>>>>>> @lloyddewolf >>>>>>>>>> http://www.pistoncloud.com/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Cheers, >>>>>>>>> Deepak >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Mailing list: https://launchpad.net/~openstack >>>>>>>>> Post to : openstack@lists.launchpad.net >>>>>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>>>>> More help : https://help.launchpad.net/ListHelp >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Cheers, >>>>>>> Deepak >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Cheers, >>>>>> Deepak >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Cheers, >>>>>> Deepak >>>>>> >>>>>> _______________________________________________ >>>>>> Mailing list: https://launchpad.net/~openstack >>>>>> Post to : openstack@lists.launchpad.net >>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>> More help : https://help.launchpad.net/ListHelp >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Cheers, >>>> Deepak >>>> >>> >>> >>> >>> -- >>> Cheers, >>> Deepak >>> >> >> >> >> -- >> Cheers, >> Deepak >> > > > > -- > Cheers, > Deepak > -- Cheers, Deepak
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
