Are there any errors in instance-00000001.log and other log files? Try to change virt_type = kvm to virt_type = qemu in /etc/nova/nova.conf to exclude kvm problems
On Mon, Jun 9, 2014 at 2:28 PM, Deepak Shetty <dpkshe...@gmail.com> wrote: > Hi sergey > Don't see the .log in qemu/instnace-XXX at all > > But there is more... > > [stack@devstack-large-vm ~]$ [ready] nova show > *bc598be6-999b-4ab9-8a0a-1c4d29dd811a* > > +--------------------------------------+----------------------------------------------------------------+ > | Property | > Value | > > +--------------------------------------+----------------------------------------------------------------+ > | OS-DCF:diskConfig | > MANUAL | > | OS-EXT-AZ:availability_zone | > nova | > | OS-EXT-SRV-ATTR:host | > devstack-large-vm.localdomain | > | OS-EXT-SRV-ATTR:hypervisor_hostname | > devstack-large-vm.localdomain | > *| OS-EXT-SRV-ATTR:instance_name | instance-0000000a * > | > | OS-EXT-STS:power_state | > 0 | > | OS-EXT-STS:task_state | > spawning | > | OS-EXT-STS:vm_state | > building | > | OS-SRV-USG:launched_at | > - | > | OS-SRV-USG:terminated_at | > - | > | accessIPv4 > | | > | accessIPv6 > | | > | config_drive > | | > | created | > 2014-06-09T10:18:53Z | > | flavor | m1.tiny > (1) | > | hostId | > 393d36ca7cc7c30957286e775b5808c8103b4b7be1af4f2163cc2fe4 | > | id | > bc598be6-999b-4ab9-8a0a-1c4d29dd811a | > | image | cirros-0.3.2-x86_64-uec > (99471623-273c-46ed-b1b0-5a58605aab76) | > | key_name | > - | > | metadata | > {} | > | name | > mynewvm | > | os-extended-volumes:volumes_attached | > [] | > | private network | > 10.0.0.2 | > | progress | > 0 | > | security_groups | > default | > | status | > BUILD | > | tenant_id | > 1d0dedb6e6f344258a1ab44df3fcd4ee | > | updated | > 2014-06-09T10:18:53Z | > | user_id | > 730ccd3aaf7140789f4929edd71d3d31 | > > +--------------------------------------+----------------------------------------------------------------+ > [stack@devstack-large-vm ~]$ [ready] ps aux| grep qemu| grep instance > stack 11388 99.1 0.9 1054588 36884 ? Sl 10:18 4:03 > /usr/bin/qemu-system-x86_64 -machine accel=kvm *-name > guestfs-tgx65m4jvhgmu8la* -S -machine pc-i440fx-1.6,accel=kvm,usb=off > -cpu host -m 500 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 > -uuid 6bd85030-c2a0-4576-82d2-ff30a9e44024 -nographic -no-user-config > -nodefaults -chardev > socket,id=charmonitor,path=/opt/stack/.config/libvirt/qemu/lib/guestfs-tgx65m4jvhgmu8la.monitor,server,nowait > -mon chardev=charmonitor,id=monitor,mode=control -rtc > base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet > -no-reboot -no-acpi -kernel /var/tmp/.guestfs-1001/appliance.d/kernel > -initrd /var/tmp/.guestfs-1001/appliance.d/initrd -append panic=1 > console=ttyS0 udevtimeout=600 no_timer_check acpi=off printk.time=1 > cgroup_disable=memory root=/dev/sdb selinux=0 TERM=screen -device > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device > virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -device > virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 *-drive > file=/opt/stack/data/nova/instances/bc598be6-999b-4ab9-8a0a-1c4d29dd811a/disk*,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=writeback > -device > scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 > -drive > file=/tmp/libguestfsagFFSQ/overlay1,if=none,id=drive-scsi0-0-1-0,format=qcow2,cache=unsafe > -device > scsi-hd,bus=scsi0.0,channel=0,scsi-id=1,lun=0,drive=drive-scsi0-0-1-0,id=scsi0-0-1-0 > -chardev socket,id=charserial0,path=/tmp/libguestfsagFFSQ/console.sock > -device isa-serial,chardev=charserial0,id=serial0 -chardev > socket,id=charchannel0,path=/tmp/libguestfsagFFSQ/guestfsd.sock -device > virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.libguestfs.channel.0 > -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 > > [root@devstack-large-vm qemu]# ls -lt > total 84 > -rw-------. 1 root root 8279 Jun 9 09:59 instance-00000001.log > -rw-------. 1 root root 2070 Jun 9 09:29 instance-00000007.log > -rw-------. 1 root root 63444 Jun 6 09:20 instance-00000004.log > -rw-------. 1 root root 2071 Jun 6 07:12 instance-00000002.log > [root@devstack-large-vm qemu]# date > Mon Jun 9 10:24:32 UTC 2014 > > ^^ No instance with 0a name above ^^ > > So the qemu process above is using the disk from my instance but its -name > is not instance-XXXXX tho' > *OS-EXT-SRV-ATTR:instance_name | instance-0000000a * > show 0a as instance name > > virsh shows it as running... > > [stack@devstack-large-vm ~]$ [ready] virsh list > Id Name State > ---------------------------------------------------- > * 2 guestfs-tgx65m4jvhgmu8la running* > > thanx, > deepak > > > On Mon, Jun 9, 2014 at 3:50 PM, Sergey Kolekonov <skoleko...@mirantis.com> > wrote: > >> Hi, >> >> have you inspected /var/log/libvirt/qemu/instance-xxxxxxxx.log files? >> Such problem can be connected with your hypervisor. Especially if you use >> nested kvm virtualization on your host machine. >> >> >> On Mon, Jun 9, 2014 at 2:06 PM, Deepak Shetty <dpkshe...@gmail.com> >> wrote: >> >>> (Hit send by mistake.. continuing below) >>> >>> >>> On Mon, Jun 9, 2014 at 3:32 PM, Deepak Shetty <dpkshe...@gmail.com> >>> wrote: >>> >>>> Hi All, >>>> The last time I sent this issue, I didn't had the right tags in the >>>> subject so sending with the right tags, in the hope that the right folks >>>> might help provide some clues. >>>> >>>> I am usign latest devstack on F20 and I see VMs stuck in 'spawning' >>>> state. >>>> No errors in n-sch, n-cpu and I checked other n-* screen logs too, no >>>> errors whatsoever >>>> >>>> There is enuf memory and disk space available for the tiny and small >>>> flavour VMs I am trying to run >>>> >>>> >>> 2014-06-09 09:57:19.935 AUDIT nova.compute.resource_tracker [-] Free ram >>> (MB): 1905 >>> 2014-06-09 09:57:19.935 AUDIT nova.compute.resource_tracker [-] Free >>> disk (GB): 46 >>> 2014-06-09 09:57:19.936 AUDIT nova.compute.resource_tracker [-] Free >>> VCPUS: 1 >>> 2014-06-09 09:57:19.936 AUDIT nova.compute.resource_tracker [-] PCI >>> stats: [] >>> >>> The only thing that i see other than INFO/DEbug is the below >>> 2014-06-09 09:57:42.700 WARNING nova.compute.manager [-] Found 3 in the >>> database and 0 on the hypervisor. >>> >>> What does that mean ? Does that say that Nova sees 3 VMs in its DB but a >>> query to the hyp (via libvirt I assume) returns none ? >>> >>> Interestingly I can see the qemu process for my VM that nova says is >>> stuck in Spawning state and virsh list also lists' it >>> But i don't see the libvirt.xml for this stuck VM (from nova's >>> perspective) in the /opt/stack/data/nova/instances/<uuid> folder. >>> >>> I was trying to figure what this means ? Where is nova stuck ? Screen >>> logs just doesn't point to anything useful to debug further. >>> >>> From the n-cpu logs i see the "Creating image" info log from >>> nova.virt.libvirt and thats the last >>> I don't see the info log "Creating isntance" from nova.virt.. so it >>> looks like its stuck somewhere between creating image and spawning >>> instance.. but virsh list and qemu process says otherwise >>> >>> Looking for some debug hints here >>> >>> thanx, >>> deepak >>> >>> _______________________________________________ >>> Mailing list: >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>> Post to : openstack@lists.openstack.org >>> Unsubscribe : >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>> >>> >> >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack