username: vdsm@ovirrt, password: shibboleth
On Thu, Apr 8, 2021 at 5:25 PM David Kerry <[email protected]> wrote: > > Hi Shani, > > I actually came across that option and attempted it at one point, > but vdsm has locked me out of using that command it seems. > > Eg: > > [root@ovirt-node217 ~]# virsh undefine vm-s2 > Please enter your authentication name: admin > Please enter your password: > error: failed to connect to the hypervisor > error: authentication failed: authentication failed > > No known username/password seems to work. > > Is there some magic user to use for this, or some way > to bypass the authentication? > > Thanks > > David > > On 2021-04-08 10:10 a.m., Shani Leviim wrote: > > Hi David, > > Yes - this one will remove completely the VM from the DB. > > > > You can use the virsh command to delete the VM guests: > > https://www.cyberciti.biz/faq/howto-linux-delete-a-running-vm-guest-on-kvm/ > > <https://www.cyberciti.biz/faq/howto-linux-delete-a-running-vm-guest-on-kvm/> > > > > *Regards, > > * > > *Shani Leviim > > * > > > > > > On Thu, Apr 8, 2021 at 4:32 PM David Kerry <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi Shani, > > > > These VMs in particular are running just fine on other hosts (and > > I'd like to keep them that way, preferably). > > > > It looks like this command would delete the whole VM from the > > entire system instead of just removing the stuck/shutdown instances > > from the hosts it's not running on any more. > > > > Can you confirm this is what it would do? If so, is there another > > option to remove these stuck "ghost" VM instances from the hosts they > > are > > no longer running on? > > > > > > Thanks > > > > David > > > > > > On 2021-04-08 3:20 a.m., Shani Leviim wrote: > > > Hi David, > > > You can delete the VM from the DB using this command: > > > SELECT DeleteVm('<vm-id>'); > > > > > > *Regards, > > > * > > > *Shani Leviim > > > * > > > > > > > > > On Wed, Apr 7, 2021 at 4:23 PM David Kerry <[email protected] > > <mailto:[email protected]> <mailto:[email protected] > > <mailto:[email protected]>>> wrote: > > > > > > Hello, > > > > > > This seems to be what the engine is trying to do, and failing at > > for some reason. > > > > > > eg: > > > > > > [root@ovirt-node217 ~]# vdsm-client Host getVMList > > fullStatus=True > > > [ > > > "8b3964bc-cd3f-4f13-84c6-1811193c93eb", > > > "132668b6-9992-451f-95ac-dbcbeb03f5f1" > > > ] > > > > > > For reference: > > > > > > [root@ovirt-node217 ~]# virsh -r list --all > > > Id Name State > > > ---------------------------------------------------- > > > - vm-s2 shut off > > > - vm-s1 shut off > > > > > > And in the console, it shows a count of "2" beside this host, > > but on the host detail > > > page, under the virtual-machine tab, the list is empty (these > > VMs are actually > > > running on a different host). > > > > > > [root@ovirt-node217 ~]# vdsm-client VM destroy > > vmID="8b3964bc-cd3f-4f13-84c6-1811193c93eb" > > > vdsm-client: Command VM.destroy with args {'vmID': > > '8b3964bc-cd3f-4f13-84c6-1811193c93eb'} failed: > > > (code=100, message=General Exception: ("'1048576'",)) > > > > > > I guess what I need is a way to remove/clean-up these VMs > > manually since ovirt > > > does not seem to be able to do it by itself. > > > > > > This condition also blocks the host from being put into > > maintenance mode. > > > > > > When I reboot the host manually and "confirm host was rebooted", > > the VMs > > > are still there and still stuck. > > > > > > Sincerely, > > > > > > David > > > > > > > > > On 2021-04-07 6:01 a.m., Shani Leviim wrote: > > >> Hi, > > >> You can try with the vdsm-client tool: > > >> https://www.ovirt.org/develop/developer-guide/vdsm/vdsm-client.html > > <https://www.ovirt.org/develop/developer-guide/vdsm/vdsm-client.html> > > <https://www.ovirt.org/develop/developer-guide/vdsm/vdsm-client.html > > <https://www.ovirt.org/develop/developer-guide/vdsm/vdsm-client.html>> > > >> > > >> Stopping a VM: > > >> 1) Get the vmId: > > >> # vdsm-client Host getVMList fullStatus=True > > >> > > >> 2) Destroy the VM > > >> # vdsm-client VM destroy vmID=<vmID> > > >> > > >> *Regards, > > >> * > > >> *Shani Leviim > > >> * > > >> > > >> > > >> On Sat, Apr 3, 2021 at 7:50 AM <[email protected] > > <mailto:[email protected]> <mailto:[email protected] > > <mailto:[email protected]>>> wrote: > > >> > > >> Hello, > > >> > > >> I've somehow gotten one of my VMs stuck in a state that > > ovirt seems to be rather confused about its > > >> existence of now. I'm running oVirt 4.3.10 and using oVirt > > Node on all the hosts. > > >> > > >> My engine and host event logs are now filling up very > > rapidly with this error: > > >> > > >> VDSM node217 command DestroyVDS failed: General Exception: > > ("'1048576'",) > > >> > > >> I was playing with hugetable support, and that error number > > or string looks suspiciously > > >> like the "hugetable size" custom property I set on the VM. > > >> > > >> This VM was migrated to another host at one point as well, > > and now that host is also > > >> generating the same error as well. > > >> > > >> When I try to move these hosts to maintenance mode, they > > get stuck in "Preparing for > > >> Maintenance" while it tries to migrate/deal with the VM > > that's not there any more. > > >> > > >> Forcibly rebooting the hosts does not change anything. The > > VM state/host seems to be > > >> captured somewhere persistent in this case. > > >> > > >> The VM in question is not running, and I can start it up on > > another host successfully, > > >> but ovirt still thinks it exists on the other 2 hosts no > > matter what I do. > > >> > > >> Is there perhaps some way to delete it from the engine > > database directly to straighten > > >> things out? > > >> > > >> Here's a dump of the vdsm log on one of the hosts. I > > haven't been able to pinpoint what > > >> the exact issue is or how to fix it, but hopefully someone > > here will have seen this before? > > >> > > >> 2021-04-03 04:40:35,515+0000 INFO (jsonrpc/1) [api.virt] > > START destroy(gracefulAttempts=1) from=::ffff:10.100.0.210,58150, > > vmId=58abf0cf-d7b9-4067-a86a-e619928368e7 (api:48) > > >> 2021-04-03 04:40:35,516+0000 INFO (jsonrpc/1) [virt.vm] > > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Release VM resources (vm:5186) > > >> 2021-04-03 04:40:35,516+0000 WARN (jsonrpc/1) [virt.vm] > > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') trying to set state to > > Powering down when already Down (vm:626) > > >> 2021-04-03 04:40:35,516+0000 INFO (jsonrpc/1) [virt.vm] > > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Stopping connection > > (guestagent:455) > > >> 2021-04-03 04:40:35,517+0000 INFO (jsonrpc/1) [vdsm.api] > > START teardownImage(sdUUID='a08af6be-3802-4bb1-9fa5-4b6a10227290', > > spUUID='78dc095a-5238-11e8-b8bf-00163e6a7af9', > > imgUUID='9c896907-59b0-4983-9478-b36b2c2eb01e', volUUID=None) > > from=::ffff:10.100.0.210,58150, task_id=fc946d20-126a-4fd0-9078-91 > > >> 4b4a64b1d9 (api:48) > > >> 2021-04-03 04:40:35,518+0000 INFO (jsonrpc/1) > > [storage.StorageDomain] Removing image rundir link > > u'/var/run/vdsm/storage/a08af6be-3802-4bb1-9fa5-4b6a10227290/9c896907-59b0-4983-9478-b36b2c2eb01e' > > (fileSD:592) > > >> 2021-04-03 04:40:35,518+0000 INFO (jsonrpc/1) [vdsm.api] > > FINISH teardownImage return=None from=::ffff:10.100.0.210,58150, > > task_id=fc946d20-126a-4fd0-9078-914b4a64b1d9 (api:54) > > >> 2021-04-03 04:40:35,519+0000 INFO (jsonrpc/1) [vdsm.api] > > START teardownImage(sdUUID='b891448d-dd92-4a7b-a51a-22abc3d7da67', > > spUUID='78dc095a-5238-11e8-b8bf-00163e6a7af9', > > imgUUID='c0e95483-35f1-4a61-958e-4e308b70d3f8', volUUID=None) > > from=::ffff:10.100.0.210,58150, task_id=77c0fdca-e13a-44b5-9a00-29 > > >> 0522b194b2 (api:48) > > >> 2021-04-03 04:40:35,520+0000 INFO (jsonrpc/1) > > [storage.StorageDomain] Removing image rundir link > > u'/var/run/vdsm/storage/b891448d-dd92-4a7b-a51a-22abc3d7da67/c0e95483-35f1-4a61-958e-4e308b70d3f8' > > (fileSD:592) > > >> 2021-04-03 04:40:35,520+0000 INFO (jsonrpc/1) [vdsm.api] > > FINISH teardownImage return=None from=::ffff:10.100.0.210,58150, > > task_id=77c0fdca-e13a-44b5-9a00-290522b194b2 (api:54) > > >> 2021-04-03 04:40:35,521+0000 INFO (jsonrpc/1) [virt.vm] > > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') Stopping connection > > (guestagent:455) > > >> 2021-04-03 04:40:35,521+0000 WARN (jsonrpc/1) [root] File: > > /var/lib/libvirt/qemu/channels/58abf0cf-d7b9-4067-a86a-e619928368e7.ovirt-guest-agent.0 > > already removed (fileutils:54) > > >> 2021-04-03 04:40:35,521+0000 WARN (jsonrpc/1) [root] > > Attempting to remove a non existing network: > > ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7 (libvirtnetwork:198) > > >> 2021-04-03 04:40:35,522+0000 WARN (jsonrpc/1) [root] > > Attempting to remove a non existing net user: > > ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7 (libvirtnetwork:205) > > >> 2021-04-03 04:40:35,526+0000 WARN (jsonrpc/1) [root] > > Attempting to remove a non existing network: > > ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7 (libvirtnetwork:198) > > >> 2021-04-03 04:40:35,526+0000 WARN (jsonrpc/1) [root] > > Attempting to remove a non existing net user: > > ovirtmgmt/58abf0cf-d7b9-4067-a86a-e619928368e7 (libvirtnetwork:205) > > >> 2021-04-03 04:40:35,527+0000 WARN (jsonrpc/1) [root] File: > > /var/lib/libvirt/qemu/channels/58abf0cf-d7b9-4067-a86a-e619928368e7.org.qemu.guest_agent.0 > > already removed (fileutils:54) > > >> 2021-04-03 04:40:35,528+0000 WARN (jsonrpc/1) [virt.vm] > > (vmId='58abf0cf-d7b9-4067-a86a-e619928368e7') timestamp already removed > > from stats cache (vm:2445) > > >> 2021-04-03 04:40:35,531+0000 ERROR (jsonrpc/1) [api] FINISH > > destroy error='1048576' (api:134) > > >> Traceback (most recent call last): > > >> File > > "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 124, in method > > >> ret = func(*args, **kwargs) > > >> File "/usr/lib/python2.7/site-packages/vdsm/API.py", line > > 333, in destroy > > >> res = self.vm.destroy(gracefulAttempts) > > >> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > > line 5283, in destroy > > >> result = self.doDestroy(gracefulAttempts, reason) > > >> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > > line 5302, in doDestroy > > >> return self.releaseVm(gracefulAttempts) > > >> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > > line 5207, in releaseVm > > >> self._cleanup() > > >> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > > line 2382, in _cleanup > > >> self._cleanup_hugepages() > > >> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", > > line 2393, in _cleanup_hugepages > > >> self.nr_hugepages, self.hugepagesz > > >> File > > "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 243, in > > calculate_required_deallocation > > >> _preallocated_hugepages(vm_hugepagesz)) > > >> File > > "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 262, in > > _preallocated_hugepages > > >> kernel_args['hugepagesz'] > > >> File > > "/usr/lib/python2.7/site-packages/vdsm/hugepages.py", line 291, in > > _cmdline_hugepagesz_to_kb > > >> }[cmdline] > > >> KeyError: '1048576' > > >> 2021-04-03 04:40:35,531+0000 INFO (jsonrpc/1) [api.virt] > > FINISH destroy return={'status': {'message': 'General Exception: > > ("\'1048576\'",)', 'code': 100}} from=::ffff:10.100.0.210,58150, > > vmId=58abf0cf-d7b9-4067-a86a-e619928368e7 (api:54) > > >> 2021-04-03 04:40:35,532+0000 INFO (jsonrpc/1) > > [jsonrpc.JsonRpcServer] RPC call VM.destroy failed (error 100) in 0.02 > > seconds (__init__:312) > > >> _______________________________________________ > > >> Users mailing list -- [email protected] > > <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>> > > >> To unsubscribe send an email to [email protected] > > <mailto:[email protected]> <mailto:[email protected] > > <mailto:[email protected]>> > > >> Privacy Statement: > > https://www.ovirt.org/privacy-policy.html > > <https://www.ovirt.org/privacy-policy.html> > > <https://www.ovirt.org/privacy-policy.html > > <https://www.ovirt.org/privacy-policy.html>> > > >> oVirt Code of Conduct: > > https://www.ovirt.org/community/about/community-guidelines/ > > <https://www.ovirt.org/community/about/community-guidelines/> > > <https://www.ovirt.org/community/about/community-guidelines/ > > <https://www.ovirt.org/community/about/community-guidelines/>> > > >> List Archives: > > https://lists.ovirt.org/archives/list/[email protected]/message/KBABFNZIJ6T6QZXGYRSROHCO7LAS4VOM/ > > > > <https://lists.ovirt.org/archives/list/[email protected]/message/KBABFNZIJ6T6QZXGYRSROHCO7LAS4VOM/> > > > > <https://lists.ovirt.org/archives/list/[email protected]/message/KBABFNZIJ6T6QZXGYRSROHCO7LAS4VOM/ > > > > <https://lists.ovirt.org/archives/list/[email protected]/message/KBABFNZIJ6T6QZXGYRSROHCO7LAS4VOM/>> > > >> > > > _______________________________________________ > Users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/KZ4YZHKUA3THMKJOKHTS25SPFJKOCVKW/ _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/6ZW5MC4IA2M6MZA3MEILY7Y54DSQIVVB/
