Re: CS 4.8 KVM VMs will not live migrate

2018-02-02 Thread Andrija Panic
If you need help (if this is production cloud) to monitor this DB inconsistencies ) together with some other DB inconsistencies) via Jenkins, ping me on PM, I might be able to provide you the script, but need to check. On 2 February 2018 at 04:46, David Mabry wrote: > Andrija, > > You were right

Re: CS 4.8 KVM VMs will not live migrate

2018-02-01 Thread David Mabry
Andrija, You were right! The isolation_uri and the broadcast_uri where both blank for the problem VMs. Once I corrected the issue, I was able to migrate them inside of CS without issue. Thanks for helping me get to the root cause of this issue. Thanks, David Mabry On 2/1/18, 3:27 PM, "D

Re: CS 4.8 KVM VMs will not live migrate

2018-02-01 Thread David Mabry
Andrija, Thanks for the tip. I'll check that out and let you know what I find. Thanks, David Mabry On 2/1/18, 2:04 PM, "Andrija Panic" wrote: The customer with serial number here :) So, another issue which I noticed, when you have KVM host disconnections (agent disconnect), t

Re: CS 4.8 KVM VMs will not live migrate

2018-02-01 Thread Andrija Panic
The customer with serial number here :) So, another issue which I noticed, when you have KVM host disconnections (agent disconnect), then in some cases in the cloud.NICs table, there will be missing broadcast URI, isolatio_URI and state or similar filed that is NULL instead of having correct value

Re: CS 4.8 KVM VMs will not live migrate

2018-01-31 Thread Tutkowski, Mike
Glad to hear you fixed the issue! :) > On Jan 31, 2018, at 7:16 AM, David Mabry wrote: > > Mike and Wei, > > Good news! I was able to manually live migrate these VMs following the steps > outlined below: > > 1.) virsh dumpxml 38 --migratable > 38.xml > 2.) Change the vnc information in 38.xm

Re: CS 4.8 KVM VMs will not live migrate

2018-01-31 Thread David Mabry
Mike and Wei, Good news! I was able to manually live migrate these VMs following the steps outlined below: 1.) virsh dumpxml 38 --migratable > 38.xml 2.) Change the vnc information in 38.xml to match destination host IP and available VNC port 3.) virsh migrate --verbose --live 38 --xml 38.xml

Re: CS 4.8 KVM VMs will not live migrate

2018-01-30 Thread David Mabry
Ah, understood. I'll take a closer look at the logs and make sure that I didn't accidentally miss those lines when I pulled together the logs for this email chain. Thanks, David Mabry On 1/30/18, 8:34 AM, "Wei ZHOU" wrote: Hi David, I encountered the UnsupportAnswer once before,

Re: CS 4.8 KVM VMs will not live migrate

2018-01-30 Thread Wei ZHOU
Hi David, I encountered the UnsupportAnswer once before, when I made some changes in the kvm plugin. Normally there should be some network configurations in the agent.log but I do not see it. -Wei 2018-01-30 15:00 GMT+01:00 David Mabry : > Hi Wei, > > I detached the iso and received the same

Re: CS 4.8 KVM VMs will not live migrate

2018-01-30 Thread David Mabry
Hi Wei, I detached the iso and received the same error. Just out of curiosity, what leads you to believe it is something in the vxlan code? I guess at this point, attaching a remote debugger to the agent in question might be the best way to get to the bottom of what is going on. Thanks in ad

Re: CS 4.8 KVM VMs will not live migrate

2018-01-30 Thread Wei ZHOU
The answer should be caused by an exception in the cloudstack agent. I tried to migrate a vm in our testing env, it is working. there are some different between our env and yours. (1) vlan VS vxlan (2) no ISO VS attached ISO (3) both of us use ceph and centos7. I suspect it is caused by codes on

Re: CS 4.8 KVM VMs will not live migrate

2018-01-29 Thread Tutkowski, Mike
Well…unfortunately, the serial-number issue that I had seen before cause an issue doesn’t seem to be the case here. On both the working and non-working (for live migration) VMs, there is a element for applicable elements (per the XML below). Anyone else have any ideas here? On 1/29/18, 4:41

Re: CS 4.8 KVM VMs will not live migrate

2018-01-29 Thread David Mabry
Mike, Thanks for the reply. As requested: Will not Migrate i-532-1392-NSVLTN f7dbf00b-2e15-4991-a407-cf27a3d65d1e Other PV Virtio-SCSI (64-bit) 4194304 4194304 2 2000 /machine Apache Software Foundation CloudStack

Re: CS 4.8 KVM VMs will not live migrate

2018-01-29 Thread Tutkowski, Mike
Hi David, So, I don’t know if what I am going to say here will at all be of use to you, but maybe. :) I had a customer one time mention to me that he had trouble with live VM migration on KVM with a VM that was created on an older version of CloudStack. Live VM migration worked fine for these