Eventually I fixed the problem, without clear understanding of the root cause.
I destroyed routerVM, and cloudstack recreated it. But I discovered, that it is version 4.11.0. Not 4.11.1! I checked and systemVM template for 4.11.1 is registered in cloudstack - all OK. I noticed however, that its "Type" property is USER, not SYSTEM. Then I wanted to delete template for 4.11.0, but cloudstack does not offer me this option. I can only add new ones. So, I ended with manipulations in the DB itself to make template for 4.11.1 the only one in the system and have Type = SYSTEM. After that I destroyed again routerVM. It was recreated and it is 4.11.1. And now everything works fine for over an hour already. I hope, thats it. On Fri, Jul 20, 2018 at 12:10 PM ilya musayev <[email protected]> wrote: > Have you tried destroying router vm and let CloudStack create new one ? > > On Fri, Jul 20, 2018 at 1:33 AM Jevgeni Zolotarjov <[email protected] > > > wrote: > > > - an ip-address conflict. > > JZ: unlikely, but not impossible. I tried to restart router VM in > > Network-Guest networks -> defaultGuestNetwork -> VirtualAppliances > > While rebooting ping to this router VM disappeared. Hence, no other > device > > is using the same IP. > > But!!! when this virtual router started, then network connection to all > > guest VMs disappeared. So, it must be something with this virtual router. > > > > - flakey hardware being one of > > -+ if card in the host > > JZ: higly unlikely > > > > -+ a router with bad firmware > > JZ: also unlikely > > > > - of course a strange cofiguration of the software router in you host > might > > be the issue as well > > JZ: I didnt do any special configuration. Just used default. > > > > by all I know this happening after upgrade sounds like an unhappy > incident > > but can't be sure. > > The iptables restart, was this on the VirtualRouter or on the host, or > > maybe on the guest? and the restart network? > > > > JZ: iptables restart on host machine. (or network restart on host) > > > > > > > > On Fri, Jul 20, 2018 at 11:14 AM Daan Hoogland <[email protected]> > > wrote: > > > > > that behaviour sound familiar from a couple of cases: > > > - an ip-address conflict. > > > - flakey hardware being one of > > > -+ if card in the host > > > -+ a router with bad firmware > > > - of course a strange cofiguration of the software router in you host > > might > > > be the issue as well > > > > > > by all I know this happening after upgrade sounds like an unhappy > > incident > > > but can't be sure. > > > The iptables restart, was this on the VirtualRouter or on the host, or > > > maybe on the guest? and the restart network? > > > > > > > > > On Fri, Jul 20, 2018 at 7:43 AM, Jevgeni Zolotarjov < > > > [email protected]> > > > wrote: > > > > > > > I updated cloudstack 4.11.0 -> 4.11.1 > > > > > > > > Everything went OK during update, but after host reboot guest VMs > lost > > > > connection after few minutes of normal work. > > > > I tried restarting network - systemctl restart network.service > > > > then connection was restored again for few minutes > > > > > > > > Finally I could restore connection by restarting iptables - systemctl > > > > restart iptables.service > > > > > > > > But then again guest VMs lost connection after few minutes of normal > > > > operation. > > > > The time of normal operation can be 5 minutes, but sometimes up to 40 > > > > minutes. > > > > > > > > Please help me to track the root cause and fix it > > > > > > > > Host OS - Centos 7.5 > > > > virtualisation - KVM > > > > > > > > > > > > > > > > -- > > > Daan > > > > > >
