On Thu, Oct 22, 2020 at 3:39 PM Gilboa Davara <[email protected]> wrote:
> Hello all,
>
Hi,
>
> I just upgraded one of my test oVirt setups to latest.
>
> Post reboot, the machine lost network and hosted engine didn't start.
> When I connected to the machine, I noticed all the
> /etc/sysconfig/network-scripts/ifcfg-* files disappeared. No idea why /
> how. (Possibly I did something wrong and forgot about it.)
>
It would be nice to know when this happens as it might be a serious
problem.
>
> Long story story short, I copied the two missing files (ifcfg-onb0
> ethernet device configuration and the ifcfg-ovirtmgmt bridge configuration)
> from another ovirt host, changed the ovirtmgmt IP address and UUID to match
> the UUID reported in logs as missing and restarted NetworkManager and
> restarted all the oVirt related services (vdsmd, supervdsmd, ovirt-*, etc).
>
> Sadly enough, even with both onb0 and ovirtmgmt up, vdsm still complains
> about the missing network (ovirtmgmt) and refuses to start the hosted
> engine.
> Reboot doesn't seem to change anything.
>
Unfortunately this won't work. From this it seems like vdsm persistence was
broken somehow during the upgrade.
>
> In the main log I see the following errors:
> Oct 22 16:33:31 office-wx-otest vdsm[2634]: WARN Attempting to remove a
> non existing network: ovirtmgmt/1da8c5b7-999c-4ada-8287-1f35de6ce21d
> Oct 22 16:33:31 office-wx-otest vdsm[2634]: WARN Attempting to remove a
> non existing net user: ovirtmgmt/1da8c5b7-999c-4ada-8287-1f35de6ce21d
> Oct 22 16:33:31 office-wx-otest vdsm[2634]: WARN Attempting to remove a
> non existing network: ovirtmgmt/1da8c5b7-999c-4ada-8287-1f35de6ce21d
> Oct 22 16:33:31 office-wx-otest vdsm[2634]: WARN Attempting to remove a
> non existing net user: ovirtmgmt/1da8c5b7-999c-4ada-8287-1f35de6ce21d
>
> As it is one of my oVirt test setup, I can simply redeploy the host and
> continue from there, but I rather use this experience to learn how to fix
> oVirt such issues in the future.
>
For start you can really ensure that the network is not saved in the vdsm
configuration.
By using vdsm-tool on the host, if you run "vdsm-tool list-nets" it won't
produce anything most likely.
To restore your previous configuration on the host you can use:
cat << EOF > ovirtmgmt.json
{
"networks": {
"ovirtmgmt": {
"netmask": "255.255.255.0",
"ipv6autoconf": false,
"nic": "onb0",
"bridged": true,
"ipaddr": "192.168.2.117",
"defaultRoute": true,
"dhcpv6": false,
"gateway": "192.168.2.100",
"mtu": 1500,
"switch": "legacy",
"stp": false,
"bootproto": "none",
"nameservers": [
"192.168.2.100",
"8.8.8.8"
]
}
},
"bondings": {},
"options": {
"connectivityCheck": false
}
}
EOF
vdsm-client -f ovirtmgmt.json Host setupNetworks
If that works, you have to persist the configuration before reboot, so
either from the engine UI
or from the host itself:
vdsm-client Host setSafeNetworkConfig
>
> Logs attached.
>
> https://drive.google.com/file/d/12ugy6CuaFaMvXYt6uGT4D_EHIW6nXttb/view?usp=sharing
>
> $ PAGER= nmcli connection show
> NAME UUID TYPE DEVICE
> ovirtmgmt 1da8c5b7-999c-4ada-8287-1f35de6ce21d bridge ovirtmgmt
> onb0 48332db3-8939-bff3-6b71-772a28c9e7b8 ethernet onb0
>
> $ PAGER= nmcli device show
> GENERAL.DEVICE: ovirtmgmt
> GENERAL.TYPE: bridge
> GENERAL.HWADDR: FC:AA:14:6B:A8:E0
> GENERAL.MTU: 1500
> GENERAL.STATE: 100 (connected)
> GENERAL.CONNECTION: ovirtmgmt
> GENERAL.CON-PATH:
> /org/freedesktop/NetworkManager/ActiveConnection/2
> IP4.ADDRESS[1]: 192.168.2.117/24
> IP4.GATEWAY: 192.168.2.100
> IP4.ROUTE[1]: dst = 192.168.2.0/24, nh =
> 0.0.0.0, mt = 425
> IP4.ROUTE[2]: dst = 0.0.0.0/0, nh =
> 192.168.2.100, mt = 425
> IP4.DNS[1]: 192.168.2.100
> IP4.DNS[2]: 8.8.8.8
> IP6.GATEWAY: --
>
> GENERAL.DEVICE: onb0
> GENERAL.TYPE: ethernet
> GENERAL.HWADDR: FC:AA:14:6B:A8:E0
> GENERAL.MTU: 1500
> GENERAL.STATE: 100 (connected)
> GENERAL.CONNECTION: onb0
> GENERAL.CON-PATH:
> /org/freedesktop/NetworkManager/ActiveConnection/3
> WIRED-PROPERTIES.CARRIER: on
> IP4.GATEWAY: --
>
> GENERAL.DEVICE: ;vdsmdummy;
> GENERAL.TYPE: bridge
> GENERAL.HWADDR: 92:8B:9A:5E:C1:3E
> GENERAL.MTU: 1500
> GENERAL.STATE: 10 (unmanaged)
> GENERAL.CONNECTION: --
> GENERAL.CON-PATH: --
> IP4.GATEWAY: --
> IP6.GATEWAY: --
>
> GENERAL.DEVICE: lo
> GENERAL.TYPE: loopback
> GENERAL.HWADDR: 00:00:00:00:00:00
> GENERAL.MTU: 65536
> GENERAL.STATE: 10 (unmanaged)
> GENERAL.CONNECTION: --
> GENERAL.CON-PATH: --
> IP4.ADDRESS[1]: 127.0.0.1/8
> IP4.GATEWAY: --
> IP6.GATEWAY: --
>
> GENERAL.DEVICE: br-int
> GENERAL.TYPE: openvswitch
> GENERAL.HWADDR: 8E:15:6A:F8:3C:45
> GENERAL.MTU: 1500
> GENERAL.STATE: 10 (unmanaged)
> GENERAL.CONNECTION: --
> GENERAL.CON-PATH: --
> IP4.GATEWAY: --
> IP6.GATEWAY: --
>
> GENERAL.DEVICE: ovs-system
> GENERAL.TYPE: openvswitch
> GENERAL.HWADDR: E2:09:EA:A2:BD:70
> GENERAL.MTU: 1500
> GENERAL.STATE: 10 (unmanaged)
> GENERAL.CONNECTION: --
> GENERAL.CON-PATH: --
> IP4.GATEWAY: --
> IP6.GATEWAY:
>
> - Gilboa
>
> office-wx-otest-vdsm.bz2
> <https://drive.google.com/file/d/12ugy6CuaFaMvXYt6uGT4D_EHIW6nXttb/view?usp=drive_web>
> _______________________________________________
> Users mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> Privacy Statement: https://www.ovirt.org/privacy-policy.html
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/[email protected]/message/V5M23EHLZ5GSL2ANQVIDJLZR6CZWKIA6/
>
Hopefully this helps.
Regards,
Ales
--
Ales Musil
Software Engineer - RHV Network
Red Hat EMEA <https://www.redhat.com>
[email protected] IM: amusil
<https://red.ht/sig>
_______________________________________________
Users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/[email protected]/message/LV3V5QA7NIR6AC76TNMNGYYHVCYV7QMD/
