On 04/04/2025 18:28, Gabriel Goller wrote: > This series allows the user to add fabrics such as OpenFabric and OSPF over > their clusters. > > This series relies on: > https://lore.proxmox.com/pve-devel/20250404135522.2603272-1-s.hanre...@proxmox.com/T/#mf4cf46c066d856cea819ac3e79d115a290f47466
Thanks for the v2, I like this feature a lot! Unfortunately, one problem I noticed while testing this is that it may break pre-existing FRR configs (such as full-mesh Ceph clusters set up according to [1]) when making seemingly unrelated SDN changes. I already quickly discussed this with Stefan, posting here in case others have input as well. Steps to reproduce: - on PVE 8.3 (without these patches), set up Ceph full mesh with OpenFabric as described in [1], includes custom /etc/frr/frr.conf - also use some SDN feature, e.g. a VLAN zone with a Vnet - install patched packages, systemctl restart pveproxy pvedaemon - make a fabric-unrelated change in the SDN config, e.g. change tag of the VLAN zone Vnet - apply SDN config => SDN stack writes out a nearly-empty /etc/frr/frr.conf on all nodes and thus takes down the full mesh: # cat /etc/frr/frr.conf frr version 10.2.1 frr defaults datacenter hostname fabric159 log syslog informational service integrated-vtysh-config ! ! line vty It seems to also disable the fabricd daemon in /etc/frr/daemons: # grep fabric /etc/frr/daemons fabricd=no fabricd_options="-A 127.0.0.1 --dummy_as_loopback" # vtysh -c 'show openfabric route' fabricd is not running It makes sense that one cannot use both our fabrics integration and custom FRR configs, but the above SDN config change is not related to fabrics, so we should probably avoid touching the frr.conf if possible. The wiki article [1] does warn that the full mesh doesn't work in combination with EVPN, but unfortunately doesn't mention an inherent incompatibility with the SDN stack as a whole. [1] https://pve.proxmox.com/wiki/Full_Mesh_Network_for_Ceph_Server#Routed_Setup_(with_Fallback) _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
