Hello Ilya -- Thank you for your message and clarification.
Could you please clarify which method/function called ovs-vswitchd and ovs-dpctl to open the kernel datapath and set flags? Thank you. [https://my.gcdn.co/images/web_3253/logo_g.png]<https://gcorelabs.com/> Vsevolod Vayner Product Director vsevolod.vay...@gcore.com<mailto:vsevolod.vay...@gcore.com> gcore.com<https://gcore.com/> [https://my.gcdn.co/images/web_3253/face.png]<https://www.facebook.com/gcorelabscom/> [https://my.gcdn.co/images/web_3253/in.png] <https://www.linkedin.com/company/g-core> [https://my.gcdn.co/images/web_3253/yout.png] <https://www.youtube.com/channel/UCFhtQE7HB097a1_Nfv-Ma_Q> [https://my.gcdn.co/images/web_3253/weib.png] <https://www.weibo.com/p/1005056986184457> This message is intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to us, and immediately and permanently delete it. Do not use, copy or disclose the information contained in this message or in any attachment. From: Ilya Maximets <i.maxim...@ovn.org> Date: Monday, 31 July 2023 at 20:27 To: Vsevolod Vayner <vsevolod.vay...@gcore.com>, b...@openvswitch.org <b...@openvswitch.org> Cc: [Partner] Fedor Tarasenko <prn.fedor.tarase...@gcore.lu>, [Partner] Ivan Kramarev <prn.ivan.krama...@gcore.lu>, Dmitri Polyakov <dmitri.polya...@gcore.com>, [Partner] Ivan Suchkov <ivan.such...@gcore.lu>, Aleksandr Fedotov <aleksandr.fedo...@gcore.com>, i.maxim...@ovn.org <i.maxim...@ovn.org> Subject: Re: [ovs-discuss] OVS Stopped forwarding On 7/28/23 18:14, Vsevolod Vayner via discuss wrote: > 1. What you did that made the problem appear. > > In our env we use openvswitch exporter to collect dump-flows and data from > production ovs. > It runs only ovs-dpctl and ovs-ofctl to collect, parse and convert data to > prometheus format. > We run vswitchd, db & exporter in separate containers. > > In our production env we have: > > Open vSwitch: > > ovs-vswitchd (Open vSwitch) 2.12.0 > DPDK 18.11.2 > ovs-vsctl (Open vSwitch) 2.12.0 > DB Schema 8.0.0 > > and we also have an installed OpenvSwitch on our exporter to collect > ovs-ofctl and ovs-dpctl data: > > ovs-vswitchd (Open vSwitch) 3.1.3 > DPDK 22.11.1 > ovs-vsctl (Open vSwitch) 3.1.3 > DB Schema 8.3.1 > > 2. What you expected to happen. > > We expected the exporter to collect statistics using the ovs-ofctl utility > with the dump-flows [br-int, br-tun, br-ex] and ovs-dpctl show to collect > lookups. > > 3. What actually happened. > > openvswitch stopped to forward traffic. I see only ingress traffic on ports > of ovs, no traffic outs. > > 4. The kernel version on which Open vSwitch is running (from /proc/version) > and the distribution and version number of your OS (e.g. "Centos 5.0"). > > Linux version 4.18.0-408.el8.x86_64 (mockbu...@kbuilder.bsys.centos.org > <mailto:mockbu...@kbuilder.bsys.centos.org>) (gcc version 8.5.0 20210514 (Red > Hat 8.5.0-14) (GCC)) #1 SMP Mon Jul 18 17:42:52 UTC 2022 > > > In some cases, only a host reboot helps. > > If this is not well-known bug, I'll try to find resource in production to > reproduce and provide more debug information. Hi, Vsevolod. The problem is that you're using drastically different versions of ovs-vswitchd and ovs-dpctl. Every time you run ovs-dpctl, it will open the kernel datapath and set flags to it in the process. In your particular case, these flags change the way upcalls are delivered to ovs-vswitchd, so no traffic can go to userspace afterwards and no new flows will be installed to a datapath as a result, because ovs-vswitchd is too old to understand a new datapath configuration. The issue should be resolved by re-starting ovs-vswitchd. In general, use of ovs-dpctl while ovs-vswitchd is running is not safe. Use ovs-appctl dpctl/* commands to get the same information via ovs-vswitchd instead. Or at least make sure that the version of ovs-dpctl is exactly the same as version of ovs-vswitchd. Best regards, Ilya Maximets.
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
