On Mon, Mar 25, 2024 at 10:54 AM 张 祖建 via discuss < ovs-discuss@openvswitch.org> wrote:
> Hi, all: > > There is an occasional memory leak detected by valgrind in ovn-controller: > > Memory leak detected in ovn-controller. > kubectl-ko-log/kube-ovn-worker/ovn/ovn-controller.valgrind.log.67100 > ==00:00:05:52.763 67100== 1 bytes in 1 blocks are possibly lost in loss > record 3 of 3,106 > ==00:00:05:52.763 67100== at 0x4848899: malloc (in > /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) > ==00:00:05:52.763 67100== by 0x4A3DC48: xmalloc__ (util.c:140) > ==00:00:05:52.763 67100== by 0x4A3DCA8: xmemdup (util.c:188) > ==00:00:05:52.763 67100== by 0x13D7F8: ovn_flow_init (ofctrl.c:1624) > ==00:00:05:52.763 67100== by 0x13D7F8: desired_flow_alloc > (ofctrl.c:1649) > ==00:00:05:52.763 67100== by 0x140E5D: > ofctrl_check_and_add_flow_metered (ofctrl.c:1230) > ==00:00:05:52.763 67100== by 0x140FF2: ofctrl_add_flow_metered > (ofctrl.c:1272) > ==00:00:05:52.763 67100== by 0x14101B: ofctrl_add_flow (ofctrl.c:1260) > ==00:00:05:52.763 67100== by 0x160C62: put_local_common_flows > (physical.c:925) > ==00:00:05:52.763 67100== by 0x162079: consider_port_binding > (physical.c:1452) > ==00:00:05:52.763 67100== by 0x164636: physical_run (physical.c:2288) > ==00:00:05:52.763 67100== by 0x15B19E: en_pflow_output_run > (ovn-controller.c:3736) > ==00:00:05:52.763 67100== by 0x494ED12: engine_recompute > (inc-proc-eng.c:415) > ==00:00:05:52.763 67100== > ==00:00:05:52.763 67100== 1 bytes in 1 blocks are possibly lost in loss > record 4 of 3,106 > ==00:00:05:52.763 67100== at 0x4848899: malloc (in > /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) > ==00:00:05:52.763 67100== by 0x4A3DC48: xmalloc__ (util.c:140) > ==00:00:05:52.763 67100== by 0x4A3DCA8: xmemdup (util.c:188) > ==00:00:05:52.763 67100== by 0x13D7F8: ovn_flow_init (ofctrl.c:1624) > ==00:00:05:52.763 67100== by 0x13D7F8: desired_flow_alloc > (ofctrl.c:1649) > ==00:00:05:52.763 67100== by 0x140E5D: > ofctrl_check_and_add_flow_metered (ofctrl.c:1230) > ==00:00:05:52.763 67100== by 0x140FF2: ofctrl_add_flow_metered > (ofctrl.c:1272) > ==00:00:05:52.763 67100== by 0x14101B: ofctrl_add_flow (ofctrl.c:1260) > ==00:00:05:52.763 67100== by 0x15FFB4: add_default_drop_flow > (physical.c:872) > ==00:00:05:52.763 67100== by 0x164BB2: physical_run (physical.c:2419) > ==00:00:05:52.763 67100== by 0x15B19E: en_pflow_output_run > (ovn-controller.c:3736) > ==00:00:05:52.763 67100== by 0x494ED12: engine_recompute > (inc-proc-eng.c:415) > ==00:00:05:52.763 67100== by 0x494F27F: engine_run_node > (inc-proc-eng.c:491) > ==00:00:05:52.763 67100== by 0x494F27F: engine_run (inc-proc-eng.c:528) > ==00:00:05:52.763 67100== > ==00:00:05:52.763 67100== 1 bytes in 1 blocks are possibly lost in loss > record 5 of 3,106 > ==00:00:05:52.763 67100== at 0x4848899: malloc (in > /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) > ==00:00:05:52.763 67100== by 0x4A3DC48: xmalloc__ (util.c:140) > ==00:00:05:52.763 67100== by 0x4A3DCA8: xmemdup (util.c:188) > ==00:00:05:52.763 67100== by 0x13D7F8: ovn_flow_init (ofctrl.c:1624) > ==00:00:05:52.763 67100== by 0x13D7F8: desired_flow_alloc > (ofctrl.c:1649) > ==00:00:05:52.763 67100== by 0x140E5D: > ofctrl_check_and_add_flow_metered (ofctrl.c:1230) > ==00:00:05:52.763 67100== by 0x140FF2: ofctrl_add_flow_metered > (ofctrl.c:1272) > ==00:00:05:52.763 67100== by 0x14101B: ofctrl_add_flow (ofctrl.c:1260) > ==00:00:05:52.763 67100== by 0x15FFB4: add_default_drop_flow > (physical.c:872) > ==00:00:05:52.763 67100== by 0x164E1A: physical_run (physical.c:2496) > ==00:00:05:52.763 67100== by 0x15B19E: en_pflow_output_run > (ovn-controller.c:3736) > ==00:00:05:52.763 67100== by 0x494ED12: engine_recompute > (inc-proc-eng.c:415) > ==00:00:05:52.763 67100== by 0x494F27F: engine_run_node > (inc-proc-eng.c:491) > ==00:00:05:52.763 67100== by 0x494F27F: engine_run (inc-proc-eng.c:528) > ==00:00:05:52.763 67100== > ==00:00:05:52.763 67100== 1 bytes in 1 blocks are possibly lost in loss > record 6 of 3,106 > ==00:00:05:52.763 67100== at 0x4848899: malloc (in > /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) > ==00:00:05:52.763 67100== by 0x4A3DC48: xmalloc__ (util.c:140) > ==00:00:05:52.763 67100== by 0x4A3DCA8: xmemdup (util.c:188) > ==00:00:05:52.763 67100== by 0x13D7F8: ovn_flow_init (ofctrl.c:1624) > ==00:00:05:52.763 67100== by 0x13D7F8: desired_flow_alloc > (ofctrl.c:1649) > ==00:00:05:52.763 67100== by 0x140E5D: > ofctrl_check_and_add_flow_metered (ofctrl.c:1230) > ==00:00:05:52.763 67100== by 0x140FF2: ofctrl_add_flow_metered > (ofctrl.c:1272) > ==00:00:05:52.763 67100== by 0x14101B: ofctrl_add_flow (ofctrl.c:1260) > ==00:00:05:52.763 67100== by 0x15FFB4: add_default_drop_flow > (physical.c:872) > ==00:00:05:52.763 67100== by 0x164EFD: physical_run (physical.c:2529) > ==00:00:05:52.763 67100== by 0x15B19E: en_pflow_output_run > (ovn-controller.c:3736) > ==00:00:05:52.763 67100== by 0x494ED12: engine_recompute > (inc-proc-eng.c:415) > ==00:00:05:52.763 67100== by 0x494F27F: engine_run_node > (inc-proc-eng.c:491) > ==00:00:05:52.763 67100== by 0x494F27F: engine_run (inc-proc-eng.c:528) > ==00:00:05:52.763 67100== > ==00:00:05:52.763 67100== 1 bytes in 1 blocks are possibly lost in loss > record 7 of 3,106 > ==00:00:05:52.763 67100== at 0x4848899: malloc (in > /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) > ==00:00:05:52.763 67100== by 0x4A3DC48: xmalloc__ (util.c:140) > ==00:00:05:52.763 67100== by 0x4A3DCA8: xmemdup (util.c:188) > ==00:00:05:52.763 67100== by 0x13D7F8: ovn_flow_init (ofctrl.c:1624) > ==00:00:05:52.763 67100== by 0x13D7F8: desired_flow_alloc > (ofctrl.c:1649) > ==00:00:05:52.763 67100== by 0x140E5D: > ofctrl_check_and_add_flow_metered (ofctrl.c:1230) > ==00:00:05:52.763 67100== by 0x140FF2: ofctrl_add_flow_metered > (ofctrl.c:1272) > ==00:00:05:52.763 67100== by 0x14101B: ofctrl_add_flow (ofctrl.c:1260) > ==00:00:05:52.763 67100== by 0x160C62: put_local_common_flows > (physical.c:925) > ==00:00:05:52.763 67100== by 0x161C9C: consider_port_binding > (physical.c:1637) > ==00:00:05:52.763 67100== by 0x163E37: physical_eval_port_binding > (physical.c:2163) > ==00:00:05:52.763 67100== by 0x1640DC: physical_handle_flows_for_lport > (physical.c:2224) > ==00:00:05:52.763 67100== by 0x15AEDA: > pflow_output_sb_port_binding_handler (ovn-controller.c:3819) > ==00:00:05:52.763 67100== > ... > > > ovs version: branch-3.1 (latest) > ovn version: branch-22.12 (latest) > > Attachment is the full report. > > _______________________________________________ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > Hi, it seems like the kube-ovn-worker is not gracefully stopping the ovn-controller. We have a several tests to ensure that there is no memory leak and that seems to be the case when I do "ovn-appctl -t ovn-controller exit", but when I kill the controller "kill $(pidof ovn-controller)" I'll get the same memory leak. Can you please check if the controller was stopped in a graceful way before this report? Best Regards, Ales -- Ales Musil Senior Software Engineer - OVN Core Red Hat EMEA <https://www.redhat.com> amu...@redhat.com <https://red.ht/sig>
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
