> > Hello > > I would like to report a regression triggered by recent change in > Intel ICE Ethernet driver in the 6.6.9 linux kernel. The problem was > bisected and the regression is triggered by > fc4d6d136d42fab207b3ce20a8ebfd61a13f931f "ice: alter feature support > check for SRIOV and LAG" commit and originally reported as part of > https://lore.kernel.org/linux-mm/cak8ffz4dy+gtba40pm7nn5xchy+51w3sfxpqkqpqaksxyyx...@mail.gmail.com/T/#m5217c62beb03b3bc75d7dd4b1d9bab64a3e68826 > thread. > > > However, after the following patch we see that more NUMA nodes have > > such a low amount of memory and that is causing constant reclaiming > > of memory because it looks like something inside of the kernel ate all > > the memory. This is right after the start of the system as well. > > I'm reporting it here as it is a different problem than the original > thread. The commit introduces a low memory problem per each numa node > of the first socket (node0 .. node3 in our case) and cause constant > kswapd* 100% CPU usage. See attached 6.6.9-kswapd_usage.png. The low > memory issue is nicely visible in "numastat -m", see attached files: > * numastat_m-6.6.10_28GB_HP_ice_revert.txt >= 6.6.9 with reverted ice commit > * numastat_m-6.6.10_28GB_HP_no_revert.txt >= 6.6.9 vanilla > the server "is fresh" (after reboot), without running any application load. > > $ grep MemFree numastat_m-6.6.10_28GB_HP_ice_revert.txt > numastat_m-6.6.10_28GB_HP_no_revert.txt > numastat_m-6.6.10_28GB_HP_ice_revert.txt:MemFree > 2756.89 2754.86 100.39 2278.43 < ice > fix is reverted, we have ~2GB free per numa, except one, like before > == no issue > numastat_m-6.6.10_28GB_HP_ice_revert.txt:MemFree > 3551.29 1530.52 2212.04 3488.09 > ... > numastat_m-6.6.10_28GB_HP_no_revert.txt:MemFree > 127.52 66.49 120.23 263.47 < > ice fix is present, we see just few MB free per each node, this will > cause kswapd utilization! > numastat_m-6.6.10_28GB_HP_no_revert.txt:MemFree > 3322.18 3134.47 195.55 879.17 > ... > > If you have some hints on how to debug what is actually occupying all > that memory and some fix of the problem will be nice. We can provide > testing and more reports if needed to analyze the issue. We reverted > the commit fc4d6d136d42fab207b3ce20a8ebfd61a13f931f as a workaround > till we know a proper fix. > > Best regards, > Jaroslav Pulchart
Hello everyone FYI: I try to use linuxk-6.7(.0) kernel and the issue is there as well. Best regards, Jaroslav Pulchart
