[Bug 271246] Kernel wired memory leak with repeated netstat

2023-12-14 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 kadir kose changed: What|Removed |Added CC||kadir.k...@timusnetworks.co

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-12 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #27 from Dave Hayes --- That will be impossible since once it does have the problem, I cannot log in and command prompts do not work, as evidenced by previous interactions with a machine with this bug. If the attached graph do

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-12 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #26 from Konstantin Belousov --- (In reply to Dave Hayes from comment #25) So there is nothing unusual in the data you shown, and more, there were no de-facto demonstration of the problem. I do not see a leak in any of the numb

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-11 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #25 from Dave Hayes --- So, as part of monitoring, the machine in question does repeated netstats and hence is always in that state. It's just a matter of figuring out what statistics are useful to you in finding the problem, as

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-11 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #24 from Konstantin Belousov --- 43426 cached pages is equal to around 170MB of cached free memory. First, this is not wired memory, second, it is allocatable on demand. There should be something else that leaks wirings. It wo

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-10 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #23 from Dave Hayes --- After 48 hours here is the top of the file containing output of the script: Total wired change: 37450 pages (vmstat -z) vm pgcache (18) grew 43426 units of whatever the 'used' field is i

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-08 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 Dave Hayes changed: What|Removed |Added Attachment #242004|0 |1 is obsolete|

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-08 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #21 from Dave Hayes --- [Mon May 8 10:23:42 2023] (842226) (wired) Changed by 7 pages (now at 4.205 Gb, total change since start 13210 pages) [Mon May 8 10:23:42 2023] (842226) (vmstat -z) vm pgcache (18

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-07 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #20 from Dave Hayes --- https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256507 ... in case you are unaware that I also initiated that one. In 12.4, this memory leak is only visible if you do specific math on the sysctls. In

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-07 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #19 from Dave Hayes --- I have manually inspected sys/vm/vm_page.c for the listed commit. It is there. -- You are receiving this mail because: You are the assignee for the bug.

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-07 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #18 from Konstantin Belousov --- Do you have the commit 6094749a1a5dafb8daf98deab23fc968070bc695 in your source tree? You should since you specified 13.2-STABLE, but the symptoms require re-checking. -- You are receiving this

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-07 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #17 from Dave Hayes --- So far, one of the vm pgcache entries in vmstat -z is leaking the most. Do I have to do extra gymnastics to find out which entry it is somehow? -- You are receiving this mail because: You are the assign

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-07 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #16 from Konstantin Belousov --- (In reply to Dave Hayes from comment #14) These are per-cpu page cache queues. -- You are receiving this mail because: You are the assignee for the bug.

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-07 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #15 from Dave Hayes --- Ok now I'm handling duplicated names (there was another one too "buffer arena-40"). The machine I have to test this on is "live" but I can let this run until it loses a fair bit of memory and do a contr

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-07 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #14 from Dave Hayes --- Are you aware of this idea? # vmstat -z | grep pgca vm pgcache:4096, 0, 632833,2480,2107607698, 0, 0, 0 vm pgcache:4096, 0, 589518,1688,865006857, 74,

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-06 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #13 from Konstantin Belousov --- (In reply to Dave Hayes from comment #12) Unit is the size of the item. Keep the thing running until you see that the machine reached the almost unusable state, then look at the vmstat -z/vmstat

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-06 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #12 from Dave Hayes --- What is wrong is likely the label "bytes". Here's a sample output from vmstat -z --libxo json (formatted for clarity): { "fail" : 0, "free" : 0, "limit" : 0,

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-06 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #11 from Konstantin Belousov --- There is something wrong in your scripts. Zones grow by the size of the items allocated, so e.g. the PROC zone cannot grow by 1 byte. -- You are receiving this mail because: You are the assign

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-06 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #10 from Dave Hayes --- Ok. Thank you for clarifying. It turns out this was easier than I thought it would be. Here's output from my script (source code on request). I stopped at > 128 pages of lost wired memory but I can run th

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-06 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #9 from Konstantin Belousov --- Yes, I am asking to check each entry. This is the way to trim down the search area for kernel memory leak. -- You are receiving this mail because: You are the assignee for the bug.

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-06 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #8 from Dave Hayes --- First, I do understand that you want me to check for "the cause" and "the outstanding growing malloc zones". Next I also do understand vmstat -z produces output related to zones, and vmstat -m produces o

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-06 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #7 from Konstantin Belousov --- (In reply to Dave Hayes from comment #6) vmstat -z output lists zones. Zones you quoted are used to back the malloc(9) allocator. Malloc types are listed with vmstat -m. Again, I am asking you t

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-05 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #6 from Dave Hayes --- When you say "malloc_type", do you mean one of these? # vmstat -z | grep malloc malloc-16: 16, 0, 22655,6073,105306080, 0, 0, 0 malloc-32: 32, 0, 21773,

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-05 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 Graham Perrin changed: What|Removed |Added CC||grahamper...@freebsd.org

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-05 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #5 from Konstantin Belousov --- If I know which zone or malloc type leaks, I would not ask you to look which one leaks. You need to find, on your machine, if any of malloc type increases steadily. -- You are receiving this ma

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-05 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #4 from Dave Hayes --- I should finally mention that, in my monitoring system, netstat runs twice every 5 seconds on the graph above. That might provide the correct context for the above data, and shed some light on how I discov

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-05 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #3 from Dave Hayes --- Created attachment 242004 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=242004&action=edit Graph over past 24 hours of vm.stats.vm.v_wire_count I've added this graph in case that helps. -- Y

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-05 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 --- Comment #2 from Dave Hayes --- So I am not a modern kernel dev. That being said, I currently data over time of: - anything in sysctl - vmstat -z ... malloc buckets and UMA I can fairly easily monitor almost any other value I can get

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-05 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 Konstantin Belousov changed: What|Removed |Added CC||k...@freebsd.org --- Comment

[Bug 271246] Kernel wired memory leak with repeated netstat

2023-05-04 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271246 Bug ID: 271246 Summary: Kernel wired memory leak with repeated netstat Product: Base System Version: 13.2-STABLE Hardware: Any OS: Any Status: New S