Here the results with a fresh boot, exec commands, iperf3 trigger and cmands again:
==> FRESH BOOT 1/5594/5595 mbufs in use (current/cache/total) 0/3556/3556/1009905 mbuf clusters in use (current/cache/total/max) 0/2794 mbuf+clusters out of packet secondary zone in use (current/cache) 0/254/254/504952 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/149615 9k jumbo clusters in use (current/cache/total/max) 0/0/0/84158 16k jumbo clusters in use (current/cache/total/max) 0K/9526K/9526K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 sendfile syscalls 0 sendfile syscalls completed without I/O request 0 requests for I/O initiated by sendfile 0 pages read by sendfile as part of a request 0 pages were valid at time of a sendfile request 0 pages were valid and substituted to bogus page 0 pages were requested for read ahead by applications 0 pages were read ahead by sendfile 0 times sendfile encountered an already busy page 0 requests for sfbufs denied 0 requests for sfbufs delayed ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP XDOM kstack_cache: 16384, 0, 481, 17, 549, 0, 0, 0 buffer arena-40: 4096, 0, 0, 0, 0, 0, 0, 0 buffer arena-81: 8192, 0, 0, 0, 0, 0, 0, 0 buffer arena-12: 12288, 0, 0, 0, 0, 0, 0, 0 buffer arena-16: 16384, 0, 10, 0, 10, 0, 0, 0 buffer arena-20: 20480, 0, 0, 0, 0, 0, 0, 0 buffer arena-24: 24576, 0, 0, 0, 0, 0, 0, 0 buffer arena-28: 28672, 0, 0, 0, 0, 0, 0, 0 buffer arena-32: 32768, 0, 1, 0, 1, 0, 0, 0 buffer arena-36: 36864, 0, 0, 0, 0, 0, 0, 0 buffer arena-40: 40960, 0, 0, 0, 0, 0, 0, 0 buffer arena-45: 45056, 0, 0, 0, 0, 0, 0, 0 buffer arena-49: 49152, 0, 0, 0, 0, 0, 0, 0 buffer arena-53: 53248, 0, 0, 0, 0, 0, 0, 0 buffer arena-57: 57344, 0, 0, 0, 0, 0, 0, 0 buffer arena-61: 61440, 0, 0, 0, 0, 0, 0, 0 buffer arena-65: 65536, 0, 0, 0, 0, 0, 0, 0 buf free cache: 432, 0, 11, 199, 11, 0, 0, 0 vm pgcache: 4096, 0, 0, 0, 0, 0, 0, 0 vm pgcache: 4096, 0, 10162, 893, 133125, 16, 0, 0 vm pgcache: 4096, 0, 45571, 1067, 197085, 0, 0, 0 iwlwifi:bc: 4294967295, 0, 0, 0, 0, 0, 0, 0 UMA Kegs: 384, 0, 166, 7, 166, 0, 0, 0 UMA Zones: 2048, 0, 194, 5, 194, 0, 0, 0 UMA Slabs 0: 80, 0, 37546, 48, 37546, 0, 0, 0 UMA Slabs 1: 112, 0, 11, 24, 11, 0, 0, 0 UMA Hash: 256, 0, 0, 0, 0, 0, 0, 0 2 Bucket: 32, 0, 397, 4769, 50194, 0, 0, 0 4 Bucket: 48, 0, 39, 3069, 168, 0, 0, 0 8 Bucket: 80, 0, 200, 1350, 2212, 0, 0, 0 16 Bucket: 144, 0, 157, 739, 1679, 3, 0, 0 32 Bucket: 256, 0, 329, 556, 1274, 3, 0, 0 64 Bucket: 512, 0, 217, 167, 3246, 110, 0, 0 128 Bucket: 1024, 0, 244, 191, 2849,1615, 0, 0 256 Bucket: 2048, 0, 315, 47, 894, 0, 0, 0 SMR SHARED: 24, 0, 8, 503, 8, 0, 0, 0 SMR CPU: 32, 0, 8, 503, 8, 0, 0, 0 vmem: 1856, 0, 2, 7, 2, 0, 0, 0 vmem btag: 56, 0, 13300, 1448, 13547, 102, 0, 0 VM OBJECT: 264, 0, 1337, 373, 64085, 0, 0, 0 RADIX NODE: 152, 0, 7621, 4180, 177415, 0, 0, 0 KMAP ENTRY: 96, 0, 199, 8, 300, 0, 0, 0 MAP ENTRY: 96, 0, 1164, 1860, 221218, 0, 0, 0 VMSPACE: 584, 0, 29, 279, 4346, 0, 0, 0 fakepg: 104, 0, 435, 345, 721, 0, 0, 0 pcpu-4: 4, 0, 0, 0, 0, 0, 0, 0 pcpu-8: 8, 0, 4436, 1708, 4438, 0, 0, 0 pcpu-16: 16, 0, 48, 2512, 48, 0, 0, 0 pcpu-32: 32, 0, 0, 0, 0, 0, 0, 0 pcpu-64: 64, 0, 509, 1539, 509, 0, 0, 0 malloc-16: 16, 0, 18921, 2499, 381981, 0, 0, 0 malloc-32: 32, 0, 12553, 2441, 81424, 0, 0, 0 malloc-64: 64, 0, 23490, 2970, 152215, 0, 0, 0 malloc-128: 128, 0, 31727, 1567, 434736, 0, 0, 0 malloc-256: 256, 0, 3227, 688, 47753, 0, 0, 0 malloc-384: 384, 0, 3102, 258, 32259, 0, 0, 0 malloc-512: 512, 0, 355, 309, 16813, 0, 0, 0 malloc-1024: 1024, 0, 2843, 149, 31468, 0, 0, 0 malloc-2048: 2048, 0, 894, 122, 12662, 0, 0, 0 malloc-4096: 4096, 0, 1248, 24, 28359, 0, 0, 0 malloc-8192: 8192, 0, 162, 14, 1232, 0, 0, 0 malloc-16384: 16384, 0, 117, 17, 2259, 0, 0, 0 malloc-32768: 32768, 0, 102, 7, 1136, 0, 0, 0 malloc-65536: 65536, 0, 46, 5, 990, 0, 0, 0 SLEEPQUEUE: 80, 0, 545, 1119, 545, 0, 0, 0 kenv: 258, 0, 23, 337, 14955, 0, 0, 0 Files: 80, 0, 105, 1695, 35143, 0, 0, 0 filedesc0: 1072, 0, 48, 141, 4363, 0, 0, 0 PWD: 40, 0, 12, 3119, 200, 0, 0, 0 rl_entry: 40, 0, 0, 0, 0, 0, 0, 0 rangeset pctrie nodes: 152, 0, 0, 0, 0, 0, 0, 0 TURNSTILE: 136, 0, 545, 463, 545, 0, 0, 0 lkpicurr: 168, 0, 6, 3186, 9, 0, 0, 0 lkpimm: 56, 0, 2, 3238, 5, 0, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0, 0 umtx_shm: 88, 0, 0, 0, 0, 0, 0, 0 ertt: 72, 0, 0, 280, 3, 0, 0, 0 ertt_txseginfo: 40, 0, 0, 0, 0, 0, 0, 0 swrbuf: 2360, 192, 0, 0, 0, 0, 0, 0 swwbuf: 2360, 96, 0, 0, 0, 0, 0, 0 nfspbuf: 2360, 192, 0, 0, 0, 0, 0, 0 vnpbuf: 2360, 768, 0, 0, 0, 0, 0, 0 clpbuf: 2360, 192, 0, 0, 0, 0, 0, 0 pbuf: 2360, 16, 0, 1460, 0, 0, 0, 0 FPU_save_area: 2696, 0, 483, 35, 612, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0, 0 PROC: 1376, 0, 47, 96, 4362, 0, 0, 0 PGRP: 120, 0, 26, 1261, 44, 0, 0, 0 THREAD: 1824, 0, 481, 63, 549, 0, 0, 0 cpuset: 200, 0, 33, 655, 33, 0, 0, 0 domainset: 40, 0, 0, 0, 0, 0, 0, 0 pkru ranges: 24, 0, 0, 0, 0, 0, 0, 0 audit_record: 1280, 0, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 0, 0, 2794, 724, 0, 0, 0 mbuf: 256,6463393, 1, 2800, 705, 0, 0, 0 mbuf_cluster: 2048,1009905, 2794, 762, 2803, 0, 0, 0 mbuf_jumbo_page: 4096, 504952, 0, 254, 2, 0, 0, 0 mbuf_jumbo_9k: 9216, 149615, 0, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 84158, 0, 0, 0, 0, 0, 0 epoch_record pcpu: 256, 0, 4, 60, 4, 0, 0, 0 ttyinq: 160, 0, 120, 505, 420, 0, 0, 0 ttyoutq: 256, 0, 64, 566, 224, 0, 0, 0 skbuff: 1808, 0, 0, 0, 0, 0, 0, 0 IOMMU_MAP_ENTRY: 112, 0, 0, 0, 0, 0, 0, 0 DEVCTL: 1024, 1000, 0, 800, 752, 0, 0, 0 g_bio: 376, 0, 0, 480, 14032, 0, 0, 0 linux_dma_pctrie: 152, 0, 0, 0, 0, 0, 0, 0 linux_dma_object: 32, 0, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 24, 0, 0, 168, 0, 0, 0, 0 cryptop: 280, 0, 0, 0, 0, 0, 0, 0 taskq_zone: 160, 0, 0, 945, 1510, 0, 0, 0 VNODE: 440, 0, 933, 363, 1409, 0, 0, 0 BUF TRIE: 152, 0, 5, 104593, 5, 0, 0, 0 S VFS Cache: 104, 0, 1359, 669, 1446, 0, 0, 0 STS VFS Cache: 144, 0, 0, 0, 0, 0, 0, 0 L VFS Cache: 320, 0, 0, 36, 1, 0, 0, 0 LTS VFS Cache: 360, 0, 0, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 192, 46690, 0, 0, 0 NCLNODE: 608, 0, 0, 0, 0, 0, 0, 0 TMPFS node: 240, 0, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 0, 0, 0, 0, 0, 0 pipe: 736, 0, 2, 174, 2969, 0, 0, 0 AIO: 208, 0, 0, 0, 0, 0, 0, 0 AIOCB: 552, 0, 0, 0, 0, 0, 0, 0 AIOLIO: 272, 0, 0, 0, 0, 0, 0, 0 Mountpoints: 2816, 0, 29, 19, 29, 0, 0, 0 zfs_btree_leaf_cache: 4096, 0, 1648, 56, 2122, 0, 0, 0 metaslab_alloc_trace_cache:72, 0, 0, 0, 0, 0, 0, 0 brt_entry_cache: 168, 0, 0, 0, 0, 0, 0, 0 ddt_cache: 29240, 0, 5, 1, 5, 0, 0, 0 ddt_entry_flat_cache: 184, 0, 0, 0, 0, 0, 0, 0 ddt_entry_trad_cache: 368, 0, 0, 0, 0, 0, 0, 0 ddt_log_entry_flat_cache:144, 0, 0, 0, 0, 0, 0, 0 ddt_log_entry_trad_cache:328, 0, 0, 0, 0, 0, 0, 0 zio_cache: 1216, 0, 16, 304, 102017, 0, 0, 0 zio_link_cache: 48, 0, 0, 4116, 40478, 0, 0, 0 zio_buf_comb_512: 512, 0, 134, 618, 56623, 0, 0, 0 zio_buf_comb_1024: 1024, 0, 105, 135, 202, 0, 0, 0 zio_buf_comb_1536: 1536, 0, 60, 45, 73, 0, 0, 0 zio_buf_comb_2048: 2048, 0, 42, 62, 44, 0, 0, 0 zio_buf_comb_3072: 3072, 0, 37, 31, 43, 0, 0, 0 zio_buf_comb_4096: 4096, 0, 1281, 21, 2763, 0, 0, 0 zio_buf_comb_6144: 6144, 0, 25, 17, 88, 0, 0, 0 zio_buf_comb_8192: 8192, 0, 15, 14, 169, 0, 0, 0 zio_buf_comb_12288: 12288, 0, 16, 21, 134, 0, 0, 0 zio_buf_comb_16384: 16384, 0, 578, 27, 1222, 0, 0, 0 zio_buf_comb_24576: 24576, 0, 14, 24, 95, 0, 0, 0 zio_buf_comb_32768: 32768, 0, 16, 10, 74, 0, 0, 0 zio_buf_comb_49152: 49152, 0, 15, 18, 247, 0, 0, 0 zio_buf_comb_65536: 65536, 0, 7, 17, 119, 0, 0, 0 zio_buf_comb_98304: 98304, 0, 5, 20, 299, 0, 0, 0 zio_buf_comb_131072: 131072, 0, 291, 36, 1662, 0, 0, 0 zio_buf_comb_196608: 196608, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_262144: 262144, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_393216: 393216, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_524288: 524288, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_786432: 786432, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_1048576:1048576, 0, 0, 1, 15, 0, 0, 0 zio_buf_comb_1572864:1572864, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_2097152:2097152, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_3145728:3145728, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_4194304:4194304, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_6291456:6291456, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_8388608:8388608, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_12582912:12582912, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_16777216:16777216, 0, 0, 0, 0, 0, 0, 0 abd_chunk: 4096, 0, 26560, 20, 88729, 0, 0, 0 sa_cache: 264, 0, 844, 206, 883, 0, 0, 0 dnode_t: 776, 0, 1510, 110, 3659, 0, 0, 0 arc_buf_hdr_t_full: 232, 0, 2997, 420, 31112, 0, 0, 0 arc_buf_hdr_t_l2only: 96, 0, 0, 0, 0, 0, 0, 0 arc_buf_t: 32, 0, 1314, 2340, 31894, 0, 0, 0 dmu_buf_impl_t: 280, 0, 2419, 283, 32336, 0, 0, 0 dbuf_dirty_record_t: 368, 0, 0, 484, 1027, 0, 0, 0 zil_lwb_cache: 376, 0, 2, 118, 8, 0, 0, 0 zil_zcw_cache: 80, 0, 0, 650, 7, 0, 0, 0 sio_cache_0: 136, 0, 0, 0, 0, 0, 0, 0 sio_cache_1: 152, 0, 0, 0, 0, 0, 0, 0 sio_cache_2: 168, 0, 0, 0, 0, 0, 0, 0 zap_name: 328, 0, 0, 360, 135610, 0, 0, 0 zap_attr_cache: 288, 0, 0, 364, 988, 0, 0, 0 zap_name_long: 1096, 0, 0, 0, 0, 0, 0, 0 zap_attr_long_cache: 1056, 0, 0, 98, 4263, 0, 0, 0 zfs_znode_cache: 464, 0, 844, 236, 883, 0, 0, 0 ksiginfo: 112, 0, 87, 1425, 181, 0, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0, 0 KNOTE: 160, 0, 14, 111, 66, 0, 0, 0 socket: 1024, 519607, 34, 3014, 6635, 0, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0, 0 unpcb: 320, 519607, 20, 3028, 6442, 0, 0, 0 ripcb: 376, 0, 1, 29, 1, 0, 0, 0 tcp_inpcb: 1304, 0, 0, 18, 3, 0, 0, 0 udp_inpcb: 408, 0, 9, 324, 106, 0, 0, 0 udplite_inpcb: 408, 0, 0, 0, 0, 0, 0, 0 routing nhops: 256, 0, 8, 307, 13, 0, 0, 0 tcpreass: 48, 63119, 0, 0, 0, 0, 0, 0 tcp_log: 408,5000000, 0, 0, 0, 0, 0, 0 tcp_log_id_bucket: 176, 0, 0, 0, 0, 0, 0, 0 tcp_log_id_node: 120, 0, 0, 0, 0, 0, 0, 0 rtentry: 168, 0, 11, 301, 13, 0, 0, 0 ipq: 56, 31559, 0, 0, 0, 0, 0, 0 syncache: 168, 15360, 0, 0, 0, 0, 0, 0 hostcache: 64, 15360, 0, 0, 0, 0, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0, 0 tfo: 4, 0, 0, 0, 0, 0, 0, 0 tfo_ccache_entries: 80, 0, 0, 0, 0, 0, 0, 0 ada_ccb: 272, 0, 0, 0, 0, 0, 0, 0 da_ccb: 536, 0, 0, 70, 225, 0, 0, 0 swpctrie: 152,2021292, 0, 0, 0, 0, 0, 0 swblk: 136,2021271, 0, 0, 0, 0, 0, 0 drm_buddy_block: 72, 0, 0, 0, 0, 0, 0, 0 drm_sched_fence: 184, 0, 0, 0, 0, 0, 0, 0 mqnode: 416, 0, 3, 33, 3, 0, 0, 0 mqueue: 248, 0, 0, 0, 0, 0, 0, 0 mvdata: 64, 0, 0, 0, 0, 0, 0, 0 mqnotifier: 216, 0, 0, 0, 0, 0, 0, 0 nvidia_stack_t: 12288, 0, 5, 5, 773, 0, 0, 0 fuse_ticket: 184, 0, 0, 0, 0, 0, 0, 0 iwl_cmd_pool:iwlwifi0: 60, 0, 0, 0, 0, 0, 0, 0 NetGraph items: 72, 4096, 0, 704, 60, 0, 0, 0 NetGraph data items: 72, 4096, 0, 864, 54, 0, 0, 0 bridge_rtnode: 88, 0, 0, 0, 0, 0, 0, 0 ==> IPERF % iperf3 -c hp -R Connecting to host hp, port 5201 Reverse mode, remote host hp is sending [ 5] local 192.168.1.82 port 12202 connected to 192.168.1.100 port 5201 rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf [ ID] Interval Transfer Bitrate [ 5] 0.00-1.06 sec 0.00 Bytes 0.00 bits/sec rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf [ 5] 1.06-2.06 sec 0.00 Bytes 0.00 bits/sec rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf [ 5] 2.06-3.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 3.06-4.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 4.06-5.01 sec 0.00 Bytes 0.00 bits/sec rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf rtwn0: rtwn_rx_copy_to_mbuf: could not allocate RX mbuf [ 5] 5.01-6.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 6.06-7.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 7.06-8.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 8.06-9.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 9.06-10.06 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - ==> AFTER 1/6359/6360 mbufs in use (current/cache/total) 0/3810/3810/1009905 mbuf clusters in use (current/cache/total/max) 0/3048 mbuf+clusters out of packet secondary zone in use (current/cache) 0/762/762/504952 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/149615 9k jumbo clusters in use (current/cache/total/max) 0/0/0/84158 16k jumbo clusters in use (current/cache/total/max) 0K/12257K/12258K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 sendfile syscalls 0 sendfile syscalls completed without I/O request 0 requests for I/O initiated by sendfile 0 pages read by sendfile as part of a request 0 pages were valid at time of a sendfile request 0 pages were valid and substituted to bogus page 0 pages were requested for read ahead by applications 0 pages were read ahead by sendfile 0 times sendfile encountered an already busy page 0 requests for sfbufs denied 0 requests for sfbufs delayed ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP XDOM kstack_cache: 16384, 0, 480, 18, 550, 0, 0, 0 buffer arena-40: 4096, 0, 0, 0, 0, 0, 0, 0 buffer arena-81: 8192, 0, 0, 0, 0, 0, 0, 0 buffer arena-12: 12288, 0, 0, 0, 0, 0, 0, 0 buffer arena-16: 16384, 0, 10, 0, 10, 0, 0, 0 buffer arena-20: 20480, 0, 0, 0, 0, 0, 0, 0 buffer arena-24: 24576, 0, 0, 0, 0, 0, 0, 0 buffer arena-28: 28672, 0, 0, 0, 0, 0, 0, 0 buffer arena-32: 32768, 0, 1, 0, 1, 0, 0, 0 buffer arena-36: 36864, 0, 0, 0, 0, 0, 0, 0 buffer arena-40: 40960, 0, 0, 0, 0, 0, 0, 0 buffer arena-45: 45056, 0, 0, 0, 0, 0, 0, 0 buffer arena-49: 49152, 0, 0, 0, 0, 0, 0, 0 buffer arena-53: 53248, 0, 0, 0, 0, 0, 0, 0 buffer arena-57: 57344, 0, 0, 0, 0, 0, 0, 0 buffer arena-61: 61440, 0, 0, 0, 0, 0, 0, 0 buffer arena-65: 65536, 0, 0, 0, 0, 0, 0, 0 buf free cache: 432, 0, 11, 199, 11, 0, 0, 0 vm pgcache: 4096, 0, 0, 0, 0, 0, 0, 0 vm pgcache: 4096, 0, 10211, 1264, 134553, 16, 0, 0 vm pgcache: 4096, 0, 46478, 1252, 199558, 0, 0, 0 iwlwifi:bc: 4294967295, 0, 0, 0, 0, 0, 0, 0 UMA Kegs: 384, 0, 166, 7, 166, 0, 0, 0 UMA Zones: 2048, 0, 194, 5, 194, 0, 0, 0 UMA Slabs 0: 80, 0, 38429, 15, 38429, 0, 0, 0 UMA Slabs 1: 112, 0, 11, 24, 11, 0, 0, 0 UMA Hash: 256, 0, 0, 0, 0, 0, 0, 0 2 Bucket: 32, 0, 414, 4752, 51039, 0, 0, 0 4 Bucket: 48, 0, 39, 3069, 168, 0, 0, 0 8 Bucket: 80, 0, 207, 1343, 2471, 0, 0, 0 16 Bucket: 144, 0, 157, 739, 1682, 3, 0, 0 32 Bucket: 256, 0, 343, 662, 1370, 3, 0, 0 64 Bucket: 512, 0, 254, 162, 3375, 118, 0, 0 128 Bucket: 1024, 0, 258, 177, 2891,1615, 0, 0 256 Bucket: 2048, 0, 331, 47, 924, 0, 0, 0 SMR SHARED: 24, 0, 8, 503, 8, 0, 0, 0 SMR CPU: 32, 0, 8, 503, 8, 0, 0, 0 vmem: 1856, 0, 2, 7, 2, 0, 0, 0 vmem btag: 56, 0, 13333, 1415, 13580, 102, 0, 0 VM OBJECT: 264, 0, 1279, 431, 64575, 0, 0, 0 RADIX NODE: 152, 0, 7556, 4245, 182023, 0, 0, 0 KMAP ENTRY: 96, 0, 199, 8, 300, 0, 0, 0 MAP ENTRY: 96, 0, 1031, 1993, 222995, 0, 0, 0 VMSPACE: 584, 0, 25, 283, 4362, 0, 0, 0 fakepg: 104, 0, 435, 345, 721, 0, 0, 0 pcpu-4: 4, 0, 0, 0, 0, 0, 0, 0 pcpu-8: 8, 0, 4436, 1708, 4438, 0, 0, 0 pcpu-16: 16, 0, 48, 2512, 48, 0, 0, 0 pcpu-32: 32, 0, 0, 0, 0, 0, 0, 0 pcpu-64: 64, 0, 509, 1539, 509, 0, 0, 0 malloc-16: 16, 0, 18918, 2502, 387787, 0, 0, 0 malloc-32: 32, 0, 12555, 2943, 83887, 0, 0, 0 malloc-64: 64, 0, 23475, 3237, 163211, 0, 0, 0 malloc-128: 128, 0, 31801, 1617, 445543, 0, 0, 0 malloc-256: 256, 0, 3252, 783, 52442, 0, 0, 0 malloc-384: 384, 0, 3103, 287, 32746, 0, 0, 0 malloc-512: 512, 0, 323, 341, 17128, 0, 0, 0 malloc-1024: 1024, 0, 2843, 165, 31696, 0, 0, 0 malloc-2048: 2048, 0, 893, 123, 15222, 0, 0, 0 malloc-4096: 4096, 0, 1249, 29, 28782, 0, 0, 0 malloc-8192: 8192, 0, 162, 14, 1232, 0, 0, 0 malloc-16384: 16384, 0, 117, 22, 3078, 0, 0, 0 malloc-32768: 32768, 0, 102, 7, 1136, 0, 0, 0 malloc-65536: 65536, 0, 47, 4, 991, 0, 0, 0 SLEEPQUEUE: 80, 0, 545, 1119, 545, 0, 0, 0 kenv: 258, 0, 23, 337, 14955, 0, 0, 0 Files: 80, 0, 95, 1705, 35503, 0, 0, 0 filedesc0: 1072, 0, 44, 145, 4379, 0, 0, 0 PWD: 40, 0, 12, 3119, 202, 0, 0, 0 rl_entry: 40, 0, 0, 0, 0, 0, 0, 0 rangeset pctrie nodes: 152, 0, 0, 0, 0, 0, 0, 0 TURNSTILE: 136, 0, 545, 463, 545, 0, 0, 0 lkpicurr: 168, 0, 6, 3186, 9, 0, 0, 0 lkpimm: 56, 0, 2, 3238, 5, 0, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0, 0 umtx_shm: 88, 0, 0, 0, 0, 0, 0, 0 ertt: 72, 0, 0, 280, 5, 0, 0, 0 ertt_txseginfo: 40, 0, 0, 606, 8, 0, 0, 0 swrbuf: 2360, 192, 0, 0, 0, 0, 0, 0 swwbuf: 2360, 96, 0, 0, 0, 0, 0, 0 nfspbuf: 2360, 192, 0, 0, 0, 0, 0, 0 vnpbuf: 2360, 768, 0, 0, 0, 0, 0, 0 clpbuf: 2360, 192, 0, 0, 0, 0, 0, 0 pbuf: 2360, 16, 0, 1460, 0, 0, 0, 0 FPU_save_area: 2696, 0, 482, 36, 802, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0, 0 PROC: 1376, 0, 43, 100, 4378, 0, 0, 0 PGRP: 120, 0, 24, 1362, 61, 0, 0, 0 THREAD: 1824, 0, 480, 64, 550, 0, 0, 0 cpuset: 200, 0, 33, 655, 33, 0, 0, 0 domainset: 40, 0, 0, 0, 0, 0, 0, 0 pkru ranges: 24, 0, 0, 0, 0, 0, 0, 0 audit_record: 1280, 0, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 0, 0, 3048, 3558, 0, 0, 0 mbuf: 256,6463393, 1, 3311, 2004, 0, 0, 0 mbuf_cluster: 2048,1009905, 3048, 762, 3057, 0, 0, 0 mbuf_jumbo_page: 4096, 504952, 0, 762, 15, 0, 0, 0 mbuf_jumbo_9k: 9216, 149615, 0, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 84158, 0, 0, 0, 0, 0, 0 epoch_record pcpu: 256, 0, 4, 60, 4, 0, 0, 0 ttyinq: 160, 0, 120, 505, 420, 0, 0, 0 ttyoutq: 256, 0, 64, 566, 224, 0, 0, 0 skbuff: 1808, 0, 0, 0, 0, 0, 0, 0 IOMMU_MAP_ENTRY: 112, 0, 0, 0, 0, 0, 0, 0 DEVCTL: 1024, 1000, 0, 800, 1012, 0, 0, 0 g_bio: 376, 0, 0, 480, 17503, 0, 0, 0 linux_dma_pctrie: 152, 0, 0, 0, 0, 0, 0, 0 linux_dma_object: 32, 0, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 24, 0, 0, 168, 0, 0, 0, 0 cryptop: 280, 0, 0, 0, 0, 0, 0, 0 taskq_zone: 160, 0, 0, 1071, 3055, 0, 0, 0 VNODE: 440, 0, 945, 405, 1507, 0, 0, 0 BUF TRIE: 152, 0, 5, 104593, 5, 0, 0, 0 S VFS Cache: 104, 0, 1391, 637, 1488, 0, 0, 0 STS VFS Cache: 144, 0, 0, 0, 0, 0, 0, 0 L VFS Cache: 320, 0, 0, 36, 1, 0, 0, 0 LTS VFS Cache: 360, 0, 0, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 192, 47261, 0, 0, 0 NCLNODE: 608, 0, 0, 0, 0, 0, 0, 0 TMPFS node: 240, 0, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 0, 0, 0, 0, 0, 0 pipe: 736, 0, 1, 175, 2972, 0, 0, 0 AIO: 208, 0, 0, 0, 0, 0, 0, 0 AIOCB: 552, 0, 0, 0, 0, 0, 0, 0 AIOLIO: 272, 0, 0, 0, 0, 0, 0, 0 Mountpoints: 2816, 0, 29, 19, 29, 0, 0, 0 zfs_btree_leaf_cache: 4096, 0, 1644, 60, 2638, 0, 0, 0 metaslab_alloc_trace_cache:72, 0, 0, 0, 0, 0, 0, 0 brt_entry_cache: 168, 0, 0, 0, 0, 0, 0, 0 ddt_cache: 29240, 0, 5, 1, 5, 0, 0, 0 ddt_entry_flat_cache: 184, 0, 0, 0, 0, 0, 0, 0 ddt_entry_trad_cache: 368, 0, 0, 0, 0, 0, 0, 0 ddt_log_entry_flat_cache:144, 0, 0, 0, 0, 0, 0, 0 ddt_log_entry_trad_cache:328, 0, 0, 0, 0, 0, 0, 0 zio_cache: 1216, 0, 16, 304, 110653, 0, 0, 0 zio_link_cache: 48, 0, 0, 4116, 45797, 0, 0, 0 zio_buf_comb_512: 512, 0, 136, 616, 57515, 0, 0, 0 zio_buf_comb_1024: 1024, 0, 105, 135, 236, 0, 0, 0 zio_buf_comb_1536: 1536, 0, 59, 61, 94, 0, 0, 0 zio_buf_comb_2048: 2048, 0, 42, 70, 50, 0, 0, 0 zio_buf_comb_3072: 3072, 0, 37, 31, 43, 0, 0, 0 zio_buf_comb_4096: 4096, 0, 1308, 18, 3609, 0, 0, 0 zio_buf_comb_6144: 6144, 0, 19, 23, 95, 0, 0, 0 zio_buf_comb_8192: 8192, 0, 12, 24, 203, 0, 0, 0 zio_buf_comb_12288: 12288, 0, 15, 22, 144, 0, 0, 0 zio_buf_comb_16384: 16384, 0, 588, 30, 1580, 0, 0, 0 zio_buf_comb_24576: 24576, 0, 9, 29, 123, 0, 0, 0 zio_buf_comb_32768: 32768, 0, 11, 17, 86, 0, 0, 0 zio_buf_comb_49152: 49152, 0, 9, 24, 265, 0, 0, 0 zio_buf_comb_65536: 65536, 0, 3, 21, 122, 0, 0, 0 zio_buf_comb_98304: 98304, 0, 4, 21, 308, 0, 0, 0 zio_buf_comb_131072: 131072, 0, 298, 31, 2030, 0, 0, 0 zio_buf_comb_196608: 196608, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_262144: 262144, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_393216: 393216, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_524288: 524288, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_786432: 786432, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_1048576:1048576, 0, 0, 1, 15, 0, 0, 0 zio_buf_comb_1572864:1572864, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_2097152:2097152, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_3145728:3145728, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_4194304:4194304, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_6291456:6291456, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_8388608:8388608, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_12582912:12582912, 0, 0, 0, 0, 0, 0, 0 zio_buf_comb_16777216:16777216, 0, 0, 0, 0, 0, 0, 0 abd_chunk: 4096, 0, 26721, 27, 89340, 0, 0, 0 sa_cache: 264, 0, 856, 194, 913, 0, 0, 0 dnode_t: 776, 0, 1540, 110, 3705, 0, 0, 0 arc_buf_hdr_t_full: 232, 0, 3040, 445, 31607, 0, 0, 0 arc_buf_hdr_t_l2only: 96, 0, 0, 0, 0, 0, 0, 0 arc_buf_t: 32, 0, 1215, 2439, 32432, 0, 0, 0 dmu_buf_impl_t: 280, 0, 2333, 369, 32856, 0, 0, 0 dbuf_dirty_record_t: 368, 0, 0, 517, 2241, 0, 0, 0 zil_lwb_cache: 376, 0, 4, 206, 27, 0, 0, 0 zil_zcw_cache: 80, 0, 0, 1050, 20, 0, 0, 0 sio_cache_0: 136, 0, 0, 0, 0, 0, 0, 0 sio_cache_1: 152, 0, 0, 0, 0, 0, 0, 0 sio_cache_2: 168, 0, 0, 0, 0, 0, 0, 0 zap_name: 328, 0, 0, 360, 136781, 0, 0, 0 zap_attr_cache: 288, 0, 0, 364, 988, 0, 0, 0 zap_name_long: 1096, 0, 0, 0, 0, 0, 0, 0 zap_attr_long_cache: 1056, 0, 0, 98, 4394, 0, 0, 0 zfs_znode_cache: 464, 0, 856, 224, 913, 0, 0, 0 ksiginfo: 112, 0, 87, 1425, 361, 0, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0, 0 KNOTE: 160, 0, 14, 111, 1118, 0, 0, 0 socket: 1024, 519607, 30, 3018, 6638, 0, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0, 0 unpcb: 320, 519607, 16, 3032, 6443, 0, 0, 0 ripcb: 376, 0, 1, 29, 1, 0, 0, 0 tcp_inpcb: 1304, 0, 0, 18, 5, 0, 0, 0 udp_inpcb: 408, 0, 9, 324, 106, 0, 0, 0 udplite_inpcb: 408, 0, 0, 0, 0, 0, 0, 0 routing nhops: 256, 0, 8, 307, 13, 0, 0, 0 tcpreass: 48, 63119, 0, 840, 4, 0, 0, 0 tcp_log: 408,5000000, 0, 0, 0, 0, 0, 0 tcp_log_id_bucket: 176, 0, 0, 0, 0, 0, 0, 0 tcp_log_id_node: 120, 0, 0, 0, 0, 0, 0, 0 rtentry: 168, 0, 11, 301, 13, 0, 0, 0 ipq: 56, 31559, 0, 0, 0, 0, 0, 0 syncache: 168, 15360, 0, 0, 0, 0, 0, 0 hostcache: 64, 15360, 1, 314, 1, 0, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0, 0 tfo: 4, 0, 0, 0, 0, 0, 0, 0 tfo_ccache_entries: 80, 0, 0, 0, 0, 0, 0, 0 ada_ccb: 272, 0, 0, 0, 0, 0, 0, 0 da_ccb: 536, 0, 0, 70, 225, 0, 0, 0 swpctrie: 152,2021292, 0, 0, 0, 0, 0, 0 swblk: 136,2021271, 0, 0, 0, 0, 0, 0 drm_buddy_block: 72, 0, 0, 0, 0, 0, 0, 0 drm_sched_fence: 184, 0, 0, 0, 0, 0, 0, 0 mqnode: 416, 0, 3, 33, 3, 0, 0, 0 mqueue: 248, 0, 0, 0, 0, 0, 0, 0 mvdata: 64, 0, 0, 0, 0, 0, 0, 0 mqnotifier: 216, 0, 0, 0, 0, 0, 0, 0 nvidia_stack_t: 12288, 0, 5, 6, 1133, 0, 0, 0 fuse_ticket: 184, 0, 0, 0, 0, 0, 0, 0 iwl_cmd_pool:iwlwifi0: 60, 0, 0, 0, 0, 0, 0, 0 NetGraph items: 72, 4096, 0, 704, 60, 0, 0, 0 NetGraph data items: 72, 4096, 0, 864, 54, 0, 0, 0 bridge_rtnode: 88, 0, 0, 0, 0, 0, 0, 0 Adrian Chadd <adrian.ch...@gmail.com> escreveu (sexta, 25/04/2025 à(s) 23:40): > > Did you see more mbuf alloc failures in between these calls? > > > > > -adrian > > > On Fri, 25 Apr 2025 at 15:37, Nuno Teixeira <edua...@freebsd.org> wrote: >> >> Before iperf3: >> >> netstat -mb >> 27/8103/8130 mbufs in use (current/cache/total) >> 1/5333/5334/1009905 mbuf clusters in use (current/cache/total/max) >> 0/4318 mbuf+clusters out of packet secondary zone in use (current/cache) >> 0/3810/3810/504952 4k (page size) jumbo clusters in use >> (current/cache/total/max) >> 0/0/0/149615 9k jumbo clusters in use (current/cache/total/max) >> 0/0/0/84158 16k jumbo clusters in use (current/cache/total/max) >> 9K/27931K/27940K bytes allocated to network (current/cache/total) >> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >> 0 sendfile syscalls >> 0 sendfile syscalls completed without I/O request >> 0 requests for I/O initiated by sendfile >> 0 pages read by sendfile as part of a request >> 0 pages were valid at time of a sendfile request >> 0 pages were valid and substituted to bogus page >> 0 pages were requested for read ahead by applications >> 0 pages were read ahead by sendfile >> 0 times sendfile encountered an already busy page >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> >> After iperf3 (trigger) >> >> % netstat -mb >> 26/8104/8130 mbufs in use (current/cache/total) >> 1/5333/5334/1009905 mbuf clusters in use (current/cache/total/max) >> 0/4318 mbuf+clusters out of packet secondary zone in use (current/cache) >> 0/3810/3810/504952 4k (page size) jumbo clusters in use >> (current/cache/total/max) >> 0/0/0/149615 9k jumbo clusters in use (current/cache/total/max) >> 0/0/0/84158 16k jumbo clusters in use (current/cache/total/max) >> 8K/27932K/27940K bytes allocated to network (current/cache/total) >> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> 0/0/0 requests for jumbo clusters denied (4k/9k/16k) >> 0 sendfile syscalls >> 0 sendfile syscalls completed without I/O request >> 0 requests for I/O initiated by sendfile >> 0 pages read by sendfile as part of a request >> 0 pages were valid at time of a sendfile request >> 0 pages were valid and substituted to bogus page >> 0 pages were requested for read ahead by applications >> 0 pages were read ahead by sendfile >> 0 times sendfile encountered an already busy page >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> >> Adrian Chadd <adrian.ch...@gmail.com> escreveu (sexta, 25/04/2025 à(s) >> 23:33): >> > >> > there's nothing odd in the stats output. >> > >> > Try netstat -mb before and after. This error really feels like it's >> > somewhere else in the kernel. >> > >> > >> > >> > -adrian >> > >> >> >> -- >> Nuno Teixeira >> FreeBSD UNIX: <edua...@freebsd.org> Web: https://FreeBSD.org -- Nuno Teixeira FreeBSD UNIX: <edua...@freebsd.org> Web: https://FreeBSD.org
