On 2/19/2019 2:35 PM, Mike Tancsa wrote:
> dd if=/dev/zero of=/tanker/test bs=1m count=100000
The box has 32G of RAM. If I do a
# sysctl -w vfs.zfs.arc_max=12224866304
vfs.zfs.arc_max: 32224866304 -> 12224866304
#
after WIRED memory is at 29G, it doesnt immediately reclaim it and there
is memory pressure. Booting the box with
vfs.zfs.arc_max=12224866304
keeps WIRED at 15G
last pid: 1049; load averages: 2.02, 2.30,
1.07 up
0+00:04:21 20:17:43
38 processes: 2 running, 36 sleeping
CPU: 3.6% user, 0.0% nice, 5.9% system, 0.0% interrupt, 90.5% idle
Mem: 39M Active, 6212K Inact, 15G Wired, 30M Buf, 16G Free
ARC: 11G Total, 363K MFU, 11G MRU, 117M Anon, 23M Header, 850K Other
11G Compressed, 11G Uncompressed, 1.01:1 Ratio
Swap:
Re-running the test but this time with swap,
# zpool create tanker raidz1 da0p1 da1p1 da2p1 da3p1 da4p1 da5p1 da6p1
da7p1 da8p1 da9p1 da10p1 da11p1 da12p1 da13p1 da14p1
# pstat -T
120/1043936 files
0M/65536M swap space
#
There is still 30G of Wired memory ?
last pid: 1096; load averages: 0.78, 2.13,
1.25 up
0+00:06:42 20:33:21
35 processes: 1 running, 34 sleeping
CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
Mem: 976K Active, 274M Inact, 88M Laundry, 30G Wired, 1341M Buf, 722M Free
ARC: 1227K Total, 775K MFU, 339K MRU, 32K Anon, 9216 Header, 69K Other
176K Compressed, 1179K Uncompressed, 6.70:1 Ratio
Swap: 64G Total, 64G Free
I start a new dd on a freshly created pool,
CPU: 0.0% user, 0.0% nice, 31.4% system, 0.8% interrupt, 67.8% idle
Mem: 1396K Active, 460K Inact, 18M Laundry, 30G Wired, 1341M Buf, 720M Free
ARC: 27G Total, 54K MFU, 26G MRU, 1380M Anon, 56M Header, 3459K Other
26G Compressed, 26G Uncompressed, 1.00:1 Ratio
Swap: 64G Total, 73M Used, 64G Free
and the box starts to swap
last pid: 1104; load averages: 11.61, 6.10,
3.04 up
0+00:09:32 20:36:11
36 processes: 2 running, 34 sleeping
CPU: 0.1% user, 0.0% nice, 95.3% system, 2.1% interrupt, 2.4% idle
Mem: 1212K Active, 124K Inact, 688K Laundry, 30G Wired, 1341M Buf, 819M Free
ARC: 27G Total, 54K MFU, 24G MRU, 1620M Anon, 58M Header, 5572K Other
24G Compressed, 24G Uncompressed, 1.01:1 Ratio
Swap: 64G Total, 90M Used, 64G Free
(avg's oneline script shows)
61,724,160 arc_buf_hdr_t_full 55,263,744 6,460,416
111,783,936 512 109,534,208 2,249,728
555,884,000 UMA_Slabs 553,374,640 2,509,360
1,280,966,656 zio_data_buf_131072 905,576,448 375,390,208
28,142,604,288 abd_chunk 27,965,505,536 177,098,752
30,431,435,848 TOTAL 29,663,389,468 768,046,380
# vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n",
$2*$5/1024/1024, $1}' | sort -k1,1 -rn | head
358 zio_data_buf_131072
168.895 abd_chunk
18.5738 zio_cache
14.25 zio_buf_786432
14 zio_buf_1048576
13.987 BUF TRIE
13.125 zio_buf_655360
9.625 zio_buf_917504
8.75 zio_buf_458752
8 zio_buf_524288
#
As the box is netbooted over nfs, when it swaps out some of the nfs
helper apps, the box gets a bit unresponsive. e.g. if I do a find
/usr/src -type f |xargs md5 the box will stall out for periods of time.
If I limit ARC to say 12G, all is well.
---Mike
--
-------------------
Mike Tancsa, tel +1 519 651 3400 x203
Sentex Communications, m...@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada
_______________________________________________
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
