Richard - the l2arc is c1t13d0. What tools can be use to show the l2arc stats?
raidz1 2.68T 580G 543 453 4.22M 3.70M
c1t1d0 - - 258 102 689K 358K
c1t2d0 - - 256 103 684K 354K
c1t3d0 - - 258 102 690K 359K
c1t4d0 - - 260 103 687K 354K
c1t5d0 - - 255 101 686K 358K
c1t6d0 - - 263 103 685K 354K
c1t7d0 - - 259 101 689K 358K
c1t8d0 - - 259 103 687K 354K
c1t9d0 - - 260 102 689K 358K
c1t10d0 - - 263 103 686K 354K
c1t11d0 - - 260 102 687K 359K
c1t12d0 - - 263 104 684K 354K
c1t14d0 396K 29.5G 0 65 7 3.61M
cache - - - - - -
c1t13d0 29.7G 11.1M 157 84 3.93M 6.45M
We've added 16GB to the box bring the overall total to 32GB.
arc_max is set to 8GB:
set zfs:zfs_arc_max = 8589934592
arc_summary output:
ARC Size:
Current Size: 8192 MB (arcsize)
Target Size (Adaptive): 8192 MB (c)
Min Size (Hard Limit): 1024 MB (zfs_arc_min)
Max Size (Hard Limit): 8192 MB (zfs_arc_max)
ARC Size Breakdown:
Most Recently Used Cache Size: 39% 3243 MB (p)
Most Frequently Used Cache Size: 60% 4948 MB (c-p)
ARC Efficency:
Cache Access Total: 154663786
Cache Hit Ratio: 41% 64221251 [Defined State for
buffer]
Cache Miss Ratio: 58% 90442535 [Undefined State for
Buffer]
REAL Hit Ratio: 41% 64221251 [MRU/MFU Hits Only]
Data Demand Efficiency: 38%
Data Prefetch Efficiency: DISABLED (zfs_prefetch_disable)
CACHE HITS BY CACHE LIST:
Anon: --% Counter Rolled.
Most Recently Used: 17% 11118906 (mru) [
Return Customer ]
Most Frequently Used: 82% 53102345 (mfu) [
Frequent Customer ]
Most Recently Used Ghost: 14% 9427708 (mru_ghost) [
Return Customer Evicted, Now Back ]
Most Frequently Used Ghost: 6% 4344287 (mfu_ghost) [
Frequent Customer Evicted, Now Back ]
CACHE HITS BY DATA TYPE:
Demand Data: 84% 54444108
Prefetch Data: 0% 0
Demand Metadata: 15% 9777143
Prefetch Metadata: 0% 0
CACHE MISSES BY DATA TYPE:
Demand Data: 96% 87542292
Prefetch Data: 0% 0
Demand Metadata: 3% 2900243
Prefetch Metadata: 0% 0
Also disabled file-level pre-fletch and vdev cache max:
set zfs:zfs_prefetch_disable = 1
set zfs:zfs_vdev_cache_max = 0x1
After reading about some issues with concurrent ios, I tweaked the setting down
from 35 to 1 and it reduced the response times greatly (2 -> 8ms):
set zfs:zfs_vdev_max_pending=1
It did increased the actv...I'm still unsure about the side-effects here:
r/s w/s Mr/s Mw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t0d0
2295.2 398.7 4.2 7.2 0.0 18.6 0.0 6.9 0 1084 c1
0.0 0.8 0.0 0.0 0.0 0.0 0.0 0.1 0 0 c1t0d0
190.3 22.9 0.4 0.0 0.0 1.5 0.0 7.0 0 87 c1t1d0
180.9 20.6 0.3 0.0 0.0 1.7 0.0 8.5 0 95 c1t2d0
195.0 43.0 0.3 0.2 0.0 1.6 0.0 6.8 0 93 c1t3d0
193.2 21.7 0.4 0.0 0.0 1.5 0.0 6.8 0 88 c1t4d0
195.7 34.8 0.3 0.1 0.0 1.7 0.0 7.5 0 97 c1t5d0
186.8 20.6 0.3 0.0 0.0 1.5 0.0 7.3 0 88 c1t6d0
188.4 21.0 0.4 0.0 0.0 1.6 0.0 7.7 0 91 c1t7d0
189.6 21.2 0.3 0.0 0.0 1.6 0.0 7.4 0 91 c1t8d0
193.8 22.6 0.4 0.0 0.0 1.5 0.0 7.1 0 91 c1t9d0
192.6 20.8 0.3 0.0 0.0 1.4 0.0 6.8 0 88 c1t10d0
195.7 22.2 0.3 0.0 0.0 1.5 0.0 6.7 0 88 c1t11d0
184.7 20.3 0.3 0.0 0.0 1.4 0.0 6.8 0 84 c1t12d0
7.3 82.4 0.1 5.5 0.0 0.0 0.0 0.2 0 1 c1t13d0
1.3 23.9 0.0 1.3 0.0 0.0 0.0 0.2 0 0 c1t14d0
I'm still in talks with the dba in seeing if we can raise the SGA from 4GB to
6GB to see if it'll help.
The changes that showed a lot of improvement is disabling file/device level
pre-fletch and reducing concurrent ios from 35 to 1 (tried 10 but it didn't
help much). Is there anything else that could be tweaked to increase write
performance? Record sizes are set according to 8K and 128K for redo logs.
--
This message posted from opensolaris.org
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
