[EMAIL PROTECTED] said: > That is the part of your setup that puzzled me. You took the same 7 disk > raid5 set and split them into 9 LUNS. The Hitachi likely splits the "virtual > disk" into 9 continuous partitions so each LUN maps back to different parts > of the 7 disks. I speculate that ZFS thinks it is talking to 9 different > disks so spreads out the writes accordingly. What ZFS thinks is sequential > writes becomes well spaced writes across the entire disk & blows your seek > time off the roof.
That's what I thought might happen before I even tried this, although it's also possible the Hitachi "stripes" each LUN across all 7 disks. Either way, one could be getting too many seeks. Note that I'm just trying to see if it was so bad that the self-healing capability wasn't worth the cost. I do realize these are 7200rpm SATA disks, so seeking isn't what they do best. > I'm interested how it looks like from the Hitachi end. If you can, > repeat the test with the Hitachi presenting all 7 disks directly to > ZFS as LUNs? The array doesn't give us that capability. > Interesting... what you are suggesting is that %b is 100% when w/s and r/s is > 0? Correct. Sometimes all "iostat -xn" columns are 0 except %b; Sometimes the asvc_t column stays at "4.0" for the duration of the quiet period. I've also observed times where all columns were 0, including %b. Sure is puzzling. [EMAIL PROTECTED] said: > IIRC, the calculation for %busy is the amount of time that an I/O is on the > device. These symptoms would occur if an I/O is dropped somewhere along the > way or at the array. Eventually, we'll timeout and retry, though by default > that should be after 60 seconds. I think we need to figure out what is going > on here before accepting the results. It could be that we're overrunning the > queue on the Hitachi. By default, ZFS will send 35 concurrent commands per > vdev and the ssd driver will send up to 256 to a target. IIRC, Hitachi has a > formula for calculating sdd_max_throttle to avoid such overruns, but I'm not > sure if that applies to this specific array. Hmm, it's true that I have made no tuning changes on the T2000 side. It would make sense if the array just stopped responding. I'll have to poke at the array and see if it has any diagnostics logged somewhere. I recall that the Hitachi docs do have some recommendations on max-throttle settings, so I'll go dig those up and see what I can find out. Thanks for the comments, Marion _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
