I note in your iostat data below that one drive (sd5) consistently performs MUCH worse than the others, even when doing less work.....
-----Original Message----- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of John J Balestrini Sent: Tuesday, May 18, 2010 8:11 AM To: OpenSolaris ZFS discuss Subject: Re: [zfs-discuss] Very serious performance degradation Howdy, Is dedup on? I was having some pretty strange problems including slow performance when dedup was on. Disabling dedup helped out a whole bunch. My system only has 4gig of ram, so that may have played a part too. Good luck! John On May 18, 2010, at 7:51 AM, Philippe wrote: > Hi, > > The 4 disks are Western Digital ATA 1TB (one is slighlty different) : > 1 x ATA-WDC WD10EACS-00D-1A01-931.51GB > 3 x ATA-WDC WD10EARS-00Y-0A80-931.51GB > > I've done lots of tests (speed tests + SMART reports) with each of these 4 > disk on another system (another computer, running Windows 2003 x64), and > everything was fine ! The 4 disks operate well, at 50-100 MB/s (tested with > Hdtune). And the access time : 14ms > > The controller is an LSI Logic SAS 1068-IR (MPT BIOS 6.12.00.00 - 31/10/2006) > > Here are some stats : > > 1) cp of a big file to a ZFS filesystem (128K recordsize) : > ============================================================= > iostat -x 30 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 0.3 0.3 17.6 2.3 0.0 0.0 19.5 0 0 > sd2 11.5 6.0 350.1 154.5 0.0 0.3 19.5 0 4 > sd3 12.5 5.7 351.4 154.5 0.0 0.5 27.1 0 5 > sd4 15.9 6.3 615.1 153.8 0.0 1.3 58.2 0 8 > sd5 15.1 8.1 600.4 150.7 0.0 7.6 326.7 0 31 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 41.3 0.0 5289.7 0.0 0.0 1.3 31.0 0 4 > sd2 4.2 24.1 214.0 1183.0 0.0 0.5 19.4 0 4 > sd3 3.7 23.6 227.2 1183.0 0.0 2.1 78.5 0 12 > sd4 6.6 26.4 374.2 1179.4 0.0 10.1 306.5 0 35 > sd5 4.3 31.0 369.6 973.3 0.0 22.0 622.0 0 96 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 17.1 0.0 2184.6 0.0 0.0 0.5 30.6 0 2 > sd2 1.6 12.3 116.4 570.9 0.0 0.6 41.3 0 3 > sd3 1.6 12.1 107.6 570.9 0.0 10.3 754.7 0 33 > sd4 2.1 12.6 187.1 569.4 0.0 9.4 634.7 0 28 > sd5 0.4 21.7 25.6 700.6 0.0 29.5 1338.1 0 96 > > > 2) cp of a big file to a ZFS filesystem (16K recordsize) : > ============================================================= > iostat -x 30 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 0.2 0.3 16.7 2.3 0.0 0.0 19.3 0 0 > sd2 11.5 6.0 350.7 154.5 0.0 0.3 19.5 0 4 > sd3 12.5 5.7 352.0 154.5 0.0 0.5 27.0 0 5 > sd4 15.9 6.3 616.2 153.8 0.0 1.3 58.0 0 8 > sd5 15.1 8.1 601.3 150.7 0.0 7.5 324.6 0 31 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 32.0 0.0 4095.9 0.0 0.0 1.0 30.8 0 3 > sd2 2.0 22.4 124.2 425.0 0.0 0.1 2.3 0 2 > sd3 1.9 19.4 115.9 425.0 0.0 0.6 28.7 0 14 > sd4 2.3 23.6 170.9 421.8 0.0 3.2 124.7 0 15 > sd5 3.2 24.5 290.6 306.6 0.0 22.5 810.5 0 94 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd2 0.0 2.0 0.0 3.0 0.0 0.0 0.7 0 0 > sd3 0.1 1.1 4.3 2.0 0.0 0.0 15.9 0 1 > sd4 0.1 1.4 4.3 1.9 0.0 0.0 2.9 0 0 > sd5 0.2 19.8 10.7 101.8 0.0 32.1 1606.9 0 100 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 8.6 0.0 1096.2 0.0 0.0 0.3 29.7 0 1 > sd2 0.2 4.8 10.7 267.2 0.0 0.0 7.8 0 0 > sd3 0.2 5.5 6.8 268.2 0.0 0.6 107.0 0 3 > sd4 0.2 9.1 11.0 265.4 0.0 6.3 678.4 0 21 > sd5 0.2 21.4 6.8 104.5 0.0 31.6 1467.8 0 92 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd5 0.0 18.9 0.0 101.7 0.0 35.0 1851.6 0 100 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd2 0.3 5.9 15.3 279.2 0.0 0.0 6.7 0 1 > sd3 0.4 5.7 23.5 279.2 0.0 1.0 161.5 0 5 > sd4 0.4 11.6 23.8 275.6 0.0 11.6 964.3 0 36 > sd5 0.2 20.6 13.1 107.2 0.0 30.2 1452.7 0 99 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd5 0.0 20.1 0.0 105.7 0.0 35.0 1741.2 0 100 > extended device statistics > device r/s w/s kr/s kw/s wait actv svc_t %w %b > sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 > sd2 0.1 4.3 6.4 196.4 0.0 0.1 13.8 0 1 > sd3 0.2 4.0 10.7 196.4 0.0 0.1 15.9 0 1 > sd4 0.3 5.5 17.1 195.6 0.0 0.2 31.3 0 3 > sd5 0.3 22.5 17.1 127.4 0.0 31.4 1378.4 0 99 > > > Thanks, > Philippe > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
