Hi, thanks for respose. As this is close-source mailserver (CommuniGate pro), I can't say 100% answer, but the writes that I see that take too much time (15-30secs) are writes from temp queue to final storage, and from my understanding, they are sync so the queue manager can guarantee they are on solid storage.
Apart from that however the interactive access to mailstore modifies filenames ( read/opened/deleted flags are part of filename for each file) a lot content of files is not changed any more. Also moves abetween directories and deletions are only dir operations ( don't know whether sync or not - you know OS internals better). >From description of the error you mentioned, and also 6440499 zil should avoid txg_wait_synced() and use dmu_sync() to issue parallel IOs when fsyncing I think that this may be also my case. So my question is: I run Sol10U2, is there a way to quicky test new ZFS, without reinstalling whole system? Please say there is.... Ivan -----Original Message----- From: eric kustarz [mailto:[EMAIL PROTECTED] Sent: Thursday, September 07, 2006 8:39 PM To: Ivan Debnár Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Performance problem of ZFS ( Sol 10U2 ) Ivan Debnár wrote: >Hi, > >I deployed ZFS on our mailserver recently, hoping for eternal peace after >running on UFS and moving files witch each TB added. > >It is mailserver - it's mdirs are on ZFS pool: > capacity operations bandwidth >pool used avail read write read write >------------------------- ----- ----- ----- ----- ----- ----- >mailstore 3.54T 2.08T 280 295 7.10M 5.24M > mirror 590G 106G 34 31 676K 786K > c6t3d0 - - 14 16 960K 773K > c8t22260001552EFE2Cd0 - - 16 18 1.06M 786K > mirror 613G 82.9G 51 37 1.44M 838K > c6t3d1 - - 20 19 1.57M 824K > c5t1d1 - - 20 24 1.40M 838K > c8t227C0001559A761Bd0 - - 5 101 403K 4.63M > mirror 618G 78.3G 133 60 6.23M 361K > c6t3d2 - - 40 27 3.21M 903K > c4t2d0 - - 23 81 1.91M 2.98M > c8t221200015599F2CFd0 - - 6 108 442K 4.71M > mirror 613G 83.2G 110 51 3.66M 337K > c6t3d3 - - 36 25 2.72M 906K > c5t2d1 - - 29 65 1.80M 2.92M > mirror 415G 29.0G 30 28 460K 278K > c6t3d4 - - 11 19 804K 268K > c4t1d2 - - 15 22 987K 278K > mirror 255G 441G 26 49 536K 1.02M > c8t22110001552F3C46d0 - - 12 27 835K 1.02M > c8t224B0001559BB471d0 - - 12 29 835K 1.02M > mirror 257G 439G 32 52 571K 1.04M > c8t22480001552D7AF8d0 - - 14 28 1003K 1.04M > c4t1d0 - - 14 32 1002K 1.04M > mirror 251G 445G 28 53 543K 1.02M > c8t227F0001552CB892d0 - - 13 28 897K 1.02M > c8t22250001559830A5d0 - - 13 30 897K 1.02M > mirror 17.4G 427G 22 38 339K 393K > c8t22FA00015529F784d0 - - 9 19 648K 393K > c5t2d2 - - 9 23 647K 393K > > >It is 3x dual-iSCSI + 2x dual SCSI DAS arrays (RAID0, 13x250). > >I have problem however: >The 2 SCSI arrays were able to handle the mail-traffic fine with UFS on them. >The new config with 3 additional arrays seem to have problem using ZFS. >The writes are waiting for 10-15 seconds to get to disk - so queue fills ver >quickly, reads are quite ok. > > Are those synchronouse writes or asynchronous? If both, what are the percentages of each? Neil just putback a fix into snv_48 for: 6413510 zfs: writing to ZFS filesystem slows down fsync() on other files in the same FS Basically the fsync/synchronous writes end up doing more work than they should - instead of writing the data and meta-data for just the file you're trying to fsync, you will write (and wait for) other files' data & meta-data too. eric >I assume this is the problem with ZFS prefering reads to writes. > >I also see in 'zpool iostat -v 1' that writes are issued to disk only once in >10 secs, and then its 2000rq one sec. >Reads are sustained at cca 800rq/s. > >Is there a way to tune this read/write ratio? Is this know problem? > >I tried to change vq_max_pending as suggested by Eric in >http://blogs.sun.com/erickustarz/entry/vq_max_pending >But no change in this write behaviour. > >Iostat shows cca 20-30ms asvc_t, 0%w, and cca 30% busy on all drives so these >are not saturated it seems. (before with UTF they had 90%busy, 1%wait). > >System is Sol 10 U2, sun x4200, 4GB RAM. > >Please if you could give me some hint to really make this working as the way >back to UFS is almost impossible on live system. > > > > > _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
