Hi roy, Hi Dan, many thanks for Your responses.
I am using napp-it to control the OpenSolaris-Systems The napp-it-interface shows a dedup factor of 1.18x on System 1 and 1.16x on System 2. Dedup is on "always" (not only at the start), also compression is activated: System 1 = compression on (lzib?) System 2 = compression on (gzip-6) compression rates: System 1 = 1.10x System 2 = 1.48x compression and dedup were some of the primary reasons to choose zfs in this situation. We tried more RAM (48GB) at the beginning to check if this would do anything on performance. But it did not, but we had only about 3-4TB of data on the storage at that time (performance was good). So I will order some RAM-modules and double the RAM to 48GB. The RAM usage is about 21GB, 3GB free (on both systems after a while). At the start (after a few hours of usage and only 3-4TB of data) the usage was identical. I read on some places, that zfs will use all memory, leaving only 1GB left (so I thought the RAM wasn't used completely). The swap isn't used by the system, Now the systems are idle and the RAM-usage is very little: top: System 1 Memory: 24G phys mem, 21G free mem, 12G total swap, 12G free swap System 2 Memory: 24G phys mem, 21G free mem, 12G total swap, 12G free swap Starting to read about 12GB from System 2 and RAM-Usage goes up to performance is at about 65-70MB/s via GigaBit (iSCSI). Memory: 24G phys mem, 3096M free mem, 12G total swap, 12G free swap ok, I understand, more RAM will be no fault.. ;) On System 1 there is no such massive change in RAM usage while copying files to and from the volume. But the Performance is only about 20MB/s via GigaBit (iSCSI). So RAM can´t be the issue on System 1 (which has more data stored). This system ist also equipped with a 240GB SSD used for L2ARC at the second LSI-controller inside the server-enclosure. Roy: But is the L2ARC also important while writing to the device? Because the storeges are used most of the time only for writing data on it, the Read-Cache (as I thought) isn´t a performance-factor... Please correct me, if my thoughts are wrong... But this is only a "small cost"-addition, to add also a 120GB/240GB OCZ Vertex 2 SSD to System 2 (≈ 150/260 Euro). I will give it a try. Would it be better to add the SSD to the LSI-Controller (put it in the JBOD-Storage) or put it in the server enclosure itself and connect it to the internal SATA-Controller? Do You have any tips for the settings of the dataset? These are the settings PROPERTY System 1 System 2 used 34.4T 19.4T available 10.7T 40.0T referenced 34.4T 19.4T compressratio 1.10x 1.43x mounted yes yes quota none none reservation none none recordsize 128K 128k mountpoint / / sharenfs off off checksum on on compression on gzip atime off off devices on on exec on on setuid on on readonly off off zoned off off snapdir hidden hidden aclinherit passthrough passthrough canmount on on xattr on on copies 1 1 version 5 5 utf8only off off normalization none none casesensitivity insensitive insensitive vscan off off nbmand off off sharesmb off off refquota none none refreservation none none primarycache metadata all secondarycache all all usedbysnapshots 0 0 usedbydataset 34.4T 19.4T usedbychildren 0 0 usedbyrefreservation 0 0 logbias latency latency dedup on on mlslabel none none sync standard standard Best regards Sven C. merckensscATmac.com On Wed, Jun 15, 2011 at 07:19:05PM +0200, Roy Sigurd Karlsbakk wrote: > > Dedup is known to require a LOT of memory and/or L2ARC, and 24GB isn't really > much with 34TBs of data. The fact that your second system lacks the l2arc cache device is absolutely your prime suspect. Am 15.06.2011 um 19:19 schrieb Roy Sigurd Karlsbakk: >> System1 (inhouse) >> ---------------- >> SuperMicro-enclosure 2U SC825TQ >> Mainboard: X8DTH-IF >> 1 x 1 x Quad-Core Intel Xeon E5620 processor, 2.4GHz, 12MB L3 Cache >> 24GB of RAM (3 x 8GB) >> 1 x LSISAS9211-8I (for the internal 8 drive-carriers) >> 1 x LSISAS9200-8E (for the JBOD) >> >> attached is a >> 1 x SC847E16-RJBOD1 with two backplanes (each backplane is connected >> to one port of the LSI-controller). > > You mention further down that you had dedup turned on for some time. Could > you do a "zpool list" and report the dedup amount? > > Dedup is known to require a LOT of memory and/or L2ARC, and 24GB isn't really > much with 34TBs of data. > > Vennlige hilsener / Best regards > > roy > -- > Roy Sigurd Karlsbakk > (+47) 97542685 > r...@karlsbakk.net > http://blogg.karlsbakk.net/ > -- > I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det > er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av > idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og > relevante synonymer på norsk. _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
