Re: [zfs-discuss] Optimal Disk configuration

2010-07-21 Thread Erik Trimble
On 7/21/2010 2:38 PM, Cindy Swearingen wrote: The answer depends on your goals: space, performance, reliability To me, optimal is best performance and reliability so use: - JBOD - ZFS mirrored pool of 22x2 + 2 spares - Mirror the disk pairs across both controllers Let ZFS protect your data. C

Re: [zfs-discuss] slog/L2ARC on a hard drive and not SSD?

2010-07-21 Thread Scott Meilicke
Another data point - I used three 15K disks striped using my RAID controller as a slog for the zil, and performance went down. I had three raidz sata vdevs holding the data, and my load was VMs, i.e. a fair amount of small, random IO (60% random, 50% write, ~16k in size). Scott -- This messag

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Erik Trimble
On 7/21/2010 7:47 AM, Eugen Leitl wrote: How badly would a dual-core 1.6 GHz Atom with 4 GBytes RAM be underpowered for serving 4-6 SATA drives? What kind of transfer speed (GBit Ethernet, Intel NICs) can I expect with raidz2 or raidz3? Thanks. For light usage (I'm assuming you want to us

Re: [zfs-discuss] Optimal Disk configuration

2010-07-21 Thread Cindy Swearingen
The answer depends on your goals: space, performance, reliability To me, optimal is best performance and reliability so use: - JBOD - ZFS mirrored pool of 22x2 + 2 spares - Mirror the disk pairs across both controllers Let ZFS protect your data. Cindy On 07/21/10 15:10, John Andrunas wrote:

Re: [zfs-discuss] carrying on [was: Legality and the future of zfs...]

2010-07-21 Thread Linder, Doug
Andrej Podzimek wrote: > 1) Btrfs does not have mature and user-friendly command-line > tools. AFAIK, you can only list your snapshots and subvolumes by > grep'ing the tree dump. ;-) I haven't looked closely at the btrfs commands recently, but from what I've seen, they're really amazingly

[zfs-discuss] Optimal Disk configuration

2010-07-21 Thread John Andrunas
I know this is potentially a loaded question, but what is generally considered the optimal disk configuration for ZFS. I have 48 disks on 2 RAID controllers (2x24). The RAID controller can do RAID 0/1/5/6/10/50/60 or JBOD. What is generally considered the optimal configuration for the disks. JB

Re: [zfs-discuss] Confused about consumer drives and zfs can someone help?

2010-07-21 Thread Freddie Cash
On Wed, Jul 21, 2010 at 1:38 PM, JavaWebDev wrote: > 1.  WD Caviar Black > > Can they be used with in raidz or mirrors? We use the 500 GB versions attached to 3Ware controllers (configured as Single Disk arrays). They work quite nicely. > With the new models the firmwar

[zfs-discuss] Confused about consumer drives and zfs can someone help?

2010-07-21 Thread JavaWebDev
I wanted to build a small back up (maybe also NAS) server using OpenSolaris and ZFS using consumer drives but after reading a number of threads and blogs I'm totally confused and was hoping I could get some questions answered since many people have been using consumer drives with zfs. When Z

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-21 Thread Chad Cantwell
It does seem to be faster now that I really installed the non-debug bits. I let it resume a scrub after reboot, and while it's not as fast as it usually is (280 - 300 MB/s vs 500+) I assume it's just presently checking a part of the filesystem currently with smaller files thus reducing the spee

Re: [zfs-discuss] L2ARC and ZIL on same SSD?

2010-07-21 Thread Garrett D'Amore
On Wed, 2010-07-21 at 09:42 -0700, Orvar Korvar wrote: > Are there any drawbacks to partition a SSD in two parts and use L2ARC on one > partition, and ZIL on the other? Any thoughts? Its probably a reasonable approach. The ZIL can be fairly small... only about 8 GB is probably sufficient for mo

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-21 Thread Chad Cantwell
Hi, My bits were originally debug because I didn't know any better. I thought I had then recompiled without debug to test again, but I didn't realize until just now the packages end up in a different directory (nightly vs nightly-nd) so I believe after compiling non-debug I just reinstalled th

[zfs-discuss] L2ARC and ZIL on same SSD?

2010-07-21 Thread Orvar Korvar
Are there any drawbacks to partition a SSD in two parts and use L2ARC on one partition, and ZIL on the other? Any thoughts? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/

Re: [zfs-discuss] How does zil work

2010-07-21 Thread Christopher George
Here is another very recent blog post from ConstantThinking: http://constantin.glez.de/blog/2010/07/solaris-zfs-synchronous-writes-and-zil-explained Very well done, a highly recommended read. Christopher George Founder/CTO www.ddrdrive.com -- This message posted from opensolaris.org ___

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-21 Thread Garrett D'Amore
On Wed, 2010-07-21 at 02:21 -0400, Richard Lowe wrote: > I built in the normal fashion, with the CBE compilers > (cc: Sun C 5.9 SunOS_i386 Patch 124868-10 2009/04/30), and 12u1 lint. > > I'm not subscribed to zfs-discuss, but have you established whether the > problematic build is DEBUG? (the bits

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Garrett D'Amore
On Wed, 2010-07-21 at 17:12 +0200, Saso Kiselkov wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > If you plan on using it as a storage server for multimedia data > (movies), don't even bother considering compression, as most media files > already come heavily compressed. Dedup might st

Re: [zfs-discuss] zpool throughput: snv 134 vs 138 vs 143

2010-07-21 Thread Richard Lowe
I built in the normal fashion, with the CBE compilers (cc: Sun C 5.9 SunOS_i386 Patch 124868-10 2009/04/30), and 12u1 lint. I'm not subscribed to zfs-discuss, but have you established whether the problematic build is DEBUG? (the bits I uploaded were non-DEBUG). -- Rich Haudy Kazemi wrote: >>> C

Re: [zfs-discuss] Deleting large amounts of files

2010-07-21 Thread Hernan Freschi
On Tue, Jul 20, 2010 at 1:40 PM, Ulrich Graef wrote: > When you are writing to a file and currently dedup is enabled, then the > Data is entered into the dedup table of the pool. > (There is one dedup table per pool not per zfs). > > Switching off the dedup does not change this data. Yes, i suppo

Re: [zfs-discuss] Deleting large amounts of files

2010-07-21 Thread Ulrich Graef
Hi, Hernan Freschi wrote: Hi, thanks for answering, How large is your ARC / your main memory? Probably too small to hold all metadata (1/1000 of the data amount). => metadata has to be read again and again Main memory is 8GB. ARC (according to arcstat.pl) usually stays at 5-7GB

Re: [zfs-discuss] zfs raidz1 and traditional raid 5 perfomrance comparision

2010-07-21 Thread Ulrich Graef
There is a common misconception about the comparison between mirror and raidz. You get the same performance, when you use the same number of disks. But the resulting filesystem has a different sizre, therefore a comparison is not applicable. Example: you have 8 disks Compare a zpool with one

Re: [zfs-discuss] slog/L2ARC on a hard drive and not SSD?

2010-07-21 Thread Garrett D'Amore
On Wed, 2010-07-21 at 07:56 -0700, Hernan F wrote: > Hi, > Out of pure curiosity, I was wondering, what would happen if one tries to use > a regular 7200RPM (or 10K) drive as slog or L2ARC (or both)? > > I know these are designed with SSDs in mind, and I know it's possible to use > anything you

Re: [zfs-discuss] Deleting large amounts of files

2010-07-21 Thread Hernan F
I have 8GB RAM, arcsz as reported by arcstat.pl is 5-7GB usually. It took about 20-30 mins to delete the files. Is there a way to see which files have been deduped, so I can copy them again an un-dedupe them? Thanks, Hernan -- This message posted from opensolaris.org __

Re: [zfs-discuss] slog/L2ARC on a hard drive and not SSD?

2010-07-21 Thread Marty Scholes
> Hi, > Out of pure curiosity, I was wondering, what would > happen if one tries to use a regular 7200RPM (or 10K) > drive as slog or L2ARC (or both)? I have done both with success. At one point my backup pool was a collection of USB attached drives (please keep the laughter down) with dedup=ver

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I do encounter situations when I (or somebody from my family) accidentally create multiple copies of photo albums. :-) - -- Saso On 07/21/2010 05:20 PM, Edward Ned Harvey wrote: >> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- >> bou

Re: [zfs-discuss] zfs raidz1 and traditional raid 5 perfomrance comparision

2010-07-21 Thread Robert Milkowski
On 21/07/2010 15:40, Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of v for zfs raidz1, I know for random io, iops of a raidz1 vdev eqaul to one physical disk iops, since raidz1 is like raid5 , so is raid5 has same perf

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Saso Kiselkov > > If you plan on using it as a storage server for multimedia data > (movies), don't even bother considering compression, as most media > files > already come heavily compressed.

Re: [zfs-discuss] slog/L2ARC on a hard drive and not SSD?

2010-07-21 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Hernan F > > Hi, > Out of pure curiosity, I was wondering, what would happen if one tries > to use a regular 7200RPM (or 10K) drive as slog or L2ARC (or both)? I tested it once, for the same r

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Darren J Moffat
On 21/07/2010 16:12, Saso Kiselkov wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If you plan on using it as a storage server for multimedia data (movies), don't even bother considering compression, as most media files already come heavily compressed. Dedup might still come in handy, thoug

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Saso Kiselkov
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If you plan on using it as a storage server for multimedia data (movies), don't even bother considering compression, as most media files already come heavily compressed. Dedup might still come in handy, though. - -- Saso On 07/21/2010 05:03 PM, Eugen

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Eugen Leitl
On Wed, Jul 21, 2010 at 04:56:26PM +0200, Roy Sigurd Karlsbakk wrote: > It'll probably be ok. If you use lzjb compresion, it'll probably suffice as > well. Give it gzip-9 compression, and you might have a cpu bottleneck, but > then, for most use, that config will probably do. What sort of traffi

Re: [zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Roy Sigurd Karlsbakk
- Original Message - > How badly would a dual-core 1.6 GHz Atom with 4 GBytes RAM > be underpowered for serving 4-6 SATA drives? What kind of > transfer speed (GBit Ethernet, Intel NICs) can I expect with > raidz2 or raidz3? It'll probably be ok. If you use lzjb compresion, it'll probably

[zfs-discuss] slog/L2ARC on a hard drive and not SSD?

2010-07-21 Thread Hernan F
Hi, Out of pure curiosity, I was wondering, what would happen if one tries to use a regular 7200RPM (or 10K) drive as slog or L2ARC (or both)? I know these are designed with SSDs in mind, and I know it's possible to use anything you want as cache. So would ZFS benefit from it? Would it be the sa

Re: [zfs-discuss] How does zil work

2010-07-21 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of v > > For synchronous write, and if io is small, will the whole io be place > on zil? or just the pointer be save into zil? what about large size io? This one doesn't have a really clear answer

[zfs-discuss] CPU requirements for zfs performance

2010-07-21 Thread Eugen Leitl
How badly would a dual-core 1.6 GHz Atom with 4 GBytes RAM be underpowered for serving 4-6 SATA drives? What kind of transfer speed (GBit Ethernet, Intel NICs) can I expect with raidz2 or raidz3? Thanks. -- Eugen* Leitl http://leitl.org";>leitl http://leitl.org

Re: [zfs-discuss] zfs raidz1 and traditional raid 5 perfomrance comparision

2010-07-21 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of v > > for zfs raidz1, I know for random io, iops of a raidz1 vdev eqaul to > one physical disk iops, since raidz1 is like raid5 , so is raid5 has > same performance like raidz1? ie. random iops

Re: [zfs-discuss] zfs send to remote any ideas for a faster way than ssh?

2010-07-21 Thread Edward Ned Harvey
> From: Giovanni Tirloni [mailto:gtirl...@sysdroid.com] > > We have hundreds of servers using LACP and so far have not noticed any > increase in the error rate. If the error rate is not zero, you have an increased error rate. In linux, you just do this: sudo /sbin/ifconfig -a | grep errors | gre

Re: [zfs-discuss] Help identify failed drive

2010-07-21 Thread Marty Scholes
> If the format utility is not displaying the WD drives > correctly, > then ZFS won't see them correctly either. You need to > find out why. > > I would export this pool and recheck all of your > device connections. I didn't see it in the postings, but are the same serial numbers showing up mult

Re: [zfs-discuss] zfs fails to import zpool

2010-07-21 Thread Erik Trimble
On 7/21/2010 1:36 AM, Jorge Montes IV wrote: I think this maybe my problem(at least I hope it is): http://opensolaris.org/jive/thread.jspa?messageID=489905񷦱 But I am not sure what this means "Everything in /dev/dsk and /dev/rdsk is a symlink or directory, so you can fake them out with a temp

Re: [zfs-discuss] zfs fails to import zpool

2010-07-21 Thread Jorge Montes IV
I think this maybe my problem(at least I hope it is): http://opensolaris.org/jive/thread.jspa?messageID=489905񷦱 But I am not sure what this means "Everything in /dev/dsk and /dev/rdsk is a symlink or directory, so you can fake them out with a temporary directory and clever use of the "zpool im