Re: [zfs-discuss] JBOD performance
Hello Peter, Saturday, December 15, 2007, 7:45:50 AM, you wrote: >> Use a faster processor or change to a mirrored configuration. >> raidz2 can become processor bound in the Reed-Soloman calculations >> for the 2nd parity set. You should be able to see this in mpstat, and to >> a coarser grain in vmstat. PS> Hmm. Is the OP's hardware *that* slow? (I don't know enough about the Sun PS> hardware models) PS> I have a 5-disk raidz2 (cheap SATA) here on my workstation, which is an X2 PS> 3800+ (i.e., one of the earlier AMD dual-core offerings). Here's me dd:ing to PS> a file on FreeBSD on ZFS running on that hardware: PS> promraid 741G 387G 0380 0 47.2M PS> promraid 741G 387G 0336 0 41.8M PS> promraid 741G 387G 0424510 51.0M PS> promraid 741G 387G 0441 0 54.5M PS> promraid 741G 387G 0514 0 19.2M PS> promraid 741G 387G 34192 4.12M 24.1M PS> promraid 741G 387G 0341 0 42.7M PS> promraid 741G 387G 0361 0 45.2M PS> promraid 741G 387G 0350 0 43.9M PS> promraid 741G 387G 0370 0 46.3M PS> promraid 741G 387G 1423 134K 51.7M PS> promraid 742G 386G 22329 2.39M 10.3M PS> promraid 742G 386G 28214 3.49M 26.8M PS> promraid 742G 386G 0347 0 43.5M PS> promraid 742G 386G 0349 0 43.7M PS> promraid 742G 386G 0354 0 44.3M PS> promraid 742G 386G 0365 0 45.7M PS> promraid 742G 386G 2460 7.49K 55.5M PS> At this point the bottleneck looks architectural rather than CPU. None of the PS> cores are saturated, and the CPU usage of the ZFS kernel threads is pretty PS> low. PS> I say architectural because writes to the underlying devices are not PS> sustained; it drops to almost zero for certain periods (this is more visible PS> in iostat -x than it is in the zpool statistics). What I think is happening PS> is that ZFS is too late to evict data in the cache, thus blocking the writing PS> process. Once a transaction group with a bunch of data gets committed the PS> application unblocks, but presumably ZFS waits for a little while before PS> resuming writes. PS> Note that this is also being run on plain hardware; it's not even PCI Express. PS> During throughput peaks, but not constantly, the bottleneck is probably the PS> PCI bus. Sequential writing problem with process throttling - there's an open bug for it for quite a while. Try to lower txg_time to 1s - should help a little bit. Can you also post iostat -xnz 1 while you're doing dd? and zpool status -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Nice chassis for ZFS server
On December 13, 2007 10:12:52 PM -0800 "can you guess?" <[EMAIL PROTECTED]> wrote: >> On December 13, 2007 12:51:55 PM -0800 "can you >> guess?" >> <[EMAIL PROTECTED]> wrote: >> > ... >> > >> >> when the difference between an unrecoverable >> single >> >> bit error is not just >> >> 1 bit but the entire file, or corruption of an >> entire >> >> database row (etc), >> >> those small and infrequent errors are an >> "extremely >> >> big" deal. >> > >> > You are confusing unrecoverable disk errors (which >> are rare but orders of >> > magnitude more common) with otherwise >> *undetectable* errors (the >> > occurrence of which is at most once in petabytes by >> the studies I've >> > seen, rather than once in terabytes), despite my >> attempt to delineate the >> > difference clearly. >> >> No I'm not. I know exactly what you are talking >> about. > > Then you misspoke in your previous post by referring to "an unrecoverable > single bit error" rather than to "an undetected single-bit error", which > I interpreted as a misunderstanding. I did misspeak. thanks. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] /usr/bin and /usr/xpg4/bin differences
Hello, I am working on open solaris bugs .. and need to change the code of df in the above two folders.. I would like to know why there are two df's with diff options in the respective folders.. /usr/bin/df is different is from /usr/xpg4/bin/df!! Why is it so?? What is this xpg4 represent? Thanks, Sasidhar. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] LSI SAS3081E = unstable drive numbers?
Kent Watsen wrote: > Given that manually tracking shifting ids doesn't sound appealing to > me, would using a SATA controller like the AOC-SAT2-MV8 resolve the > issue? Given that I currently only have one LSI HBA - I'd need to get 2 > more for all 24 drives ---or--- I could get 3 of these SATA controllers > plus 6 discrete-to-8087 reverse breakout cables. Going down the > LSI-route would cost about $600 while going down the AOC-SAT2-MV8 would > cost about $400. I understand that the SATA controllers are less > performant, but I'd gladly exchange some performance that I'm likely to > never need to simplify my administrative overhead... > So, I picked up an AOC-SAT2-MV8 off eBay for not too much and then I got a 4xSATA to one SFF-8087 cable to connect it to one one my six backplanes. But, as fortune would have it, the cable I bought has SATA connectors that are physically too big to plug into the AOC-SAT2-MV8 - since the AOC-SAT2-MV8 stacks two SATA connectors on top of each other... Any chance anyone knows of a 4xSATA to SFF-8087 reverse breakout cable that will plug into the AOC-SAT2-MV8? Alternatively, anyone know how hard it would be to splice the SATA cables that came with the AOC-SAT2-MV8 into the cable I bought? Also, the cable I got actually had 5 cables going into the SFF-8087 - 4 SATA and another that is rectangular and has holes for either 7 or 8 pins (not sure if its for 7 or 8 pins as one of the holes appears a bit different). Any idea what this fifth cable is for? Thanks, Kent ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] LSI SAS3081E = unstable drive numbers?
Eric Schrock wrote:
> For x86 systems, you can use ipmitool to manipulate the led state
> (ipmitool sunoem led ...). On older galaxy systems, you can only set the
> fail LED ('io.hdd0.led'), as the ok2rm LED is not physically connected
> to anything. On newer systems, you can set both the 'fail' and 'okrm'
> LEDs. You cannot change the activity LED except by manually sending the
> 'set sensor reading' IPMI command (not available via impitool).
>
> For external enclosures, you'll need a SES control program.
>
> Both of these problems are being worked on under the FMA sensor
> framework to create a unified view through libtopo. Until that's
> complete, you'll be stuck using ad hoc methods.
>
Hi Eric,
I've looked at your blog and have tried your suggestions, but I need a
little more advice.
I am on an x86 system running SXCE svn_74 - the system has 6 SAS
backplanes but, according to the integrator, no "real" scsi enclosure
services. according to the man page, I found that I could use `sdr
elist generic` to list LEDs, but that command doesn't return any output:
# ipmitool sdr elist generic
# /* there was no output */
Is it not returning sensor ids because I don't have real scsi enclosure
services? Is there anything I can do or am I doomed to ad hoc methods
forever?
Thanks,
Kent
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] /usr/bin and /usr/xpg4/bin differences
Hi, It's a different version in terms of the Unix standard it complies to: http://docs.sun.com/app/docs/doc/816-5221/6mbcm38u8?l=en&a=view On 12/16/07, KASTURI VENKATA SESHA SASIDHAR <[EMAIL PROTECTED]> wrote: > Hello, > I am working on open solaris bugs .. and need to change the code of > df in the above two folders.. > > I would like to know why there are two df's with diff options in the > respective folders.. > /usr/bin/df is different is from /usr/xpg4/bin/df!! > > Why is it so?? What is this xpg4 represent? > > > Thanks, > Sasidhar. > > > This message posted from opensolaris.org > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > -- _/ sengork.blogspot.com / ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] /usr/bin and /usr/xpg4/bin differences
On Dec 15, 2007 11:31 PM, KASTURI VENKATA SESHA SASIDHAR <[EMAIL PROTECTED]> wrote: > Hello, > I am working on open solaris bugs .. and need to change the code of > df in the above two folders.. > > I would like to know why there are two df's with diff options in the > respective folders.. > /usr/bin/df is different is from /usr/xpg4/bin/df!! The code for both variants of df come from the same source (usr/src/cmd/fs.d/df.c). The xpg4 variant is compiled with -DXPG4. After a build in usr/src/cmd/fs.d is complete you will see the following: $ ls df* df df.odf.po.xpg4 df.xpg4 df.cdf.po df.xcl df.xpg4.o It looks to me as though df becomes /usr/bin/df and df.xpg4 becomes /usr/xpg4/bin/df. -- Mike Gerdts http://mgerdts.blogspot.com/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
