Re: [zfs-discuss] Single disk parity

2009-07-10 Thread Haudy Kazemi
Richard Elling wrote: There are many error correcting codes available. RAID2 used Hamming codes, but that's just one of many options out there. Par2 uses configurable strength Reed-Solomon to get multi bit error correction. The par2 source is available, although from a ZFS perspective is hi

Re: [zfs-discuss] Single disk parity

2009-07-09 Thread Richard Elling
Christian Auby wrote: On Wed, 8 Jul 2009, Moore, Joe wrote: That's true for the worst case, but zfs mitigates that somewhat by batching i/o into a transaction group. This means that i/o is done every 30 seconds (or 5 seconds, depending on the version you're running), allowing multiple writes

Re: [zfs-discuss] Single disk parity

2009-07-09 Thread Christian Auby
> On Wed, 8 Jul 2009, Moore, Joe wrote: > That's true for the worst case, but zfs mitigates > that somewhat by > batching i/o into a transaction group. This means > that i/o is done every > 30 seconds (or 5 seconds, depending on the version > you're running), > allowing multiple writes to be wr

Re: [zfs-discuss] Single disk parity

2009-07-09 Thread Richard Elling
Haudy Kazemi wrote: Adding additional data protection options are commendable. On the other hand I feel there are important gaps in the existing feature set that are worthy of a higher priority, not the least of which is the automatic recovery of uberblock / transaction group problems (see

Re: [zfs-discuss] Single disk parity

2009-07-08 Thread Haudy Kazemi
Adding additional data protection options are commendable. On the other hand I feel there are important gaps in the existing feature set that are worthy of a higher priority, not the least of which is the automatic recovery of uberblock / transaction group problems (see Victor Latushkin's

Re: [zfs-discuss] Single disk parity

2009-07-08 Thread Richard Elling
Haudy Kazemi wrote: Daniel Carosone wrote: Sorry, don't have a thread reference to hand just now. http://www.opensolaris.org/jive/thread.jspa?threadID=100296 Note that there's little empirical evidence that this is directly applicable to the kinds of errors (single bit, or otherwise) th

Re: [zfs-discuss] Single disk parity

2009-07-08 Thread Haudy Kazemi
Daniel Carosone wrote: Sorry, don't have a thread reference to hand just now. http://www.opensolaris.org/jive/thread.jspa?threadID=100296 Note that there's little empirical evidence that this is directly applicable to the kinds of errors (single bit, or otherwise) that a single failing d

Re: [zfs-discuss] Single disk parity

2009-07-08 Thread Mark J Musante
On Wed, 8 Jul 2009, Moore, Joe wrote: The copies code is nice because it tries to put each copy "far away" from the others. This does have a significant performance impact when on a single spindle, however, because each logical write will be written "here" and then a disk seek to write it to

Re: [zfs-discuss] Single disk parity

2009-07-08 Thread Moore, Joe
Christian Auby wrote: > It's not quite like copies as it's not actually a copy of the data I'm > talking about. 10% parity or even 5% could easily fix most disk errors > that won't result in a total disk loss. (snip) > I don't see a performance issue if it's not enabled by default though. The co

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Daniel Carosone
> Sorry, don't have a thread reference > to hand just now. http://www.opensolaris.org/jive/thread.jspa?threadID=100296 Note that there's little empirical evidence that this is directly applicable to the kinds of errors (single bit, or otherwise) that a single failing disk medium would produce.

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Christian Auby
> Do you have data to back this up? It's more of a logical observation. The random data corruption I've had up through the years have generally either involved a single sector or two or a full disk failure. 5% parity on a 128KB block size would allow you to lose 6.4KB, or ~10 512 byte sectors.

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Richard Elling
Christian Auby wrote: You are describing the copies parameter. It really helps to describe it in pictures, rather than words. So I did that. http://blogs.sun.com/relling/entry/zfs_copies_and_data _protection -- richard It's not quite like copies as it's not actually a copy of the data I

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Christian Auby
> You are describing the copies parameter. It really > helps to describe > it in pictures, rather than words. So I did that. > http://blogs.sun.com/relling/entry/zfs_copies_and_data > _protection > -- richard It's not quite like copies as it's not actually a copy of the data I'm talking about.

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Daniel Carosone
There was a discussion in zfs-code around error-correcting (rather than just -detecting) properties of the checksums currently kept, an of potential additional checksum methods with stronger properties. It came out of another discussion about fletcher2 being both weaker than desired, and flawed

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Louis-Frédéric Feuillette
On Tue, 2009-07-07 at 17:42 -0700, Richard Elling wrote: > Christian Auby wrote: > > ZFS is able to detect corruption thanks to checksumming, but for single > > drives (regular folk-pcs) it doesn't help much unless it can correct them. > > I've been searching and can't find anything on the topic,

Re: [zfs-discuss] Single disk parity

2009-07-07 Thread Richard Elling
Christian Auby wrote: ZFS is able to detect corruption thanks to checksumming, but for single drives (regular folk-pcs) it doesn't help much unless it can correct them. I've been searching and can't find anything on the topic, so here goes: 1. Can ZFS do parity data on a single drive? e.g. x%