some business do not accept any kind of risk and hence will try hard (i.e spend a lot of money) to eliminate it (create 2, 3, 4 copies, read-verify, cksum...)
at the moment only ZFS can give this assurance, plus the ability to self correct detected errors. It's a good things that ZFS can help people store and manage safely their jpgs on their usb disk..the real target customers here are companies that rely a lot on their data: their goal is create value out of it. It is the case for CERN (corrupted file might imply a missed higgs particule) and any mature company as matter of fact (finance, governement). these are the business ZFS gives a real data storage assurance Selim -- ------------------------------------------------------ Blog: http://fakoli.blogspot.com/ On Nov 13, 2007 12:53 AM, can you guess? <[EMAIL PROTECTED]> wrote: > Thanks for taking the time to flesh these points out. Comments below: > > ... > > > The compression I see varies from something like 30% > > to 50%, very > > roughly (files reduced *by* 30%, not files reduced > > *to* 30%). This is > > with the Nikon D200, compressed NEF option. On some > > of the lower-level > > bodies, I believe the compression can't be turned > > off. Smaller files > > will of course get hit less often -- or it'll take > > longer to accumulate > > the terrabyte, is how I'd prefer to think of it. > > Either viewpoint works. And since the compression is not that great, you > still wind up consuming a lot of space. Effectively, you're trading (at > least if compression is an option rather than something that you're stuck > with) the possibility that a picture will become completely useless should a > bit get flipped for a storage space reduction of 30% - 50% - and that's a > good trade, since it effectively allows you to maintain a complete backup > copy on disk (for archiving, preferably off line) almost for free compared > with the uncompressed option. > > > > > Damage that's fixable is still damage; I think of > > this in archivist > > mindset, with the disadvantage of not having an > > external budget to be my > > own archivist. > > There will *always* be the potential for damage, so the key is to make sure > that any damage is easily fixable. The best way to do this is to a) keep > multiple copies, b) keep them isolated from each other (that's why RAID is > not a suitable approach to archiving), and c) check (scrub) them periodically > to ensure that if you lose a piece (whether a bit or a sector) you can > restore the affected data from another copy and thus return your redundancy > to full strength. > > For serious archiving, you probably want to maintain at least 3 such copies > (possibly more if some are on media of questionable longevity). For normal > use, there's probably negligible risk of losing any data if you maintain only > two on reasonably reliable media: 'MAID' experience suggests that scrubbing > as little as every few months reduces the likelihood of encountering > detectable errors while restoring redundancy by several orders of magnitude > (i.e., down to something like once in a PB at worst for disks - becoming > comparable to the levels of bit-flip errors that the disk fails to detect at > all). > > Which is what I've been getting at w.r.t. ZFS in this particular application > (leaving aside whether it can reasonably be termed a 'consumer' application - > because bulk video storage is becoming one and it not only uses a similar > amount of storage space but should probably be protected using similar > strategies): unless you're seriously worried about errors in the once-per-PB > range, ZFS primarily just gives you automated (rather than > manually-scheduled) scrubbing (and only for your on-line copy). Yes, it will > help detect hardware faults as well if they happen to occur between RAM and > the disk (and aren't otherwise detected - I'd still like to know whether the > 'bad cable' experiences reported here occurred before ATA started CRCing its > transfers), but while there's anecdotal evidence of such problems presented > here it doesn't seem to be corroborated by the few actual studies that I'm > familiar with, so that risk is difficult to quantify. > > Getting back to 'consumer' use for a moment, though, given that something > like 90% of consumers entrust their PC data to the tender mercies of Windows, > and a large percentage of those neither back up their data, nor use RAID to > guard against media failures, nor protect it effectively from the perils of > Internet infection, it would seem difficult to assert that whatever > additional protection ZFS may provide would make any noticeable difference in > the consumer space - and that was the kind of reasoning behind my comment > that began this sub-discussion. > > By George, we've managed to get around to having a substantive discussion > after all: thanks for persisting until that occurred. > > - bill > > > This message posted from opensolaris.org > _______________________________________________ > > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > -- ------------------------------------------------------ Blog: http://fakoli.blogspot.com/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
