can you guess? wrote: >> >> I have to comment here. As a bloke with a bit of a >> photography >> habit - I have a 10Mpx camera and I shoot in RAW mode >> - it is >> very, very easy to acquire 1Tb of image files in >> short order. >> > > So please respond to the question that I raised above (and that you yourself > quoted): just how much damage will a single-bit error do to such a RAW file? >
In a compressed raw file, it'll affect the rest of the file generally; so it essentially renders the whole thing useless, unless it happens to hit towards the end and you can crop around it. If it hits in metadata (statistically unlikely, the bulk of the file is image data) it's probably at worst annoying, but it *might* hit one of the bits software uses to recognize and validate the file, too. In an uncompressed raw file, if it hits in image data it'll affect probably 9 pixels; it's easily fixed. >> Each of the photos I take is between 8 and 11Mb, and >> if I'm >> at a sporting event or I'm travelling for work or >> pleasure, >> it is *incredibly* easy to amass several hundred Mb >> of photos >> every single day. >> > > Even assuming that you meant 'MB' rather than 'Mb' above, that suggests that > it would take you well over a decade to amass 1 TB of RAW data (assuming > that, as you suggest both above and later, you didn't accumulate several > hundred MB of pictures *every* day but just on those days when you were > traveling, at a sporting event, etc.). > I seem to come up with a DVD full every month or two these days, myself. I mean, it varies; there was this one weekend I filled 4 or some such; but it varies both ways, and that average isn't too far off. 25GB a year seems to take 40 years to reach 1TB. However, my rate has increased so dramatically in the last 7 years that I'm not at all sure what to expect; is it time for the curve to level off yet, for me? Who knows! Then again, I'm *also* working on scanning in the *last* 40 years worth of photos, and those tend to be bigger (scans are less good pixels so you need more of them), and *that* runs the numbers up, in chunks when I take time to do a big scanning batch. >> I'm by no means a professional photographer (so I'm >> not out >> taking photos every single day), although a very >> close friend >> of mine is. My photo storage is protected by ZFS with >> mirroring >> and backups to dvd media. My profotog friend has 3 >> copies of >> all her data - working set, immediate copy on >> usb-attached disk, >> and second backup also on usb-attached disk but >> disconnected. >> > > Sounds wise on both your parts - and probably makes ZFS's extra protection > pretty irrelevant (I won't bother repeating why here). > > >> Even if you've got your original file archived, you >> still need >> your working copies available, and Adobe Photoshop >> can turn that >> RAW file into a PSD of nearly 60Mb in some cases. >> > > If you really amass all your pictures this way (rather than, e.g., use > Photoshop on some of them and then save the result in a less verbose format), > I'll suggest that this takes you well beyond the 'consumer' range of behavior. > It's not snapshot usage, but it's common amateur usage. Amateurs tend to do lots of the same things professionals do (and sometimes better, though not usually). Hobbies are like that. The argument for the full Photoshop file is the concept of "nondestructive editing". I do retouching on new layers instead of erasing what I already have with the new stuff. I use adjustment layers with layer masks for curve adjustments. I can go back and improve the mask, or nudge the curves, without having to start over from scratch. It's a huge win. And it may be more valuable for amateurs, actually; professionals tend to have the experience to know their minds better and know when they have it right, so many of them may do less revisiting old stuff and improving it a bit. Also, when the job is done and sent to the client, they tend not to care about it any more. >> It is very easy for the storage medium to acquire >> some degree >> of corruption - whether it's a CF or SD card, they >> all use >> FAT32. I have been in the position of losing photos >> due to >> this. Not many - perhaps a dozen over the course of >> 12 months. >> > > So in those cases you didn't maintain multiple copies. Bad move, and usually > nothing that using ZFS could help with. While I'm not intimately acquainted > with flash storage, my impression is that data loss usually occurs due to bad > writes (since once written the data just sits there persistently and AFAIK is > not subject ot the kinds of 'bit rot' that disk and tape data can > experience). So if the loss occurs to the original image captured on flash > before it can be copied elsewhere, you're just SOL and nothing ZFS offers > could help you. > I don't know a whole lot about flash cards either. The cameras I know the dataflow on are writing from an internal ram buffer to the card, so if the write error was reported at the time of write, the block could be mapped out and rewritten, since the data is still available. But anything after the file is reported successfully written to the card would, I believe, be too late. I've never lost a file to a card problem, or even to a human glitch involving card handling (though I did have to use a recovery utility once, for a human glitch). So far. I use cards generally until I get a camera that makes files so much bigger than the old camera that I have to buy new cards :-). I think there are more sequences of reasonable-seeming things to do with a card that result in corruption of FAT32 filesystem than there would be for a better filesystem. But the cards need to be formatted with a pretty least-common-denominator (or at least very very widely supported) filesystem, and ZFS clearly isn't that right now. And of course when working with existing cameras, one needs to use whatever the camera understands (and that's universally FAT32, so far as I can remember). >> That flipped bit which you seem to be dismissing as >> "hardly... >> a disaster" can in fact make your photo file totally >> useless, >> because not only will you probably not be able to get >> the file >> off the media card, but whatever software you're >> using to keep >> track of your catalog will also be unable to show you >> the >> entire contents. That might be the image itself, or >> it might >> be the equally important EXIF information. >> > > Here come those pesky numbers again, I'm afraid. Because given that the size > difference between your image data and the metadata (including EXIF > information, if that's what I suspect it is) is at least several orders of > magnitude, the chance that the bad bit will be in something other than the > image data is pretty negligible. > > So even if you can format your card to use ZFS (can you? if not, what > possible relevance does your comment above have to this discussion?), doing > so won't help at all: the affected file will still be inaccessible (unless > you use ZFS to create a redundant pool across multiple such cards: is that > really what you're suggesting should be done?) both to normal extraction > (though couldn't dd normally get off everything but the bad sector?) and to > your cataloging software. > Hmmm; the new Nikon D3 supports dual CF slots, and I do believe that one of the modes is mirroring across the two slots. I think of that as for really *really* paranoid people. Personally, I *did* worry about my pictures being only on the one roll of film back when I shot film. The cameras currently in the field do not, of course, support writing to cards formatted with ZFS, so it's not a practical choice today. -- David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
