On 05/26/09 13:07, Kjetil Torgrim Homme wrote:
also thank you, all ZFS developers, for your great job :-)
I'll second that! A great achievement - puts Solaris in a league of its own, so much so, you'd want to run it on all your hardware, however crappy the hardware might be ;-) There are too many branches in this thread now. Going to summarize here without responding to some of the less than helpful comments, although death and taxes seems an ironic metaphor in the current climate :-) In some ways this isn't a technical issue. This much maligned machine and its ilk are running Solaris and ZFS quite happily and the users are pleased with the stability and performance. But their applications are running on machines (via xdmcp) with ECC, and ZFS mirror/raidz doesn't have a problem there. Picture a new convert with enthusiasm for ZFS, but has a less than perfect PC which has otherwise been apparently quite reliable. Perhaps it already has mirrored drives. He/she installs Solaris from the live CD (and finds that the installer doesn't support mirroring). The install fails, or worse, afterwords he/she loses that movie of Aunt Minnie playing golf, because a checksum error makes the file unrecoverable. This could be very frustrating and make the blogosphere go crazy, especially if the PC passes every diagnostic. Be even worse if a file is lost on a mirror. Unrecoverable files on mirrored drives simply shouldn't happen. What kind of hardware error (other than a rare bit flip) could conceivably cause 5 out of 15 checksum errors to be unrecoverable when mirrored during the write of around 20*10^10 bits? ZFS has both a larger spatial and temporal footprint than other file systems, so it is slightly more vulnerable to the once-a-month on average bit flip that will afflict many a PC with 4GB of memory. Perhaps someone with a statistical bent could step in and actually calculate the probability of random errors, perhaps assuming that half of available memory is used to queue writes, that there is a 95% chance of one bit flip per month per 4GB, and there is a (say) 10% duty cycle over say a period of a year. Alternatively, the chance of a 1 bit flip over a period of 6 hours at a 100% duty cycle repeated 1461 times (1461 installs per year at 100%). Seems to me intuitively that 6 out of 1461 installs will fail due to an unrecoverable checksum failure, but I'm not a statistician. Multiply that failure rate by the number of Live CD installs you expect over the next year (noting that *all* checksum failures are unrecoverable without mirroring) and you'll count quite a few frustrated would-be installers. Maybe ZFS without ECC and no mirroring should disable checksumming by default - it would be a little worse than UFS and ext3 (due to its larger spatial and temporal footprints) but still provide all the other great features. Proposed RFE #1 Add option to make files with unrecoverable checksum failures readable and to pass the best image possible back to the application. [How much do you bet most folks would select this option?] If both sides of the mirror could be read, it might help to diagnose the problem, which obviously must be in the hardware somewhere. If both images are identical, then it surely must be memory. If they differ, then what could it be? Proposed RFE#2 Add an option for machines with mirrored drives but without ECC to double buffer and only then calculate the checksums (for those who are reasonably paranoid about cosmic rays). Proposed RFE#3 (or is this a bug report?) Add diagnostics to the ZFS recv to help understand why a perfectly good ZFS send can't be received when the same machine can successfully compute a md5sum over the same stream. Even something like "recv failed at block nnnnnnn" would help. For example, it seems to fail suspiciously close to 2GB on a 32 bit machine. Proposed RFE #4 Disable checksumming by default if no mirroring and no ECC is detected. (Of course this assumes a install to mirror option). If it could still checksum, but make it a warning instead of an error, this could turn into a great feature for cheapskates with machines that have no ECC. --- 1 and #2 above could be fixed in the documentation. "Random memory bit flips can theoretically cause unrecoverable checksum failures, even if the data is mirrored. Either disable the checksum feature or only run ZFS on systems with ECC memory if you have any data you don't want to risk losing [even with a 1 bit error]". None of this is meant as a criticism of ZFS, just suggestions to help make a merely superb file system into the unbeatable one it should be. (I suppose it really is a system of file systems, but ZFS it is...) Regards -- Frank _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
