On Sun, Sep 14, 2014 at 05:21:10PM +0200, Andres Freund wrote: > On 2014-09-13 20:27:51 -0500, k...@rice.edu wrote: > > > Also, while I understand that CRC has a very venerable history and > > is well studied for transmission type errors, I have been unable to find > > any research on its applicability to validating file/block writes to a > > disk drive. > > Which incidentally doesn't really match what the CRC is used for > here. It's used for individual WAL records. Usually these are pretty > small, far smaller than disk/postgres' blocks on average. There's a > couple scenarios where they can get large, true, but most of them are > small. > The primary reason they're important is to correctly detect the end of > the WAL. To ensure we're interpreting half written records, or records > from before the WAL file was overwritten. > > > > While it is to quote you "unbeaten collision wise", xxhash, > > both the 32-bit and 64-bit version are its equal. > > Aha? You take that from the smhasher results?
Yes. > > > Since there seems to be a lack of research on disk based error > > detection versus CRC polynomials, it seems likely that any of the > > proposed hash functions are on an equal footing in this regard. As > > Andres commented up-thread, xxhash comes along for "free" with lz4. > > This is pure handwaving. Yes. But without research to support the use of CRC32 in this same environment, it is handwaving in the other direction. :) Regards, Ken -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
