On Mon, Jun 13, 2022 at 12:59 PM Aleksander Alekseev <aleksan...@timescale.com> wrote: > So, to clarify, what we are trying to achieve here is to reduce the > probability of an event when a page gets corrupted but the checksum is > accidentally the same as it was before the corruption, correct? And we > also assume that neither file system nor hardware catched this > corruption.
Yeah, I think so, although it also depends on what the filesystem and the hardware would do if they did catch the corruption. If they would have made our read() or write() operation fail, then any checksum feature at the PostgreSQL level is superfluous. If they would have noticed the operation but not caused a failure, and say just logged something in the system log, a PostgreSQL check could still be useful, because the PostgreSQL user might not be looking at the system log, but will definitely notice if they get an ERROR rather than a query result from PostgreSQL. And if the lower-level systems wouldn't have caught the failure at all, then checksums are useful in that case, too. > If that's the case I would say that using something like SHA256 would > be an overkill, not only because of the consumed disk space but also > because SHA256 is expensive. Allowing the user to choose from 16-bit, > 32-bit and maybe 64-bit checksums should be enough. I would also > suggest that no matter how we do it, if the user chooses 16-bit > checksums the performance and the disk consumption should remain as > they currently are. If the user wants 16-bit checksums, the feature we've already got seems good enough -- and, as you say, it doesn't use any extra disk space. This proposal is just about making people happy if they want a bigger checksum. On the topic of which algorithm to use, I'd be inclined to think that it is going to be more useful to offer checksums that are 64 bits or more, since IMHO 32 is not all that much more than 16, and I still think there are going to be alignment issues. Beyond that I don't have anything against your specific suggestions, but I'd like to hear what other people think. -- Robert Haas EDB: http://www.enterprisedb.com
