Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Thomas Munro
On Thu, Aug 2, 2018 at 1:20 PM, Alvaro Herrera wrote: > On 2018-Aug-02, Thomas Munro wrote: >> PostgreSQL only requires atomic writes of 512 bytes (see >> PG_CONTROL_MAX_SAFE_SIZE), the traditional sector size for disks made >> approximately 1980-2010, though as far as I know spinning disks made >

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andres Freund
On 2018-08-01 21:20:22 -0400, Alvaro Herrera wrote: > On 2018-Aug-02, Thomas Munro wrote: > > > PostgreSQL only requires atomic writes of 512 bytes (see > > PG_CONTROL_MAX_SAFE_SIZE), the traditional sector size for disks made > > approximately 1980-2010, though as far as I know spinning disks mad

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Alvaro Herrera
On 2018-Aug-02, Thomas Munro wrote: > PostgreSQL only requires atomic writes of 512 bytes (see > PG_CONTROL_MAX_SAFE_SIZE), the traditional sector size for disks made > approximately 1980-2010, though as far as I know spinning disks made > this decade use 4KB sectors, and for SSDs there is more va

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Thomas Munro
On Wed, Aug 1, 2018 at 11:06 PM, Andrey Borodin wrote: >> 1 авг. 2018 г., в 13:49, Thomas Munro >> написал(а): >> Hmm. This proposal doesn't seem to deal with torn writes. > > That's true, but it's a bit orthogonal to problem solved with checksums. > Checksums provide way to avoid reading bad p

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Thomas Munro
On Thu, Aug 2, 2018 at 6:17 AM, Andres Freund wrote: > On 2018-08-01 11:14:18 -0700, Shawn Debnath wrote: >> On Wed, Aug 01, 2018 at 11:00:46AM -0700, Andres Freund wrote: >> > I believe you also planned to work on that, do I remember that >> > correctly, or is that just wishful thinking? >> >> Ye

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andres Freund
On 2018-08-01 11:14:18 -0700, Shawn Debnath wrote: > On Wed, Aug 01, 2018 at 11:00:46AM -0700, Andres Freund wrote: > > I believe you also planned to work on that, do I remember that > > correctly, or is that just wishful thinking? > > Yep - I am actively working on this at the moment, planning on

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andres Freund
Hi, On 2018-08-01 10:58:03 -0700, Shawn Debnath wrote: > On Wed, Aug 01, 2018 at 02:06:44PM +0300, Andrey Borodin wrote: > > But adding LSNs, and whole regular PageHeader is quite easy in this > > patch. Do you think we should really go that way? > > > > Putting SLRUs into usual shared buffers a

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andrey Borodin
Hi! > 1 авг. 2018 г., в 13:49, Thomas Munro > написал(а): > > Hmm. This proposal doesn't seem to deal with torn writes. That's true, but it's a bit orthogonal to problem solved with checksums. Checksums provide way to avoid reading bad page, torn pages - is about preventing writing bad writes

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Thomas Munro
On Wed, Jul 18, 2018 at 5:54 PM, Thomas Munro wrote: > On Wed, Jul 18, 2018 at 5:41 PM, Andrey Borodin wrote: >>> I think we'd want pg_upgrade tests showing an example of each SLRU >>> growing past one segment, and then being upgraded, and then being >>> accessed in various different pages and se

Re: [Patch] Checksums for SLRU files

2018-07-17 Thread Thomas Munro
On Wed, Jul 18, 2018 at 5:41 PM, Andrey Borodin wrote: >> I think we'd want pg_upgrade tests showing an example of each SLRU >> growing past one segment, and then being upgraded, and then being >> accessed in various different pages and segment files, so that we can >> see that we're able to shift

Re: [Patch] Checksums for SLRU files

2018-07-17 Thread Andrey Borodin
Hi, Tomas! > > I think we'd want pg_upgrade tests showing an example of each SLRU > growing past one segment, and then being upgraded, and then being > accessed in various different pages and segment files, so that we can > see that we're able to shift the data to the right place successfully. >

Re: [Patch] Checksums for SLRU files

2018-05-16 Thread Andrey Borodin
Hi, Ivan! > 19 марта 2018 г., в 15:32, Andrey Borodin написал(а): > I was looking into this patch mainly because I was reviewing other checksums > patch in different thread. But the purpose of this patch seems viable for me. > After looking into the patch I'd like to propose some editorializatio

Re: [Patch] Checksums for SLRU files

2018-03-19 Thread Andrey Borodin
Hi, Ivan! > 5 марта 2018 г., в 20:58, Andrey Borodin написал(а): > > I've found that there are few more places with SLRU items per page I was looking into this patch mainly because I was reviewing other checksums patch in different thread. But the purpose of this patch seems viable for me. Aft

Re: [Patch] Checksums for SLRU files

2018-03-05 Thread Andrey Borodin
Hi, Ivan! I've found that there are few more places with SLRU items per page, where you have to update usable page size. Please find the diff attached. I agree that there is a little chance to get this commitable quickly, but still, the feature worth working on, from my point of view. Best rega

Re: [Patch] Checksums for SLRU files

2018-03-02 Thread Andres Freund
On 2018-03-02 11:49:05 -0500, Robert Haas wrote: > On Thu, Mar 1, 2018 at 8:25 PM, Andres Freund wrote: > > On 2018-02-02 11:37:34 +1300, Thomas Munro wrote: > >> > 3. pg_upgrade isn't considered. This patch should provide upgrading > >> > SLRUs > >> > to adopt changed useful size of page. That

Re: [Patch] Checksums for SLRU files

2018-03-02 Thread Robert Haas
On Thu, Mar 1, 2018 at 8:25 PM, Andres Freund wrote: > On 2018-02-02 11:37:34 +1300, Thomas Munro wrote: >> > 3. pg_upgrade isn't considered. This patch should provide upgrading SLRUs >> > to adopt changed useful size of page. That seems to be hardest patch of >> > this patch to be written. >> >

Re: [Patch] Checksums for SLRU files

2018-03-01 Thread Andres Freund
On 2018-02-02 11:37:34 +1300, Thomas Munro wrote: > > 3. pg_upgrade isn't considered. This patch should provide upgrading SLRUs > > to adopt changed useful size of page. That seems to be hardest patch of > > this patch to be written. > > +1 > > I think we'd want pg_upgrade tests showing an exam

Re: [Patch] Checksums for SLRU files

2018-02-01 Thread Thomas Munro
On Wed, Jan 3, 2018 at 11:21 AM, Alexander Korotkov wrote: > On Mon, Jan 1, 2018 at 9:19 PM, Andrey Borodin wrote: >> > 31 дек. 2017 г., в 22:30, Ivan Kartyshov >> > написал(а): >> > >> > Hello, I`d like to show my implementation of SLRU file protection with >> > checksums. >> > . >> > I wou

Re: [Patch] Checksums for SLRU files

2018-01-02 Thread Alexander Korotkov
On Mon, Jan 1, 2018 at 9:19 PM, Andrey Borodin wrote: > > 31 дек. 2017 г., в 22:30, Ivan Kartyshov > написал(а): > > > > Hello, I`d like to show my implementation of SLRU file protection with > checksums. > > . > > I would like to hear your thoughts over my patch. > > As far as I can see, th

Re: [Patch] Checksums for SLRU files

2018-01-01 Thread Andrey Borodin
Hi, Ivan! > 31 дек. 2017 г., в 22:30, Ivan Kartyshov > написал(а): > > Hello, I`d like to show my implementation of SLRU file protection with > checksums. > . > I would like to hear your thoughts over my patch. As far as I can see, the patch solves problem of hardware corruption in SLRU. T

[Patch] Checksums for SLRU files

2017-12-31 Thread Ivan Kartyshov
Hello, I`d like to show my implementation of SLRU file protection with checksums. It only has effect if checksums on database are enabled. Detection of a checksum failure during a read normally causes PostgreSQL to report an error. Setting ignore_slru_checksum_failure to on causes the system t