Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-04-03 Thread Heikki Linnakangas
On 02/04/2019 08:58, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: I had another quick look. I still think using the "generic xlog AM" for this is a wrong level of abstraction, and we should use the XLOG_FPI records for this directly. We can extend XLOG_FPI so that it ca

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-04-01 Thread Andrey Lepikhov
On 25/03/2019 15:21, Heikki Linnakangas wrote: I had another quick look. I still think using the "generic xlog AM" for this is a wrong level of abstraction, and we should use the XLOG_FPI records for this directly. We can extend XLOG_FPI so that it can store multiple pages in a single recor

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-04-01 Thread Andrey Lepikhov
On 26/03/2019 15:59, Heikki Linnakangas wrote: On 26/03/2019 11:29, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: Hmm. When do we create all-zero pages during index build? That seems pretty surprising. GIST uses buffered pages. During GIST build it is possible (very

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-03-26 Thread Heikki Linnakangas
On 26/03/2019 11:29, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: Hmm. When do we create all-zero pages during index build? That seems pretty surprising. GIST uses buffered pages. During GIST build it is possible (very rarely) what no one index tuple was written to the

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-03-26 Thread Andrey Lepikhov
On 25/03/2019 15:21, Heikki Linnakangas wrote: On 25/03/2019 09:57, David Steele wrote: On 2/6/19 2:08 PM, Andrey Lepikhov wrote: The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So  lsn field

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-03-25 Thread Heikki Linnakangas
On 25/03/2019 09:57, David Steele wrote: On 2/6/19 2:08 PM, Andrey Lepikhov wrote: The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So lsn field at all-zero page was initialized and the PageIsVeri

Re: Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-03-25 Thread David Steele
On 2/6/19 2:08 PM, Andrey Lepikhov wrote: The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So lsn field at all-zero page was initialized and the PageIsVerified() routine detects it as a bad page. T

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-02-06 Thread Andrey Lepikhov
The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So lsn field at all-zero page was initialized and the PageIsVerified() routine detects it as a bad page. The solution may be: 1. To improve index buil

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-02-04 Thread Andrey Lepikhov
On 04.02.2019 10:04, Michael Paquier wrote: > On Tue, Dec 18, 2018 at 10:41:48AM +0500, Andrey Lepikhov wrote: >> Ok. It is used only for demonstration. > > The latest patch set needs a rebase, so moved to next CF, waiting on > author as this got no reviews. The new version in attachment. > -- > M

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2019-02-03 Thread Michael Paquier
On Tue, Dec 18, 2018 at 10:41:48AM +0500, Andrey Lepikhov wrote: > Ok. It is used only for demonstration. The latest patch set needs a rebase, so moved to next CF, waiting on author as this got no reviews. -- Michael signature.asc Description: PGP signature

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-12-17 Thread Andrey Lepikhov
On 30.11.2018 15:10, Dmitry Dolgov wrote: Thank you. I played a bit with this patch, and can confirm visible WAL size reduction (it's rather obvious, but still). Although, probably due to lack of my knowledge here, I wonder how does it work when an index is build concurrently? Routine genera

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-11-30 Thread Dmitry Dolgov
> On Tue, Jul 31, 2018 at 6:36 AM Andrey Lepikhov > wrote: > > With the consent of Anastasia I will improving this patch further. > Attachment contains next version of the patch set. Thank you. I played a bit with this patch, and can confirm visible WAL size reduction (it's rather obvious, but s

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-07-30 Thread Andrey Lepikhov
With the consent of Anastasia I will improving this patch further. Attachment contains next version of the patch set. 11.07.2018 00:03, Heikki Linnakangas пишет: On 28/02/18 18:03, Anastasia Lubennikova wrote: Implementation is based on generic_xlog. Why? I think we should just add a log_re

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-07-22 Thread Alexander Korotkov
On Tue, Jul 10, 2018 at 10:03 PM Heikki Linnakangas wrote: > > On 28/02/18 18:03, Anastasia Lubennikova wrote: > > I want to propose a bunch of patches which allow to reduce WAL traffic > > generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree > > and RUM, we can now log index p

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-07-10 Thread Heikki Linnakangas
On 28/02/18 18:03, Anastasia Lubennikova wrote: Hi, I want to propose a bunch of patches which allow to reduce WAL traffic generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree and RUM, we can now log index pages of other access methods only once in the end of indexbuild proce

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-03-01 Thread David Steele
Hi Anastasia, On 2/28/18 11:03 AM, Anastasia Lubennikova wrote: > I want to propose a bunch of patches which allow to reduce WAL traffic > generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree > and RUM, we can now log index pages of other access methods only once > in the end o

Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-02-28 Thread Anastasia Lubennikova
Hi, I want to propose a bunch of patches which allow to reduce WAL traffic generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree and RUM, we can now log index pages of other access methods only once in the end of indexbuild process. Implementation is based on generic_xlog. Not