Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-30 Thread Chapman Flack
On 03/30/18 19:57, Michael Paquier wrote: > look for tools using XLP_BKP_REMOVABLE. One is pglesslog that we > already know about. However I have to be honest, I have not been able > to find its source code, I'm glad it wasn't just me. -Chap

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-30 Thread Michael Paquier
On Fri, Mar 30, 2018 at 07:11:02PM -0400, Tom Lane wrote: > Chapman Flack writes: > > On 03/30/18 16:21, Tom Lane wrote: > >> I did not like the proposed test case too much, particularly not its > >> undocumented API change for check_pg_config, > > > Other than that API change, was there somethin

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-30 Thread Tom Lane
Chapman Flack writes: > On 03/30/18 16:21, Tom Lane wrote: >> I did not like the proposed test case too much, particularly not its >> undocumented API change for check_pg_config, > Other than that API change, was there something the test case could have > done differently to make you like it more

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-30 Thread Chapman Flack
On 03/30/18 16:21, Tom Lane wrote: > Yup. Pushed with some rewriting of the comments. Thanks! > I did not like the proposed test case too much, particularly not its > undocumented API change for check_pg_config, Other than that API change, was there something the test case could have done diff

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-30 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> A potentially stronger complaint is that WAL-reading tools might fail >> outright on a page with an invalid header, but I'd say that's a robustness >> issue that they'd need to address anyway. There's never been any >> guarantee th

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-30 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Chapman Flack writes: > > On 03/27/18 22:10, Michael Paquier wrote: > >> Here you go for one example: > >> https://sourceforge.net/projects/pglesslog/ > > > In any case, from my study of the commit, it is hard for me to see an issue. > > The co

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-30 Thread Tom Lane
Chapman Flack writes: > On 03/27/18 22:10, Michael Paquier wrote: >> Here you go for one example: >> https://sourceforge.net/projects/pglesslog/ > In any case, from my study of the commit, it is hard for me to see an issue. > The code comment says: "mark the header to indicate that WAL records >

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-29 Thread Chapman Flack
On 03/27/18 20:09, Tomas Vondra wrote: > Not sure what's up with gitweb, but git finds it without any issue: > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=2dd9322ba6eea76800b38bfea0599fbc459458f2 Thanks, that worked. On 03/27/18 22:10, Michael Paquier wrote: > Here you go fo

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-27 Thread Michael Paquier
On Tue, Mar 27, 2018 at 06:32:08PM -0400, Chapman Flack wrote: > Is 2dd9322 a commit? I'm having difficulty finding it: > > https://git.postgresql.org/gitweb/?p=postgresql.git&a=search&h=HEAD&st=commit&s=2dd9322 > > Am I searching wrong? > > I probably won't have more time to look at this tonigh

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-27 Thread Tomas Vondra
On 03/28/2018 12:32 AM, Chapman Flack wrote: > On 03/26/18 01:43, Michael Paquier wrote: > >> Have a look at BKP_REMOVABLE then. This was moved to page headers in >> 2dd9322, still it seems to me that the small benefits outlined on this >> thread don't justify breaking tools relying on this fla

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-27 Thread Chapman Flack
On 03/26/18 01:43, Michael Paquier wrote: > Have a look at BKP_REMOVABLE then. This was moved to page headers in > 2dd9322, still it seems to me that the small benefits outlined on this > thread don't justify breaking tools relying on this flag set, especially > if there is no replacement for it.

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-26 Thread Chapman Flack
On 03/26/18 12:34, Tom Lane wrote: > If that's the argument, why is the WALInsertLockUpdateInsertingAt(CurrPos) > call still there? GetXLogBuffer() would do that too. "Because I hadn't noticed that," he said, sheepishly. > In any case, the new comment ... fails to > explain what's going on, and

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-26 Thread Tom Lane
Chapman Flack writes: > On 03/25/18 23:27, Stephen Frost wrote: >> AdvanceXLInsertBuffer() does quite a bit, so I'm a bit surprised to see >> this simply removing that call, you're confident there's nothing done >> which still needs doing..? > My belief from looking at the code was that AdvanceXL

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-26 Thread Chapman Flack
On 03/25/18 23:27, Stephen Frost wrote: >> .travis.yml | 47 >> > > ... not something that I think we're going to add into the main tree. Looks like that got in by mistake, sorry. > - AdvanceXLInsertBuffer(CurrPos

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-25 Thread Michael Paquier
On Sun, Mar 25, 2018 at 11:27:31PM -0400, Stephen Frost wrote: > AdvanceXLInsertBuffer() does quite a bit, so I'm a bit surprised to see > this simply removing that call, you're confident there's nothing done > which still needs doing..? Have a look at BKP_REMOVABLE then. This was moved to page h

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-25 Thread Stephen Frost
Greetings, * Chapman Flack (c...@anastigmatix.net) wrote: > On 03/18/18 19:28, Daniel Gustafsson wrote: > > It seems expensive to regex over BLCKSZ, but it’s probably the safest option > > and it’s not a performance critical codepath. Feel free to whack the test > > patch over the head with the a

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-18 Thread Chapman Flack
On 03/18/18 19:28, Daniel Gustafsson wrote: > It seems expensive to regex over BLCKSZ, but it’s probably the safest option > and it’s not a performance critical codepath. Feel free to whack the test > patch over the head with the above diff. Both patches in a single email for cfbot's enjoyment, c

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-18 Thread Daniel Gustafsson
> On 18 Mar 2018, at 22:54, Chapman Flack wrote: > > On 03/18/18 16:56, Daniel Gustafsson wrote: >> sorry about that. Now we know that the proposed test fails without the patch >> applied and clears with it, that was at least an interesting side effect =) > > It was, and it got me looking at th

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-18 Thread Chapman Flack
On 03/18/18 16:56, Daniel Gustafsson wrote: > sorry about that. Now we know that the proposed test fails without the patch > applied and clears with it, that was at least an interesting side effect =) It was, and it got me looking at the test, and even though it does detect the difference between

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-18 Thread Daniel Gustafsson
> On 18 Mar 2018, at 03:09, Tom Lane wrote: > > Chapman Flack writes: >> Thanks for the review. I notice that cfbot has now flagged the patch as >> failing, and when I look into it, it appears that cfbot is building with >> your test patch, and without the xlog.c patch, and so the test naturally

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-17 Thread Tom Lane
Chapman Flack writes: > Thanks for the review. I notice that cfbot has now flagged the patch as > failing, and when I look into it, it appears that cfbot is building with > your test patch, and without the xlog.c patch, and so the test naturally > fails. Does the cfbot require both patches to be a

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-17 Thread Chapman Flack
On 03/16/18 17:14, Daniel Gustafsson wrote: > The attached patch adds the test, and a neccessary extension to > check_pg_config > to allow for extracting values from pg_config.h as opposed to just returning > the number of regex matches. (needed for XLOG_BLCKSZ.) Thanks for the review. I notice t

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-03-16 Thread Daniel Gustafsson
> On 25 Feb 2018, at 18:22, Chapman Flack wrote: > Here is a patch implementing the simpler approach Heikki suggested > (though my original leaning had been to wrench on AdvanceXLInsertBuffer > as Michael suggests). The sheer simplicity of the smaller change > eventually won me over, unless there

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2018-02-25 Thread Chapman Flack
On 07/17/17 11:29, Michael Paquier wrote: > On Thu, Jul 6, 2017 at 3:48 PM, Heikki Linnakangas wrote: >> On 07/03/2017 06:30 PM, Chapman Flack wrote: >>> Although it's moot in the straightforward approach of re-zeroing in >>> the loop, it would still help my understanding of the system to know >>>