Re: [HACKERS] pg_xlogdump fails to handle WAL file with multi-page XLP_FIRST_IS_CONTRECORD data

2016-08-28 Thread Fujii Masao
On Fri, Aug 26, 2016 at 10:03 PM, Fujii Masao wrote: > On Wed, Mar 23, 2016 at 7:04 PM, Pavan Deolasee > wrote: >> >> >> On Wed, Mar 23, 2016 at 1:13 PM, Michael Paquier >> wrote: >>> >>> >>> + /* >>> +* Compute targetRecOff. It should typically be greater than short >>> +*

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-28 Thread Michael Paquier
On Mon, Aug 29, 2016 at 2:36 PM, Craig Ringer wrote: > I don't care if it comes as part of some greater reorg or not but I'll be > really annoyed if scope creep lands up killing the original proposal to just > rename these dirs. I think that a simple rename should be done first. Then > if some gre

[HACKERS] ecpg -? option doesn't work in windows

2016-08-28 Thread Haribabu Kommi
ecpg option --help alternative -? doesn't work in windows. In windows, the PG provided getopt_long function is used for reading the provided options. The getopt_long function returns '?' for invalid characters also but it sets optopt option to 0 in case if the character itself is a '?'. But this w

[HACKERS] GiST penalty functions [PoC]

2016-08-28 Thread Andrew Borodin
Hi, hackers! Some time ago I put a patch to commitfest that optimizes slightly GiST inserts and updates. It's effect in general is quite modest (15% perf improved), but for sorted data inserts are 3 times quicker. This effect I attribute to mitigation of deficiencies of old split algorithm. Alexa

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-08-28 Thread Robert Haas
On Sun, Aug 28, 2016 at 11:25 PM, Craig Ringer wrote: > What I'd really like is to be able to ask transam.c to handle the > xid_in_recent_past logic, treating an attempt to read an xid from > beyond the clog truncation threshold as a soft error indicating > unknown xact state. But that involves de

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-28 Thread Craig Ringer
On 29 Aug 2016 12:10 PM, "Jim Nasby" wrote: > > On 8/26/16 4:08 PM, Andres Freund wrote: >> >> Splitting of ephemeral data seems to have a benefit, the rest seems more >> like rather noisy busywork to me. > > > People accidentally blowing away pg_clog or pg_xlog is a pretty common occurrence, and

Re: [HACKERS] patch proposal

2016-08-28 Thread Venkata B Nagothi
On Fri, Aug 26, 2016 at 10:58 PM, Stephen Frost wrote: > * Venkata B Nagothi (nag1...@gmail.com) wrote: > > On Fri, Aug 26, 2016 at 12:30 PM, Stephen Frost > wrote: > > > * Venkata B Nagothi (nag1...@gmail.com) wrote: > > > > This behaviour will be similar to that of > recovery_target="immediate

Re: [HACKERS] WAL consistency check facility

2016-08-28 Thread Kuntal Ghosh
Thank you. I've updated it accordingly. On Sun, Aug 28, 2016 at 11:20 AM, Peter Geoghegan wrote: > On Sat, Aug 27, 2016 at 9:47 PM, Amit Kapila wrote: >> Right, I think there is no need to mask all the flags. However apart >> from BTP_HAS_GARBAGE, it seems we should mask BTP_SPLIT_END as that i

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-28 Thread Jim Nasby
On 8/26/16 4:08 PM, Andres Freund wrote: Splitting of ephemeral data seems to have a benefit, the rest seems more like rather noisy busywork to me. People accidentally blowing away pg_clog or pg_xlog is a pretty common occurrence, and I don't think there's all that many tools that reference t

Re: [HACKERS] Why is a newly created index contains the invalid LSN?

2016-08-28 Thread Jim Nasby
On 8/26/16 4:17 PM, Andres Freund wrote: On 2016-08-26 18:46:42 +0300, Yury Zhuravlev wrote: Thanks all. Now understand LSN strongly connected with WAL. However how difficult put last system LSN instead 0? It's not so important but will allow make use LSN more consistent. Maybe explain why you

Re: [HACKERS] PostgreSQL Version 10, missing minor version

2016-08-28 Thread Andres Freund
On 2016-08-29 11:41:00 +0800, Craig Ringer wrote: > On 29 August 2016 at 02:52, Tom Lane wrote: > > "Regina Obe" writes: > >> The routine in PostGIS to parse out the version number from pg_config is > >> breaking in the 10 cycle > > > > TBH, I wonder why you are doing that in the first place; it

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-08-28 Thread Andres Freund
Hi, On 2016-08-29 11:25:39 +0800, Craig Ringer wrote: > ERROR: could not access status of transaction 778793573 > DETAIL: could not open file "pg_clog/02E6": No such file or directory > > What I'd really like is to be able to ask transam.c to handle the > xid_in_recent_past logic, treating

Re: [HACKERS] PostgreSQL Version 10, missing minor version

2016-08-28 Thread Craig Ringer
On 29 August 2016 at 02:52, Tom Lane wrote: > "Regina Obe" writes: >> The routine in PostGIS to parse out the version number from pg_config is >> breaking in the 10 cycle > > TBH, I wonder why you are doing that in the first place; it does not > seem like the most reliable source of version infor

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-08-28 Thread Craig Ringer
On 24 August 2016 at 03:10, Robert Haas wrote: > > On Tue, Aug 23, 2016 at 12:59 PM, Craig Ringer wrote: > > Also fine by me. You're right, keep it simple. It means the potential set of > > values isn't discoverable the same way, but ... meh. Using it usefully means > > reading the docs anyway. >

[HACKERS] VACUUM's ancillary tasks

2016-08-28 Thread Masahiko Sawada
On Monday, 29 August 2016, Andres Freund > wrote: > Hi, > > On 2016-08-29 03:26:06 +0200, Vik Fearing wrote: > > The attached two patches scratch two itches I've been having for a > > while. I'm attaching them together because the second depends on the > first. > > > > Both deal with the fact tha

Re: [HACKERS] VACUUM's ancillary tasks

2016-08-28 Thread Andres Freund
Hi, On 2016-08-29 03:26:06 +0200, Vik Fearing wrote: > The attached two patches scratch two itches I've been having for a > while. I'm attaching them together because the second depends on the first. > > Both deal with the fact that [auto]vacuum has taken on more roles than > its original purpos

[HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-08-28 Thread Kouhei Kaigai
Hello, The attached patch adds an optional callback to support special optimization if ForeignScan/CustomScan are located under the Limit node in plan-tree. Our sort node wisely switches the behavior when we can preliminary know exact number of rows to be produced, because all the Sort node has t

[HACKERS] Sample configuration files

2016-08-28 Thread Vik Fearing
We have sample configuration files for postgresql.conf and recovery.conf, but we do not have them for contrib modules. This patch attempts to add them. Although the patch puts the sample configuration files in the tree, it doesn't try to install them. That's partly because I think it would need

[HACKERS] VACUUM's ancillary tasks

2016-08-28 Thread Vik Fearing
The attached two patches scratch two itches I've been having for a while. I'm attaching them together because the second depends on the first. Both deal with the fact that [auto]vacuum has taken on more roles than its original purpose. Patch One: autovacuum insert-heavy tables If you have a ta

[HACKERS] comment fix for CUSTOMPATH_* flags

2016-08-28 Thread Kouhei Kaigai
Hello, I noticed the source code comment around CustomPath structure says "see above" for definition of CUSTOMPATH_* flags. It was originally right, but it was moved to nodes/extensible.h on the further development. So, no comments are above. The attached patch corrects the comment for the right l

[HACKERS] Reminder: 9.6rc1 wraps tomorrow

2016-08-28 Thread Tom Lane
Just making sure everyone's on the same page. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] distinct estimate of a hard-coded VALUES list

2016-08-28 Thread Jeff Janes
On Mon, Aug 22, 2016 at 10:19 AM, Robert Haas wrote: > On Sat, Aug 20, 2016 at 4:58 PM, Tom Lane wrote: > > Jeff Janes writes: > >> On Thu, Aug 18, 2016 at 2:25 PM, Tom Lane wrote: > >>> It does know it, what it doesn't know is how many duplicates there are. > > > >> Does it know whether the c

Re: [HACKERS] src/include/catalog/pg_foreign_table.h still refers genbki.sh

2016-08-28 Thread Tom Lane
Tomas Vondra writes: > I've noticed the comment in src/include/catalog/pg_foreign_table.h still > talks about genbki.sh, but AFAIK it was replaced with genbki.pl. Hmm, somebody must have copied-and-pasted this header comment from some old catalog include file. Fixed, thanks.

Re: [HACKERS] RLS related docs

2016-08-28 Thread Joe Conway
On 05/30/2016 01:56 PM, Joe Conway wrote: > On 05/26/2016 12:26 AM, Dean Rasheed wrote: >> On 25 May 2016 at 02:04, Joe Conway wrote: >>> Please see attached two proposed patches for the docs related to RLS: >>> >>> 1) Correction to pg_restore >>> 2) Additional mentions that "COPY FROM" does not a

Re: [HACKERS] PostgreSQL Version 10, missing minor version

2016-08-28 Thread Tom Lane
"Regina Obe" writes: > The routine in PostGIS to parse out the version number from pg_config is > breaking in the 10 cycle TBH, I wonder why you are doing that in the first place; it does not seem like the most reliable source of version information. If you need to do compile-time tests, PG_CATA

Re: [HACKERS] PostgreSQL Version 10, missing minor version

2016-08-28 Thread Joe Conway
On 08/28/2016 09:55 AM, Regina Obe wrote: > The routine in PostGIS to parse out the version number from pg_config is > breaking in the 10 cycle. > > Issue seems to be because there is no minor specified. > > e.g. > > pgconfig --version > > returns: > > PostgreSQL 10devel > > Instead of expec

[HACKERS] PostgreSQL Version 10, missing minor version

2016-08-28 Thread Regina Obe
The routine in PostGIS to parse out the version number from pg_config is breaking in the 10 cycle. Issue seems to be because there is no minor specified. e.g. pgconfig --version returns: PostgreSQL 10devel Instead of expected PostgreSQL 10.0devel Is this the way it's going to be or will th

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Tomas Vondra
On 08/28/2016 04:59 PM, Tom Lane wrote: Tomas Vondra writes: I'm however pretty sure this will have absolutely no impact on profiles. Yeah, I do not believe that either. Also, I think that the intent of the existing code is to defend against bad parameters even in non-assert builds. Which m

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Tom Lane
Tomas Vondra writes: > I'm however pretty sure this will have absolutely no impact on profiles. Yeah, I do not believe that either. Also, I think that the intent of the existing code is to defend against bad parameters even in non-assert builds. Which might argue for turning these into test-an

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Tomas Vondra
On 08/28/2016 04:41 PM, Tom Lane wrote: Robert Haas writes: Also, I think we ought to replace this code in aset.c: initBlockSize = MAXALIGN(initBlockSize); if (initBlockSize < 1024) initBlockSize = 1024; maxBlockSize = MAXALIGN(maxBlockSize); With this: Assert(i

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Tomas Vondra
On 08/28/2016 04:41 PM, Tom Lane wrote: Robert Haas writes: Also, I think we ought to replace this code in aset.c: initBlockSize = MAXALIGN(initBlockSize); if (initBlockSize < 1024) initBlockSize = 1024; maxBlockSize = MAXALIGN(maxBlockSize); With this: Assert(i

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Tom Lane
Robert Haas writes: > Also, I think we ought to replace this code in aset.c: > initBlockSize = MAXALIGN(initBlockSize); > if (initBlockSize < 1024) > initBlockSize = 1024; > maxBlockSize = MAXALIGN(maxBlockSize); > With this: > Assert(initBlockSize >= 1024 && initBlockSi

Re: [HACKERS] increasing the default WAL segment size

2016-08-28 Thread Robert Haas
On Fri, Aug 26, 2016 at 4:45 AM, Eduardo Morras wrote: > From my ignorance, could block size affect this WAL size increase? > > In past (didn't tried with >9 versions) you can change disk block page size > from 8KB to 16 KB or 32KB (or 4) modifing src/include/pg_config.h BLCKSZ 8192 > and recomp

Re: [HACKERS] increasing the default WAL segment size

2016-08-28 Thread Robert Haas
On Fri, Aug 26, 2016 at 12:39 AM, Andres Freund wrote: > Maybe I'm missing something here - but why would we need to do any of > that? The WAL already isn't compatible between versions, and we don't > reuse the old server's WAL anyway? Isn't all that's needed relaxing some > error check? Yeah. I

Re: [HACKERS] Checksum error and VACUUM FULL

2016-08-28 Thread Robert Haas
On Thu, Aug 25, 2016 at 5:45 AM, Tatsuki Kadomoto < tatsuki.kadom...@proceranetworks.com> wrote: > I faced incorrect checksum on "global/pg_filenode.map" at the right > timing "VACUUM FULL" is executed and session was aborted. > > Aug 16 20:51:19 postgres[22329]: [2-1] FATAL: relation mapping fil

Re: [HACKERS] Bogus sizing parameters in some AllocSetContextCreate calls

2016-08-28 Thread Robert Haas
On Sat, Aug 27, 2016 at 2:08 PM, Tom Lane wrote: > The standard calling pattern for AllocSetContextCreate is > > cxt = AllocSetContextCreate(parent, name, > ALLOCSET_DEFAULT_MINSIZE, > ALLOCSET_DEFAULT_INITSIZE, >