Re: [HACKERS] Small improvement to parallel query docs

2017-02-14 Thread Amit Kapila
On Tue, Feb 14, 2017 at 3:33 AM, David Rowley wrote: > On 14 February 2017 at 10:56, Brad DeJong wrote: >> David Rowley wrote: >>> I propose we just remove the whole paragraph, and mention about >>> the planning and estimated number of groups stuff in another new paragraph. >>> >>> I've attached

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-14 Thread Pavel Stehule
2017-02-14 3:36 GMT+01:00 Andres Freund : > On 2017-02-13 20:59:43 +0100, Pavel Stehule wrote: > > Hi > > > > 2017-02-13 6:46 GMT+01:00 Michael Paquier : > > > > > On Sat, Feb 11, 2017 at 3:03 PM, Pavel Stehule < > pavel.steh...@gmail.com> > > > wrote: > > > > here is new update - check is done be

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-14 Thread Seki, Eiji
Michael Paquier wrote: > It seems to me that it would be far less confusing to just replace the > boolean argument of GetOldestXmin by a uint8 and get those flags declared in > procarray.h, no? There are a couple of code path calling > GetOldestXmin() that do not include proc.h, so it would be b

Re: [HACKERS] parallelize queries containing subplans

2017-02-14 Thread Amit Kapila
On Tue, Jan 3, 2017 at 4:19 PM, Amit Kapila wrote: > On Wed, Dec 28, 2016 at 11:47 AM, Amit Kapila wrote: >> >> Now, we can further extend this to parallelize queries containing >> correlated subplans like below: >> >> explain select * from t1 where t1.i in (select t2.i from t2 where t2.i=t1.i);

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-14 Thread Corey Huinker
> > @in't gonna execute it? >> > > Hmmm... This is too much of an Americanism, IMHO. The @ looks like a handwritten 'a'. @in't gonna => ain't gonna => will not. It's a bad joke, made as a way of saying that I also could not think of a good mnemonic for '@' or ','. > I'm here all week, try the

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-14 Thread Amit Kapila
On Tue, Feb 14, 2017 at 11:49 AM, Seki, Eiji wrote: > Hi all, > > I propose the patch that adds a function GetOldestXminExtended that is like > GetOldestXmin but can ignore arbitrary vacuum flags. And then, rewrite > GetOldestXmin to use it. Note that this is done so as not to change the > beha

[HACKERS] Sync message

2017-02-14 Thread Tatsuo Ishii
Sync message causes backend to return a "Ready for query" response even if there's no query previously sent to the backend. I don't think this is a bug but I'd think it would be better to write this behavior in the doc, because it might help someone who want to write an API which needs to handle fr

Re: [HACKERS] Small improvement to parallel query docs

2017-02-14 Thread David Rowley
On 14 February 2017 at 21:25, Amit Kapila wrote: > +Aggregate stage. For such cases there is clearly going to be no > +performance benefit to using parallel aggregation. > > A comma is required after "For such cases" Added >> The query planner takes >> +this into account during the p

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-14 Thread Fabien COELHO
For two states: * for being executed (beware, it is ***important***) It does lend importance, but that's also the line continuation marker for "comment". Would that be a problem? Argh. Indeed, even if people seldom type C comments in psql interactive mode... Remaining ASCII characters I

[HACKERS] PostgreSQL Code of Conduct Draft

2017-02-14 Thread Dave Page
The revised draft of the proposed Code of Conduct for the PostgreSQL community is at https://wiki.postgresql.org/wiki/Code_of_Conduct. This updated draft incorporates comments and suggestions from the community received at PgCon Ottawa and subsequent discussion. We will not be monitoring the mail

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2017-02-14 Thread Kyotaro HORIGUCHI
Thank you for the comment. At Mon, 6 Feb 2017 17:10:43 +0100, Pavel Stehule wrote in > > 0001-Refactoring-tab-complete-to-make-psql_completion-cod.patch > > - Just a refactoring of psql_completion > > > > 0002-Make-keywords-case-follow-to-input.patch > > - The letter case of additional sugg

Re: [HACKERS] Measuring replay lag

2017-02-14 Thread Thomas Munro
On Wed, Feb 1, 2017 at 5:21 PM, Michael Paquier wrote: > On Sat, Jan 21, 2017 at 10:49 AM, Thomas Munro > wrote: >> Ok. I see that there is a new compelling reason to move the ring >> buffer to the sender side: then I think lag tracking will work >> automatically for the new logical replication

Re: [HACKERS] Measuring replay lag

2017-02-14 Thread Simon Riggs
On 14 February 2017 at 11:48, Thomas Munro wrote: > Here is a new version with the buffer on the sender side as requested. Thanks, I will definitely review in good time to get this in PG10 -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DB

Re: [HACKERS] Proposal : Parallel Merge Join

2017-02-14 Thread Dilip Kumar
On Tue, Feb 14, 2017 at 12:25 PM, Amit Kapila wrote: > Few review comments on the latest version of the patch: > 1. > - if (joinrel->consider_parallel && nestjoinOK && > - save_jointype != JOIN_UNIQUE_OUTER && > - bms_is_empty(joinrel->lateral_relids)) > + if (outerrel->partial_pathlist == NIL ||

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-14 Thread Alexander Korotkov
On Mon, Feb 13, 2017 at 7:07 PM, Alvaro Herrera wrote: > Alexander Korotkov wrote: > > > Yes, influence seems to be low. But nevertheless it's important to > insure > > that there is no regression here. > > Despite pg_prewarm'ing and running tests 300s, there is still significant > > variation.

Re: [HACKERS] WIP: Covering + unique indexes.

2017-02-14 Thread Amit Kapila
On Mon, Jan 9, 2017 at 8:32 PM, Anastasia Lubennikova wrote: > Updated version of the patch is attached. Besides code itself, it contains > new regression test, > documentation updates and a paragraph in nbtree/README. > The latest patch doesn't apply cleanly. Few assorted comments: 1. @@ -4806,

Re: [HACKERS] pg_waldump's inclusion of backend headers is a mess

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 12:23 AM, Michael Paquier wrote: > On Tue, Feb 14, 2017 at 12:42 PM, Robert Haas wrote: >> Therefore, I proposed the attached patch, which splits spgxlog.h out >> of spgist_private.h, nbtxlog.h out of nbtree.h, gistxlog.h out of >> gist_private.h, and ginxlog.h and ginbloc

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-02-14 Thread Alexander Korotkov
On Mon, Feb 13, 2017 at 10:17 PM, Tomas Vondra wrote: > On 02/13/2017 03:16 PM, Bernd Helmle wrote: > >> Am Samstag, den 11.02.2017, 15:42 +0300 schrieb Alexander Korotkov: >> >>> Thus, I see reasons why in your tests absolute results are lower than >>> in my >>> previous tests. >>> 1. You use 2

Re: [HACKERS] pg_waldump's inclusion of backend headers is a mess

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 2:45 AM, Fabien COELHO wrote: >> You may want to name the new headers dedicated to WAL records with _xlog.h >> as suffix though, like gin_xlog.h instead of ginxlog.h. > > Should not it be more consistent to use "*_wal.h", after all these efforts > to move "xlog" to "wal" ev

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-14 Thread Alvaro Herrera
Alexander Korotkov wrote: > On Mon, Feb 13, 2017 at 7:07 PM, Alvaro Herrera > wrote: > > Re the coding of the padding computation, seems it'd be better to use > > our standard "offsetof(last-struct-member) + sizeof(last-struct-member)" > > rather than adding each of the members' sizes individuall

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-14 Thread Alexander Korotkov
On Tue, Feb 14, 2017 at 3:57 PM, Alvaro Herrera wrote: > Alexander Korotkov wrote: > > On Mon, Feb 13, 2017 at 7:07 PM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > Re the coding of the padding computation, seems it'd be better to use > > > our standard "offsetof(last-struct-me

Re: [HACKERS] pg_waldump's inclusion of backend headers is a mess

2017-02-14 Thread Fabien COELHO
I believe that what was agreed was to eliminate "xlog" from user-facing parts of the system, not internal details. [...] Ok, I get it. So xlog stays xlog if it is hidden from the user, eg file names and probably unexposed functions names, structures or whatever, but everything else has been

Re: [HACKERS] removing tsearch2

2017-02-14 Thread Jim Nasby
On 2/13/17 2:37 AM, Magnus Hagander wrote: That's based on an assumption that PGXN shouldn't be treated as part of the community effort, which I think is a mistake. Having a robust, community run extension/package/module framework has proven to be extremely valuable for other prog

Re: [HACKERS] pg_waldump's inclusion of backend headers is a mess

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 8:15 AM, Fabien COELHO wrote: >> I believe that what was agreed was to eliminate "xlog" from >> user-facing parts of the system, not internal details. [...] > Ok, I get it. So xlog stays xlog if it is hidden from the user, eg file > names and probably unexposed functions na

Re: [HACKERS] Sum aggregate calculation for single precsion real

2017-02-14 Thread Jim Nasby
On 2/13/17 10:45 AM, Konstantin Knizhnik wrote: It is not true - please notice query execution time of this two queries: I bet you'd get even less difference if you simply cast to float8 instead of adding 0.0. Same result, no floating point addition. The expectation for SUM(float4) is that

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-14 Thread Alvaro Herrera
Alexander Korotkov wrote: > On Tue, Feb 14, 2017 at 3:57 PM, Alvaro Herrera > wrote: > > Yes, I understand. I just mean that it could be done something like > > this: > > > > #define PGXACTPadSize (PG_CACHE_LINE_SIZE - (offsetof(PGXACT, nxid) + > > sizeof(uint8))) > > Yes, but I can't use such

Re: [HACKERS] libpq Alternate Row Processor

2017-02-14 Thread Jim Nasby
On 2/13/17 8:46 AM, Kyle Gearhart wrote: profile_filler.txt 61,410,901 ???:_int_malloc [/usr/lib64/libc-2.17.so] 38,321,887 ???:_int_free [/usr/lib64/libc-2.17.so] 31,400,139 ???:pqResultAlloc [/usr/local/pgsql/lib/libpq.so.5.10] 22,839,505 ???:pqParseInput3 [/usr/local/pgsql/lib/libpq.so.5.1

Re: [HACKERS] removing tsearch2

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 8:37 AM, Jim Nasby wrote: > Right; I think we need at least some amount of pgxn buildfarm coverage. > There probably also needs to be a way to officially bless certain > distributions. Unless there's a pretty significant need for an official > extension to be in contrib, it

Re: [HACKERS] Small improvement to parallel query docs

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 5:17 AM, David Rowley wrote: > On 14 February 2017 at 21:25, Amit Kapila wrote: >> +Aggregate stage. For such cases there is clearly going to be no >> +performance benefit to using parallel aggregation. >> >> A comma is required after "For such cases" > > Added > >

Re: [HACKERS] UPDATE of partition key

2017-02-14 Thread Robert Haas
On Mon, Feb 13, 2017 at 7:01 AM, Amit Khandekar wrote: > partspartitioned inheritance no. of rows subpartitions > ==== === === = > > 500 10 sec 3 min 02 sec 1,000,000 0 > 1000 10 sec 3 min 05 sec 1,000,000

Re: [HACKERS] Parallel bitmap heap scan

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 12:18 AM, Dilip Kumar wrote: > Fixed. Thanks, the external interface to this looks much cleaner now. Scrutinizing the internals: What is the point of having a TBMSharedIterator contain a TIDBitmap pointer? All the values in that TIDBitmap are populated from the TBMShared

Re: [HACKERS] Add doc advice about systemd RemoveIPC

2017-02-14 Thread Magnus Hagander
On Fri, Feb 10, 2017 at 10:36 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 12/31/16 11:43 AM, Tom Lane wrote: > > Magnus Hagander writes: > >> I still think that some wording in the direction of the fact that the > >> majority of all users won't actually have this problem

Re: [HACKERS] UPDATE of partition key

2017-02-14 Thread David Fetter
On Mon, Feb 13, 2017 at 05:31:56PM +0530, Amit Khandekar wrote: > Currently, an update of a partition key of a partition is not > allowed, since it requires to move the row(s) into the applicable > partition. > > Attached is a WIP patch (update-partition-key.patch) that removes > this restriction.

Re: [HACKERS] Parallel Append implementation

2017-02-14 Thread Robert Haas
On Mon, Feb 6, 2017 at 12:36 AM, Amit Khandekar wrote: >> Now that I think of that, I think for implementing above, we need to >> keep track of per-subplan max_workers in the Append path; and with >> that, the bitmap will be redundant. Instead, it can be replaced with >> max_workers. Let me check

Re: [HACKERS] Parallel Append implementation

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 12:05 PM, Robert Haas wrote: > Having the extra > 1-2 workers exist does not seem better. Err, exit, not exist. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-14 Thread Magnus Hagander
On Mon, Feb 13, 2017 at 10:33 AM, Michael Banck wrote: > Hi, > > Am Montag, den 13.02.2017, 09:31 +0100 schrieb Magnus Hagander: > > On Mon, Feb 13, 2017 at 3:29 AM, Jim Nasby > > wrote: > > On 2/11/17 4:36 AM, Michael Banck wrote: > > I guess you're right, I've moved it

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove all references to "xlog" from SQL-callable functions in p

2017-02-14 Thread Robert Haas
On Mon, Feb 13, 2017 at 5:00 PM, Michael Paquier wrote: > It seems like the previous review I provided for the set of renaming > patches has been ignored: > https://www.postgresql.org/message-id/CAB7nPqSm=PNSe3EfvnEResdFzpQMcXXgapFBcF=EFdxVW4=f...@mail.gmail.com > Good that somebody else checked..

Re: [HACKERS] Removal of deprecated views pg_user, pg_group, pg_shadow

2017-02-14 Thread Magnus Hagander
On Mon, Feb 13, 2017 at 11:38 PM, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > > On Mon, Feb 13, 2017 at 2:42 PM, Alvaro Herrera > > wrote: > > > Well, we can remove them from PG10 and pgAdmin3 (and others) be > adjusted > > > to use the new ones, conditionally on serv

Re: [HACKERS] removing tsearch2

2017-02-14 Thread David E. Wheeler
On Feb 14, 2017, at 5:37 AM, Jim Nasby wrote: >> Until pgxn has a way of helping users on for example Windows (or other >> platforms where they don't have a pgxs system and a compiler around), >> it's always going to be a "second class citizen". > > I view that as more of a failing of pgxs than

Re: [HACKERS] [PROPOSAL] timestamp informations to pg_stat_statements

2017-02-14 Thread Robert Haas
On Sun, Jul 17, 2016 at 7:15 PM, Tom Lane wrote: > Peter Geoghegan writes: >> On Sun, Jul 17, 2016 at 12:22 AM, Jun Cheol Gim wrote: >>> If we have timestamp of first and last executed, we can easily gather thess >>> informations and there are tons of more use cases. > >> -1 from me. > >> I thin

Re: [HACKERS] AT detach partition is broken

2017-02-14 Thread Robert Haas
On Mon, Feb 13, 2017 at 2:30 AM, Amit Langote wrote: > I noticed that running ALTER TABLE table_name DETACH PARTITION crashes, if > table_name is not a partitioned table. That's because of an Assert in > ATExecDetachPartition(). We really should error out much sooner in this > case, IOW during

Re: [HACKERS] removing tsearch2

2017-02-14 Thread Magnus Hagander
On Feb 14, 2017 18:26, "David E. Wheeler" wrote: On Feb 14, 2017, at 5:37 AM, Jim Nasby wrote: >> Until pgxn has a way of helping users on for example Windows (or other >> platforms where they don't have a pgxs system and a compiler around), >> it's always going to be a "second class citizen".

Re: [HACKERS] pg_basebackup -R

2017-02-14 Thread Robert Haas
On Thu, Feb 9, 2017 at 9:40 PM, Michael Paquier wrote: > On Thu, Feb 9, 2017 at 8:17 PM, Amit Kapila wrote: >> +1. Sounds sensible thing to do. > > Hm. It seems to me that PGPASSFILE still needs to be treated as an > exception because it is set to $HOME/.pgpass without any value set in > PQconni

Re: [HACKERS] Declarative partitioning - another take

2017-02-14 Thread Robert Haas
On Fri, Feb 10, 2017 at 12:54 AM, Amit Langote wrote: > On 2017/02/09 15:22, amul sul wrote: >> About 0001-Check-partition-strategy-in-ATExecDropNotNull.patch, >> following test is already covered in alter_table.sql @ Line # 1918, >> instead of this kindly add test for list_partition: >> >> 77 +-

Re: [HACKERS] WAL consistency check facility

2017-02-14 Thread Robert Haas
On Thu, Feb 9, 2017 at 8:17 PM, Michael Paquier wrote: > Please find attached a patch with those fixes. Committed, but I changed the copyright dates to 2016-2017 rather than just 2017 since surely some of the code was originally written before 2017. Even that might not really be going back far e

Re: [HACKERS] Set of fixes for WAL consistency check facility

2017-02-14 Thread Robert Haas
On Mon, Feb 13, 2017 at 8:00 PM, Michael Paquier wrote: > Beginning a new thread to raise awareness... As already reported here, > I had a look at what has been committed in a507b869: > https://www.postgresql.org/message-id/CAB7nPqRzCQb=vdfhvmtp0hmlbhu6z1agdo4gjsup-hp8jx+...@mail.gmail.com > > Her

Re: [HACKERS] Parallel Index Scans

2017-02-14 Thread Robert Haas
On Mon, Feb 13, 2017 at 9:04 PM, Amit Kapila wrote: > I think the comment at that place is not as clear as it should be. So > how about changing it as below: > > Existing comment: > -- > /* > * For parallel scans, get the last page scanned as it is quite > * possible that

Re: [HACKERS] removing tsearch2

2017-02-14 Thread David E. Wheeler
On Feb 14, 2017, at 9:37 AM, Magnus Hagander wrote: > It's a failing in one of the two at least. It either needs to be easier to > build the things on windows, or pgxn would need to learn to do binary > distributions. PGXN makes no effort to support installation on any platform at all. Happy

Re: [HACKERS] Possible TODO: allow arbitrary expressions in event trigger WHEN

2017-02-14 Thread Jim Nasby
On 2/13/17 9:34 PM, Tom Lane wrote: Jim Nasby writes: Is there a reason not to allow $SUBJECT? Specifically, it'd be nice to be able to do something like WHEN tag LIKE 'ALTER%'. Seems like it would be a seriously bad idea for such an expression to be able to invoke arbitrary SQL code. What i

[HACKERS] Add checklist item for psql completion to commitfest review

2017-02-14 Thread Jim Nasby
After seeing Yet Another Missing Psql Tab Completion it occurred to me... why not add a checklist item to the commitfest review page? I realize most regular contributors don't use the form, but a fair number of people do. I don't see how it could hurt. Another possible idea is a git hook that

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-14 Thread Jim Nasby
On 2/13/17 9:36 PM, Michael Paquier wrote: Probably I failed to get Peter's point... Anyway IMO that we can expose all the columns except the sensitive information (i.e., subconninfo field) in pg_subscription to even non-superusers. Then we can use pg_subscription for the tab-completion for ALTER

Re: [HACKERS] log_autovacuum_min_duration doesn't log VACUUMs

2017-02-14 Thread Jim Nasby
On 2/13/17 11:12 AM, Robert Haas wrote: FWIW, this is a significant problem outside of DDL. Once you're past 1-2 levels of nesting SET client_min_messages = DEBUG becomes completely useless. I think the ability to filter logging based on context would be very valuable. AFAIK you could actually d

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-14 Thread Robert Haas
On Mon, Feb 13, 2017 at 3:40 PM, Fabien COELHO wrote: > What would be the mnemonic for "," an "@"? Oh, I just picked it because control-@ is the nul character, and your commands would be nullified. I realize that's pretty weak, but we're talking about finding a punctuation mark to represent the

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-14 Thread Jim Nasby
On 2/13/17 8:50 PM, Andres Freund wrote: On 2017-02-14 11:46:52 +0900, Michael Paquier wrote: I still fail to see why --use-existing as suggested in https://www.postgresql.org/message-id/20170208002900.vkldujzfkwbvq...@alap3.anarazel.de isn't sufficient. Some tests create objects without remov

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-14 Thread Jim Nasby
On 2/14/17 3:13 AM, Seki, Eiji wrote: +extern TransactionId GetOldestXmin(Relation rel, uint8 ignoreFlags); My impression is that most other places that do this sort of thing just call the argument 'flags', so as not to "lock in" a single idea of what the flags are for. I can't readily thin

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 12:06 PM, Magnus Hagander wrote: > However, outputing this info by default will make it show up in things like > everybodys cronjobs by default. Right now a successful pg_basebackup run > will come out with no output at all, which is how most Unix commands work, > and bring

Re: [HACKERS] renaming pg_resetxlog to pg_resetwal has broken pg_upgrade.

2017-02-14 Thread Jeff Janes
On Mon, Feb 13, 2017 at 6:19 PM, Michael Paquier wrote: > On Tue, Feb 14, 2017 at 9:09 AM, Jeff Janes wrote: > > check for "/usr/local/pgsql9_6/bin/pg_resetwal" failed: No such file or > > directory > > > > This looks somewhat complicated to fix. Should check_bin_dir test the > old > > cluster

Re: [HACKERS] PATCH: two slab-like memory allocators

2017-02-14 Thread Tomas Vondra
On 02/14/2017 03:22 AM, Andres Freund wrote: Hi, On 2017-02-11 14:40:18 +0100, Tomas Vondra wrote: On 02/11/2017 02:33 AM, Andres Freund wrote: I have a hard time believing this the cache efficiency of linked lists (which may or may not be real in this case) out-weights this, but if you want t

Re: [HACKERS] Improve OR conditions on joined columns (common star schema problem)

2017-02-14 Thread Tom Lane
I wrote: > The main remaining piece of work here is that, as you can see from the > above, it fails to eliminate joins to tables that we don't actually need > in a particular UNION arm. This is because the references to those > tables' ctid columns prevent analyzejoins.c from removing the joins. >

[HACKERS] Official adoption of PGXN (was: removing tsearch2)

2017-02-14 Thread Jim Nasby
First, just to clarify: my reasons for proposing "core adoption" of PGXN are not technical in nature. My desire is to have an extension/add-on system that's officially endorsed and embraced by the official community, similar to CPAN, pypy, npm, etc. There's no technical reason we need PGXN to b

Re: [HACKERS] pg_waldump's inclusion of backend headers is a mess

2017-02-14 Thread Andres Freund
Hi, On 2017-02-13 22:42:00 -0500, Robert Haas wrote: > I dug into the problem and discovered that pg_waldump is slurping up a > tremendous crapload of backend headers. It includes gin.h, > gist_private.h, hash_xlog.h, nbtree.h, and spgist.h, which all end up > including amapi.h, which includes ge

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-14 Thread Andres Freund
On 2017-02-14 12:33:35 -0600, Jim Nasby wrote: > On 2/13/17 8:50 PM, Andres Freund wrote: > > On 2017-02-14 11:46:52 +0900, Michael Paquier wrote: > > > > I still fail to see why --use-existing as suggested in > > > > https://www.postgresql.org/message-id/20170208002900.vkldujzfkwbvq...@alap3.anara

Re: [HACKERS] Official adoption of PGXN (was: removing tsearch2)

2017-02-14 Thread Tom Lane
Jim Nasby writes: > First, just to clarify: my reasons for proposing "core adoption" of PGXN > are not technical in nature. What do you think "core adoption" means? Surely not that anything associated with PGXN would be in the core distro. > Right now contrib is serving two completely separate

Re: [HACKERS] Official adoption of PGXN

2017-02-14 Thread Jim Nasby
On 2/14/17 2:05 PM, Tom Lane wrote: Jim Nasby writes: First, just to clarify: my reasons for proposing "core adoption" of PGXN are not technical in nature. What do you think "core adoption" means? Surely not that anything associated with PGXN would be in the core distro. No, certainly not.

Re: [HACKERS] Official adoption of PGXN

2017-02-14 Thread Josh Berkus
On 02/14/2017 12:05 PM, Tom Lane wrote: > Jim Nasby writes: >> First, just to clarify: my reasons for proposing "core adoption" of PGXN >> are not technical in nature. > > What do you think "core adoption" means? Surely not that anything > associated with PGXN would be in the core distro. One

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-14 Thread Jim Nasby
On 2/14/17 1:59 PM, Andres Freund wrote: AFAIK if you're doing make check (as opposed to installcheck) it's significantly more complicated than that since you'd have to create a temp cluster/install yourself. > But in that case you can't have useful templates in the regression test either, so t

Re: [HACKERS] Parallel Index Scans

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 12:48 PM, Robert Haas wrote: > That sounds way better. Here's an updated patch. Please review my changes, which include: * Various comment updates. * _bt_parallel_seize now unconditionally sets *pageno to P_NONE at the beginning, instead of doing it conditionally at the

Re: [HACKERS] Official adoption of PGXN

2017-02-14 Thread Jim Nasby
On 2/14/17 2:19 PM, Josh Berkus wrote: One part of this would need to be having a designated committee of the Postgres community pick a set of "blessed" extensions for packagers to package. Right now, contrib serves that purpose (badly). One of the reasons we haven't dealt with the extension di

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-14 Thread Andres Freund
On 2017-02-14 14:29:56 -0600, Jim Nasby wrote: > On 2/14/17 1:59 PM, Andres Freund wrote: > > > AFAIK if you're doing make check (as opposed to installcheck) it's > > > significantly more complicated than that since you'd have to create a temp > > > cluster/install yourself. > > > > But in that cas

Re: [HACKERS] Official adoption of PGXN

2017-02-14 Thread Andres Freund
On 2017-02-14 12:19:56 -0800, Josh Berkus wrote: > On 02/14/2017 12:05 PM, Tom Lane wrote: > > Jim Nasby writes: > >> First, just to clarify: my reasons for proposing "core adoption" of PGXN > >> are not technical in nature. > > > > What do you think "core adoption" means? Surely not that anyth

Re: [HACKERS] Small improvement to parallel query docs

2017-02-14 Thread David Rowley
On 15 February 2017 at 03:41, Robert Haas wrote: > On Tue, Feb 14, 2017 at 5:17 AM, David Rowley > wrote: >> Updated patch attached. > > Committed and back-patched to 9.6. Great. Thanks Robert. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Suppor

Re: [HACKERS] Improve OR conditions on joined columns (common star schema problem)

2017-02-14 Thread Jim Nasby
On 2/14/17 1:18 PM, Tom Lane wrote: I think this might be code-complete, modulo the question of whether we want an enabling GUC for it. I'm still concerned about the question of whether it adds more planning time than it's worth for most users. (Obviously it needs some regression test cases too,

Re: [HACKERS] pg_waldump's inclusion of backend headers is a mess

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 2:54 PM, Andres Freund wrote: >> Thoughts, comments, objections, better ideas? > > No better ideas. I'm a bit concerned about declarations needed both by > normal and xlog related routines, but I guess that can be solved by a > third header as you did. Yeah, that doesn't

Re: [HACKERS] possibility to specify template database for pg_regress

2017-02-14 Thread Pavel Stehule
Dne 14. 2. 2017 21:35 napsal uživatel "Andres Freund" : On 2017-02-14 14:29:56 -0600, Jim Nasby wrote: > On 2/14/17 1:59 PM, Andres Freund wrote: > > > AFAIK if you're doing make check (as opposed to installcheck) it's > > > significantly more complicated than that since you'd have to create a tem

Re: [HACKERS] pg_waldump's inclusion of backend headers is a mess

2017-02-14 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Feb 14, 2017 at 2:54 PM, Andres Freund wrote: > >> Thoughts, comments, objections, better ideas? > > > > No better ideas. I'm a bit concerned about declarations needed both by > > normal and xlog related routines, but I guess that can be solved by a > > third header a

Re: [HACKERS] Improve OR conditions on joined columns (common star schema problem)

2017-02-14 Thread Tom Lane
Jim Nasby writes: > On 2/14/17 1:18 PM, Tom Lane wrote: >> One point that could use further review is whether the de-duplication >> algorithm is actually correct. I'm only about 95% convinced by the >> argument I wrote in planunionor.c's header comment. > I'll put some thought into it and see if

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-14 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Feb 14, 2017 at 12:06 PM, Magnus Hagander wrote: > > However, outputing this info by default will make it show up in things like > > everybodys cronjobs by default. Right now a successful pg_basebackup run > > will come out with no output at all, which is how most Unix

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-14 Thread Tom Lane
Corey Huinker writes: > So moving the conditional stack back into PsqlScanState has some side > effects: conditional.[ch] have to move to the fe_utils/ dirs, and now > pgbench, which does not use conditionals, would have to link to them. Is > that a small price to pay for modularity and easier-to-

[HACKERS] bytea_output vs make installcheck

2017-02-14 Thread Jeff Janes
make installcheck currently fails against a server running with bytea_output = escape. Making it succeed is fairly easy, and I think it is worth doing. Attached are two options for doing that. One overrides bytea_output locally where-ever needed, and the other overrides it for the entire 'regres

Re: [HACKERS] Set of fixes for WAL consistency check facility

2017-02-14 Thread Michael Paquier
On Wed, Feb 15, 2017 at 2:44 AM, Robert Haas wrote: > I committed the patch posted to the other thread. Hopefully that > closes this issue. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] renaming pg_resetxlog to pg_resetwal has broken pg_upgrade.

2017-02-14 Thread Michael Paquier
On Wed, Feb 15, 2017 at 3:55 AM, Jeff Janes wrote: > I thought people would object to checking the version number in two > different places to make the same fundamental decision, and would want that > refactored somehow. But if you are OK with it, then I am. The binary versions are checked only

Re: [HACKERS] WAL consistency check facility

2017-02-14 Thread Michael Paquier
On Wed, Feb 15, 2017 at 2:43 AM, Robert Haas wrote: > On Thu, Feb 9, 2017 at 8:17 PM, Michael Paquier > wrote: >> Please find attached a patch with those fixes. > > Committed, but I changed the copyright dates to 2016-2017 rather than > just 2017 since surely some of the code was originally writt

Re: [HACKERS] Skipping PgStat_FunctionCallUsage for many expressions

2017-02-14 Thread Andres Freund
On 2016-11-26 08:41:28 -0800, Andres Freund wrote: > On November 26, 2016 8:06:26 AM PST, Tom Lane wrote: > >Robert Haas writes: > >> On Fri, Nov 25, 2016 at 11:12 PM, Andres Freund > >wrote: > >>> while working on my faster expression evaluation stuff I noticed > >that a > >>> lot of expression

Re: [HACKERS] Skipping PgStat_FunctionCallUsage for many expressions

2017-02-14 Thread Tom Lane
Andres Freund writes: > On 2016-11-26 08:41:28 -0800, Andres Freund wrote: >> On November 26, 2016 8:06:26 AM PST, Tom Lane wrote: >>> Those don't call functions, they call operators. Yes, I know that an >>> operator has a function underlying it, but the user-level expectation >>> for track_func

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-14 Thread Corey Huinker
On Tue, Feb 14, 2017 at 4:44 PM, Tom Lane wrote: > Corey Huinker writes: > > So moving the conditional stack back into PsqlScanState has some side > > effects: conditional.[ch] have to move to the fe_utils/ dirs, and now > > pgbench, which does not use conditionals, would have to link to them. I

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-14 Thread Fabien COELHO
Hello Tom, So moving the conditional stack back into PsqlScanState has some side effects: conditional.[ch] have to move to the fe_utils/ dirs, and now pgbench, which does not use conditionals, would have to link to them. Is that a small price to pay for modularity and easier-to-find code? Or sh

Re: [HACKERS] bytea_output vs make installcheck

2017-02-14 Thread neha khatri
On Wed, Feb 15, 2017 at 8:50 AM, Jeff Janes wrote: > make installcheck currently fails against a server running > with bytea_output = escape. > > Making it succeed is fairly easy, and I think it is worth doing. > > Attached are two options for doing that. One overrides bytea_output > locally whe

Re: [HACKERS] parallelize queries containing subplans

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 4:24 AM, Amit Kapila wrote: > On further evaluation, it seems this patch has one big problem which > is that it will allow forming parallel plans which can't be supported > with current infrastructure. For ex. marking immediate level params > as parallel safe can generate

Re: [HACKERS] Skipping PgStat_FunctionCallUsage for many expressions

2017-02-14 Thread Andres Freund
On 2017-02-14 17:58:23 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2016-11-26 08:41:28 -0800, Andres Freund wrote: > >> On November 26, 2016 8:06:26 AM PST, Tom Lane wrote: > >>> Those don't call functions, they call operators. Yes, I know that an > >>> operator has a function underlyi

Re: [HACKERS] parallelize queries containing subplans

2017-02-14 Thread Robert Haas
On Mon, Feb 13, 2017 at 11:07 PM, Amit Kapila wrote: > I have removed the check of AlternativeSubPlan so that it can be > handled by expression_tree_walker. ... > Attached patch fixes all the comments raised till now. Committed after removing the reference to AlternativeSubPlan from the comment.

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera wrote: > I'd rather have a --quiet mode instead. If you're running it by hand, > you're likely to omit the switch, whereas when writing the cron job > you're going to notice lack of switch even before you let the job run > once. Well, that might've

Re: [HACKERS] WAL consistency check facility

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 5:16 PM, Michael Paquier wrote: > On Wed, Feb 15, 2017 at 2:43 AM, Robert Haas wrote: >> On Thu, Feb 9, 2017 at 8:17 PM, Michael Paquier >> wrote: >>> Please find attached a patch with those fixes. >> >> Committed, but I changed the copyright dates to 2016-2017 rather tha

[HACKERS] new high availability feature for the system with both asynchronous and synchronous replication

2017-02-14 Thread Higuchi, Daisuke
Hi all, I propose a new feature for high availability. This configuration is effective for following configuration: 1. Primary and synchronous standby are in the same center; called main center. 2. Asynchronous standby is in the another center; called backup center. (The backup center is l

Re: [HACKERS] Possible TODO: allow arbitrary expressions in event trigger WHEN

2017-02-14 Thread Robert Haas
On Mon, Feb 13, 2017 at 10:34 PM, Tom Lane wrote: > Jim Nasby writes: >> Is there a reason not to allow $SUBJECT? Specifically, it'd be nice to >> be able to do something like WHEN tag LIKE 'ALTER%'. > > Seems like it would be a seriously bad idea for such an expression to be > able to invoke arb

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove all references to "xlog" from SQL-callable functions in p

2017-02-14 Thread Michael Paquier
On Wed, Feb 15, 2017 at 2:08 AM, Robert Haas wrote: > On Mon, Feb 13, 2017 at 5:00 PM, Michael Paquier > wrote: >> It seems like the previous review I provided for the set of renaming >> patches has been ignored: >> https://www.postgresql.org/message-id/CAB7nPqSm=PNSe3EfvnEResdFzpQMcXXgapFBcF=EFd

Re: [HACKERS] WAL consistency check facility

2017-02-14 Thread Tom Lane
Robert Haas writes: > On Tue, Feb 14, 2017 at 5:16 PM, Michael Paquier > wrote: >> Just for curiosity: does the moment when the code has been written or >> committed counts? It's no big deal seeing how liberal the Postgres >> license is, but this makes me wonder... > IANAL, but I think if you as

[HACKERS] Missing CHECK_FOR_INTERRUPTS in hash joins

2017-02-14 Thread Tom Lane
I ran into a case where a hash join took a really long time to respond to a cancel request --- long enough that I gave up and kill -9'd it, because its memory usage was also growing to the point where the kernel would likely soon choose to do that for me. The culprit seems to be that there's no CH

Re: [HACKERS] [Bug fix] PQsendQuery occurs error when target_session_attrs is set to read-write

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 12:11 AM, Michael Paquier wrote: > On Thu, Feb 2, 2017 at 3:01 PM, Higuchi, Daisuke > wrote: >> From: Michael Paquier [mailto:michael.paqu...@gmail.com] >>>This has not been added yet to the next CF. As Ashutosh mentioned >>>things tend to be easily forgotten. So I have ad

Re: [HACKERS] WAL consistency check facility

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 7:12 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Feb 14, 2017 at 5:16 PM, Michael Paquier >> wrote: >>> Just for curiosity: does the moment when the code has been written or >>> committed counts? It's no big deal seeing how liberal the Postgres >>> license is, bu

Re: [HACKERS] WAL consistency check facility

2017-02-14 Thread Michael Paquier
On Wed, Feb 15, 2017 at 11:08 AM, Robert Haas wrote: > On Tue, Feb 14, 2017 at 7:12 PM, Tom Lane wrote: >> FWIW, my own habit when creating new PG files is generally to write >> >> * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group >> * Portions Copyright (c) 1994, Regents

  1   2   >