Re: [HACKERS] [COMMITTERS] pgsql: Add new function dsa_allocate0.

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 6:31 PM, Thomas Munro wrote: >> Agreed. Here's a patch like that. > > Oops, that had a typo in a comment. Here's a better one. This looked fine, except that the new comment in dsa_allocate contained an extremely long sentence which happened to contradict the last sentenc

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Devrim Gündüz
Hi, On Thu, 2017-02-16 at 08:21 -0500, Peter Eisentraut wrote: > I will wait for that before proceeding. Sorry for the delay, I also had to build a newer flex RPM before proceeding. arapaima(x86) and aholehole(x86_64) are the new animals. They are running the buildfarm script now. Regards, --

[HACKERS] About adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread 李跃森
Hi hackers: When using pg_stat_statements to collect running SQL of PG, we find it is hard for our program to get exact operation type of the SQL, such as SELECT, DELETE, UPDATE, INSERT, and so on. So we modify the the source code of pg_stat_statements and add another output paramet

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 11:53 PM, Tom Lane wrote: >> It's a little surprising to me that the delay we're seeing here is >> significant, because the death of a child should cause an immediate >> SIGCHLD, resulting in a call to reaper(), resulting in a call to >> waitpid(). Why's that not working?

Re: [HACKERS] Gather Merge

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 6:43 PM, Amit Kapila wrote: > I think there is a value in supporting mark/restore position for any > node which produces sorted results, however, if you don't want to > support it, then I think we should update above comment in the code to > note this fact. Also, you might

Re: [HACKERS] Suppressing that pesky warning with older flex versions

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 10:40 PM, Tom Lane wrote: > While experimenting with enabling -Werror in the buildfarm, I got > annoyed about the fact that we have to apply -Wno-error while > building some of the flex scanners, because with flex versions > before 2.5.36 you get > > scan.c: In function 'yy

[HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread 李跃森
Hi hackers: When using pg_stat_statements to collect running SQL of PG, we find it is hard for our program to get exact operation type of the SQL, such as SELECT, DELETE, UPDATE, INSERT, and so on. So we modify the the source code of pg_stat_statements and add another output paramet

Re: [HACKERS] Parallel Append implementation

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 11:44 AM, Ashutosh Bapat wrote: > That's true for a partitioned table, but not necessarily for every > append relation. Amit's patch is generic for all append relations. If > the child plans are joins or subquery segments of set operations, I > doubt if the same logic works

[HACKERS]About adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread 李跃森
Hi hackers: When using pg_stat_statements to collect running SQL of PG, we find it is hard for our program to get exact operation type of the SQL, such as SELECT, DELETE, UPDATE, INSERT, and so on. So we modify the the source code of pg_stat_statements and add another output paramet

[HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread 李跃森
Hi PG hackers: When using pg_stat_statements to collect running SQL of PG, we find it is hard for our program to get exact operation type of the SQL, such as SELECT, DELETE, UPDATE, INSERT, and so on. So we modify the the source code of pg_stat_statements and add another output para

Re: [HACKERS] Parallel Append implementation

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 2:56 PM, Amit Khandekar wrote: > The log2(num_children)+1 formula which you proposed does not take into > account the number of workers for each of the subplans, that's why I > am a bit more inclined to look for some other logic. May be, treat the > children as if they belo

Re: [HACKERS] Reporting xmin from VACUUMs

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 12:30 AM, Jim Nasby wrote: >> What? > > There's a bunch of information reported by vacuum logging but not in > pg_stat*, such as all-visible/frozen skipping, unable to get cleanup lock, > last freeze scan, times autovac has been interrupted. There's been > resistance in the

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 3:31 AM, Tom Lane wrote: > Thoughts? Should we double down on trying to make this work according > to the "all integer timestamps" protocol specs, or cut our losses and > change the specs? I vote for doubling down. It's bad enough that we have so many internal details th

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 1:12 PM, Amit Langote wrote: >> I agree. But, we need to be careful that a database-wide VACUUM or >> ANALYZE doesn't hit the partitions multiple times, once for the parent >> and again for each child. Actually, a database-wide VACUUM should hit >> each partition individu

Re: [HACKERS] SCRAM authentication, take three

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 7:29 PM, Michael Paquier wrote: > On Wed, Feb 15, 2017 at 9:27 PM, Michael Paquier > wrote: >> On Wed, Feb 15, 2017 at 7:58 PM, Heikki Linnakangas wrote: >>> On 02/09/2017 09:33 AM, Michael Paquier wrote: Now regarding the shape of the implementation for SCRAM, we ne

Re: [HACKERS] SUBSCRIPTIONS and pg_upgrade

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 7:34 AM, Stephen Frost wrote: > I'm not entirely sure about the reasoning behind requiring a flag to > include subscriptions in pg_dump output, as the documentation doesn't > actually provide one, but if we are going to require that, shouldn't > pg_upgrade use it, to make s

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 11:15 PM, Tom Lane wrote: > I wrote: >> However, you might be able to find it without so much random I/O. >> I'm envisioning a seqscan over the table, in which you simply look for >> HOT chains in which the indexed columns aren't all the same. When you >> find one, you'd h

Re: [HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 12:33 AM, Jim Nasby wrote: > On 2/15/17 1:37 PM, Ryan Murphy wrote: >> attcacheoff can only be set positive for fields preceding any varlena >> (typlen<0, but including the first such) or nullable values. I don't >> know how much faster it is with the cache; yo

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 4:52 AM, Tomas Vondra wrote: > I have my doubts about this actually addressing gitlab-like mistakes, > though, because it's a helluva jump from "It's waiting and not doing > anything," to "We need to remove the datadir." (One of the reasons being > that non-empty directory

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

2017-02-19 Thread Pavan Deolasee
On Sun, Feb 19, 2017 at 3:43 PM, Robert Haas wrote: > On Fri, Feb 17, 2017 at 11:15 PM, Tom Lane wrote: > > > Ah, nah, scratch that. If any post-index-build pruning had occurred on a > > page, the evidence would be gone --- the non-matching older tuples would > > be removed and what remained wo

Re: [HACKERS] Parallel Index-only scan

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 12:02 PM, Amit Kapila wrote: > + /* > +* If we are here to just update the scan keys, then don't > reset parallel > +* scan. For detailed reason behind this look in the comments for > +* ExecReScanIndexScan. > +*/ > > You can phrase the

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Magnus Hagander
On Sun, Feb 19, 2017 at 2:01 AM, Michael Paquier wrote: > On Sun, Feb 19, 2017 at 9:50 AM, Michael Paquier > wrote: > > I have been poking at it, and yeah... I missed the fact that > > pg_subcription is not a view. I thought that check_conninfo was being > > called in this context only.. > > Sti

Re: [HACKERS] [PATCH] Add pg_disable_checksums() and supporting infrastructure

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 2:28 AM, David Christensen wrote: > - Change "data_checksums" from a simple boolean to "data_checksum_state", an > enum type for all of > the potentially-required states for this feature (as well as enabling). Color me skeptical. I don't know what CHECKSUMS_ENABLING, C

Re: [HACKERS] [PATCH] Incremental sort (was: PoC: Partial sort)

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 4:01 PM, Alexander Korotkov wrote: > I decided to start new thread for this patch for following two reasons. > * It's renamed from "Partial sort" to "Incremental sort" per suggestion by > Robert Haas [1]. New name much better characterizes the essence of > algorithm. > *

Re: [HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 2:58 PM, jasonysli(李跃森) wrote: > Hi PG hackers: > > When using pg_stat_statements to collect running SQL of PG, we find > it is hard for our program to get exact operation type of the SQL, such as > SELECT, DELETE, UPDATE, INSERT, and so on. > >So we modify th

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 3:52 PM, Pavan Deolasee wrote: > This particular case of corruption results in a heap tuple getting indexed > by a wrong key (or to be precise, indexed by its old value). So the only way > to detect the corruption is to look at each index key and check if it > matches with

Re: [HACKERS] SCRAM authentication, take three

2017-02-19 Thread Michael Paquier
On Sun, Feb 19, 2017 at 6:55 PM, Robert Haas wrote: > Gosh, this SCRAM stuff seems to be taking us pretty deeply into > dealing with encoding details which apparently we haven't formerly > needed to worry about. That is a little surprising and maybe > something we should try to avoid? The RFC of

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Petr Jelinek
On 19/02/17 12:03, Magnus Hagander wrote: > > > On Sun, Feb 19, 2017 at 2:01 AM, Michael Paquier > mailto:michael.paqu...@gmail.com>> wrote: > > On Sun, Feb 19, 2017 at 9:50 AM, Michael Paquier > mailto:michael.paqu...@gmail.com>> wrote: > > I have been poking at it, and yeah... I mi

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 2:17 PM, Robert Haas wrote: > Such a change can be made, but as I pointed out in the part you didn't > quote, there are reasons to wonder whether that will be a constructive > change in real life even if it's better for the regression tests. > Optimizing PostgreSQL for the

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Magnus Hagander
On Sun, Feb 19, 2017 at 1:03 PM, Petr Jelinek wrote: > On 19/02/17 12:03, Magnus Hagander wrote: > > > > > > On Sun, Feb 19, 2017 at 2:01 AM, Michael Paquier > > mailto:michael.paqu...@gmail.com>> wrote: > > > > On Sun, Feb 19, 2017 at 9:50 AM, Michael Paquier > > mailto:michael.paqu...@g

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 5:54 PM, Robert Haas wrote: > However, it looks like there's a race condition here, because the slot > doesn't get freed up at the same time that the PID gets set to 0. > That actually happens later, when the postmaster calls > maybe_start_bgworker() or DetermineSleepTime()

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Michael Paquier
On Sun, Feb 19, 2017 at 8:03 PM, Magnus Hagander wrote: > If password auth is used, we have to store the password in plaintext > equivalent somewhere. Meaning it's by definition going to be exposed to > superusers and replication downstreams. Another possibility is to mention the use of the new p

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Magnus Hagander
On Sun, Feb 19, 2017 at 1:43 PM, Michael Paquier wrote: > On Sun, Feb 19, 2017 at 8:03 PM, Magnus Hagander > wrote: > > If password auth is used, we have to store the password in plaintext > > equivalent somewhere. Meaning it's by definition going to be exposed to > > superusers and replication

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 6:13 PM, Michael Paquier wrote: > On Sun, Feb 19, 2017 at 8:03 PM, Magnus Hagander wrote: >> If password auth is used, we have to store the password in plaintext >> equivalent somewhere. Meaning it's by definition going to be exposed to >> superusers and replication downst

Re: [HACKERS] SCRAM authentication, take three

2017-02-19 Thread Michael Paquier
On Wed, Feb 15, 2017 at 8:28 PM, Heikki Linnakangas wrote: > On 02/07/2017 04:20 AM, Michael Paquier wrote: >> --- a/src/backend/utils/errcodes.txt >> +++ b/src/backend/utils/errcodes.txt >> @@ -247,6 +247,7 @@ Section: Class 28 - Invalid Authorization >> Specification >> >> 28000EERRCODE

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Amit Kapila
On Sun, Feb 19, 2017 at 5:54 PM, Robert Haas wrote: > On Sun, Feb 19, 2017 at 2:17 PM, Robert Haas wrote: >> Such a change can be made, but as I pointed out in the part you didn't >> quote, there are reasons to wonder whether that will be a constructive >> change in real life even if it's better

Re: [HACKERS] Avoiding OOM in a hash join with many duplicate inner keys

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 8:13 PM, Thomas Munro wrote: > Obviously there are vanishing returns here as we add more defences > making it increasingly unlikely that we hit "fail" mode. But it > bothers me that hash joins in general are not 100% guaranteed to be > able to complete unless you have infi

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Dilip Kumar
On Sat, Feb 18, 2017 at 10:45 PM, Dilip Kumar wrote: > in 0002: > - Improved comments. > - Code refactoring in BitmapHeapNext. > - Removed local tbm creation in BitmapHeapNext : as per new tidbitmap > it's of no use. I have observed one problem with 0002 and I though of sharing that before fixing

Re: [HACKERS] Gather Merge

2017-02-19 Thread Amit Kapila
On Sun, Feb 19, 2017 at 2:22 PM, Robert Haas wrote: > On Sat, Feb 18, 2017 at 6:43 PM, Amit Kapila wrote: >> I think there is a value in supporting mark/restore position for any >> node which produces sorted results, however, if you don't want to >> support it, then I think we should update above

Re: [HACKERS] COPY IN/BOTH vs. extended query mode

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 1:58 PM, Corey Huinker wrote: > Forgive my ignorance, but is this issue related to the Catch-22 I had with > "COPY as a set returning function", wherein a function that invokes > BeginCopyFrom() basically starts a result set, but then ends it to do the > BeginCopyFrom() hav

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Amit Kapila
On Sun, Feb 19, 2017 at 7:18 PM, Dilip Kumar wrote: > On Sat, Feb 18, 2017 at 10:45 PM, Dilip Kumar wrote: >> in 0002: >> - Improved comments. >> - Code refactoring in BitmapHeapNext. >> - Removed local tbm creation in BitmapHeapNext : as per new tidbitmap >> it's of no use. > > I have observed o

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 7:18 PM, Dilip Kumar wrote: > I have observed one problem with 0002 and I though of sharing that > before fixing the same because we might have encountered the same > problem in some other patches i.e parallel shared hash and there might > be already a way to handle that. >

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-02-19 Thread Robins Tharakan
On 19 February 2017 at 17:02, Robins Tharakan wrote: > > On Sun, 19 Feb 2017 at 10:08 Stephen Frost wrote: > >> If anything, it should use pg_roles, not pg_user. >> >> I don't really like the "--avoid-pgauthid" option, but "--no-passwords" >> would probably work. >> >> > '--no-passwords' is a goo

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 6:50 PM, Amit Kapila wrote: > To close the remaining gap, don't you think we can check slot->in_use > flag when generation number for handle and slot are same. That doesn't completely fix it either, because ForgetBackgroundWorker() also does BackgroundWorkerData->parallel_

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > arapaima(x86) and aholehole(x86_64) are the new animals. They are running the > buildfarm script now. ... and failing. I wonder what is wrong with tcl_date_week()? Will look for the explanation in a bit. regards, tom lane --

Re: [HACKERS] Suppressing that pesky warning with older flex versions

2017-02-19 Thread Tom Lane
Robert Haas writes: > On Sat, Feb 18, 2017 at 10:40 PM, Tom Lane wrote: >> It seems like it would be quite simple and reliable to apply a patch >> that inserts "(void) yyg;" into this function. (Which, indeed, is >> essentially how flex 2.5.36 and later fixed it.) > Sounds fine as a master-only

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 11:25 AM, Ashutosh Bapat wrote: > Forgot to attach the patch. Thanks Rajkumar for notifying me. I think this is overexplaining what is anyway obvious. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Tom Lane
I wrote: > Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: >> arapaima(x86) and aholehole(x86_64) are the new animals. They are running the >> buildfarm script now. > ... and failing. I wonder what is wrong with tcl_date_week()? > Will look for the explanation in a bit. Relevant question: what versi

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-19 Thread Tom Lane
Robert Haas writes: > On Sun, Feb 19, 2017 at 3:31 AM, Tom Lane wrote: >> Thoughts? Should we double down on trying to make this work according >> to the "all integer timestamps" protocol specs, or cut our losses and >> change the specs? > I vote for doubling down. It's bad enough that we have

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 7:15 AM, Amit Langote wrote: >> I think 0001 is better than the status quo, but I'm wondering whether >> we should try to do something slightly different. Maybe it should >> always work for the child table to specify neither WITH OIDS nor >> WITHOUT OIDS, but if you do spe

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 12:43 PM, Amit Langote wrote: > On 2017/02/16 2:08, Robert Haas wrote: >> On Wed, Feb 15, 2017 at 11:34 AM, Alvaro Herrera >> wrote: >>> I think new-style partitioning is supposed to consider each partition as >>> an implementation detail of the table; the fact that you ca

Re: [HACKERS] Logical replication existing data copy - comments origin.c

2017-02-19 Thread Erik Rijkers
0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.patch 0001-Logical-replication-support-for-in

[HACKERS] tab completion for partitioning

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 7:15 AM, Amit Langote wrote: > Also attaching 0002 (unchanged) for tab-completion support for the new > partitioning syntax. At one point you have this: +/* Limited completion support for partition bound specification */ +else if (TailMatches3("ATTACH", "PARTITION

Re: [HACKERS] Logical replication existing data copy - comments origin.c

2017-02-19 Thread Erik Rijkers
On 2017-02-19 17:21, Erik Rijkers wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.pat

Re: [HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-19 Thread Ryan Murphy
> > You'll probably want to do those at a C level, bypassing the executor. I > > would guess that executor overhead will completely swamp the effect of > the > > cache in most cases. > > That seems like it's kind of missing the point. If the tupleDesc > cache saves so little that it's irrelevant w

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Dilip Kumar
On Sun, Feb 19, 2017 at 7:44 PM, Robert Haas wrote: > It's probably OK if tbm_free() doesn't free the memory allocated from > DSA, and we just let cleanup at end of query do it. However, that > could cause some trouble if the Parallel Bitmap Heap Scan gets > executed over and over and keeps alloc

Re: [HACKERS] Passing query string to workers

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 6:41 PM, Kuntal Ghosh wrote: > On Thu, Feb 16, 2017 at 5:47 PM, Rafia Sabih > wrote: >> Other that that I updated some comments and other cleanup things. Please >> find the attached patch for the revised version. > Looks good. > > It has passed the regression tests (with a

Re: [HACKERS] New CORRESPONDING clause design

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 8:28 PM, Surafel Temsgen wrote: > Here is the implementation of the clause with the slight change, instead of > doing column mapping for each side of leaf Queries in planner I make the > projection nodes output to corresponding column lists only. > > This patch compiles and

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 11:16 PM, Joel Jacobson wrote: >> The short answer is that nobody can see a way to modify the identifier >> case-folding rules that isn't going to add more pain than it subtracts. >> And much of the added pain will be felt by people who aren't getting >> any benefit, who wi

Re: [HACKERS] [PATCH] Add pg_disable_checksums() and supporting infrastructure

2017-02-19 Thread David Christensen
> On Feb 19, 2017, at 5:05 AM, Robert Haas wrote: > > On Fri, Feb 17, 2017 at 2:28 AM, David Christensen wrote: >> - Change "data_checksums" from a simple boolean to "data_checksum_state", an >> enum type for all of >> the potentially-required states for this feature (as well as enabling). >

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 9:59 PM, Dilip Kumar wrote: > I can imagine it can get executed over and over if plan is something like > below. > > NestLoopJoin > -> SeqScan > -> Gather > -> Parallel Bitmap Heap Scan > > But in such case every time the Inner node of the NLJ will be > res

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 9:19 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Feb 19, 2017 at 3:31 AM, Tom Lane wrote: >>> Thoughts? Should we double down on trying to make this work according >>> to the "all integer timestamps" protocol specs, or cut our losses and >>> change the specs? >

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 12:46 PM, Kouhei Kaigai wrote: > The attached patch is revised one. > > Invocation of Exec(Foreign|Custom)ParallelFinish was moved to > ExecParallelRetrieveInstrumentation() not to walk on the plan- > state tree twice. > One (hypothetical) downside is, FDW/CSP can retrieve

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-02-19 Thread Fabien COELHO
-f filename The filename in the newer html appears much larger under chrome, seemingly because of the within a . Maybe a bug in chrome CSS interpretation, because CSS on code seems to indicate "font-size: 1.3em", but it seems to do 1.3**2 instead for "filename"... However it

Re: [HACKERS] COPY IN/BOTH vs. extended query mode

2017-02-19 Thread Corey Huinker
On Sun, Feb 19, 2017 at 9:04 AM, Robert Haas wrote: If you tried to write an SQL-callable function that internally started > and ended a copy from the client, then I think you would run into this > problem, and probably some others. > > That's it. I had a PoC patch submitted that allowed someone

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Devrim Gündüz
Hi Tom, On Sun, 2017-02-19 at 10:42 -0500, Tom Lane wrote: > Relevant question: what version of tcl is installed on those? 8.4.13 is installed. Regards, -- Devrim Gündüz EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > On Sun, 2017-02-19 at 10:42 -0500, Tom Lane wrote: >> Relevant question: what version of tcl is installed on those? > 8.4.13 is installed. Hmph. I can't see any relevant-looking source changes between 8.4.13 and 8.4.15, which I have laying about here

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Devrim Gündüz
Hi Tom, On Sun, 2017-02-19 at 13:48 -0500, Tom Lane wrote: > Hmph.  I can't see any relevant-looking source changes between 8.4.13 > and 8.4.15, which I have laying about here and which works fine. > I wonder if Red Hat is carrying some distro-specific patch that > breaks this case?  Just downlo

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-19 Thread Simon Riggs
On 16 February 2017 at 11:32, Simon Riggs wrote: > On 10 February 2017 at 06:19, Amit Langote > wrote: > >> the "right thing" here being that the >> command's code either throws an error or warning (in some cases) if the >> specified table is a partitioned table or ignores any partitioned tables

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Simon Riggs
On 15 February 2017 at 15:46, Robert Haas wrote: >> It leaves me asking what else is missing. > > There is certainly a lot of room for improvement here but I don't > understand your persistent negativity about what's been done thus far. > I think it's pretty clearly a huge step forward, and I thi

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > On Sun, 2017-02-19 at 13:48 -0500, Tom Lane wrote: >> Or conceivably it's timezone dependent? > FWIW, the timezone of the server is GMT+3, if that is what you are asking. Well, that test is checking which week-of-the-year a Sunday midnight is consider

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-02-19 Thread Stephen Frost
Robins, * Robins Tharakan (thara...@gmail.com) wrote: > On 19 February 2017 at 17:02, Robins Tharakan wrote: > > On Sun, 19 Feb 2017 at 10:08 Stephen Frost wrote: > >> If anything, it should use pg_roles, not pg_user. > >> > >> I don't really like the "--avoid-pgauthid" option, but "--no-passwor

[HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread husttripper
Hi PG hackers: When using pg_stat_statements to collect running SQL of PG, we find it is hard for our program to get exact operation type of the SQL, such as SELECT, DELETE, UPDATE, INSERT, and so on. So we modify the the source code of pg_stat_statements and add another output param

[HACKERS] Deprecate floating point timestamps.

2017-02-19 Thread Jim Nasby
Over in the "Keeping pg_recvlogical's "feTimestamp" separate from TimestampTz"... On 2/17/17 12:15 PM, Tom Lane wrote: > I am not sure that it was really a good design to pretend that the > replication protocol is independent of --disable-integer-datetimes > when the underlying WAL stream most c

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Feb 19, 2017 at 6:13 PM, Michael Paquier > wrote: > > On Sun, Feb 19, 2017 at 8:03 PM, Magnus Hagander > > wrote: > >> If password auth is used, we have to store the password in plaintext > >> equivalent somewhere. Meaning it's by definition

Re: [HACKERS] Logical replication existing data copy - comments snapbuild.c

2017-02-19 Thread Erik Rijkers
0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.patch 0001-Logical-replication-support-for-in

Re: [HACKERS] Logical replication existing data copy - comments snapbuild.c

2017-02-19 Thread Erik Rijkers
On 2017-02-19 23:24, Erik Rijkers wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.pat

Re: [HACKERS] Reporting xmin from VACUUMs

2017-02-19 Thread Jim Nasby
On 2/19/17 3:43 AM, Robert Haas wrote: This is the kind of information that you really want to see once per autovac, though, not just the most recent autovac or some kind of cumulative total. Knowing that I've done 301 index scans in my last 300 vacuums is not nearly as useful as knowing which a

[HACKERS] Resolved typo in a comment

2017-02-19 Thread neha khatri
Hi, Attached is a patch that fixes a comment typo in varlena.c Thanks, Neha typo_correction.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-19 Thread Joel Jacobson
On Sun, Feb 19, 2017 at 5:58 PM, Robert Haas wrote: >> When case preservation by default is on, then simply enforce >> UNIQUE(LOWER(object_name)), to prevent ambiguity. > > That (1) breaks backward compatibility, because people might have > objects with names identical except for case in existing

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-02-19 Thread Gilles Darold
Le 16/02/2017 à 16:13, Robert Haas a écrit : > On Wed, Feb 15, 2017 at 4:03 PM, Alvaro Herrera > wrote: >> So what is going on here is that SysLogger_Start() wants to unlink the >> current-logfile file if the collector is not enabled. This should >> probably be split out into a separate new funct

Re: [HACKERS] pg_get_object_address() doesn't support composites

2017-02-19 Thread Jim Nasby
On 2/18/17 4:26 PM, Jim Nasby wrote: On 2/17/17 9:53 PM, Alvaro Herrera wrote: Jim Nasby wrote: See below. ISTM that pg_get_object_address should support everything pg_identify_object_as_address can output, no? I'm guessing the answer here is to have pg_identify_object_as_address complain if y

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Thomas Munro
On Sun, Feb 19, 2017 at 10:34 PM, Robert Haas wrote: > On Sun, Feb 19, 2017 at 9:59 PM, Dilip Kumar wrote: >> I can imagine it can get executed over and over if plan is something like >> below. >> >> NestLoopJoin >> -> SeqScan >> -> Gather >> -> Parallel Bitmap Heap Scan >> >> Bu

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-02-19 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Monday, February 20, 2017 2:20 AM > To: Kaigai Kouhei(海外 浩平) > Cc: Claudio Freire ; Amit Kapila > ; pgsql-hackers > Subject: Re: ParallelFinish-ho

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-19 Thread Masahiko Sawada
On Fri, Feb 17, 2017 at 3:41 AM, Robert Haas wrote: > On Thu, Feb 16, 2017 at 6:17 AM, Simon Riggs wrote: >> On 15 February 2017 at 08:07, Masahiko Sawada wrote: >>> It's a bug. Attached latest version patch, which passed make check. >> >> In its current form, I'm not sure this is a good idea. P

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-19 Thread Jim Nasby
On 2/19/17 4:51 PM, Joel Jacobson wrote: But once you've already decided to have a hard-and-fast rule that the names must be unique after lower-casing, there's no obvious benefit to rejecting queries that mention the same name with different case. Exactly, that trade-off is necessary, otherwise

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Tom Lane
Thomas Munro writes: > One practical problem that came up was the need for executor nodes to > get a chance to do that kind of cleanup before the DSM segment is > detached. In my patch series I introduced a new node API > ExecNodeDetach to allow for that. Andres objected that the need for > that

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-19 Thread Tom Lane
Joel Jacobson writes: > I think a good general philosophy for the PostgreSQL project would be to > try to look at how to meed the needs for new users of new projects > in a way that don't impair things for existing users, Yeah, exactly, and the problem here is that claiming that something like th

Re: [HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of > husttrip...@vip.sina.com > When using pg_stat_statements to collect running SQL of PG, we > find it is hard for our program to get exact operation type of the SQL, > such as SELECT, DELETE

Re: [HACKERS] [PATCH] Add pg_disable_checksums() and supporting infrastructure

2017-02-19 Thread Jim Nasby
On 2/19/17 11:02 AM, David Christensen wrote: My design notes for the patch were submitted to the list with little comment; see: https://www.postgresql.org/message-id/1E6E64E9-634B-43F4-8AA2-CD85AD92D2F8%40endpoint.com I have since added the WAL logging of checksum states, however I’d be glad

[HACKERS] dropping partitioned tables without CASCADE

2017-02-19 Thread Amit Langote
Re-posting the patch I posted in a nearby thread [0]. On 2017/02/16 2:08, Robert Haas wrote: > On Wed, Feb 15, 2017 at 11:34 AM, Alvaro Herrera > wrote: >> I think new-style partitioning is supposed to consider each partition as >> an implementation detail of the table; the fact that you can mani

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Amit Langote
On 2017/02/20 1:04, Robert Haas wrote: > On Thu, Feb 16, 2017 at 12:43 PM, Amit Langote wrote: >> So I count more than a few votes saying that we should be able to DROP >> partitioned tables without specifying CASCADE. >> >> I tried to implement that using the attached patch by having >> StoreCatal

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Amit Kapila
On Sun, Feb 19, 2017 at 8:32 PM, Robert Haas wrote: > On Sun, Feb 19, 2017 at 6:50 PM, Amit Kapila wrote: >> To close the remaining gap, don't you think we can check slot->in_use >> flag when generation number for handle and slot are same. > > That doesn't completely fix it either, because > Forg

Re: [HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread Jim Nasby
On 2/19/17 6:34 PM, Tsunakawa, Takayuki wrote: We have done the job and are willing to post a patch. I sent one through my work mail, but it seems that my mail didn't reach the maillist, so I try again by using my personal mail account. A view for counting the number of executions per operation

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-19 Thread Amit Langote
On 2017/02/20 5:31, Simon Riggs wrote: > On 16 February 2017 at 11:32, Simon Riggs wrote: >> On 10 February 2017 at 06:19, Amit Langote >> wrote: >> >>> the "right thing" here being that the >>> command's code either throws an error or warning (in some cases) if the >>> specified table is a part

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-19 Thread Jim Nasby
On 2/19/17 7:56 PM, Masahiko Sawada wrote: The half-dead pages are never cleaned up if the ratio of pages containing garbage is always lower than threshold. Also in gin index the pending list is never cleared, which become big problem. I guess that we should take action for each type of indexes.

Re: [HACKERS] tab completion for partitioning

2017-02-19 Thread Amit Langote
On 2017/02/20 1:22, Robert Haas wrote: > On Thu, Feb 16, 2017 at 7:15 AM, Amit Langote > wrote: >> Also attaching 0002 (unchanged) for tab-completion support for the new >> partitioning syntax. > > At one point you have this: > > +/* Limited completion support for partition bound specificati

Re: [HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread Tom Lane
Jim Nasby writes: > Something that needs to be considered with doing this in > pg_stat_statement is that a query that's reported there can contain > multiple SQL statements. I don't remember offhand if all statements get > parsed as a whole before anything else happens; if that's the case th

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-02-19 Thread Amit Langote
Hi Stephen, On 2017/02/17 22:32, Stephen Frost wrote: > Amit, > > * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: >> In certain cases, pg_dump's dumpTableSchema() emits a separate ALTER TABLE >> command for those schema elements of a table that could not be included >> directly in the CREAT

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Devrim Gündüz
Hi, On Sun, 2017-02-19 at 16:20 -0500, Tom Lane wrote: > Well, that test is checking which week-of-the-year a Sunday midnight is > considered to fall into.  There could be an edge-case bug in Tcl itself, > or a problem with the time zone data, or maybe if you're setting LC_TIME > to tr_TR, that c

  1   2   >