Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-03-07 Thread Andres Freund
Hi, 0001: Do hash join work_mem accounting in chunks. Don't think there's much left to say. 0002: Check hash join work_mem usage at the point of chunk allocation. Modify the existing hash join code to detect work_mem exhaustion at the point where chunks are allocated, instead of checking after

Re: [HACKERS] Bizarre choice of case for RELKIND_PARTITIONED_TABLE

2017-03-07 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 7, 2017 at 6:03 PM, Tom Lane wrote: >> Hm, one would hope that the vast majority of code references are neither >> of those, but rather "RELKIND_PARTITIONED_TABLE". > For reasons which must've seemed good to whoever instituted the > policy, pg_dump refers to rel

Re: [HACKERS] Statement-level rollback

2017-03-07 Thread legrand legrand
There was a mistake in my driver definition, this works fine with autosave=always (but not with autoSave ...) Thanks Again De : Vladimir Sitnikov [via PostgreSQL] Envoyé : mardi 7 mars 2017 22:32:27 À : legrand legrand Objet : Re: Statement-level rollback Ple

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-07 Thread Andres Freund
Hi, On 2017-03-07 18:46:31 -0500, Peter Eisentraut wrote: > I looked over > 0001-Add-expression-dependencies-on-composite-type-whole-.patch. That > seems useful independent of the things you have following. > > Even though it is presented more as a preparatory change, it appears to > have notice

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-03-07 Thread Tom Lane
Andres Freund writes: > +++ b/src/include/storage/barrier.h > +#include "postgres.h" > Huh, that normally shouldn't be in a header. I see you introduced that > in a bunch of other places too - that really doesn't look right to me. That is absolutely not project style and is not acceptable. The

Re: [HACKERS] delta relations in AFTER triggers

2017-03-07 Thread Kevin Grittner
On Mon, Feb 20, 2017 at 7:44 PM, Thomas Munro wrote: > Was it intentional that this test doesn't include any statements that > reach the case where the trigger does RAISE EXCEPTION 'RI error'? > From the output generated there doesn't seem to be any evidence that > the triggers run at all, though

Re: [HACKERS] Avoiding OOM in a hash join with many duplicate inner keys

2017-03-07 Thread Tom Lane
Thomas Munro writes: > I have been wondering about a couple of different worst case execution > strategies that would be better than throwing our hands up and > potentially exploding memory once we detect that further partitioning > is not going to help, if we still manage to reach that case despi

Re: [HACKERS] Example Custom Scan Provider Implementation?

2017-03-07 Thread Amit Langote
On 2017/03/08 7:10, Eric Ridge wrote: > Hi all! > > Does anyone know of a simple, example Custom Scan Provider implementation > for 9.6+? > > I found pg_strom by searching GitHub. Its gpuscan.c looks like maybe it > implements a pattern similar to what I want to do, but there's a lot of > extran

Re: [HACKERS] WARNING: relcache reference leak: relation "p1" not closed

2017-03-07 Thread Amit Langote
On 2017/03/08 1:34, Robert Haas wrote: > On Tue, Mar 7, 2017 at 1:43 AM, Amit Langote wrote: >> On 2017/03/07 14:04, Tom Lane wrote: >>> Therefore, there should definitely be a partitioned table, hopefully with >>> a less generic name than "p1", in the final regression DB state. Whether >>> this p

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-07 Thread Craig Ringer
On 8 March 2017 at 00:52, Daniel Verite wrote: > Vaishnavi Prabakaran wrote: > >> Yes, I have created a new patch entry into the commitfest 2017-03 and >> attached the latest patch with this e-mail. > > Please find attached a companion patch implementing the batch API in > pgbench, exposed

Re: [HACKERS] Patch to improve performance of replay of AccessExclusiveLock

2017-03-07 Thread David Rowley
On 8 March 2017 at 01:13, Simon Riggs wrote: > Don't understand this. I'm talking about setting a flag on > commit/abort WAL records, like the attached. There's nothing setting a flag in the attached. I only see the checking of the flag. > We just need to track info so we can set the flag at EOX

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Robert Haas
On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: > Right, that's what I thought he was getting at and my general thinking > was that we would need a way to discover if a CIC is ongoing on the > relation and therefore heap_page_prune(), or anything else, would know > that it can't twiddle the b

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-07 Thread Vaishnavi Prabakaran
On Wed, Mar 8, 2017 at 3:52 AM, Daniel Verite wrote: > Vaishnavi Prabakaran wrote: > > > Yes, I have created a new patch entry into the commitfest 2017-03 and > > attached the latest patch with this e-mail. > > Please find attached a companion patch implementing the batch API in > pgbench

Re: [HACKERS] foreign partition DDL regression tests

2017-03-07 Thread Robert Haas
On Tue, Mar 7, 2017 at 7:14 AM, Ashutosh Bapat wrote: > Hi Amit, > Thanks for adding testcases. Overall the testcases look good. > > The testcase is using ALTER TABLE to modify foreign table schema. > Though this works, I think a better option is to use ALTER FOREIGN > TABLE. > > Something not rel

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-07 Thread Amit Kapila
On Wed, Mar 8, 2017 at 3:38 AM, Robert Haas wrote: > On Wed, Mar 1, 2017 at 4:18 AM, Robert Haas wrote: >> On Tue, Feb 28, 2017 at 7:31 PM, Amit Kapila wrote: >>> Yeah, actually those were added later in Enable-WAL-for-Hash* patch, >>> but I think as this patch is standalone, so we should not re

[HACKERS] CREATE/ALTER ROLE PASSWORD ('value' USING 'method')

2017-03-07 Thread Michael Paquier
Hi all, As discussed on the thread dedicated to SCRAM (https://www.postgresql.org/message-id/243d8c11-6149-a4bb-0909-136992f74...@iki.fi), here is a separate thread dedicated to the following extension for CREATE/ALTER ROLE: PASSWORD ('value' USING 'method'). Now that password_encryption has been

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Andres Freund
On 2017-03-07 21:03:53 -0500, Robert Haas wrote: > On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: > > Right, that's what I thought he was getting at and my general thinking > > was that we would need a way to discover if a CIC is ongoing on the > > relation and therefore heap_page_prune(), o

Re: [HACKERS] PATCH: two slab-like memory allocators

2017-03-07 Thread David Rowley
On 28 February 2017 at 01:02, Andres Freund wrote: > Hi, > > On 2017-02-27 03:17:32 -0800, Andres Freund wrote: >> I'll work on getting slab committed first, and then review / edit / >> commit generation.c later. One first note there is that I'm wondering >> if generation.c is a too generic filen

Re: [HACKERS] Parallel seq. plan is not coming against inheritance or partition table

2017-03-07 Thread Amit Kapila
On Tue, Mar 7, 2017 at 10:12 PM, Robert Haas wrote: > On Mon, Mar 6, 2017 at 10:28 PM, Amit Kapila wrote: >> I also think that commit didn't intend to change the behavior, >> however, the point is how sensible is it to keep such behavior after >> Parallel Append. I am not sure if this is the rig

Re: [HACKERS] Enabling replication connections by default in pg_hba.conf

2017-03-07 Thread Peter Eisentraut
On 3/6/17 21:11, Michael Paquier wrote: > I think that the documentation of initdb should mention that > pg_hba.conf entries are configured for replication connections as > well, something like a sentence in the Description paragraph: > initdb sets pg_hba.conf entries using the specified authentica

Re: [HACKERS] wait events for disk I/O

2017-03-07 Thread Amit Kapila
On Tue, Mar 7, 2017 at 9:16 PM, Robert Haas wrote: > On Mon, Mar 6, 2017 at 9:09 PM, Amit Kapila wrote: >> Sure, if you think both Writes and Reads at OS level can have some >> chance of blocking in obscure cases, then we should add a wait event >> for them. > > I think writes have a chance of bl

Re: [HACKERS] adding an immutable variant of to_date

2017-03-07 Thread Robert Haas
On Tue, Mar 7, 2017 at 3:56 PM, Sven R. Kunze wrote: >> 2) As far as I can tell from reading the code to_date currently ignores >> the M suffix which indicates that you want localized month/day names, so i >> guess that to_date is currently immutable. Maybe it is stable due to the >> idea that we

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Robert Haas
On Tue, Mar 7, 2017 at 9:12 PM, Andres Freund wrote: > On 2017-03-07 21:03:53 -0500, Robert Haas wrote: >> On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: >> > Right, that's what I thought he was getting at and my general thinking >> > was that we would need a way to discover if a CIC is ong

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Pavan Deolasee
On Wed, Mar 8, 2017 at 7:33 AM, Robert Haas wrote: > On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: > > Right, that's what I thought he was getting at and my general thinking > > was that we would need a way to discover if a CIC is ongoing on the > > relation and therefore heap_page_prune(

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2017-03-07 Thread Robert Haas
On Sun, Mar 5, 2017 at 7:13 PM, Andreas Karlsson wrote: > And I would argue that his feature is useful for quite many, based on my > experience running a semi-large database. Index bloat happens and without > REINDEX CONCURRENTLY it can be really annoying to solve, especially for > primary keys. C

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Pavan Deolasee
On Wed, Mar 8, 2017 at 8:08 AM, Robert Haas wrote: > On Tue, Mar 7, 2017 at 9:12 PM, Andres Freund wrote: > > > > > I wonder however, if careful snapshot managment couldn't solve this as > > well. I have *not* thought a lot about this, but afaics we can easily > > prevent all-visible from being

Re: [HACKERS] wait events for disk I/O

2017-03-07 Thread Robert Haas
On Tue, Mar 7, 2017 at 9:32 PM, Amit Kapila wrote: > On Tue, Mar 7, 2017 at 9:16 PM, Robert Haas wrote: >> On Mon, Mar 6, 2017 at 9:09 PM, Amit Kapila wrote: >>> Sure, if you think both Writes and Reads at OS level can have some >>> chance of blocking in obscure cases, then we should add a wait

Re: [HACKERS] Parallel seq. plan is not coming against inheritance or partition table

2017-03-07 Thread Robert Haas
On Tue, Mar 7, 2017 at 9:24 PM, Amit Kapila wrote: > If we have lesser index pages and more heap pages, then we limit the > parallelism based on index pages. Kinda. In most cases, we figure out the degree of parallelism based on heap pages and then we figure out the degree of parallelism based o

[HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-03-07 Thread Michael Paquier
Hi all, Here is another thread for an issue related to SCRAM (https://www.postgresql.org/message-id/243d8c11-6149-a4bb-0909-136992f74...@iki.fi), that can be discussed independently: SASLprep. RFC5802 (https://tools.ietf.org/html/rfc5802) regarding the implementation of SCRAM, needs to have passw

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-07 Thread Andres Freund
On 2017-03-07 21:38:40 -0500, Robert Haas wrote: > > I wonder however, if careful snapshot managment couldn't solve this as > > well. I have *not* thought a lot about this, but afaics we can easily > > prevent all-visible from being set in cases it'd bother us by having an > > "appropriate" xmin /

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2017-03-07 Thread Andres Freund
On 2017-03-07 21:48:23 -0500, Robert Haas wrote: > On Sun, Mar 5, 2017 at 7:13 PM, Andreas Karlsson wrote: > > And I would argue that his feature is useful for quite many, based on my > > experience running a semi-large database. Index bloat happens and without > > REINDEX CONCURRENTLY it can be r

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-03-07 Thread David Steele
On 3/3/17 4:54 PM, David Steele wrote: On 2/1/17 1:25 AM, Kyotaro HORIGUCHI wrote: Hello, thank you for moving this to the next CF. At Wed, 1 Feb 2017 13:09:51 +0900, Michael Paquier wrote in On Tue, Jan 24, 2017 at 4:58 PM, Kyotaro HORIGUCHI wrote: Six new syscaches in 665d1fa was confl

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

2017-03-07 Thread Haribabu Kommi
On Wed, Mar 1, 2017 at 12:59 PM, Haribabu Kommi wrote: > > Patch attached. Still some more docs needs to be added. > Updated patch attached to resolve the conflicts with following commit. commit 9a83d56b38c870ce47b7651385ff2add583bf136 Author: Simon Riggs Date: Tue Mar 7 22:00:54 2017 +0800

Re: [HACKERS] SCRAM authentication, take three

2017-03-07 Thread Magnus Hagander
On Tue, Mar 7, 2017 at 4:36 AM, Heikki Linnakangas wrote: > On 03/02/2017 08:50 AM, Michael Paquier wrote: > >> Attached is a new patch set. I have combined SASLprep with the rest >> and fixed some conflicts. At the same time when going through NFKC >> this morning I have noticed that the impleme

Re: [HACKERS] New SQL counter statistics view (pg_stat_sql)

2017-03-07 Thread Haribabu Kommi
On Wed, Feb 1, 2017 at 3:13 PM, Michael Paquier wrote: > On Fri, Jan 27, 2017 at 10:26 AM, Haribabu Kommi > wrote: > > Thanks for the review. > > Let's wait for the committer's opinion. > > I have moved this patch to CF 2017-03 to wait for this to happen. > Attached a rebased patch to resolve t

Re: [HACKERS] ALTER PUBLICATION and segmentation fault

2017-03-07 Thread Peter Eisentraut
On 3/7/17 11:56, Masahiko Sawada wrote: > This issue happen even ALTER SUBSCRIPTION. I guess the main cause is > that acl_kind of pg_publication and pg_subscription of ObjectProperty > array are not correct. These values should be set ACL_KIND_PUBLICATION > and ACL_KIND_SUBSCRIPTION instead of -1.

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-07 Thread Magnus Hagander
On Tue, Mar 7, 2017 at 7:24 AM, Andrew Dunstan < andrew.duns...@2ndquadrant.com> wrote: > > > On 03/07/2017 07:58 AM, Simon Riggs wrote: > > On 7 March 2017 at 20:36, Alvaro Herrera > wrote: > > > >> FWIW, +1 on improving matters here. > > +1 also. > > > > I don't see what's wrong with relying on

Re: [HACKERS] Bizarre choice of case for RELKIND_PARTITIONED_TABLE

2017-03-07 Thread Amit Langote
On 2017/03/08 2:55, Tom Lane wrote: > Is there a good reason why RELKIND_PARTITIONED_TABLE is 'P' not 'p'? > It looks rather out of place considering that seven of the eight > pre-existing relkind codes are lower case. (And no, I don't especially > approve of RELKIND_SEQUENCE being 'S' either, but

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-03-07 Thread David Steele
On 3/7/17 11:38 AM, Andres Freund wrote: <...> We have a plenty of time and we dedicate one full-time developer for this project. How about having that, and perhaps others, developer participate in reviewing patches and getting to the bottom of the commitfest? Should we end up being done ear

Re: [HACKERS] foreign partition DDL regression tests

2017-03-07 Thread Ashutosh Bapat
On Wed, Mar 8, 2017 at 7:36 AM, Robert Haas wrote: > On Tue, Mar 7, 2017 at 7:14 AM, Ashutosh Bapat > wrote: >> Hi Amit, >> Thanks for adding testcases. Overall the testcases look good. >> >> The testcase is using ALTER TABLE to modify foreign table schema. >> Though this works, I think a better

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-07 Thread Andres Freund
On 2017-03-07 20:58:35 +0800, Simon Riggs wrote: > On 7 March 2017 at 20:36, Alvaro Herrera wrote: > > > FWIW, +1 on improving matters here. > > +1 also. > > I don't see what's wrong with relying on buildfarm though; testing is > exactly what its there for. > > If we had a two-stage process, w

Re: [HACKERS] CREATE/ALTER ROLE PASSWORD ('value' USING 'method')

2017-03-07 Thread Tom Lane
Michael Paquier writes: > here is a separate thread dedicated to the following extension for > CREATE/ALTER ROLE: PASSWORD ('value' USING 'method'). The parentheses seem weird ... do we really need those? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] CREATE/ALTER ROLE PASSWORD ('value' USING 'method')

2017-03-07 Thread Michael Paquier
On Wed, Mar 8, 2017 at 1:29 PM, Tom Lane wrote: > Michael Paquier writes: >> here is a separate thread dedicated to the following extension for >> CREATE/ALTER ROLE: PASSWORD ('value' USING 'method'). > > The parentheses seem weird ... do we really need those? A grammar without parenthesis works

Re: [HACKERS] some dblink refactoring

2017-03-07 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Peter Eisentraut > Here is a patch to refactor some macro hell in dblink. > > This patch was discussed in the background sessions thread as a prerequisite > for some work there, but I figure I'll ma

Re: [HACKERS] some dblink refactoring

2017-03-07 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tsunakawa, > Takayuki > How about applying the attached small patch for another refactoring? This > merely changes makeStringInfo() to initStringInfo() at two sites just other > places in the same f

Re: [HACKERS] CREATE/ALTER ROLE PASSWORD ('value' USING 'method')

2017-03-07 Thread Rushabh Lathia
On Wed, Mar 8, 2017 at 9:59 AM, Tom Lane wrote: > Michael Paquier writes: > > here is a separate thread dedicated to the following extension for > > CREATE/ALTER ROLE: PASSWORD ('value' USING 'method'). > > The parentheses seem weird ... do we really need those? > +1 I had quick glance to patc

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-03-07 Thread Amit Langote
Hi Ashutosh, On 2017/03/06 18:19, Ashutosh Bapat wrote: > On Mon, Mar 6, 2017 at 11:12 AM, Ashutosh Bapat wrote: >> On Mon, Mar 6, 2017 at 11:05 AM, Simon Riggs wrote: >>> "has partitions" is not part of the DDL, whereas "FOR VALUES FROM (0) >>> TO (100)" is. So ISTM sensible to differentiate bet

Re: [HACKERS] Reporting planning time with EXPLAIN

2017-03-07 Thread Ashutosh Bapat
On Tue, Mar 7, 2017 at 9:23 PM, Stephen Frost wrote: > Ashutosh, > > * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: >> Here are patches for follwing >> 1. pg_explain_plan_time_v3 adds SUMMARY option which behaves as: >> SUMMARY when ON prints planning time. With ANALYZE ON, it also prin

Re: [HACKERS] Parallel bitmap heap scan

2017-03-07 Thread Dilip Kumar
On Tue, Mar 7, 2017 at 10:10 PM, Dilip Kumar wrote: >> It's not about speed. It's about not forgetting to prefetch. Suppose >> that worker 1 becomes the prefetch worker but then doesn't return to >> the Bitmap Heap Scan node for a long time because it's busy in some >> other part of the plan tre

Re: [HACKERS] CREATE/ALTER ROLE PASSWORD ('value' USING 'method')

2017-03-07 Thread Michael Paquier
On Wed, Mar 8, 2017 at 2:32 PM, Rushabh Lathia wrote: > One quick comments: > > +| PASSWORD '(' Sconst USING Sconst ')' > +{ > +$$ = makeDefElem("methodPassword", > + (Node *)list_make2(makeString($3), > +

[HACKERS] [bug fix] dblink leaks unnamed connections

2017-03-07 Thread Tsunakawa, Takayuki
Hello, dblink fails to close the unnamed connection as follows when a new unnamed connection is requested. The attached patch fixes this. postgres=# select count(*) from pg_stat_activity; count --- 1 (1 row) postgres=# select dblink_connect('dbname=postgres'); dblink_connect -

Re: [HACKERS] Parallel Append implementation

2017-03-07 Thread Amit Khandekar
On 19 February 2017 at 14:59, Robert Haas wrote: > On Fri, Feb 17, 2017 at 2:56 PM, Amit Khandekar > wrote: >> The log2(num_children)+1 formula which you proposed does not take into >> account the number of workers for each of the subplans, that's why I >> am a bit more inclined to look for some

Re: [HACKERS] parallel index(-only) scan breaks when run without parallelism

2017-03-07 Thread Amit Kapila
On Wed, Mar 8, 2017 at 12:27 AM, Robert Haas wrote: > Amit, Rafia, > > nodeIndexscan.c, unlike nodeSeqscan.c, thinks that a parallel-aware > scan will always be executed in parallel mode. But that's not true: > an Execute message with a non-zero row count could cause us to abandon > planned paral

Re: [HACKERS] Enabling replication connections by default in pg_hba.conf

2017-03-07 Thread Michael Paquier
On Wed, Mar 8, 2017 at 11:29 AM, Peter Eisentraut wrote: > On 3/6/17 21:11, Michael Paquier wrote: >> I think that the documentation of initdb should mention that >> pg_hba.conf entries are configured for replication connections as >> well, something like a sentence in the Description paragraph: >

Re: [HACKERS] contrib modules and relkind check

2017-03-07 Thread Michael Paquier
On Tue, Mar 7, 2017 at 4:15 PM, Amit Langote wrote: > Sorry about the absence on this thread. No problems! Thanks for showing up with an updated patch. > On 2017/02/14 15:30, Michael Paquier wrote: >> On Tue, Feb 14, 2017 at 3:18 PM, Amit Langote wrote: >>> >>> Added more tests in pgstattuple an

<    1   2