Re: [HACKERS] GSoC 2017 : Patch for predicate locking in Gist index

2017-06-21 Thread Heikki Linnakangas
On 06/16/2017 01:24 PM, Shubham Barai wrote: @@ -497,6 +499,13 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (ptr = dist->next; ptr; ptr = ptr->next) UnlockReleaseBuffer(ptr->buffer); } + +

Re: [HACKERS] GSoC 2017 : Patch for predicate locking in Gist index

2017-06-21 Thread Heikki Linnakangas
On 06/21/2017 10:41 AM, Heikki Linnakangas wrote: On 06/16/2017 01:24 PM, Shubham Barai wrote: @@ -497,6 +499,13 @@ gistplacetopage(Relation rel, Size freespace, GISTSTATE *giststate, for (ptr = dist->next; ptr; ptr = ptr->next) UnlockRele

Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

2017-06-21 Thread Mahendranath Gurram
Hi Thomas, I like the whole idea. In fact, i understood this in your very first response itself. Only thing is every time i have to check for dsa_attached or not. I mean get_my_shared_state() is NULL or not. To avoid that check, i tried creating it in _PG_Init(postmaster process itself) a

Re: [HACKERS] GSoC 2017 : Patch for predicate locking in Gist index

2017-06-21 Thread Shubham Barai
Hi, On 21 June 2017 at 13:11, Heikki Linnakangas wrote: > On 06/16/2017 01:24 PM, Shubham Barai wrote: > >> @@ -497,6 +499,13 @@ gistplacetopage(Relation rel, Size freespace, >> GISTSTATE *giststate, >> for (ptr = dist->next; ptr; ptr = ptr->next) >>

Re: [HACKERS] ASOF join

2017-06-21 Thread Thomas Munro
On Mon, Jun 19, 2017 at 11:57 PM, Konstantin Knizhnik wrote: > I attached simple patch adding ASOF join to Postgres. Right now it support > only outer join and requires USING clause (consequently it is not possible > to join two tables which joi keys has different names. May be it is also > possib

[HACKERS] Useless code in ExecInitModifyTable

2017-06-21 Thread Etsuro Fujita
Commit d3cc37f1d801a6b5cad9bf179274a8d767f1ee50 added this to ExecInitModifyTable: + /* The root table RT index is at the head of the partitioned_rels list */ + if (node->partitioned_rels) + { + Index root_rti; + Oid root_oid; + + root_rti = linitial_int(node->par

Re: [HACKERS] [GSOC][weekly report 3] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-06-21 Thread Mengxing Liu
> From: "Heikki Linnakangas" > > Hmm. The hash table ought to speed up the RWConflictExists() function > right? Where in the flame graph is RWConflictExists()? If it only > accounts for a small amount of the overall runtime, even drastic speedup > there won't make much difference to the total.

[HACKERS] Comment typo in execMain.c

2017-06-21 Thread Etsuro Fujita
Here is a patch to fix a typo in a comment in ExecWithCheckOptions(): s/as as/as/. Best regards, Etsuro Fujita diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c index 7f460bd..9dbe175 100644 --- a/src/backend/executor/execMain.c +++ b/src/backend/executor/execMain.c

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2017-06-21 Thread Heikki Linnakangas
(I'm cleaning up my inbox, hence the delayed reply) On 08/02/2016 10:51 PM, Robert Haas wrote: On Tue, Aug 2, 2016 at 2:33 PM, Bruce Momjian wrote: On Tue, Jul 26, 2016 at 05:42:43PM -0400, Chapman Flack wrote: Even so, I'd be curious whether it would break anything to have xlp_pageaddr simpl

Re: [HACKERS] Comment typo in execMain.c

2017-06-21 Thread Heikki Linnakangas
On 06/21/2017 11:35 AM, Etsuro Fujita wrote: Here is a patch to fix a typo in a comment in ExecWithCheckOptions(): s/as as/as/. Fixed, thanks! - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailp

Re: [HACKERS] Useless code in ExecInitModifyTable

2017-06-21 Thread Amit Langote
Fujita-san, On 2017/06/21 16:59, Etsuro Fujita wrote: > Commit d3cc37f1d801a6b5cad9bf179274a8d767f1ee50 added this to > ExecInitModifyTable: > > + /* The root table RT index is at the head of the partitioned_rels list */ > + if (node->partitioned_rels) > + { > + Index root_rti; > +

Re: [HACKERS] UPDATE of partition key

2017-06-21 Thread Amit Langote
On 2017/06/21 3:53, Robert Haas wrote: > On Tue, Jun 20, 2017 at 2:54 AM, Amit Khandekar > wrote: >>> I guess I don't see why it should work like this. In the INSERT case, >>> we must build withCheckOption objects for each partition because those >>> partitions don't appear in the plan otherwise

Re: [HACKERS] Logical decoding on standby

2017-06-21 Thread sanyam jain
Hi, After changing sendTimeLineIsHistoric = state->currTLI == ThisTimeLineID; to sendTimeLineIsHistoric = state->currTLI != ThisTimeLineID; I was facing another issue. On promotion of a cascaded server ThisTimeLineID in the standby server having logical slot becomes 0. Then i added a function cal

Re: [HACKERS] ASOF join

2017-06-21 Thread Konstantin Knizhnik
On 21.06.2017 11:00, Thomas Munro wrote: Hmm. Yeah, I see the notational problem. It's hard to come up with a new syntax that has SQL nature. What if... we didn't use a new syntax at all, but recognised existing queries that are executable with this strategy? Queries like this: WITH ticks(

Re: [HACKERS] Rules on table partitions

2017-06-21 Thread Dean Rasheed
On 20 June 2017 at 03:01, Amit Langote wrote: > Hmm, yes. The following exercise convinced me. > > create table r (a int) partition by range (a); > create table r1 partition of r for values from (1) to (10); > create rule "_RETURN" as on select to r1 do instead select * from r; > > insert into r

Re: [HACKERS] Proposal for CSN based snapshots

2017-06-21 Thread Alexander Kuzmenkov
On 21.06.2017 04:48, Michael Paquier wrote: There has not been much activity on this thread for some time, and I mentioned my intentions to some developers at the last PGCon. But I am planning to study more the work that has been done here, with as envisaged goal to present a patch for the first

[HACKERS] pg_terminate_backend can terminate background workers and autovacuum launchers

2017-06-21 Thread Yugo Nagata
Hi, I have found that we can cancel/terminate autovacuum launchers and background worker processes by pg_cancel/terminate_backend function. I'm wondering this behavior is not expected and if not I want to fix it. The current pg_stat_activity shows background workers and autovacuum lancher as bel

[HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Yugo Nagata
Hi, As I report in another thread[1], I found the autovacuum launcher occurs the following error in PG 10 when this received SIGINT. I can repuroduce this by pg_cancel_backend or `kill -2 `. 2017-06-21 13:56:07.010 JST [32483] ERROR: canceling statement due to user request 2017-06-21 13:56:08.0

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-21 Thread Andrew Dunstan
On 06/20/2017 08:30 PM, Tom Lane wrote: > Michael Paquier writes: >> On Wed, Jun 21, 2017 at 8:43 AM, Tom Lane wrote: >>> Yeah, I thought it would work fine with Makefile-using Windows toolchains. >>> But people who use MSVC need something else, no? >> Are there that many anyway who care? > Wel

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-21 Thread Jeevan Ladhe
Hi, Sorry for being away from here. I had some issues with my laptop, and I have resumed working on this. On Thu, Jun 15, 2017 at 1:21 AM, Robert Haas wrote: > On Wed, Jun 14, 2017 at 8:02 AM, Jeevan Ladhe > wrote: > > Here are the details of the patches in attached zip. > > 0001. refactoring

[HACKERS] Incorrect documentation about pg_stat_activity

2017-06-21 Thread Yugo Nagata
Hi, In the documentation[1], there is the following description: "pg_stat_activity does not show an entry for the Startup process" However, the current pg_stat_activity show startup process's entry. postgres=# select pid, backend_type from pg_stat_activity ; pid | backend_type ---

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-21 Thread Jeevan Ladhe
Hi Amit, On Thu, Jun 15, 2017 at 12:31 PM, Amit Langote < langote_amit...@lab.ntt.co.jp> wrote: > Oops, I meant to send one more comment. > > On 2017/06/15 15:48, Amit Langote wrote: > > BTW, I noticed the following in 0002 > +errmsg("there exists a default

Re: [HACKERS] Adding support for Default partition in partitioning

2017-06-21 Thread Jeevan Ladhe
Thanks Ashutosh and Kyotaro for reviewing further. I shall address your comments in next version of my patch. Regards, Jeevan Ladhe On Fri, Jun 16, 2017 at 1:46 PM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, I'd like to review this but it doesn't fit the master, as > Ro

Re: [HACKERS] An attempt to reduce WALWriteLock contention

2017-06-21 Thread jasrajd
We are also seeing contention on the walwritelock and repeated writes to the same offset if we move the flush outside the lock in the Azure environment. pgbench doesn't scale beyond ~8 cores without saturating the IOPs or bandwidth. Is there more work being done in this area? -- View this messag

Re: [HACKERS] Default Partition for Range

2017-06-21 Thread Dilip Kumar
On Tue, Jun 20, 2017 at 6:57 PM, Dilip Kumar wrote: > This is basically crashing in RelationBuildPartitionDesc, so I think > we don't have any test case for testing DEFAULT range partition where > partition key has more than one attribute. So I suggest we can add > such test case. Some more comm

[HACKERS] Logical replication: stuck spinlock at ReplicationSlotRelease

2017-06-21 Thread Albe Laurenz
While playing with HEAD as of d14c85ed, I ran into the following: CREATE DATABASE source; CREATE DATABASE recipient; \c source CREATE TABLE repli(id integer PRIMARY KEY, val text NOT NULL); INSERT INTO repli VALUES (1, 'one'); CREATE PUBLICATION repsend FOR TABLE repli; SELECT pg_create_logical_r

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Kuntal Ghosh
On Wed, Jun 21, 2017 at 5:45 PM, Yugo Nagata wrote: > Hi, > > As I report in another thread[1], I found the autovacuum launcher occurs > the following error in PG 10 when this received SIGINT. I can repuroduce > this by pg_cancel_backend or `kill -2 `. > > 2017-06-21 13:56:07.010 JST [32483] ERROR

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Dilip Kumar
On Wed, Jun 21, 2017 at 6:50 PM, Kuntal Ghosh wrote: > I think we can just check dsm_find_mapping() to check whether the dsm > handle is already attached. Something like, > > } > - else > + else if(!dsm_find_mapping(AutoVacuumShmem->av_dsa_handle)) > { >

Re: [HACKERS] Incorrect documentation about pg_stat_activity

2017-06-21 Thread Kuntal Ghosh
On Wed, Jun 21, 2017 at 6:05 PM, Yugo Nagata wrote: > > Attached is a patch for the documentation fix. > Please attach the patch as well. :-) -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-21 Thread Yugo Nagata
Hi, Here are some comments for the patch. +Datum +pg_cancel_backend(PG_FUNCTION_ARGS) +{ + PG_RETURN_BOOL(pg_cancel_backend_internal(PG_GETARG_INT32(0), NULL)); +} +Datum +pg_cancel_backend_msg(PG_FUNCTION_ARGS) +{ + pid_t pid = PG_GETARG_INT32(0); + char *msg = text_to_cstring(

Re: [HACKERS] Logical replication in the same cluster

2017-06-21 Thread Albe Laurenz
Tom Lane wrote: > Petr Jelinek writes: >> On 26/04/17 18:59, Bruce Momjian wrote: >>> ... it just hangs. My server logs say: > >> Yes that's result of how logical replication slots work, the transaction >> that needs to finish is your transaction. It can be worked around by >> creating the slot

Re: [HACKERS] Incorrect documentation about pg_stat_activity

2017-06-21 Thread Yugo Nagata
On Wed, 21 Jun 2017 19:08:35 +0530 Kuntal Ghosh wrote: > On Wed, Jun 21, 2017 at 6:05 PM, Yugo Nagata wrote: > > > > Attached is a patch for the documentation fix. > > > Please attach the patch as well. :-) I'm sorry, I forgot it. I attahed this. > > > -- > Thanks & Regards, > Kuntal Ghosh

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Kuntal Ghosh
On Wed, Jun 21, 2017 at 7:07 PM, Dilip Kumar wrote: > On Wed, Jun 21, 2017 at 6:50 PM, Kuntal Ghosh > wrote: >> I think we can just check dsm_find_mapping() to check whether the dsm >> handle is already attached. Something like, >> >> } >> - else >> + else if(!dsm_find_mapping

Re: [HACKERS] Incorrect documentation about pg_stat_activity

2017-06-21 Thread Simon Riggs
On 21 June 2017 at 16:15, Yugo Nagata wrote: > On Wed, 21 Jun 2017 19:08:35 +0530 > Kuntal Ghosh wrote: > >> On Wed, Jun 21, 2017 at 6:05 PM, Yugo Nagata wrote: >> > >> > Attached is a patch for the documentation fix. >> > >> Please attach the patch as well. :-) > > I'm sorry, I forgot it. I att

Re: [HACKERS] Broken hint bits (freeze)

2017-06-21 Thread Amit Kapila
On Tue, Jun 20, 2017 at 7:24 PM, Amit Kapila wrote: > On Tue, Jun 20, 2017 at 7:05 PM, Sergey Burladyan wrote: >> Amit Kapila writes: >> >>> On Tue, Jun 20, 2017 at 3:40 PM, Sergey Burladyan >>> wrote: >> I use pg 9.2 and "skipping restartpoint, already performed at" is from >> src/backend/acc

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Dilip Kumar
On Wed, Jun 21, 2017 at 7:47 PM, Kuntal Ghosh wrote: >> IMHO, It's not a good idea to use DSM call to verify the DSA handle. >> > Okay. Is there any particular scenario you've in mind where this may fail? It's not about failure, but about the abstraction. When we are using the DSA we should not

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-21 Thread Yugo Nagata
On Wed, 21 Jun 2017 12:06:33 +0900 Michael Paquier wrote: > On Tue, Jun 20, 2017 at 3:24 AM, Daniel Gustafsson wrote: > > The message is stored in a new shmem area which is checked when the session > > is > > aborted. To keep things simple a small buffer is kept per backend for the > > message

Re: [HACKERS] Useless code in ExecInitModifyTable

2017-06-21 Thread Tom Lane
Amit Langote writes: > On 2017/06/21 16:59, Etsuro Fujita wrote: >> but I noticed that that function doesn't use the relation descriptor at >> all. Since partitioned_rels is given in case of an UPDATE/DELETE on a >> partitioned table, the relation is opened in that case, but the relation >> descr

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-21 Thread Andrew Dunstan
On 06/21/2017 08:20 AM, Andrew Dunstan wrote: > > On 06/20/2017 08:30 PM, Tom Lane wrote: >> Michael Paquier writes: >>> On Wed, Jun 21, 2017 at 8:43 AM, Tom Lane wrote: Yeah, I thought it would work fine with Makefile-using Windows toolchains. But people who use MSVC need something e

Re: [HACKERS] Default Partition for Range

2017-06-21 Thread Robert Haas
On Wed, Jun 21, 2017 at 8:57 AM, Dilip Kumar wrote: > For the default partition we are only setting bound->content[0] to > default, but content for others key > attributes are not initialized. But later in the code, if the content > of the first key is RANGE_DATUM_DEFAULT then it should not acces

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-21 Thread Daniel Gustafsson
> On 21 Jun 2017, at 16:30, Yugo Nagata wrote: > > On Wed, 21 Jun 2017 12:06:33 +0900 > Michael Paquier wrote: > >> On Tue, Jun 20, 2017 at 3:24 AM, Daniel Gustafsson wrote: >>> The message is stored in a new shmem area which is checked when the session >>> is >>> aborted. To keep things sim

Re: [HACKERS] UPDATE of partition key

2017-06-21 Thread Robert Haas
On Wed, Jun 21, 2017 at 5:28 AM, Amit Langote wrote:>> The comment "UPDATE/DELETE cases are handled above" is referring to >> the code that initializes the WCOs generated by the planner. You've >> modified the comment in your patch, but the associated code: your >> updated comment says that only

Re: [HACKERS] Useless code in ExecInitModifyTable

2017-06-21 Thread Robert Haas
On Wed, Jun 21, 2017 at 10:33 AM, Tom Lane wrote: > Amit Langote writes: >> On 2017/06/21 16:59, Etsuro Fujita wrote: >>> but I noticed that that function doesn't use the relation descriptor at >>> all. Since partitioned_rels is given in case of an UPDATE/DELETE on a >>> partitioned table, the r

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-06-21 Thread Masahiko Sawada
On Fri, May 19, 2017 at 11:12 AM, Masahiko Sawada wrote: > On Wed, May 17, 2017 at 1:30 AM, Robert Haas wrote: >> On Sat, May 13, 2017 at 7:27 AM, Amit Kapila wrote: >>> On Fri, May 12, 2017 at 9:14 AM, Tom Lane wrote: Robert Haas writes: > On Wed, May 10, 2017 at 8:39 PM, Masahiko Sa

Re: [HACKERS] pg_terminate_backend can terminate background workers and autovacuum launchers

2017-06-21 Thread Robert Haas
On Wed, Jun 21, 2017 at 7:56 AM, Yugo Nagata wrote: > I have found that we can cancel/terminate autovacuum launchers and > background worker processes by pg_cancel/terminate_backend function. > I'm wondering this behavior is not expected and if not I want to fix it. I think it is expected. Even

Re: [HACKERS] pg_terminate_backend can terminate background workers and autovacuum launchers

2017-06-21 Thread Alvaro Herrera
Yugo Nagata wrote: > However, we can terminate background workers by pg_terminate_backend. > In the following example, I terminated the logical replication launcher, > and this process did not appear again[1]. > > postgres=# select pg_terminate_backend(30902); > pg_terminate_backend >

Re: [HACKERS] Shortened URLs for commit messages

2017-06-21 Thread Alvaro Herrera
Bruce Momjian wrote: > Oh, here is a fixed version that requires an @ sign, which all message > id's have: > > sed '/http/!s;^\(Discussion: *\)\(.*@.*\)$;\1https://postgr.es/m/\2;' So how do you actually use this? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL D

Re: [HACKERS] Default Partition for Range

2017-06-21 Thread Dilip Kumar
On Wed, Jun 21, 2017 at 8:08 PM, Robert Haas wrote: > I think somebody should do some testing of the existing code with > valgrind. And then apply the list-partitioning patch and this patch, > and do some more testing with valgrind. It seems to be really easy to > miss these uninitialized access

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-21 Thread Tom Lane
Andrew Dunstan writes: > Unfortunately this seems precluded by the use of the non-standard > "err.h". It looks like that will trip us up on mingw also. Hm, why? Doesn't the -I search path get the right thing? regards, tom lane -- Sent via pgsql-hackers mailing list (p

Re: [HACKERS] Useless code in ExecInitModifyTable

2017-06-21 Thread Tom Lane
Robert Haas writes: > It appears to me that create_modifytable_path() has a partitioned_rels > argument and it looks like inheritance_planner not only passes it but > guarantees that it's non-empty when the target is a partitioned table. Oh, somehow I missed the call in inheritance_planner. Righ

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-21 Thread Andres Freund
On 2017-06-20 20:30:34 -0400, Tom Lane wrote: > Michael Paquier writes: > > On Wed, Jun 21, 2017 at 8:43 AM, Tom Lane wrote: > >> Yeah, I thought it would work fine with Makefile-using Windows toolchains. > >> But people who use MSVC need something else, no? > > > Are there that many anyway who

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-21 Thread Tom Lane
Andres Freund writes: > On 2017-06-20 20:30:34 -0400, Tom Lane wrote: >> Well, *I* don't care, but I thought we wanted to support Windows-using >> developers as reasonably first-class citizens. > The old one didn't have windows support either, right? Not that I was aware of, but a large part of

Re: [HACKERS] Shortened URLs for commit messages

2017-06-21 Thread Bruce Momjian
On Wed, Jun 21, 2017 at 11:11:57AM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Oh, here is a fixed version that requires an @ sign, which all message > > id's have: > > > > sed '/http/!s;^\(Discussion: *\)\(.*@.*\)$;\1https://postgr.es/m/\2;' > > So how do you actually use this

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Kuntal Ghosh
On Wed, Jun 21, 2017 at 7:52 PM, Dilip Kumar wrote: > On Wed, Jun 21, 2017 at 7:47 PM, Kuntal Ghosh > wrote: >>> IMHO, It's not a good idea to use DSM call to verify the DSA handle. >>> >> Okay. Is there any particular scenario you've in mind where this may fail? > > It's not about failure, but a

Re: [HACKERS] Broken hint bits (freeze)

2017-06-21 Thread Bruce Momjian
On Wed, Jun 21, 2017 at 07:49:21PM +0530, Amit Kapila wrote: > On Tue, Jun 20, 2017 at 7:24 PM, Amit Kapila wrote: > > Hmm. I think we need something that works with lesser effort because > > not all users will be as knowledgeable as you are, so if they make any > > mistakes in copying the file m

Re: [HACKERS] Incorrect documentation about pg_stat_activity

2017-06-21 Thread Yugo Nagata
On Wed, 21 Jun 2017 16:18:50 +0200 Simon Riggs wrote: > On 21 June 2017 at 16:15, Yugo Nagata wrote: > > On Wed, 21 Jun 2017 19:08:35 +0530 > > Kuntal Ghosh wrote: > > > >> On Wed, Jun 21, 2017 at 6:05 PM, Yugo Nagata wrote: > >> > > >> > Attached is a patch for the documentation fix. > >> > >

Re: [HACKERS] initdb initalization failure for collation "ja_JP"

2017-06-21 Thread Marco Atzeri
On 20/06/2017 17:37, Tom Lane wrote: I wrote: Marco Atzeri writes: Building on Cygwin latest 10 beta1 or head sourece, make check fails as: ... performing post-bootstrap initialization ... 2017-05-31 23:23:22.214 CEST [16860] FATAL: collation "ja_JP" for encoding "EUC_JP" already exists H

[HACKERS] Logical replication launcher never been restarted when terminated

2017-06-21 Thread Yugo Nagata
Hi, As I report in another thread[1], when the logical replication launcher is terminated by SIGTERM, it never been restarted and we need to restart the server to enable logical replication again. This is because the logical replication launcher exits with exitstatus 0, so if it exits with statu

Re: [HACKERS] UPDATE of partition key

2017-06-21 Thread Amit Khandekar
On 21 June 2017 at 00:23, Robert Haas wrote: > On Tue, Jun 20, 2017 at 2:54 AM, Amit Khandekar > wrote: >>> I guess I don't see why it should work like this. In the INSERT case, >>> we must build withCheckOption objects for each partition because those >>> partitions don't appear in the plan ot

Re: [HACKERS] UPDATE of partition key

2017-06-21 Thread Amit Khandekar
On 21 June 2017 at 20:14, Robert Haas wrote: > On Wed, Jun 21, 2017 at 5:28 AM, Amit Langote > wrote:>> The comment "UPDATE/DELETE > cases are handled above" is referring to >>> the code that initializes the WCOs generated by the planner. You've >>> modified the comment in your patch, but the as

[HACKERS] archive_timeout ignored directly after promotion

2017-06-21 Thread Andres Freund
Hi, When promoting a standby without using the "fast promotion" logic, e.g. by using recovery targets (which doesn't use fast promotion for unbeknownst to me reasons), checkpointer doesn't necessarily respect archive timeout for the first cycle after promotion. The reason for that is that it dete

[HACKERS] Fast promotion not used when doing a recovery_target PITR restore?

2017-06-21 Thread Andres Freund
Hi, When doing a PITR style recovery, with recovery target set, we're currently not doing a fast promotion, in contrast to the handling when doing a pg_ctl or trigger file based promotion. That can prolong making the server available for writes. I can't really see a reason for this? Greetings,

Re: [HACKERS] CREATE SUBSCRIPTION log noise

2017-06-21 Thread Peter Eisentraut
On 6/20/17 22:54, Jeff Janes wrote: > I think this should go away: > > ereport(NOTICE, > (errmsg("created replication slot \"%s\" on > publisher", > slotname))); > > It doesn't appear to be contingent on anything other than the c

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-21 Thread Andrew Dunstan
On 06/21/2017 11:30 AM, Tom Lane wrote: > Andrew Dunstan writes: >> Unfortunately this seems precluded by the use of the non-standard >> "err.h". It looks like that will trip us up on mingw also. > Hm, why? Doesn't the -I search path get the right thing? > > The file is

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-21 Thread Tom Lane
Andrew Dunstan writes: > On 06/21/2017 11:30 AM, Tom Lane wrote: >> Andrew Dunstan writes: >>> Unfortunately this seems precluded by the use of the non-standard >>> "err.h". It looks like that will trip us up on mingw also. >> Hm, why? Doesn't the -I search path get the right thing? > The file

Re: [HACKERS] Logical replication: stuck spinlock at ReplicationSlotRelease

2017-06-21 Thread Peter Eisentraut
On 6/21/17 09:02, Albe Laurenz wrote: > 2017-06-21 14:55:12.033 CEST [23124] LOG: could not send data to client: > Broken pipe > 2017-06-21 14:55:12.033 CEST [23124] FATAL: connection to client lost > 2017-06-21 14:55:17.032 CEST [23133] LOG: logical replication apply worker > for subscription

Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org

2017-06-21 Thread Andrew Dunstan
On 06/21/2017 02:25 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 06/21/2017 11:30 AM, Tom Lane wrote: >>> Andrew Dunstan writes: Unfortunately this seems precluded by the use of the non-standard "err.h". It looks like that will trip us up on mingw also. >>> Hm, why? Doesn't the

[HACKERS] GSoC 2017 Proposal for predicate locking in hash index

2017-06-21 Thread Shubham Barai
Hi, Now that hash index support write-ahead logging, it will be great if we add support for predicate locking to it. Implementation of predicate locking in hash index seems very simple. I have already made changes in the code. I am currently working on testing. Here is my approach 1) PredicateLo

Re: [HACKERS] Adding connection id in the startup message

2017-06-21 Thread Robert Haas
On Thu, Jun 15, 2017 at 3:11 AM, Satyanarayana Narlapuram wrote: > The proposal is to tweak the connectivity wire protocol, and add a > connection id (GUID) filed in the startup message. We can trace the > connection using this GUID and investigate further on where the connection > failed. Wire p

Re: [HACKERS] Adding connection id in the startup message

2017-06-21 Thread Robert Haas
On Thu, Jun 15, 2017 at 9:50 AM, Tom Lane wrote: > Can you give a concrete example where this would have > helped above and beyond knowing, eg, the source and time of the connection > attempt? I can imagine that in really high-volume use cases (such as the OP apparently has) the number of client

Re: [HACKERS] Logical replication launcher never been restarted when terminated

2017-06-21 Thread Peter Eisentraut
On 6/21/17 13:03, Yugo Nagata wrote: > As I report in another thread[1], when the logical replication launcher > is terminated by SIGTERM, it never been restarted and we need to restart > the server to enable logical replication again. > > This is because the logical replication launcher exits wi

Re: [HACKERS] Making server name part of the startup message

2017-06-21 Thread Satyanarayana Narlapuram
> I should think that in such cases, the end client is exactly not what you > want to be choosing which server it gets redirected to. You'd be wanting to > base that on >policies defined at the pooler. There are already plenty of > client-supplied attributes you could use as inputs for such po

Re: [HACKERS] Making server name part of the startup message

2017-06-21 Thread Satyanarayana Narlapuram
PgBouncer for example assumes that the database names are unique across the database clusters it is serving. Our front-end Gateways can serve tens of thousands of Postgres servers spanning multiple customers, and organizations, and enforcing the database names being unique is not possible for th

Re: [HACKERS] Missing comment for ResultRelInfo in execnodes.h

2017-06-21 Thread Peter Eisentraut
On 6/20/17 22:37, Etsuro Fujita wrote: > On 2017/06/21 3:30, Peter Eisentraut wrote: >> On 6/20/17 05:50, Etsuro Fujita wrote: >>> Here is a small patch to add a comment on its new member PartitionRoot. >> >> The existing comment style is kind of unusual. How about the attached >> to clean it up a

Re: [HACKERS] Adding connection id in the startup message

2017-06-21 Thread David G. Johnston
On Wed, Jun 21, 2017 at 12:15 PM, Robert Haas wrote: > I think the problem is real, > but I'm not sure that this is the best solution. On the other hand, > I'm also not entirely sure I understand the proposal yet. Given the problems with changing the protocol it does seem like generalizing away

Re: [HACKERS] Pluggable storage

2017-06-21 Thread Robert Haas
On Mon, Jun 12, 2017 at 9:50 PM, Haribabu Kommi wrote: > Open Items: > > 1. The BitmapHeapScan and TableSampleScan are tightly coupled with > HeapTuple and HeapScanDesc, So these scans are directly operating > on those structures and providing the result. > > These scan types may not be applicable

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-21 Thread Tom Lane
I wrote: > Barring objections, I'd like to reindent HEAD with the new version > of pg_bsd_indent (and correspondingly updated pgindent script) > tomorrow, say around 1800 UTC. ... and it's done. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] postgresql transactons not fully isolated

2017-06-21 Thread Merlin Moncure
On Tue, Jun 20, 2017 at 2:58 PM, Merlin Moncure wrote: > On Tue, Jun 20, 2017 at 2:34 PM, David G. Johnston > wrote: >> On Tue, Jun 20, 2017 at 12:22 PM, Chapman Flack >> wrote: >>> I get the reported result (DELETE 0 and a table containing 2 and 3) >>> in both 'read committed' and 'read uncomm

Re: [HACKERS] UPDATE of partition key

2017-06-21 Thread Robert Haas
On Wed, Jun 21, 2017 at 1:37 PM, Amit Khandekar wrote: >> e.g. the table is partitioned on order number, and you do UPDATE >> lineitem SET product_code = 'K372B' WHERE product_code = 'K372'. > > This query does not update order number, so here there is no > partition-key-update. Are you thinking t

Re: [HACKERS] Making server name part of the startup message

2017-06-21 Thread Tom Lane
Satyanarayana Narlapuram writes: >> Why do we need to incur a protocol break to add another one? > This is optional and is not a protocol break. Yes, it is. We've been around on this sort of thing before and we understand the consequences. If the option is carried in the startup message, the c

Re: [HACKERS] UPDATE of partition key

2017-06-21 Thread Robert Haas
On Wed, Jun 21, 2017 at 1:38 PM, Amit Khandekar wrote: >>> Yep, it's more appropriate to use >>> ModifyTableState->rootResultRelationInfo->ri_RelationDesc somehow. That >>> is, if answer to the question I raised above is positive. > > From what I had checked earlier when coding that part, > rootR

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-21 Thread Bruce Momjian
On Wed, Jun 21, 2017 at 04:07:30PM -0400, Tom Lane wrote: > I wrote: > > Barring objections, I'd like to reindent HEAD with the new version > > of pg_bsd_indent (and correspondingly updated pgindent script) > > tomorrow, say around 1800 UTC. > > ... and it's done. You are eventually doing all act

[HACKERS] Broken O(n^2) avoidance in wal segment recycling.

2017-06-21 Thread Andres Freund
Hi, Author: Heikki Linnakangas Branch: master Release: REL9_5_BR [b2a5545bd] 2015-04-13 16:53:49 +0300 Branch: REL9_4_STABLE Release: REL9_4_2 [d72792d02] 2015-04-13 17:22:21 +0300 Branch: REL9_3_STABLE Release: REL9_3_7 [a800267e4] 2015-04-13 17:22:35 +0300 Branch: REL9_2_STABLE Release: REL9_2_

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-21 Thread Tom Lane
Bruce Momjian writes: > On Wed, Jun 21, 2017 at 04:07:30PM -0400, Tom Lane wrote: >> ... and it's done. > You are eventually doing all active branches, right? I don't think we'd entirely decided that we should do that, or when to do it. I'm not in a huge hurry; we might find some more tweaks we

Re: [HACKERS] Phantom segment upon promotion causing troubles.

2017-06-21 Thread Andres Freund
Hi, On 2017-06-20 16:11:32 +0300, Heikki Linnakangas wrote: > On 06/19/2017 10:30 AM, Andres Freund wrote: > > Greg Burek from Heroku (CCed) reported a weird issue on IM, that was > > weird enough to be interesting. What he'd observed was that he promoted > > some PITR standby, and early clones o

Re: [HACKERS] CREATE SUBSCRIPTION log noise

2017-06-21 Thread Euler Taveira
2017-06-21 15:14 GMT-03:00 Peter Eisentraut < peter.eisentr...@2ndquadrant.com>: > > It doesn't appear to be contingent on anything other than the content of > > the command you just gave it. I don't think we need a NOTICE saying > > that it did that thing I just told it to do--that should be imp

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-21 Thread Andres Freund
On 2017-06-21 17:28:32 -0400, Tom Lane wrote: > Right now we're really just speculating about how much pain there will > be, on either end of this. So it'd be interesting for somebody who's > carrying large out-of-tree patches (EDB? Citus?) to try the new > pgindent version on a back branch and se

Re: [HACKERS] Re-indent HEAD tomorrow?

2017-06-21 Thread Tom Lane
Andres Freund writes: > On 2017-06-21 17:28:32 -0400, Tom Lane wrote: >> And I think it'd make sense to wait a few >> months and garner some experience with back-patching from v10 into the >> older branches, so we have more than guesses about how much pain not >> reindenting will be for us. > Hm.

Re: [HACKERS] intermittent failures in Cygwin from select_parallel tests

2017-06-21 Thread Tom Lane
Today, lorikeet failed with a new variant on the bgworker start crash: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lorikeet&dt=2017-06-21%2020%3A29%3A10 This one is even more exciting than the last one, because it sure looks like the crashing bgworker took the postmaster down with it.

[HACKERS] PG 10beta2 schedule

2017-06-21 Thread Tom Lane
The release team has agreed that a good time to put out beta2 will be the second week of July, ie wrap tarballs 10 July for announcement 13 July. I imagine beta3 will appear along with the scheduled back-branch releases in the second week of August, but that's not positively decided yet.

[HACKERS] RLS in CTE incorrect permission failure

2017-06-21 Thread Rod Taylor
In the attached script, the second insert into t2 (as part of the CTE) should succeed. My actual use case isn't much more complex; the function is used primarily to allow peaking at columns that the function definer has access to but a typical user does not. Function also makes it easy to copy this

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-21 Thread Andres Freund
On 2017-06-20 17:51:23 +0200, Daniel Verite wrote: > Andres Freund wrote: > > > FWIW, I still think this needs a pgbench or similar example integration, > > so we can actually properly measure the benefits. > > Here's an updated version of the patch I made during review, > adding \beginbatc

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-21 Thread Craig Ringer
On 22 Jun. 2017 07:40, "Andres Freund" wrote: On 2017-06-20 17:51:23 +0200, Daniel Verite wrote: > Andres Freund wrote: > > > FWIW, I still think this needs a pgbench or similar example integration, > > so we can actually properly measure the benefits. > > Here's an updated version of the p

Re: [HACKERS] RLS in CTE incorrect permission failure

2017-06-21 Thread Tom Lane
Rod Taylor writes: > In the attached script, the second insert into t2 (as part of the CTE) > should succeed. No, I don't think so. You declared the check function as STABLE which means it is confined to seeing the same snapshot as the surrounding query. So it can't see anything inserted by that

Re: [HACKERS] [COMMITTERS] pgsql: Restart logical replication launcher when killed

2017-06-21 Thread Michael Paquier
On Thu, Jun 22, 2017 at 4:16 AM, Peter Eisentraut wrote: > Restart logical replication launcher when killed - /* The logical replication launcher can be stopped at any time. */ - proc_exit(0); + /* The logical replication launcher can be stopped at any time. +

Re: [HACKERS] [COMMITTERS] pgsql: Restart logical replication launcher when killed

2017-06-21 Thread Andres Freund
On 2017-06-22 08:46:35 +0900, Michael Paquier wrote: > On Thu, Jun 22, 2017 at 4:16 AM, Peter Eisentraut wrote: > > Restart logical replication launcher when killed > > - /* The logical replication launcher can be stopped at any time. */ > - proc_exit(0); > + /* The

Re: [HACKERS] RLS in CTE incorrect permission failure

2017-06-21 Thread Rod Taylor
On Wed, Jun 21, 2017 at 7:46 PM, Tom Lane wrote: > Rod Taylor writes: > > In the attached script, the second insert into t2 (as part of the CTE) > > should succeed. > > No, I don't think so. You declared the check function as STABLE which > means it is confined to seeing the same snapshot as th

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-21 Thread Michael Paquier
On Wed, Jun 21, 2017 at 11:42 PM, Daniel Gustafsson wrote: > The message is truncated in SetBackendCancelMessage() for safety, but > pg_{cancel|terminate}_backend() could throw an error on too long message, or > warning truncation, to the caller as well. Personally I think a warning is > the > a

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-21 Thread Andres Freund
On 2017-06-21 16:40:48 -0700, Andres Freund wrote: > On 2017-06-20 17:51:23 +0200, Daniel Verite wrote: > > Andres Freund wrote: > > > > > FWIW, I still think this needs a pgbench or similar example integration, > > > so we can actually properly measure the benefits. > > > > Here's an updated

Re: [HACKERS] Rules on table partitions

2017-06-21 Thread Amit Langote
On 2017/06/21 18:49, Dean Rasheed wrote: > On 20 June 2017 at 03:01, Amit Langote wrote: >> Hmm, yes. The following exercise convinced me. >> >> create table r (a int) partition by range (a); >> create table r1 partition of r for values from (1) to (10); >> create rule "_RETURN" as on select to r

  1   2   >