Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

2017-05-11 Thread Michael Paquier
On Wed, May 10, 2017 at 11:57 AM, Masahiko Sawada wrote: > Barring any objections, I'll add these two issues to open item. It seems to me that those open items have not been added yet to the list. If I am following correctly, they could be defined as follows: - Dropping subscription may stuck if

Re: [HACKERS] SCRAM in the PG 10 release notes

2017-05-11 Thread Heikki Linnakangas
On 05/11/2017 07:03 AM, Michael Paquier wrote: On Thu, May 11, 2017 at 11:50 AM, Bruce Momjian wrote: I have added this as an open item because we will have to wait to see where we are with driver support as the release gets closer. As Postgres ODBC now has a hard dependency with libpq, no ac

Re: [HACKERS] Re: logical replication syntax (was DROP SUBSCRIPTION, query cancellations and slot handling)

2017-05-11 Thread Noah Misch
On Mon, May 08, 2017 at 05:01:00PM -0400, Peter Eisentraut wrote: > On 5/7/17 04:21, Noah Misch wrote: > > The above-described topic is currently a PostgreSQL 10 open item. Peter, > > since you committed the patch believed to have created it, you own this open > > item. If some other commit is mo

Re: [HACKERS] Time based lag tracking for logical replication

2017-05-11 Thread Noah Misch
On Sun, Apr 23, 2017 at 01:10:32AM +0200, Petr Jelinek wrote: > The time based lag tracking commit [1] added interface for logging > progress of replication so that we can report lag as time interval > instead of just bytes. But the patch didn't contain patch for the > builtin logical replication.

[HACKERS] Re: transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-11 Thread Noah Misch
On Sat, May 06, 2017 at 06:54:37PM +, Noah Misch wrote: > On Mon, May 01, 2017 at 11:10:52AM -0500, Kevin Grittner wrote: > > On Mon, May 1, 2017 at 10:01 AM, Robert Haas wrote: > > > > > It seems pretty clear to me that this is busted. > > > > I don't think you actually tested anything that

Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

2017-05-11 Thread Masahiko Sawada
On Thu, May 11, 2017 at 4:06 PM, Michael Paquier wrote: > On Wed, May 10, 2017 at 11:57 AM, Masahiko Sawada > wrote: >> Barring any objections, I'll add these two issues to open item. > > It seems to me that those open items have not been added yet to the > list. If I am following correctly, the

Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

2017-05-11 Thread Petr Jelinek
On 11/05/17 10:10, Masahiko Sawada wrote: > On Thu, May 11, 2017 at 4:06 PM, Michael Paquier > wrote: >> On Wed, May 10, 2017 at 11:57 AM, Masahiko Sawada >> wrote: >>> Barring any objections, I'll add these two issues to open item. >> >> It seems to me that those open items have not been added

[HACKERS] alter table..drop constraint pkey, left not null un-dropped

2017-05-11 Thread Rajkumar Raghuwanshi
Hi All, I have created a table with primary key, and then dropped primary key from table. But table still have not null constraint added by primary key. Is there any other statement to delete primary key with not null? or this is an expected behaviour of pg? postgres=# create table tbl (c1 int p

[HACKERS] feature wish: filter log_min_duration_statement according to the context (parse|bind|execute|...)

2017-05-11 Thread Marc Mamin
Hello, setting log_min_duration_statement to 0 is usefull on test or development system, but this may lead to huge log files. It would often be useful to discard the parse and bind entries. maybe a new parameter like "log_min_duration_execute"? I don't have the skills to implement this though.

Re: [HACKERS] Bug in pg_dump --table and --exclude-table for declarative partition table handling.

2017-05-11 Thread Ashutosh Bapat
On Thu, May 11, 2017 at 2:06 AM, Robert Haas wrote: > On Tue, May 9, 2017 at 4:21 AM, Jeevan Ladhe > wrote: >>> Current pg_dump --exclude-table option excludes partitioned relation >>> and dumps all its child relations and vice versa for --table option, which >>> I think is incorrect. >>> >>> In

Re: [HACKERS] Bug in pg_dump --table and --exclude-table for declarative partition table handling.

2017-05-11 Thread Ashutosh Bapat
On Thu, May 11, 2017 at 3:05 AM, Tom Lane wrote: > > We should make sure that pg_dump behaves sanely when dumping just > some elements of a partition tree, of course. And for that matter, > pg_restore ought to be able to successfully restore just some elements > out of a an archive containing mo

Re: [HACKERS] alter table..drop constraint pkey, left not null un-dropped

2017-05-11 Thread Ashutosh Bapat
On Thu, May 11, 2017 at 3:03 PM, Rajkumar Raghuwanshi wrote: > Hi All, > > I have created a table with primary key, and then dropped primary key from > table. But table still have not null constraint added by primary key. > > Is there any other statement to delete primary key with not null? > or t

Re: [HACKERS] Re: logical replication syntax (was DROP SUBSCRIPTION, query cancellations and slot handling)

2017-05-11 Thread Petr Jelinek
On 11/05/17 09:27, Noah Misch wrote: > On Mon, May 08, 2017 at 05:01:00PM -0400, Peter Eisentraut wrote: >> On 5/7/17 04:21, Noah Misch wrote: >>> The above-described topic is currently a PostgreSQL 10 open item. Peter, >>> since you committed the patch believed to have created it, you own this op

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Kapila
On Fri, Mar 17, 2017 at 4:07 PM, Amit Khandekar wrote: > On 4 March 2017 at 12:49, Robert Haas wrote: >> On Thu, Mar 2, 2017 at 11:53 AM, Amit Khandekar >> wrote: >>> I think it does not make sense running after row triggers in case of >>> row-movement. There is no update happened on that leaf

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Kapila
On Wed, May 3, 2017 at 11:22 AM, Amit Khandekar wrote: > On 2 May 2017 at 18:17, Robert Haas wrote: >> On Tue, Apr 4, 2017 at 7:11 AM, Amit Khandekar >> wrote: >>> Attached updated patch v7 has the above changes. >> > > Attached is the rebased patch, which resolves the above conflicts. > Few c

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Robert Haas
On Thu, May 11, 2017 at 7:54 AM, Amit Kapila wrote: > Few comments: > 1. > Operating directly on partition doesn't allow update to move row. > Refer below example: > create table t1(c1 int) partition by range(c1); > create table t1_part_1 partition of t1 for values from (1) to (100); > create tabl

[HACKERS] Logical decoding truncate

2017-05-11 Thread Friedrich, Steffen
Hi, I am writing a logical decoding output plugin decoding WAL to SQL which is finally applied to target database. Is it possible to decode a TRUNCATE statement and the tables involved? Assuming the SQL statement "TRUNCATE x, y;", I am interested in decoding the operation TRUNCATE and the corr

Re: [HACKERS] feature wish: filter log_min_duration_statement according to the context (parse|bind|execute|...)

2017-05-11 Thread Stephen Frost
Greetings, * Marc Mamin (m.ma...@intershop.de) wrote: > setting log_min_duration_statement to 0 is usefull on test or development > system, but this may lead to huge log files. > It would often be useful to discard the parse and bind entries. > maybe a new parameter like "log_min_duration_execut

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Khandekar
On 11 May 2017 at 17:23, Amit Kapila wrote: > On Fri, Mar 17, 2017 at 4:07 PM, Amit Khandekar > wrote: >> On 4 March 2017 at 12:49, Robert Haas wrote: >>> On Thu, Mar 2, 2017 at 11:53 AM, Amit Khandekar >>> wrote: I think it does not make sense running after row triggers in case of

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Khandekar
On 11 May 2017 at 17:24, Amit Kapila wrote: > Few comments: > 1. > Operating directly on partition doesn't allow update to move row. > Refer below example: > create table t1(c1 int) partition by range(c1); > create table t1_part_1 partition of t1 for values from (1) to (100); > create table t1_par

[HACKERS] If subscription to foreign table valid ?

2017-05-11 Thread tushar
Hi, I observed that -we cannot publish "foreign table" in Publication postgres=# create foreign table t (n int) server db1_server options (table_name 't1'); CREATE FOREIGN TABLE postgres=# create publication pub for table t; ERROR: "t" is not a table DETAIL: Only tables can be added to publ

Re: [HACKERS] alter table..drop constraint pkey, left not null un-dropped

2017-05-11 Thread Tom Lane
Rajkumar Raghuwanshi writes: > I have created a table with primary key, and then dropped primary key from > table. But table still have not null constraint added by primary key. > Is there any other statement to delete primary key with not null? > or this is an expected behaviour of pg? Yes, it'

Re: [HACKERS] Bug in pg_dump --table and --exclude-table for declarative partition table handling.

2017-05-11 Thread Jeevan Ladhe
On Thu, May 11, 2017 at 4:47 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > We add PARTITION OF clause for a table which is partition, so if the > parent is not present while restoring, the restore is going to fail. +1 But, similarly for inheritance if we dump a child table, it

Re: [HACKERS] Time based lag tracking for logical replication

2017-05-11 Thread Simon Riggs
On 11 May 2017 at 08:32, Noah Misch wrote: > On Sun, Apr 23, 2017 at 01:10:32AM +0200, Petr Jelinek wrote: >> The time based lag tracking commit [1] added interface for logging >> progress of replication so that we can report lag as time interval >> instead of just bytes. But the patch didn't cont

Re: [HACKERS] Bug in pg_dump --table and --exclude-table for declarative partition table handling.

2017-05-11 Thread Tom Lane
Jeevan Ladhe writes: > On Thu, May 11, 2017 at 4:47 PM, Ashutosh Bapat < >> We add PARTITION OF clause for a table which is partition, so if the >> parent is not present while restoring, the restore is going to fail. > +1 > But, similarly for inheritance if we dump a child table, it's dump is > b

Re: [HACKERS] Time based lag tracking for logical replication

2017-05-11 Thread Petr Jelinek
On 11/05/17 15:01, Simon Riggs wrote: > On 11 May 2017 at 08:32, Noah Misch wrote: >> On Sun, Apr 23, 2017 at 01:10:32AM +0200, Petr Jelinek wrote: >>> The time based lag tracking commit [1] added interface for logging >>> progress of replication so that we can report lag as time interval >>> inst

Re: [HACKERS] Bug in pg_dump --table and --exclude-table for declarative partition table handling.

2017-05-11 Thread Robert Haas
On Thu, May 11, 2017 at 9:02 AM, Tom Lane wrote: > Jeevan Ladhe writes: >> On Thu, May 11, 2017 at 4:47 PM, Ashutosh Bapat < >>> We add PARTITION OF clause for a table which is partition, so if the >>> parent is not present while restoring, the restore is going to fail. > >> +1 >> But, similarly

Re: [HACKERS] SCRAM in the PG 10 release notes

2017-05-11 Thread Tom Lane
Heikki Linnakangas writes: > I updated the List of Drivers in the Wiki. I added a few drivers that > were missing, like the ODBC driver, and the pgtclng driver, as well as a > Go and Rust driver that I'm aware of. I reformatted it, and added a > column to indicate whether each driver uses libpq

[HACKERS] Re: transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-11 Thread Robert Haas
On Thu, May 11, 2017 at 3:38 AM, Noah Misch wrote: > This PostgreSQL 10 open item is past due for your status update. Kindly send > a status update within 24 hours, and include a date for your subsequent status > update. Refer to the policy on open item ownership: > https://www.postgresql.org/me

Re: [HACKERS] [PATCH] New command to monitor progression of long running queries

2017-05-11 Thread Remi Colinet
Do you have more details about the failed tests? Regards, Remi 2017-05-06 5:38 GMT+02:00 Vinayak Pokale : > > On Mon, Apr 17, 2017 at 9:09 PM, Remi Colinet > wrote: > >> Hello, >> >> I've implemented a new command named PROGRESS to monitor progression of >> long running SQL queries in a backend

Re: [HACKERS] If subscription to foreign table valid ?

2017-05-11 Thread Robert Haas
On Thu, May 11, 2017 at 8:25 AM, tushar wrote: > I observed that -we cannot publish "foreign table" in Publication > > postgres=# create foreign table t (n int) server db1_server options > (table_name 't1'); > CREATE FOREIGN TABLE > > postgres=# create publication pub for table t; > ERROR: "t" is

Re: [HACKERS] Bug in pg_dump --table and --exclude-table for declarative partition table handling.

2017-05-11 Thread Tom Lane
Robert Haas writes: > On Thu, May 11, 2017 at 9:02 AM, Tom Lane wrote: >> You could argue that it would be better for pg_dump to emit something >> like >> >> CREATE TABLE c (...); >> ALTER TABLE c INHERIT p; >> >> so that if p isn't around, at least c gets created. And I think it >> *would* be

Re: [HACKERS] [PATCH] New command to monitor progression of long running queries

2017-05-11 Thread Remi Colinet
That's a good point. A command is more straightforward because it targets only one backend. The user is supposed to know which backend pid is taking a long time to complete based on pg_stat_activity(). This is somehow the same approach as EXPLAIN command. But the use is limited to psql utility. A

Re: [HACKERS] If subscription to foreign table valid ?

2017-05-11 Thread Petr Jelinek
Hi, On 11/05/17 14:25, tushar wrote: > Hi, > > I observed that -we cannot publish "foreign table" in Publication > > postgres=# create foreign table t (n int) server db1_server options > (table_name 't1'); > CREATE FOREIGN TABLE > > postgres=# create publication pub for table t; > ERROR: "t" i

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-11 Thread Rahila Syed
Hello, Please find attached an updated patch with review comments and bugs reported till date implemented. >1. >In following block, we can just do with def_index, and we do not need found_def >flag. We can check if def_index is -1 or not to decide if default partition is >present. found_def is us

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Stas Kelvich
> On 10 May 2017, at 11:43, Petr Jelinek wrote: > > On 09/05/17 22:11, Erik Rijkers wrote: >> On 2017-05-09 21:00, Petr Jelinek wrote: >>> On 09/05/17 19:54, Erik Rijkers wrote: On 2017-05-09 11:50, Petr Jelinek wrote: >>> >>> Ah okay, so this is same issue that's reported by both Ma

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-05-11 Thread Tom Lane
Michael Paquier writes: > On Thu, May 11, 2017 at 9:15 AM, Bruce Momjian wrote: >> On Wed, May 10, 2017 at 01:09:36PM -0700, Joe Conway wrote: > On 05/10/2017 12:22 PM, Tom Lane wrote: >>> Hm, well, anybody else want to vote? >>> +1 for #2 >> Same, +1 for #2 (apply this patch) > #1, do nothing

Re: [HACKERS] Should pg_current_wal_location() become pg_current_wal_lsn()

2017-05-11 Thread Tom Lane
Neha Khatri writes: > [In case forgotten] pg_controdata and pg_waldump interfaces should also be > considered for this standardization. > Following are pg_controldata interfaces that might require change: > Latest checkpoint location: > Prior checkpoint location: > Latest checkpoint's REDO

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Petr Jelinek
On 11/05/17 16:33, Stas Kelvich wrote: > >> On 10 May 2017, at 11:43, Petr Jelinek wrote: >> >> On 09/05/17 22:11, Erik Rijkers wrote: >>> On 2017-05-09 21:00, Petr Jelinek wrote: On 09/05/17 19:54, Erik Rijkers wrote: > On 2017-05-09 11:50, Petr Jelinek wrote: > Ah okay, s

[HACKERS] eval_const_expresisions and ScalarArrayOpExpr

2017-05-11 Thread Heikki Linnakangas
Eval_const_expressions() doesn't know about ScalarArrayOpExpr. We simplify the arguments, but if all the arguments are booleans, we don't take the obvious step of replacing the whole expression with a boolean Const. For example: postgres=# explain select * from foo where 1 IN (1,2,3);

Re: [HACKERS] Time based lag tracking for logical replication

2017-05-11 Thread Simon Riggs
On 11 May 2017 at 14:12, Petr Jelinek wrote: >> Attached patch is Petr's patch, slightly rebased with added pacing >> delay, similar to that used by HSFeedback. >> > > This looks reasonable. I would perhaps change: >> + /* >> +* Track lag no more than once per >> WALSND_LOGICAL_LAG

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Bruce Momjian
On Wed, May 10, 2017 at 07:11:07PM +0300, Konstantin Knizhnik wrote: > I am going to continue work on this patch I will be glad to receive any > feedback and suggestions for its improvement. > In most cases, applications are not accessing Postgres directly, but using > some connection pooling layer

Re: [HACKERS] export import bytea from psql

2017-05-11 Thread Daniel Verite
Pavel Stehule wrote: > It is similar to my first or second proposal - rejected by Tom :( Doesn't it differ? ISTM that in the patches/discussion related to: https://commitfest.postgresql.org/11/787/ it was proposed to change \set in one way or another, and also in the point #1 of this pres

Re: [HACKERS] eval_const_expresisions and ScalarArrayOpExpr

2017-05-11 Thread Tom Lane
Heikki Linnakangas writes: > Eval_const_expressions() doesn't know about ScalarArrayOpExpr. > ... > That seems like an oversight. I guess that scenario doesn't happen very > often in practice, but there's no reason not to do it when it does. > Patch attached. Yeah, I think it's a lack-of-round-

Re: [HACKERS] [PATCH v2] Progress command to monitor progression of long running SQL queries

2017-05-11 Thread Remi Colinet
That's good point. I will probably convert the new command to a SQL function. I was fumbling between a table or a SQL function. Oracle uses v$session_longops to track progression of long running SQL queries which have run for more than 6 seconds. But a function is much better to provide parameter

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Peter Eisentraut
On 5/10/17 12:24, Andres Freund wrote: > The issue isn't the strength, but that we currently have this weird > hackery around open_share_lock(): > /* > * Open the sequence and acquire AccessShareLock if needed > * > * If we haven't touched the sequence already in this transaction, > * we need t

Re: [HACKERS] renaming "transaction log"

2017-05-11 Thread Tom Lane
Peter Eisentraut writes: > Most documentation and error messages still uses the term "transaction > log" to refer to the write-ahead log. Here is a patch to rename that, > which I think should be done, to match the xlog -> wal renaming in APIs. This will need to be rebased over d10c626de, which

Re: [HACKERS] [POC] hash partitioning

2017-05-11 Thread Dilip Kumar
On Wed, May 3, 2017 at 6:39 PM, amul sul wrote: > On Thu, Apr 27, 2017 at 1:42 AM, Robert Haas wrote: > >>I spent some time today looking at these patches. It seems like there >>is some more work still needed here to produce something committable >>regardless of which way we go, but I am incline

Re: [HACKERS] [PATCH v2] Progress command to monitor progression of long running SQL queries

2017-05-11 Thread Remi Colinet
2017-05-11 4:05 GMT+02:00 Michael Paquier : > On Thu, May 11, 2017 at 1:40 AM, Remi Colinet > wrote: > > This is version 2 of the new command name PROGRESS which I wrote in > order to > > monitor long running SQL queries in a Postgres backend process. > > It would be a good idea to add this patch

Re: [HACKERS] [PATCH v2] Progress command to monitor progression of long running SQL queries

2017-05-11 Thread Remi Colinet
Thx for your comment. Based on other comments, I will probably convert the command into a SQL function. This allows passing arguments such as the one which can be used in the current command (VERBOSE, FORMAT). This will avoid keyword collisions. Regards Remi 2017-05-10 22:04 GMT+02:00 Pavel Ste

Re: [HACKERS] eval_const_expresisions and ScalarArrayOpExpr

2017-05-11 Thread Heikki Linnakangas
On 05/11/2017 06:21 PM, Tom Lane wrote: Heikki Linnakangas writes: Eval_const_expressions() doesn't know about ScalarArrayOpExpr. ... That seems like an oversight. I guess that scenario doesn't happen very often in practice, but there's no reason not to do it when it does. Patch attached. Yea

[HACKERS] Safer and faster get_attstatsslot()

2017-05-11 Thread Tom Lane
Monday's round of security patches was a lot more exciting than I would have liked, because code that worked fine for Peter and me failed erratically in the buildfarm. What eventually emerged was that I'd added some missing free_attstatsslot() calls in rangetypes_selfuncs.c, and naively copied the

Re: [HACKERS] eval_const_expresisions and ScalarArrayOpExpr

2017-05-11 Thread Tom Lane
Heikki Linnakangas writes: > On 05/11/2017 06:21 PM, Tom Lane wrote: >>> On a side-note, I find it a bit awkward that ScalarArrayOpExpr uses a >>> 2-element List to hold the scalar and array arguments. Wouldn't it be >>> much more straightforward to have explicit "Expr *scalararg" and "Expr >>> *a

Re: [HACKERS] export import bytea from psql

2017-05-11 Thread Pavel Stehule
2017-05-11 17:16 GMT+02:00 Daniel Verite : > Pavel Stehule wrote: > > > It is similar to my first or second proposal - rejected by Tom :( > > Doesn't it differ? ISTM that in the patches/discussion related to: > https://commitfest.postgresql.org/11/787/ > it was proposed to change \set in o

Re: [HACKERS] Safer and faster get_attstatsslot()

2017-05-11 Thread Simon Riggs
On 11 May 2017 at 17:41, Tom Lane wrote: > ...because code that worked fine for Peter and me failed > erratically in the buildfarm. I think its always a little bit too exciting for me also. I suggest we have a commit tree and a main tree, with automatic copying from commit -> main either 1. 24 h

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-11 Thread Alvaro Herrera
Tom Lane wrote: > Hmm ... I'm not sure that I buy that particular argument. If you're > concerned that the grammar could not handle "FROM x JOIN y USING (z)", > wouldn't it also have a problem with "FROM x JOIN y ON (z)"? > > It might work anyway, since the grammar should know whether ON or USIN

Re: [HACKERS] Time based lag tracking for logical replication

2017-05-11 Thread Andres Freund
On May 11, 2017 8:08:11 AM PDT, Simon Riggs wrote: >On 11 May 2017 at 14:12, Petr Jelinek >wrote: > >>> Attached patch is Petr's patch, slightly rebased with added pacing >>> delay, similar to that used by HSFeedback. >>> >> >> This looks reasonable. I would perhaps change: >>> + /* >>> +

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-11 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Hmm ... I'm not sure that I buy that particular argument. If you're >> concerned that the grammar could not handle "FROM x JOIN y USING (z)", >> wouldn't it also have a problem with "FROM x JOIN y ON (z)"? > Tomas spent some time trying to shoehorn the

Re: [HACKERS] Safer and faster get_attstatsslot()

2017-05-11 Thread Tom Lane
Simon Riggs writes: > On 11 May 2017 at 17:41, Tom Lane wrote: >> ...because code that worked fine for Peter and me failed >> erratically in the buildfarm. > I think its always a little bit too exciting for me also. > I suggest we have a commit tree and a main tree, with automatic > copying fro

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-11 Thread Alvaro Herrera
Tom Lane wrote: > Have you thought further about the upthread suggestion to just borrow > SELECT's syntax lock stock and barrel? That is, it'd look something > like > > CREATE STATISTICS name [(list of stats types)] expression-list FROM ... > [ WHERE ... ] [ WITH (options) ] > > This would

Re: [HACKERS] Time based lag tracking for logical replication

2017-05-11 Thread Simon Riggs
On 11 May 2017 at 18:13, Andres Freund wrote: >>New patch, v3. >> >>Applying in 90 minutes, barring objections. > > Could you please wait till tomorrow? I've bigger pending fixes for related > code pending/being tested that I plan to push today. I'd also like to take a > look before... Sure.

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Douglas Doole
> > One interesting idea from Doug Doole was to do it between the tokenizer > and parser. I think they are glued together so you would need a way to run > the tokenizer separately and compare that to the tokens you stored for the > cached plan. > When I did this, we had the same problem that the

Re: [HACKERS] [PATCH v2] Progress command to monitor progression of long running SQL queries

2017-05-11 Thread David Fetter
On Thu, May 11, 2017 at 05:24:16PM +0200, Remi Colinet wrote: > 2017-05-10 21:52 GMT+02:00 David Fetter : > > > On Wed, May 10, 2017 at 06:40:31PM +0200, Remi Colinet wrote: > > > Hello, > > > > > > This is version 2 of the new command name PROGRESS which I wrote in > > > order to monitor long run

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Bruce Momjian
On Thu, May 11, 2017 at 05:39:58PM +, Douglas Doole wrote: > One interesting idea from Doug Doole was to do it between the tokenizer > and > parser.  I think they are glued together so you would need a way to run > the > tokenizer separately and compare that to the tokens you stor

Re: [HACKERS] Logical decoding truncate

2017-05-11 Thread Euler Taveira
2017-05-11 4:23 GMT-03:00 Friedrich, Steffen < steffen.friedr...@dieboldnixdorf.com>: > I am writing a logical decoding output plugin decoding WAL to SQL which is > finally applied to target database. > > > > Is it possible to decode a TRUNCATE statement and the tables involved? > > Yes, use event

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Tom Lane
Bruce Momjian writes: > Good point. I think we need to do some measurements to see if the > parser-only stage is actually significant. I have a hunch that > commercial databases have much heavier parsers than we do. FWIW, gram.y does show up as significant in many of the profiles I take. I spec

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Andres Freund
On May 11, 2017 11:31:02 AM PDT, Tom Lane wrote: >Bruce Momjian writes: >> Good point. I think we need to do some measurements to see if the >> parser-only stage is actually significant. I have a hunch that >> commercial databases have much heavier parsers than we do. > >FWIW, gram.y does sho

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-11 Thread Alvaro Herrera
Tom Lane wrote: > Have you thought further about the upthread suggestion to just borrow > SELECT's syntax lock stock and barrel? That is, it'd look something > like > > CREATE STATISTICS name [(list of stats types)] expression-list FROM ... > [ WHERE ... ] [ WITH (options) ] > > This would

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Konstantin Knizhnik
On 05/11/2017 06:12 PM, Bruce Momjian wrote: On Wed, May 10, 2017 at 07:11:07PM +0300, Konstantin Knizhnik wrote: I am going to continue work on this patch I will be glad to receive any feedback and suggestions for its improvement. In most cases, applications are not accessing Postgres directly,

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Konstantin Knizhnik
On 05/11/2017 09:31 PM, Tom Lane wrote: Bruce Momjian writes: Good point. I think we need to do some measurements to see if the parser-only stage is actually significant. I have a hunch that commercial databases have much heavier parsers than we do. FWIW, gram.y does show up as significant i

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-11 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Have you thought further about the upthread suggestion to just borrow >> SELECT's syntax lock stock and barrel? > Bison seems to like the productions below. Is this what you had in > mind? These mostly mimic joined_table and table_ref, stripping out t

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Andres Freund
On 2017-05-11 22:48:26 +0300, Konstantin Knizhnik wrote: > On 05/11/2017 09:31 PM, Tom Lane wrote: > > Bruce Momjian writes: > > > Good point. I think we need to do some measurements to see if the > > > parser-only stage is actually significant. I have a hunch that > > > commercial databases hav

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Peter Eisentraut
On 5/10/17 12:24, Andres Freund wrote: > Upthread I theorized whether > that's actually still meaningful given fastpath locking and such, but I > guess we'll have to evaluate that. I did some testing. I ran this script CREATE SEQUENCE seq1; DO LANGUAGE plpythonu $$ plan = plpy.prepare("SELECT n

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Konstantin Knizhnik
On 05/11/2017 10:52 PM, Andres Freund wrote: On 2017-05-11 22:48:26 +0300, Konstantin Knizhnik wrote: On 05/11/2017 09:31 PM, Tom Lane wrote: Bruce Momjian writes: Good point. I think we need to do some measurements to see if the parser-only stage is actually significant. I have a hunch tha

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Andres Freund
On 2017-05-11 11:35:22 -0400, Peter Eisentraut wrote: > On 5/10/17 12:24, Andres Freund wrote: > > The issue isn't the strength, but that we currently have this weird > > hackery around open_share_lock(): > > /* > > * Open the sequence and acquire AccessShareLock if needed > > * > > * If we have

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Tom Lane
Peter Eisentraut writes: > On 5/10/17 12:24, Andres Freund wrote: >> Upthread I theorized whether >> that's actually still meaningful given fastpath locking and such, but I >> guess we'll have to evaluate that. > [ with or without contention, fast-path locking beats the extra dance that > open_sh

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Andres Freund
Hi, On 2017-05-11 16:27:48 -0400, Peter Eisentraut wrote: > On 5/10/17 12:24, Andres Freund wrote: > > Upthread I theorized whether > > that's actually still meaningful given fastpath locking and such, but I > > guess we'll have to evaluate that. > > I did some testing. That's with the open_sha

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Tom Lane
Peter Eisentraut writes: > I ran this script > CREATE SEQUENCE seq1; > DO LANGUAGE plpythonu $$ > plan = plpy.prepare("SELECT nextval('seq1')") > for i in range(0, 1000): > plpy.execute(plan) > $$; > and timed the "DO". It occurred to me that plpy.execute is going to run a subtransacti

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Tom Lane
Andres Freund writes: > On 2017-05-11 16:27:48 -0400, Peter Eisentraut wrote: >> (So without contention fast-path locking beats the extra dance that >> open_share_lock() does.) > That's kind of surprising, I really wouldn't have thought it'd be faster > without. I guess it's the overhead of sigs

Re: [HACKERS] [BUGS] Concurrent ALTER SEQUENCE RESTART Regression

2017-05-11 Thread Andres Freund
On 2017-05-11 17:21:18 -0400, Tom Lane wrote: > Peter Eisentraut writes: > > I ran this script > > > CREATE SEQUENCE seq1; > > > DO LANGUAGE plpythonu $$ > > plan = plpy.prepare("SELECT nextval('seq1')") > > for i in range(0, 1000): > > plpy.execute(plan) > > $$; > > > and timed the "DO

Re: [HACKERS] CTE inlining

2017-05-11 Thread Yaroslav
Ilya Shkuratov wrote > First of all, to such replacement to be valid, the CTE must be > 1. non-writable (e.g. be of form: SELECT ...), > 2. do not use VOLATILE or STABLE functions, > 3. ... (maybe there must be more restrictions?) What about simple things like this? CREATE OR REPLAC

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Andres Freund
On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > I plan to commit the next pending patch after the back branch releases > are cut - it's an invasive fix and the issue doesn't cause corruption > "just" slow slot creation. So it seems better to wait for a few days, > rather than hurry it into the

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Peter Geoghegan
On Thu, May 11, 2017 at 2:51 PM, Andres Freund wrote: > Now that that's done, here's an updated version of that patch. Note the > new logic to trigger xl_running_xact's to be logged at the right spot. > Works well in my testing. You forgot the patch. :-) -- Peter Geoghegan VMware vCenter Ser

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Andres Freund
On 2017-05-11 14:51:55 -0700, wrote: > On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > > I plan to commit the next pending patch after the back branch releases > > are cut - it's an invasive fix and the issue doesn't cause corruption > > "just" slow slot creation. So it seems better to wait f

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Andres Freund
On 2017-04-15 05:18:49 +0200, Petr Jelinek wrote: > >>> From 3318a929e691870f3c1ca665bec3bfa8ea2af2a8 Mon Sep 17 00:00:00 2001 > >>> From: Petr Jelinek > >>> Date: Sun, 26 Feb 2017 01:07:33 +0100 > >>> Subject: [PATCH 3/5] Prevent snapshot builder xmin from going backwards > >> > >> A bit more com

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-11 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> Have you thought further about the upthread suggestion to just borrow > >> SELECT's syntax lock stock and barrel? > > > Bison seems to like the productions below. Is this what you had in > > mind? These mostly mimic joined_table

Re: [HACKERS] WITH clause in CREATE STATISTICS

2017-05-11 Thread Tom Lane
Alvaro Herrera writes: > BTW the new castNode() family of macros don't work with Value nodes > (because the tags are different depending on what's stored, but each > type does not have its own struct. Oh well.) Yeah. Value nodes are pretty much of a wart --- it's not clear to me that they add a

Re: [HACKERS] A design for amcheck heapam verification

2017-05-11 Thread Peter Geoghegan
On Mon, May 1, 2017 at 6:39 PM, Peter Geoghegan wrote: > On Mon, May 1, 2017 at 6:20 PM, Tom Lane wrote: >> Maybe you can fix this by assuming that your own session's advertised xmin >> is a safe upper bound on everybody else's RecentGlobalXmin. But I'm not >> sure if that rule does what you wan

[HACKERS] Row Level Security Documentation

2017-05-11 Thread Rod Taylor
I think the biggest piece missing is something to summarize the giant blocks of text. Attached is a table that has commands and policy types, and a "yes" if it applies. -- Rod Taylor diff --git a/doc/src/sgml/ref/create_policy.sgml b/doc/src/sgml/ref/create_policy.sgml index 3b24e5e95e..c737f9e8

Re: [HACKERS] Cached plans and statement generalization

2017-05-11 Thread Bruce Momjian
On Thu, May 11, 2017 at 10:41:45PM +0300, Konstantin Knizhnik wrote: > This is why I have provided second implementation which replace > literals with parameters after raw parsing. Certainly it is slower > than first approach. But still provide significant advantage in > performance: more than two

Re: [HACKERS] snapbuild woes

2017-05-11 Thread Andres Freund
On 2017-05-11 14:54:26 -0700, Andres Freund wrote: > On 2017-05-11 14:51:55 -0700, wrote: > > On 2017-05-08 00:10:12 -0700, Andres Freund wrote: > > > I plan to commit the next pending patch after the back branch releases > > > are cut - it's an invasive fix and the issue doesn't cause corruption

Re: [HACKERS] [Proposal] Allow users to specify multiple tables in VACUUM commands

2017-05-11 Thread Tom Lane
"Bossart, Nathan" writes: > Looking forward to any feedback that you have. You probably won't get much in the near term, because we're in stabilize-the-release mode not develop-new-features mode. Please add your patch to the pending commitfest https://commitfest.postgresql.org/14/ so that we reme

Re: [HACKERS] Row Level Security Documentation

2017-05-11 Thread Rod Taylor
Of course, better thoughts appear immediately after hitting the send button. This version of the table attempts to stipulate which section of the process the rule applies to. On Thu, May 11, 2017 at 8:40 PM, Rod Taylor wrote: > I think the biggest piece missing is something to summarize the gia

Re: [HACKERS] [POC] hash partitioning

2017-05-11 Thread Robert Haas
On Thu, May 11, 2017 at 12:02 PM, Dilip Kumar wrote: > We need to add PARTITION_STRATEGY_HASH as well, we don't support NULL > for hash also, right? I think it should. Actually, I think that not supporting nulls for range partitioning may have been a fairly bad decision. -- Robert Haas Enterpr

Re: [HACKERS] [Proposal] Allow users to specify multiple tables in VACUUM commands

2017-05-11 Thread Bossart, Nathan
On 5/11/17, 6:32 PM, "Tom Lane" wrote: > You probably won't get much in the near term, because we're in > stabilize-the-release mode not develop-new-features mode. > Please add your patch to the pending commitfest > https://commitfest.postgresql.org/14/ > so that we remember to look at it when the

Re: [HACKERS] Bug in pg_dump --table and --exclude-table for declarative partition table handling.

2017-05-11 Thread Robert Haas
On Thu, May 11, 2017 at 9:33 AM, Tom Lane wrote: > Uh ... what in that is creating the already-extant parent? /me looks embarrassed. Never mind. I didn't read what you wrote carefully enough. >> I think one answer to the original complaint might be to add a new >> flag to pg_dump, something li

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-05-11 Thread Robert Haas
On Wed, May 10, 2017 at 8:39 PM, Masahiko Sawada wrote: > Currently, the relation extension lock is implemented using > heavyweight lock manager and almost functions (except for > brin_page_cleanup) using LockRelationForExntesion use it with > ExclusiveLock mode. But actually it doesn't need multi

Re: [HACKERS] Addition of pg_dump --no-publications

2017-05-11 Thread Michael Paquier
On Thu, May 11, 2017 at 3:19 PM, Michael Paquier wrote: > I imagine that pg_dump -s would be the basic operation that users > would do first before creating a subcription on a secondary node, but > what I find surprising is that publications are dumped by default. I > don't find confusing that tho

Re: [HACKERS] [POC] hash partitioning

2017-05-11 Thread Amit Langote
On 2017/05/12 10:42, Robert Haas wrote: > On Thu, May 11, 2017 at 12:02 PM, Dilip Kumar wrote: >> We need to add PARTITION_STRATEGY_HASH as well, we don't support NULL >> for hash also, right? > > I think it should. > > Actually, I think that not supporting nulls for range partitioning may > hav

Re: [HACKERS] [Proposal] Allow users to specify multiple tables in VACUUM commands

2017-05-11 Thread Michael Paquier
On Fri, May 12, 2017 at 9:47 AM, Bossart, Nathan wrote: > Attached is a more complete first attempt at adding this functionality. I > added two node types: one for parsing the “relation and columns” list in the > grammar, and one for holding the relation information we need for each call > to

  1   2   >