Re: [HACKERS] Merge join for GiST

2017-04-12 Thread Jeff Davis
On Wed, Apr 12, 2017 at 10:44 PM, Andrew Borodin wrote: >> How do you think we should proceed? Which projects do you think should >> eventually be in core, versus which are fine as extensions? > > Some points in favor of Range joins via nbtree: My patch doesn't require indexes, it can sort the in

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-12 Thread Kyotaro HORIGUCHI
I'd like to put a supplimentary explanation. At Tue, 11 Apr 2017 17:38:12 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170411.173812.133964522.horiguchi.kyot...@lab.ntt.co.jp> > Sorry, what I have just sent was broken. > > At Tue, 11 Apr 2017 17:33:41 +0900 (Tokyo Standard Time),

Re: [HACKERS] Merge join for GiST

2017-04-12 Thread Andrew Borodin
2017-04-13 7:01 GMT+05:00 Jeff Davis : > On Tue, Apr 11, 2017 at 8:35 AM, Alexander Korotkov > wrote: >> On Tue, Apr 11, 2017 at 5:46 PM, Jeff Davis wrote: >>> Do you have a sense of how this might compare with range merge join? >> >> >> If you have GiST indexes over ranges for both sides of join

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-12 Thread Pavan Deolasee
On Wed, Apr 12, 2017 at 10:42 PM, Robert Haas wrote: > On Tue, Apr 11, 2017 at 1:20 PM, Pavan Deolasee > > > 5. Added code to set a CLEAR pointer to a WARM pointer when we know that > the > > entire chain is WARM. This should address the workload Dilip ran and > found > > regression (I don't th

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-12 Thread Pavan Deolasee
On Thu, Apr 13, 2017 at 2:04 AM, Peter Geoghegan wrote: > On Wed, Apr 12, 2017 at 10:12 AM, Robert Haas > wrote: > >> I may have missed something, but there is no intention to ignore known > >> regressions/reviews. Of course, I don't think that every regression > will be > >> solvable, like if y

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Andres Freund
On April 12, 2017 9:58:12 PM PDT, Noah Misch wrote: >On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: >> On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: >> > On 4/12/17 02:31, Noah Misch wrote: >> > >>> But I hope you mean to commit these snapbuild patches before >the postgre

Re: [HACKERS] Logical replication and inheritance

2017-04-12 Thread Noah Misch
On Wed, Apr 12, 2017 at 11:02:44AM -0400, Peter Eisentraut wrote: > On 4/9/17 22:16, Noah Misch wrote: > > On Wed, Apr 05, 2017 at 08:25:56AM -0400, Peter Eisentraut wrote: > >> After thinking about it some more, I think the behavior we want would be > >> that changes to inheritance would reflect i

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Noah Misch
On Wed, Apr 12, 2017 at 10:21:51AM -0700, Andres Freund wrote: > On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > > On 4/12/17 02:31, Noah Misch wrote: > > >>> But I hope you mean to commit these snapbuild patches before the > > >>> postgres 10 > > >>> release? As far as I know, logical re

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

2017-04-12 Thread Noah Misch
On Tue, Apr 11, 2017 at 02:28:44AM +0900, Fujii Masao wrote: > src/backend/replication/logical/launcher.c > * Worker started and attached to our shmem. This check is safe > * because only launcher ever starts the workers, so nobody can steal > * the worker slot.

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-12 Thread Michael Paquier
On Tue, Apr 11, 2017 at 5:38 PM, Kyotaro HORIGUCHI wrote: > Sorry, what I have just sent was broken. You can use PROVE_TESTS when running make check to select a subset of tests you want to run. I use that all the time when working on patches dedicated to certain code paths. >> - Relation has new

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-12 Thread Amit Kapila
On Wed, Apr 12, 2017 at 10:30 PM, Tom Lane wrote: > Amit Kapila writes: >> On Wed, Apr 12, 2017 at 12:40 AM, Tom Lane wrote: >>> Anyone want to draft a patch for this? > >> Please find patch attached based on above discussion. > > This patch seems fairly incomplete: you can't just whack around m

Re: [HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-12 Thread Amit Kapila
On Thu, Apr 13, 2017 at 1:05 AM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Apr 12, 2017 at 2:41 PM, Tom Lane wrote: >>> This is 100% wrong. It's failing to recurse into the subexpressions of >>> the SubPlan, which could very easily include parallel-unsafe function >>> calls. > >> My unde

Re: [HACKERS] Remove pg_stat_progress_vacuum from Table 28.2

2017-04-12 Thread Amit Langote
On 2017/04/13 12:11, Fujii Masao wrote: > On Wed, Apr 12, 2017 at 10:32 AM, Amit Langote > wrote: >> On 2017/04/12 0:22, Fujii Masao wrote: >>> On Fri, Apr 7, 2017 at 9:53 AM, Amit Langote >>> wrote: On 2017/04/07 0:56, Fujii Masao wrote: > On Tue, Apr 4, 2017 at 10:19 AM, Amit Langote >

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-12 Thread Michael Paquier
On Thu, Apr 13, 2017 at 12:28 PM, Fujii Masao wrote: > On Thu, Apr 13, 2017 at 5:25 AM, Peter Eisentraut > wrote: >> On 4/12/17 09:55, Fujii Masao wrote: >>> To fix this issue, we should terminate walsender for logical replication >>> before shutdown checkpoint starts. Of course walsender for phy

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-12 Thread Craig Ringer
On 13 April 2017 at 01:57, Stas Kelvich wrote: > However I think it worth of quick research whether it is possible to create > special > code path for COPY in which errors don’t cancel transaction. Not really. Anything at any layer of the system expects to be able to ERROR: * datatype input fu

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-12 Thread Fujii Masao
On Thu, Apr 13, 2017 at 5:25 AM, Peter Eisentraut wrote: > On 4/12/17 09:55, Fujii Masao wrote: >> To fix this issue, we should terminate walsender for logical replication >> before shutdown checkpoint starts. Of course walsender for physical >> replication still needs to keep running until shutdo

Re: [HACKERS] Remove pg_stat_progress_vacuum from Table 28.2

2017-04-12 Thread Fujii Masao
On Wed, Apr 12, 2017 at 10:32 AM, Amit Langote wrote: > On 2017/04/12 0:22, Fujii Masao wrote: >> On Fri, Apr 7, 2017 at 9:53 AM, Amit Langote >> wrote: >>> On 2017/04/07 0:56, Fujii Masao wrote: On Tue, Apr 4, 2017 at 10:19 AM, Amit Langote wrote: > It seems pg_stat_progress_vacuu

[HACKERS] Different table schema in logical replication crashes

2017-04-12 Thread Euler Taveira
Hi, If a certain table has different schemas and the subscriber table has an unmatched column with a not null constraint, the logical replication crashes with the above stack trace. -- publisher CREATE TABLE test (a integer, b varchar not null, c numeric not null, PRIMARY KEY(a)); -- subscriber C

Re: [HACKERS] delta relations in AFTER triggers

2017-04-12 Thread Corey Huinker
> > Great. Thanks. I wonder if there is some way we can automatically > include code fragments in the documentation without keeping them in > sync manually. > > In whatever extra docs you add, could you include an example of an INSERT ON CONFLICT, and potentially a CTE query that does two operati

Re: [HACKERS] pg_upgrade vs extension upgrades

2017-04-12 Thread Bruce Momjian
On Wed, Apr 12, 2017 at 06:59:32PM -0400, Robert Haas wrote: > On Wed, Apr 12, 2017 at 6:30 PM, Peter Eisentraut > > If I restore a dump into another instance, I need to upgrade all my > > extensions to that installations's versions, no? That's not particular > > to pg_upgrade. > > No, it's an op

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-12 Thread Michael Paquier
On Thu, Apr 13, 2017 at 6:37 AM, Álvaro Hernández Tortosa wrote: > By looking at the them, and unless I'm missing something, I don't see > how the extra information for the future implementation of channel binding > would be added (without changing the protocol). Relevant part is: > > The mess

Re: [HACKERS] Interval for launching the table sync worker

2017-04-12 Thread Masahiko Sawada
On Wed, Apr 12, 2017 at 11:46 PM, Peter Eisentraut wrote: > On 4/12/17 00:48, Masahiko Sawada wrote: >> On Wed, Apr 12, 2017 at 1:28 PM, Peter Eisentraut >>> Perhaps instead of a global last_start_time, we store a per relation >>> last_start_time in SubscriptionRelState? >> >> I was thinking the s

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-12 Thread Michael Paquier
On Thu, Apr 13, 2017 at 2:34 AM, Heikki Linnakangas wrote: > On 04/11/2017 02:32 PM, Álvaro Hernández Tortosa wrote: >> >> So I still see your proposal more awkward and less clear, mixing >> things that are separate. But again, your choice :) > > > So, here's my more full-fledged proposal. >

Re: [HACKERS] Tab completion support for ALTER SUBSCRIPTION REFRESH PUBLICATION

2017-04-12 Thread Fujii Masao
On Thu, Apr 13, 2017 at 11:05 AM, Masahiko Sawada wrote: > On Wed, Apr 12, 2017 at 11:48 PM, Fujii Masao wrote: >> On Wed, Apr 12, 2017 at 5:12 PM, Masahiko Sawada >> wrote: >>> Hi, >>> >>> I've attached a patch for $subject. Please check it. >> >> + COMPLETE_WITH_LIST8("WITH", "CONNECTION", "S

Re: [HACKERS] SUBSCRIPTIONS and pg_upgrade

2017-04-12 Thread Peter Eisentraut
On 4/11/17 22:16, Peter Eisentraut wrote: > On 4/10/17 13:58, Peter Eisentraut wrote: >> Proposal: Dump subscriptions if running as superuser. If not, check if >> there are subscriptions and warn about that. Remove current pg_dump >> --include-subscriptions option. > > Patch for this part. And

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Michael Paquier
On Thu, Apr 13, 2017 at 3:21 AM, Robert Haas wrote: > On Wed, Apr 12, 2017 at 2:09 PM, Heikki Linnakangas wrote: >> Yes, we need to nail down the protocol and \password before beta. I am >> working on them now. > > Good to hear. FWIW, there are patches for each issue. -- Michael -- Sent via

Re: [HACKERS] Tab completion support for ALTER SUBSCRIPTION REFRESH PUBLICATION

2017-04-12 Thread Masahiko Sawada
On Wed, Apr 12, 2017 at 11:48 PM, Fujii Masao wrote: > On Wed, Apr 12, 2017 at 5:12 PM, Masahiko Sawada > wrote: >> Hi, >> >> I've attached a patch for $subject. Please check it. > > + COMPLETE_WITH_LIST8("WITH", "CONNECTION", "SET PUBLICATION", "ENABLE", > + "DISABLE", "OWNER TO", "RENAME TO",

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-04-12 Thread David Rowley
On 13 April 2017 at 11:22, Peter Eisentraut wrote: > Is this patch considered ready for review as a backpatch candidate? Yes, however, the v5 patch is based on master. The v4 patch should apply to 9.6. Diffing the two patches I see another tiny change to a comment, of which I think needs re-worde

Re: [HACKERS] Merge join for GiST

2017-04-12 Thread Jeff Davis
On Tue, Apr 11, 2017 at 8:35 AM, Alexander Korotkov wrote: > On Tue, Apr 11, 2017 at 5:46 PM, Jeff Davis wrote: >> Do you have a sense of how this might compare with range merge join? > > > If you have GiST indexes over ranges for both sides of join, then this > method could be used for range joi

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-12 Thread Amit Langote
On 2017/04/13 6:22, Robert Haas wrote: > On Wed, Apr 12, 2017 at 3:29 PM, Stephen Frost wrote: >> I'm not following what you're getting at here. >> >> There's already a constraint on the table, and ALTER TABLE ONLY doesn't >> say anything about what happens later on (certainly it doesn't make new

Re: [HACKERS] Function to control physical replication slot

2017-04-12 Thread Andres Freund
On 2017-04-12 20:15:52 -0400, Peter Eisentraut wrote: > On 4/11/17 05:15, Magnus Hagander wrote: > > Is there a particular reason we don't have a function to *set* the > > restart_lsn of a replication slot, other than to drop it and recreate it? > > I suppose there could be lots of problems if the

Re: [HACKERS] Partitioned tables and relfilenode

2017-04-12 Thread Amit Langote
On 2017/04/13 0:36, Robert Haas wrote: > On Tue, Apr 11, 2017 at 10:15 PM, Amit Langote > wrote: >> Alright. So I made it into two patches instead: 0001 fixes the bug that >> validateCheckConstraint() tries to scan partitioned tables and 0002 makes >> trying to convert a partitioned table to a vi

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Andres Freund
Hi, On 2017-03-03 01:30:11 +0100, Petr Jelinek wrote: > From 7d5b48c8cb80e7c867b2096c999d08feda50b197 Mon Sep 17 00:00:00 2001 > From: Petr Jelinek > Date: Fri, 24 Feb 2017 21:39:03 +0100 > Subject: [PATCH 1/5] Reserve global xmin for create slot snasphot export > > Otherwise the VACUUM or pruni

Re: [HACKERS] Function to control physical replication slot

2017-04-12 Thread Peter Eisentraut
On 4/11/17 05:15, Magnus Hagander wrote: > Is there a particular reason we don't have a function to *set* the > restart_lsn of a replication slot, other than to drop it and recreate it? I suppose there could be lots of problems if the LSN you specify isn't valid. And it might be hard to determine

Re: [HACKERS] pg_upgrade vs extension upgrades

2017-04-12 Thread Peter Eisentraut
On 4/12/17 18:59, Robert Haas wrote: > I do think there might be some value in a tool that looked for old > extensions and tried to update them, but I'm not sure it should be > pg_dump. This reminds me a bit of the problem of upgrading all collations after an upgrade. Perhaps we can find similar

Re: [HACKERS] pg_statistic_ext.staenabled might not be the best column name

2017-04-12 Thread Tomas Vondra
On 04/12/2017 03:36 PM, David Rowley wrote: "stakind" seems like a better name. I'd have personally gone with "statype" but pg_statistic already thinks stakind is better. +1 to stakind -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote D

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-04-12 Thread Peter Eisentraut
Is this patch considered ready for review as a backpatch candidate? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your s

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-12 Thread Peter Eisentraut
On 4/12/17 09:50, Bruce Momjian wrote: > On Wed, Apr 12, 2017 at 01:31:51PM +0200, Magnus Hagander wrote: >> I think that only leaves the change to the javascript code that Bruce sent. >> Let's see if we can figure out a way to do that one without requiring >> javascript, but after that we have cov

Re: [HACKERS] pg_upgrade vs extension upgrades

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 6:30 PM, Peter Eisentraut wrote: > On 4/10/17 11:30, Magnus Hagander wrote: >> After you've run pg_upgrade, you have to loop through all your databases >> and do an "ALTER EXTENSION abc UPDATE" once for each extension. >> >> Is there a reason we shouldn't have pg_upgrade em

Re: [HACKERS] Re: Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-12 Thread Andres Freund
On 2017-04-05 09:39:37 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-05 02:47:55 -0400, Noah Misch wrote: > >> [Action required within three days. This is a generic notification.] > > > I've a very preliminary patch. I'd like to only start polishing it up > > once the code freeze

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-12 Thread Peter Eisentraut
On 4/12/17 15:43, Peter Eisentraut wrote: > On 4/12/17 07:31, Magnus Hagander wrote: >> Once difference I notice is that for example the "note boxes" are no >> longer centered, but they do now get the new formatting. > > I have committed something for that. The issue was that the generated > HTML

Re: [HACKERS] SUBSCRIPTIONS and pg_upgrade

2017-04-12 Thread Peter Eisentraut
On 4/11/17 23:41, Noah Misch wrote: > On Tue, Apr 11, 2017 at 11:21:24PM -0400, Peter Eisentraut wrote: >> On 4/9/17 22:16, Noah Misch wrote: >>> [Action required within three days. This is a generic notification.] >> >> Patches have been posted. Discussion is still going on a bit. > > By what d

Re: [HACKERS] pg_upgrade vs extension upgrades

2017-04-12 Thread Peter Eisentraut
On 4/10/17 11:30, Magnus Hagander wrote: > After you've run pg_upgrade, you have to loop through all your databases > and do an "ALTER EXTENSION abc UPDATE" once for each extension. > > Is there a reason we shouldn't have pg_upgrade emit a script that does > this, similar to how it emits a script

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-12 Thread Álvaro Hernández Tortosa
On 12/04/17 19:34, Heikki Linnakangas wrote: On 04/11/2017 02:32 PM, Álvaro Hernández Tortosa wrote: So I still see your proposal more awkward and less clear, mixing things that are separate. But again, your choice :) So, here's my more full-fledged proposal. The first patch refactors

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 3:29 PM, Stephen Frost wrote: > I'm not following what you're getting at here. > > There's already a constraint on the table, and ALTER TABLE ONLY doesn't > say anything about what happens later on (certainly it doesn't make new > tables created with 'LIKE' have bits omitte

Re: [HACKERS] pg_statistic_ext.staenabled might not be the best column name

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 9:36 AM, David Rowley wrote: > I'd been thinking that staenabled is not the most suitable column name > for storing the types of statistics that are defined for the extended > statistics. +1. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgre

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-12 Thread Peter Geoghegan
On Wed, Apr 12, 2017 at 10:12 AM, Robert Haas wrote: >> I may have missed something, but there is no intention to ignore known >> regressions/reviews. Of course, I don't think that every regression will be >> solvable, like if you run a CPU-bound workload, setting it up in a way such >> that you r

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-12 Thread Peter Eisentraut
On 4/12/17 09:55, Fujii Masao wrote: > To fix this issue, we should terminate walsender for logical replication > before shutdown checkpoint starts. Of course walsender for physical > replication still needs to keep running until shutdown checkpoint ends, > though. Can we turn it into a kind of re

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 3:29 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Wed, Apr 12, 2017 at 6:29 AM, Amit Langote >> wrote: >> > Actually, p1 is a partitioned table, so the error. And I realize that >> > that's a wrong behavior. Currently the check is perform

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-04-12 Thread Peter Eisentraut
On 4/12/17 07:31, Magnus Hagander wrote: > Once difference I notice is that for example the "note boxes" are no > longer centered, but they do now get the new formatting. I have committed something for that. The issue was that the generated HTML contained hard-coded style attributes. -- Peter E

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Stephen Frost
Andrew, * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > On 04/12/2017 08:40 AM, Andrew Dunstan wrote: > > On 04/12/2017 01:27 AM, Craig Ringer wrote: > >> BTW, I suggest adding --timer to our default PROVE_FLAGS, so we can > >> collect more data from the buildfarm on what takes a while.

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 4:38 PM, Claudio Freire wrote: > In essence, the patch as it is proposed, doesn't *need* a binary > search, because the segment list can only grow up to 15 segments at > its biggest, and that's a size small enough that linear search will > outperform (or at least perform as

Re: [HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-12 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 12, 2017 at 2:41 PM, Tom Lane wrote: >> This is 100% wrong. It's failing to recurse into the subexpressions of >> the SubPlan, which could very easily include parallel-unsafe function >> calls. > My understanding (apparently flawed?) is that the parallel_safe f

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Andrew Dunstan
On 04/12/2017 08:40 AM, Andrew Dunstan wrote: > > On 04/12/2017 01:27 AM, Craig Ringer wrote: >> BTW, I suggest adding --timer to our default PROVE_FLAGS, so we can >> collect more data from the buildfarm on what takes a while. Sample >> output: >> > > I'll add that to the commandline the buildfa

Re: [HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 2:41 PM, Tom Lane wrote: > While poking at the question of parallel_safe marking for Plans, > I noticed that max_parallel_hazard_walker() does this: > > /* We can push the subplans only if they are parallel-safe. */ > else if (IsA(node, SubPlan)) > return !(

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-12 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Apr 12, 2017 at 6:29 AM, Amit Langote > wrote: > > Actually, p1 is a partitioned table, so the error. And I realize that > > that's a wrong behavior. Currently the check is performed using only the > > relkind, which is bogus. Spec

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-12 Thread Robert Haas
On Thu, Apr 6, 2017 at 7:30 AM, Rahila Syed wrote: > Thanks a lot for testing and reporting this. Please find attached an updated > patch with the fix. The patch also contains a fix > regarding operator used at the time of creating expression as default > partition constraint. This was notified of

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-12 Thread Andreas Karlsson
On 04/12/2017 05:00 PM, Andreas Karlsson wrote: Looked at this an option 1 seems simple enough if I am not missing something. I might hack something up later tonight. Either way I think this improvement can be done separately from the proposed replacement of the catalog header files. Trying to fi

Re: [HACKERS] Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-12 Thread Andres Freund
On 2017-04-11 17:42:42 -0400, Tom Lane wrote: > Now, that old behavior matches what you got in the RangeFunction case: > > regression96=# select * from int4_tbl, cast(case when f1>0 then > generate_series(1,2) else null end as int); > f1 | int4 > -+-- >0 |

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-12 Thread Robert Haas
On Thu, Apr 6, 2017 at 1:17 AM, Rushabh Lathia wrote: > I like the idea about having DEFAULT partition for the range partition. With > the > way partition is designed it can have holes into range partition. I think > DEFAULT > for the range partition is a good idea, generally when the range having

[HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-12 Thread Tom Lane
While poking at the question of parallel_safe marking for Plans, I noticed that max_parallel_hazard_walker() does this: /* We can push the subplans only if they are parallel-safe. */ else if (IsA(node, SubPlan)) return !((SubPlan *) node)->parallel_safe; This is 100% wrong. It's

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 9:41 AM, Jeevan Ladhe wrote: > I have checked for NULLs too, and the default partition can be created even > when there are partitions for each TRUE, FALSE and NULL. > > Consider the example below: > > postgres=# CREATE TABLE list_partitioned ( > a bool > ) PARTITION BY

Re: [HACKERS] Undefined psql variables

2017-04-12 Thread Pavel Stehule
2017-04-12 17:05 GMT+02:00 Robert Haas : > On Sun, Apr 2, 2017 at 3:56 PM, Tom Lane wrote: > > So my view of this is that "send the expression to the server" ought > > to be just one option for \if, not the only way to do it. > > I heartily agree. There should be some kind of client-side express

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 2:09 PM, Heikki Linnakangas wrote: > That is very much appreciated! You writing a second implementation of the > client-side support (libpq being the first) is very, very helpful, to > validate that the protocol is sane, unambiguous, and adequately documented. +1. > Yes,

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 6:29 AM, Amit Langote wrote: > Actually, p1 is a partitioned table, so the error. And I realize that > that's a wrong behavior. Currently the check is performed using only the > relkind, which is bogus. Specifying ONLY should cause an error only when > the table has part

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Heikki Linnakangas
On 04/12/2017 08:38 PM, Álvaro Hernández Tortosa wrote: - Even though I don't really care about SCRAM, and without having any prior knowledge about SCRAM, I volunteered some time ago to study SCRAM, give a lightning talk about SCRAM and later write a client implementation for the jdbc driver. And

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-12 Thread Stas Kelvich
> On 12 Apr 2017, at 20:23, Robert Haas wrote: > > On Wed, Apr 12, 2017 at 1:18 PM, Nicolas Barbier > wrote: >> 2017-04-11 Robert Haas : >>> If the data quality is poor (say, 50% of lines have errors) it's >>> almost impossible to avoid runaway XID consumption. >> >> Yup, that seems difficult

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Álvaro Hernández Tortosa
On 12/04/17 18:38, Robert Haas wrote: On Tue, Apr 11, 2017 at 9:20 AM, Álvaro Hernández Tortosa wrote: LOL. Do you really want a half-baked Java programmer writing a patch in C for PostgreSQL? I once tried that and Magnus said my code was Java code that happened to compile with a C compi

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-12 Thread Heikki Linnakangas
On 04/11/2017 02:32 PM, Álvaro Hernández Tortosa wrote: So I still see your proposal more awkward and less clear, mixing things that are separate. But again, your choice :) So, here's my more full-fledged proposal. The first patch refactors libpq code, by moving the responsibility of re

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-12 Thread Andres Freund
On 2017-04-12 10:12:47 -0400, Tom Lane wrote: > Andres mentioned, and I've confirmed locally, that a large chunk of > initdb's runtime goes into regprocin's brute-force lookups of function > OIDs from function names. The recent discussion about cutting TAP test > time prompted me to look into that

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Alexander Kuzmenkov
On 12.04.2017 12:29, Alexander Korotkov wrote: That's a cool feature for FTS users! Please, register this patch to the next commitfest. I've added this to the 2017-07 commitfest: https://commitfest.postgresql.org/14/1117/ Also, what is planning overhead of this patch? That's worth t

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 1:18 PM, Nicolas Barbier wrote: > 2017-04-11 Robert Haas : >> There's a nasty trade-off here between XID consumption (and the >> aggressive vacuums it eventually causes) and preserving performance in >> the face of errors - e.g. if you make k = 100,000 you consume 100x >> f

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Andres Freund
On 2017-04-12 11:03:57 -0400, Peter Eisentraut wrote: > On 4/12/17 02:31, Noah Misch wrote: > >>> But I hope you mean to commit these snapbuild patches before the postgres > >>> 10 > >>> release? As far as I know, logical replication is still very broken > >>> without > >>> them (or at least som

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-12 Thread Nicolas Barbier
2017-04-11 Robert Haas : > There's a nasty trade-off here between XID consumption (and the > aggressive vacuums it eventually causes) and preserving performance in > the face of errors - e.g. if you make k = 100,000 you consume 100x > fewer XIDs than if you make k = 1000, but you also have 100x th

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Álvaro Hernández Tortosa
On 12/04/17 18:09, Tom Lane wrote: Heikki Linnakangas writes: On 04/12/2017 06:26 PM, Bruce Momjian wrote: How does it do that? Good question, crypto magic? I don't know the details, but the basic idea is that you extract a blob of data that uniquely identifies the TLS connection. Using som

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 1:20 PM, Pavan Deolasee wrote: > I don't know why you say that regressions are not addressed. Here are a few > things I did to address the regressions/reviews/concerns, apart from fixing > all the bugs discovered, but please let me know if there are things I've not > addres

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> ... which the user can't tell apart from having fat-fingered the password, >> I suppose? Doesn't sound terribly friendly. A report of a certificate >> mismatch is far more likely to lead people to realize there's a MITM. > We mig

Re: [HACKERS] index-only count(*) for indexes supporting bitmap scans

2017-04-12 Thread Alexander Kuzmenkov
On 12.04.2017 17:24, Tom Lane wrote: TBH, I'm not sure you need to do any of that work. Have you got evidence that the planner will fail to choose the right plan regardless? I'm particularly unconvinced that choose_bitmap_and is a critical problem, because once you're into having to AND multiple

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-12 Thread Tom Lane
Amit Kapila writes: > On Wed, Apr 12, 2017 at 12:40 AM, Tom Lane wrote: >> Anyone want to draft a patch for this? > Please find patch attached based on above discussion. This patch seems fairly incomplete: you can't just whack around major data structures like PlannedStmt and PlannerGlobal with

Re: [HACKERS] the need to finish

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 11:41 AM, Simon Riggs wrote: > On 12 April 2017 at 16:26, Tom Lane wrote: >> Erik Rijkers writes: >>> Logical replication emits logmessages like these: >>> DETAIL: 90 transactions need to finish. >>> I would prefer the line to be more terse: >>> DETAIL: 90 transactions

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Stephen Frost
Tom, all, * Tom Lane (t...@sss.pgh.pa.us) wrote: > ... which the user can't tell apart from having fat-fingered the password, > I suppose? Doesn't sound terribly friendly. A report of a certificate > mismatch is far more likely to lead people to realize there's a MITM. We might be able to impro

Re: [HACKERS] Possible problem in Custom Scan API

2017-04-12 Thread Dmitry Ivanov
Tom Lane wrote: I'm coming around to the idea that it'd be better to disable physical tlists for custom scans. I've been thinking about this all along, and it seems that this is a decent decision. However, I've made a tiny bugfix patch which allows CustomScans to notify the core code that the

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 9:20 AM, Álvaro Hernández Tortosa wrote: > LOL. Do you really want a half-baked Java programmer writing a patch in > C for PostgreSQL? I once tried that and Magnus said my code was Java code > that happened to compile with a C compiler ^_^ > > Having said that,

Re: [HACKERS] Add pgstathashindex() to get hash index table statistics.

2017-04-12 Thread Robert Haas
On Wed, Apr 12, 2017 at 2:28 AM, Noah Misch wrote: > [Action required within three days. This is a generic notification.] > > The above-described topic is currently a PostgreSQL 10 open item. Robert, > since you committed the patch believed to have created it, you own this open > item. If some

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Tom Lane
Heikki Linnakangas writes: > On 04/12/2017 06:26 PM, Bruce Momjian wrote: >> How does it do that? > Good question, crypto magic? I don't know the details, but the basic > idea is that you extract a blob of data that uniquely identifies the TLS > connection. Using some OpenSSL functions, in this

Re: [HACKERS] Possible problem in Custom Scan API

2017-04-12 Thread Tom Lane
Dmitry Ivanov writes: >> Uh, no, construction of a custom plan node is entirely driven by the >> PlanCustomPath method as far as I can see. You're free to ignore what >> create_scan_plan did and insert your own tlist. > Are you sure? Even if it's true, this targetlist should still contain each

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Heikki Linnakangas
On 04/12/2017 06:26 PM, Bruce Momjian wrote: On Wed, Apr 12, 2017 at 12:13:03PM +0300, Heikki Linnakangas wrote: That said, I stand by my comment that I don't think it's the enterprises that need or want the channel binding. If they care about it, they have already put certificate validation in

Re: [HACKERS] error handling in RegisterBackgroundWorker

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 10:13 PM, Noah Misch wrote: > On Tue, Apr 11, 2017 at 11:33:34AM -0400, Tom Lane wrote: >> Peter Eisentraut writes: >> > I think there is no clear agreement here, and no historically consistent >> > behavior. I'm prepared to let it go and cross it off the list of open >>

Re: [HACKERS] the need to finish

2017-04-12 Thread Simon Riggs
On 12 April 2017 at 16:26, Tom Lane wrote: > Erik Rijkers writes: >> Logical replication emits logmessages like these: >> DETAIL: 90 transactions need to finish. >> DETAIL: 87 transactions need to finish. >> DETAIL: 70 transactions need to finish. > >> Could we get rid of that 'need'? It str

Re: [HACKERS] Partitioned tables and relfilenode

2017-04-12 Thread Robert Haas
On Tue, Apr 11, 2017 at 10:15 PM, Amit Langote wrote: > Alright. So I made it into two patches instead: 0001 fixes the bug that > validateCheckConstraint() tries to scan partitioned tables and 0002 makes > trying to convert a partitioned table to a view a user error. Committed together, after up

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Simon Riggs
On 3 March 2017 at 00:30, Petr Jelinek wrote: >> 0004 - Changes handling of the xl_running_xacts in initial snapshot >> build to what I wrote above and removes the extra locking from >> LogStandbySnapshot introduced by logical decoding. This seems OK and unlikely to have wider impact. The "race

Re: [HACKERS] GCC 7 warnings

2017-04-12 Thread Tom Lane
Peter Eisentraut writes: > On 4/12/17 00:12, Tom Lane wrote: >> Now a human can see that saved_timeval.tv_usec must be 0..99, so >> that the %d format item must always emit exactly 3 characters, which >> means that really 5 bytes would be enough. I wouldn't expect a >> compiler to know that,

Re: [HACKERS] the need to finish

2017-04-12 Thread Tom Lane
Erik Rijkers writes: > Logical replication emits logmessages like these: > DETAIL: 90 transactions need to finish. > DETAIL: 87 transactions need to finish. > DETAIL: 70 transactions need to finish. > Could we get rid of that 'need'? It strikes me as a bit off; something > that people would

Re: [HACKERS] Some thoughts about SCRAM implementation

2017-04-12 Thread Bruce Momjian
On Wed, Apr 12, 2017 at 12:13:03PM +0300, Heikki Linnakangas wrote: > >That said, I stand by my comment that I don't think it's the enterprises > >that need or want the channel binding. If they care about it, they have > >already put certificate validation in place, and it won't buy them anything.

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Tom Lane
Mithun Cy writes: > I have tried to optimize the tests maintaining the same coverage we were > able to get with it. This patch looks good to me: on my workstation, it reduces the total runtime of the parallel regression tests from ~20.5 to ~16.5 seconds. I concur that it doesn't look like it woul

Re: [HACKERS] GCC 7 warnings

2017-04-12 Thread Peter Eisentraut
On 4/12/17 00:12, Tom Lane wrote: > The change in setup_formatted_log_time seems a bit weird: > > - charmsbuf[8]; > + charmsbuf[10]; > > The associated call is > > sprintf(msbuf, ".%03d", (int) (saved_timeval.tv_usec / 1000)); > > Now a human can see that s

Re: [HACKERS] Logical replication and inheritance

2017-04-12 Thread Robert Haas
On Wed, Apr 5, 2017 at 8:25 AM, Peter Eisentraut wrote: > After thinking about it some more, I think the behavior we want would be > that changes to inheritance would reflect in the publication membership. > So if you have a partitioned table, adding more partitions over time > would automaticall

[HACKERS] the need to finish

2017-04-12 Thread Erik Rijkers
Logical replication emits logmessages like these: DETAIL: 90 transactions need to finish. DETAIL: 87 transactions need to finish. DETAIL: 70 transactions need to finish. Could we get rid of that 'need'? It strikes me as a bit off; something that people would say but not a mechanical messag

Re: [HACKERS] Undefined psql variables

2017-04-12 Thread Robert Haas
On Sun, Apr 2, 2017 at 3:56 PM, Tom Lane wrote: > So my view of this is that "send the expression to the server" ought > to be just one option for \if, not the only way to do it. I heartily agree. There should be some kind of client-side expression language, and one thing it should allow is call

Re: [HACKERS] snapbuild woes

2017-04-12 Thread Peter Eisentraut
On 4/12/17 02:31, Noah Misch wrote: >>> But I hope you mean to commit these snapbuild patches before the postgres 10 >>> release? As far as I know, logical replication is still very broken without >>> them (or at least some of that set of 5 patches - I don't know which ones >>> are essential and w

  1   2   >