Re: [HACKERS] [COMMITTERS] pgsql: Collect and use multi-column dependency stats

2017-04-06 Thread David Rowley
On 6 April 2017 at 18:03, Kyotaro HORIGUCHI wrote: > At Thu, 6 Apr 2017 13:10:48 +1200, David Rowley > wrote in > >> On 6 April 2017 at 13:05, David Rowley wrote: >> > I tested with the attached, and it does not seem to hurt planner >> > performance executing: >> >> Here's it again, this time

Re: [HACKERS] Declarative partitioning vs. information_schema

2017-04-06 Thread Noah Misch
On Wed, Jan 25, 2017 at 01:19:00PM -0500, Robert Haas wrote: > On Wed, Jan 25, 2017 at 1:04 PM, Peter Eisentraut > wrote: > > On 1/18/17 2:32 PM, Robert Haas wrote: > >> Unless we can find something official, I suppose we should just > >> display BASE TABLE in that case as we do in other cases. I

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-04-06 Thread Petr Jelinek
On 05/04/17 23:22, Tom Lane wrote: > Andres Freund writes: >> I'd like some input from other committers whether we want this. I'm >> somewhat doubtful, but don't have particularly strong feelings. > > I don't really want to expose the workings of the plancache at user level. > The heuristics it

[HACKERS] Duplicate usage of tablespace location?

2017-04-06 Thread Kyotaro HORIGUCHI
Hello, I noticed by the following report, PostgreSQL can share the same directory as tablespaces of two servers with different pg-versions. https://www.postgresql.org/message-id/2008148.rxBNyNRHPZ@peanuts2 8.4 checked that the tablespace location is empty, but from 9.0, the check is replaced wit

Re: [HACKERS] Declarative partitioning vs. information_schema

2017-04-06 Thread Amit Langote
On 2017/04/06 16:02, Noah Misch wrote: > On Wed, Jan 25, 2017 at 01:19:00PM -0500, Robert Haas wrote: >> On Wed, Jan 25, 2017 at 1:04 PM, Peter Eisentraut >> wrote: >>> On 1/18/17 2:32 PM, Robert Haas wrote: Unless we can find something official, I suppose we should just display BASE TAB

Re: [HACKERS] Interval for launching the table sync worker

2017-04-06 Thread Masahiko Sawada
On Thu, Apr 6, 2017 at 3:45 PM, Kyotaro HORIGUCHI wrote: > I was thinking the same. > > At Thu, 6 Apr 2017 11:33:22 +0900, Masahiko Sawada > wrote in >> Hi all, >> >> While testing table sync worker for logical replication I noticed that >> if the table sync worker of logical replication failed

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-06 Thread Masahiko Sawada
On Thu, Apr 6, 2017 at 10:51 AM, Noah Misch wrote: > On Thu, Apr 06, 2017 at 12:48:56AM +0900, Fujii Masao wrote: >> On Wed, Apr 5, 2017 at 3:45 PM, Noah Misch wrote: >> > On Mon, Dec 19, 2016 at 09:49:58PM +0900, Fujii Masao wrote: >> >> Regarding this feature, there are some loose ends. We shou

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-06 Thread Michael Paquier
On Thu, Apr 6, 2017 at 1:33 AM, Heikki Linnakangas wrote: > Attached is a new version. Notable changes since yesterday: > > * Implemented the rest of the SASLPrep, mapping some characters to spaces, > leaving out others, and checking for prohibited characters and bidirectional > strings. > > * Mov

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Craig Ringer
On 6 April 2017 at 11:50, Jim Nasby wrote: > Attached is a complete series of patches that includes the docs patch. + SPI_execute_callback is the same as + SPI_execute, except that instead of returning results + via SPITupleTable, the user-supplied callback + is used. Unlike + SPI_exec

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Craig Ringer
On 6 April 2017 at 15:38, Craig Ringer wrote: > Notes on the docs aside, I am pretty happy with this and think it's > reasonable to proceed with it for Pg 10. Actually, I'm a bit hesitant about returning a static struct that you expect callers to copy and modify. But it seems to be an issue wit

Re: [HACKERS] [COMMITTERS] pgsql: Collect and use multi-column dependency stats

2017-04-06 Thread Kyotaro HORIGUCHI
At Thu, 6 Apr 2017 18:59:35 +1200, David Rowley wrote in > On 6 April 2017 at 18:03, Kyotaro HORIGUCHI > wrote: > > At Thu, 6 Apr 2017 13:10:48 +1200, David Rowley > > wrote in > > > >> On 6 April 2017 at 13:05, David Rowley > >> wrote: > I'm not all that sure why the number of columns in

Re: [HACKERS] BRIN cost estimate

2017-04-06 Thread Emre Hasegeli
> Good point. That's wrong, but I'm confused at why you kept the: > > + *indexTotalCost += selec * numTuples * cpu_index_tuple_cost; > > at all if that's the case. All the BRIN scan is going to do is build a > bitmap of the matching ranges found. My mind was not clear when I was working on it a ye

Re: [HACKERS] Interval for launching the table sync worker

2017-04-06 Thread Kyotaro HORIGUCHI
At Thu, 6 Apr 2017 16:15:33 +0900, Masahiko Sawada wrote in > On Thu, Apr 6, 2017 at 3:45 PM, Kyotaro HORIGUCHI > wrote: > > I was thinking the same. > > > > At Thu, 6 Apr 2017 11:33:22 +0900, Masahiko Sawada > > wrote in > > > >> Hi all, > >> > >> While testing table sync worker for logica

[HACKERS] Range Merge Join v1

2017-04-06 Thread Jeff Davis
Example: Find different people using the same website at the same time: create table session(sessionid text, username text, during tstzrange); SELECT s1.username, s2.username, s1.during * s2.during FROM session s1, session s2 WHERE s1.during && s2.during AND s1.username < s2

[HACKERS] No-op case in ExecEvalConvertRowtype

2017-04-06 Thread Ashutosh Bapat
Hi, In ExecEvalConvertRowtype(), if the input row doesn't require any conversion, we simply return that row as is. 2820 /* 2821 * No-op if no conversion needed (not clear this can happen here). 2822 */ 2823 if (op->d.convert_rowtype.map == NULL) 2824 return; If the type o

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-06 Thread Kuntal Ghosh
On Wed, Apr 5, 2017 at 6:49 PM, Amit Kapila wrote: > On Wed, Apr 5, 2017 at 12:35 PM, Kuntal Ghosh > wrote: >> On Tue, Apr 4, 2017 at 11:22 PM, Tomas Vondra >>> I'm probably missing something, but I don't quite understand how these >>> values actually survive the crash. I mean, what I observed is

Re: [HACKERS] [COMMITTERS] pgsql: Collect and use multi-column dependency stats

2017-04-06 Thread David Rowley
On 6 April 2017 at 19:50, Kyotaro HORIGUCHI wrote: > At Thu, 6 Apr 2017 18:59:35 +1200, David Rowley > wrote in > >> On 6 April 2017 at 18:03, Kyotaro HORIGUCHI >> wrote: >> > At Thu, 6 Apr 2017 13:10:48 +1200, David Rowley >> > wrote in >> > >> >> On 6 April 2017 at 13:05, David Rowley

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-04-06 Thread Andrew Dunstan
On 04/05/2017 05:41 PM, Andres Freund wrote: > On 2017-04-05 17:22:34 -0400, Tom Lane wrote: >> Andres Freund writes: >>> I'd like some input from other committers whether we want this. I'm >>> somewhat doubtful, but don't have particularly strong feelings. >> I don't really want to expose the

Re: [HACKERS] Bug with pg_basebackup and 'shared' tablespace

2017-04-06 Thread Pierre Ducroquet
On Thursday, April 6, 2017 2:00:55 PM CEST Kyotaro HORIGUCHI wrote: > At Thu, 06 Apr 2017 00:59:49 +0200, Pierre Ducroquet > wrote in <2008148.rxBNyNRHPZ@peanuts2> > > But it all gets messy when we want to create a streaming standby server > > using pg_basebackup. When backuping Pg 9.5, there is n

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-04-06 Thread Andres Freund
On April 5, 2017 11:19:04 PM PDT, "Jan Michálek" wrote: >2017-04-06 0:55 GMT+02:00 Andres Freund : > >> Hi, >> >> On 2017-04-02 22:28:40 +0200, Jan Michálek wrote: >> > 2017-03-23 17:26 GMT+01:00 Pierre Ducroquet : >> > > The new status of this patch is: Waiting on Author >> > > >> > >> > Corre

Re: [HACKERS] [COMMITTERS] pgsql: Collect and use multi-column dependency stats

2017-04-06 Thread Kyotaro HORIGUCHI
At Thu, 6 Apr 2017 21:55:43 +1200, David Rowley wrote in > On 6 April 2017 at 19:50, Kyotaro HORIGUCHI > wrote: > > At Thu, 6 Apr 2017 18:59:35 +1200, David Rowley > > wrote in > > > >> On 6 April 2017 at 18:03, Kyotaro HORIGUCHI > >> wrote: > >> > At Thu, 6 Apr 2017 13:10:48 +1200, David

[HACKERS] Constraint exclusion for partitioned tables

2017-04-06 Thread Ashutosh Bapat
Hi, In relation_excluded_by_constraints(), we do not apply constraint exclusion if rte->inh is true. /* Only plain relations have constraints */ if (rte->rtekind != RTE_RELATION || rte->inh) return false; Thus every partitioned table will not benefit from the constraint exclusion,

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread Beena Emerson
Hello, On Wed, Apr 5, 2017 at 4:59 PM, Simon Riggs wrote: > On 5 April 2017 at 06:04, Beena Emerson wrote: > > I see various issues raised but not properly addressed > > 1. we would need to drop support for segment sizes < 16MB unless we > adopt a new incompatible filename format. > I think at

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-06 Thread Rahila Syed
Hello, Thanks a lot for testing and reporting this. Please find attached an updated patch with the fix. The patch also contains a fix regarding operator used at the time of creating expression as default partition constraint. This was notified offlist by Amit Langote. Thank you, Rahila Syed On

Re: [HACKERS] Logical decoding on standby

2017-04-06 Thread Craig Ringer
On 5 April 2017 at 23:25, Robert Haas wrote: > On Wed, Apr 5, 2017 at 10:32 AM, Andres Freund wrote: >> On 2017-04-05 17:18:24 +0800, Craig Ringer wrote: >>> On 5 April 2017 at 04:19, Andres Freund wrote: >>> > On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: >>> >> I'm much happier with this.

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread Beena Emerson
Hello, On Wed, Apr 5, 2017 at 6:06 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote:e format and expand the range? > > > I don't think me saying it felt a bit slow around 256 MB is a proper > technical analysis that should lead to the conclusion that that upper > limit should be 128

Re: [HACKERS] Interval for launching the table sync worker

2017-04-06 Thread Kyotaro HORIGUCHI
At Thu, 06 Apr 2017 17:02:14 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170406.170214.263553093.horiguchi.kyot...@lab.ntt.co.jp> > At Thu, 6 Apr 2017 16:15:33 +0900, Masahiko Sawada > wrote in > > On Thu, Apr 6, 2017 at 3:45 PM, Kyotaro HORIGUCHI > > wrote: > > > I was thinki

Re: [HACKERS] BRIN cost estimate

2017-04-06 Thread David Rowley
On 6 April 2017 at 20:01, Emre Hasegeli wrote: >> + /* >> + * Charge a small amount per range tuple which we expect to match to. >> This >> + * is meant to reflect the costs of manipulating the bitmap. The BRIN >> scan >> + * will set a bit for each page in the range when we find a matching >> + *

[HACKERS] subscription worker doesn't start immediately on eabled

2017-04-06 Thread Kyotaro HORIGUCHI
Hello. I found dubious behavior while playing with logical replication. When we disable a subscription, replication worker immediately stops. =# ALTER SUBSCRIPTION s1 DISABLE; On the other hand even if we enable a subscription, worker doesn't start immediately. It takes 3 minutes in the worst ca

Re: [HACKERS] [PATCH] Remove unused argument in btree_xlog_split

2017-04-06 Thread Aleksander Alekseev
Hi Robert, > Hmm. I don't see anything wrong with that, particularly, but it seems > we also don't need the distinction between XLOG_BTREE_SPLIT_L and > XLOG_BTREE_SPLIT_L_ROOT or likewise between XLOG_BTREE_SPLIT_R and > XLOG_BTREE_SPLIT_R_ROOT -- in which case I think this patch should go > a l

Re: [HACKERS] identity columns

2017-04-06 Thread Peter Eisentraut
On 4/4/17 22:53, Vitaly Burovoy wrote: > The next nitpickings to the last patch. I try to get places with > lacking of variables' initialization. > All other things seem good for me now. I'll continue to review the > patch while you're fixing the current notes. Committed with your changes (see bel

Re: [HACKERS] [COMMITTERS] pgsql: Collect and use multi-column dependency stats

2017-04-06 Thread Simon Riggs
On 5 April 2017 at 18:48, Tom Lane wrote: > Simon Riggs writes: >> Collect and use multi-column dependency stats > > The buildfarm is unhappy about the fact that this changed the API > for clauselist_selectivity(). I am not convinced that that change > was a good idea, so before telling FDW auth

Re: [HACKERS] Re: new set of psql patches for loading (saving) data from (to) text, binary files

2017-04-06 Thread Stephen Frost
Greetings, * Pavel Stehule (pavel.steh...@gmail.com) wrote: > 2017-04-06 3:34 GMT+02:00 Stephen Frost : > > Having the template not require the row/column place-holders included > > strikes me as more likely to be confusing. My initial thinking around > > this was that users who actually want ind

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-04-06 Thread Alexander Korotkov
On Sun, Apr 2, 2017 at 12:13 AM, Arseny Sher wrote: > Time is short, student's application deadline is on 3rd April. I decided > to reformulate the project scope myself. Here is the proposal: > > https://docs.google.com/document/d/1dvBETE6IJA9AcXd11XJNPsF_ > VPcDhSjy7rlsxj262l8/edit?usp=sharing >

Re: [HACKERS] [PATCH] Generic type subscripting

2017-04-06 Thread Arthur Zakirov
On 05.04.2017 16:06, Arthur Zakirov wrote: I'd like to focus on "refevalfunc" and "refnestedfunc" fields as I did earlier. I think using Oid type for them is a bad approach. "..._fetch" and "..._assign" functions in catalog is unnecessary movement to me. User of subscript of his type may think t

Re: [HACKERS] GSoC 2017 Proposal

2017-04-06 Thread Alexander Korotkov
Hello, Mark! On Mon, Apr 3, 2017 at 7:00 PM, Mark Rofail wrote: > Kindly find my proposal attached to this email. > I'd like to ask what do you mean in research item number 3? 3. Making the full-table sequential scan GIN-indexable​ instead seems very > reasonable since GIN is primarily used to

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-04-06 Thread Kevin Grittner
On Thu, Apr 6, 2017 at 8:11 AM, Alexander Korotkov wrote: >> https://docs.google.com/document/d/1dvBETE6IJA9AcXd11XJNPsF_VPcDhSjy7rlsxj262l8/edit?usp=sharing > I'd love to see a comment from Andres Freund who is leading executor > performance improvements. Note that the final proposal is here:

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-06 Thread Alexander Korotkov
Hi, Alexey! On Tue, Mar 28, 2017 at 1:54 AM, Alexey Kondratov < kondratov.alek...@gmail.com> wrote: > Thank you for your responses and valuable comments! > > I have written draft proposal https://docs.google.com/document/d/1Y4mc_ > PCvRTjLsae-_fhevYfepv4sxaqwhOo4rlxvK1c/edit > > It seems that COP

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-04-06 Thread Alexander Korotkov
On Thu, Apr 6, 2017 at 2:16 AM, Andres Freund wrote: > On 2017-04-03 11:56:13 -0700, Andres Freund wrote: > > > > > +/* > > > + * Generic implementation of pg_atomic_fetch_mask_add_u32() via loop > > > + * of compare & exchange. > > > + */ > > > +static inline uint32 > > > +pg_atomic_fetch_mask_a

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-04-06 Thread Alexander Korotkov
On Thu, Apr 6, 2017 at 5:37 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Thu, Apr 6, 2017 at 2:16 AM, Andres Freund wrote: > >> On 2017-04-03 11:56:13 -0700, Andres Freund wrote: >> > Have you done x86 benchmarking? >> >> I think unless such benchmarking is done in the next 24h

Re: [HACKERS] Interval for launching the table sync worker

2017-04-06 Thread Masahiko Sawada
On Thu, Apr 6, 2017 at 9:06 PM, Kyotaro HORIGUCHI wrote: > At Thu, 06 Apr 2017 17:02:14 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI > wrote in > <20170406.170214.263553093.horiguchi.kyot...@lab.ntt.co.jp> >> At Thu, 6 Apr 2017 16:15:33 +0900, Masahiko Sawada >> wrote in >> > On Thu, Apr 6

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-06 Thread Keith Fiske
On Thu, Apr 6, 2017 at 1:18 AM, Amit Langote wrote: > On 2017/04/06 13:08, Keith Fiske wrote: > > On Wed, Apr 5, 2017 at 2:51 PM, Keith Fiske wrote: > >> Only issue I see with this, and I'm not sure if it is an issue, is what > >> happens to that default constraint clause when 1000s of partitions

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-04-06 Thread Tom Lane
Kevin Grittner writes: > Note that the final proposal is here: > https://summerofcode.withgoogle.com/serve/5874530240167936/ I'm just getting a blank page at that URL? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

[HACKERS] Uninitialized variable introduced in 3217327053638085d24dd4d276e7c1f7ac2c4c6b

2017-04-06 Thread Mark Dilger
Peter, Can you perhaps initialize the variable 'address' to suppress the warning? Thanks. Mark Dilger tablecmds.c:5984:6: warning: variable 'address' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] if (generatedEl) ^~~ tablecmds.c

Re: [HACKERS] [GSoC] Push-based query executor discussion

2017-04-06 Thread Kevin Grittner
Sorry, I didn't notice that this was going to a public list. That URL is only available to people who signed up as mentors for PostgreSQL GSoC participation this year. Does the link to the draft work for you? -- Kevin Grittner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-06 Thread Joe Conway
On 04/05/2017 02:29 PM, Mike Palmiotto wrote: > I'm going to hold the partition table regression changes for a > separate patch and include some ORDER BY fixes. Will post tomorrow > > In the meantime, attached are the latest and greatest patches. I'm going to push the attached in a few hours unle

Re: [HACKERS] No-op case in ExecEvalConvertRowtype

2017-04-06 Thread Tom Lane
Ashutosh Bapat writes: > In ExecEvalConvertRowtype(), if the input row doesn't require any > conversion, we simply return that row as is. Huh. That's been like that for a very long time. > I tried to create a testcase where this assertion would fail without > multi-level partitioned table, but

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-06 Thread Tom Lane
Joe Conway writes: > I'm going to push the attached in a few hours unless there is any > additional discussion. As stated above we'll do the regression changes > in a separate patch once that is sorted. I used Tom's approach and > comment wording for 0001a. Looks generally sane, but I noticed a g

[HACKERS] Re: Uninitialized variable introduced in 3217327053638085d24dd4d276e7c1f7ac2c4c6b

2017-04-06 Thread Peter Eisentraut
On 4/6/17 10:59, Mark Dilger wrote: > Can you perhaps initialize the variable 'address' to suppress the warning? > Thanks. A potential fix for this has been pushed. > tablecmds.c:5984:6: warning: variable 'address' is used uninitialized > whenever 'if' condition is false [-Wsometimes-uninitial

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread Peter Eisentraut
On 4/6/17 07:13, Beena Emerson wrote: > Does the options 16, 64 and 1024 seem good. > We can remove sizes below 16 as most have agreed and as per the > discussion, 64MB and 1GB seems favoured. We could probably allow 32MB > since it was an already allowed size? I don't see the need to remove any

Re: [HACKERS] Uninitialized variable introduced in 3217327053638085d24dd4d276e7c1f7ac2c4c6b

2017-04-06 Thread Mark Dilger
> On Apr 6, 2017, at 8:33 AM, Peter Eisentraut > wrote: > > On 4/6/17 10:59, Mark Dilger wrote: >> Can you perhaps initialize the variable 'address' to suppress the warning? >> Thanks. > > A potential fix for this has been pushed. It works for me. Thanks again. Mark Dilger -- Sent via p

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-04-06 Thread Tom Lane
Robert Haas writes: > ... But the underlying point here is that > the only thing you really know about the function is that it's got to > be a strategy-3 operator in some btree opclass; if that guarantees > strictness, then so be it -- but I wasn't able to find anything in the > code or documentat

Re: [HACKERS] pg_stat_wal_write statistics view

2017-04-06 Thread Fujii Masao
On Thu, Apr 6, 2017 at 5:51 AM, Andres Freund wrote: > Hi, > > On 2017-03-30 13:10:41 +1100, Haribabu Kommi wrote: >> diff --git a/src/backend/access/transam/xlog.c >> b/src/backend/access/transam/xlog.c >> index 5d58f09..a29c108 100644 >> --- a/src/backend/access/transam/xlog.c >> +++ b/src/back

Re: [HACKERS] Logical Replication and Character encoding

2017-04-06 Thread Peter Eisentraut
On 4/5/17 21:32, Kyotaro HORIGUCHI wrote: > At Wed, 5 Apr 2017 11:33:51 -0400, Peter Eisentraut > wrote in > <5401fef6-c0c0-7e8a-d8b1-169e30cbd...@2ndquadrant.com> >> After further thinking, I prefer the alternative approach of using >> pq_sendcountedtext() as is and sticking the trailing zero b

Re: [HACKERS] BRIN cost estimate

2017-04-06 Thread Tom Lane
David Rowley writes: > + *indexTotalCost += 0.1 * cpu_operator_cost * estimatedRanges * > + pagesPerRange; > This is trying to cost up the following code in bringetbitmap() > if (addrange) > { > BlockNumber pageno; > for (pageno = heapBlk; > pageno <= heapBlk + opaque->bo_pagesPerRange - 1; > p

Re: [HACKERS] Remove pg_stat_progress_vacuum from Table 28.2

2017-04-06 Thread Fujii Masao
On Tue, Apr 4, 2017 at 10:19 AM, Amit Langote wrote: > It seems pg_stat_progress_vacuum is not supposed to appear in the table > titled "Collected Statistics Views". It was added by c16dc1aca. Attached > patch fixes that. Instead, it should appear in the table of "Dynamic Statistics Views" beca

Re: [HACKERS] Duplicate usage of tablespace location?

2017-04-06 Thread Tom Lane
Kyotaro HORIGUCHI writes: > I noticed by the following report, PostgreSQL can share the same > directory as tablespaces of two servers with different > pg-versions. > https://www.postgresql.org/message-id/2008148.rxBNyNRHPZ@peanuts2 > 8.4 checked that the tablespace location is empty, but from 9

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Peter Eisentraut
On 4/6/17 03:50, Craig Ringer wrote: > But otherwise, pending docs changes, I think it's ready for committer. My opinion is still that this is ultimately the wrong approach. The right fix for performance issues in PL/Python is to change PL/Python not to materialize the list of tuples. Now with t

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-06 Thread Keith Fiske
On Thu, Apr 6, 2017 at 7:30 AM, Rahila Syed wrote: > Hello, > > Thanks a lot for testing and reporting this. Please find attached an > updated patch with the fix. The patch also contains a fix > regarding operator used at the time of creating expression as default > partition constraint. This was

Re: [HACKERS] background sessions

2017-04-06 Thread Peter Eisentraut
On 4/4/17 10:28, Robert Haas wrote: > So is this patch going anywhere? Not right now. It will take some time to sort out your feedback and do some refactoring. I will close the patch for now. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Rem

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Jim Nasby
On 4/6/17 9:04 AM, Peter Eisentraut wrote: On 4/6/17 03:50, Craig Ringer wrote: But otherwise, pending docs changes, I think it's ready for committer. My opinion is still that this is ultimately the wrong approach. The right fix for performance issues in PL/Python is to change PL/Python not t

Re: [HACKERS] subscription worker doesn't start immediately on eabled

2017-04-06 Thread Petr Jelinek
On 06/04/17 14:24, Kyotaro HORIGUCHI wrote: > Hello. I found dubious behavior while playing with logical > replication. > > When we disable a subscription, replication worker immediately stops. > > =# ALTER SUBSCRIPTION s1 DISABLE; > > On the other hand even if we enable a subscription, worker >

Re: [HACKERS] pg_background contrib module proposal

2017-04-06 Thread Peter Eisentraut
On 2/1/17 22:03, Michael Paquier wrote: > On Fri, Jan 27, 2017 at 11:38 PM, Robert Haas wrote: >> On Fri, Jan 27, 2017 at 9:14 AM, Peter Eisentraut >> wrote: >>> On 1/19/17 12:47 PM, Andrey Borodin wrote: 4. There is some controversy on where implemented feature shall be: in separate e

Re: [HACKERS] Interval for launching the table sync worker

2017-04-06 Thread Petr Jelinek
On 06/04/17 16:44, Masahiko Sawada wrote: > On Thu, Apr 6, 2017 at 9:06 PM, Kyotaro HORIGUCHI > wrote: >> At Thu, 06 Apr 2017 17:02:14 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI >> wrote in >> <20170406.170214.263553093.horiguchi.kyot...@lab.ntt.co.jp> >>> At Thu, 6 Apr 2017 16:15:33 +0900,

[HACKERS] Postgresql10 Bug report. (pg_catalog.pg_statistic_ext does not exist)

2017-04-06 Thread mark
apologies if someone has already reported this. steps to reproduce. install PG10 rpms. create table. using psql 10 \d the table. note the error below. m=# create table mytable (myid serial, mytext text); CREATE TABLE m=# \d mytable ERROR: relation "pg_catalog.pg_statistic_ext" does not exi

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Tom Lane
Peter Eisentraut writes: > On 4/6/17 03:50, Craig Ringer wrote: >> But otherwise, pending docs changes, I think it's ready for committer. > My opinion is still that this is ultimately the wrong approach. The > right fix for performance issues in PL/Python is to change PL/Python not > to material

Re: [HACKERS] Postgresql10 Bug report. (pg_catalog.pg_statistic_ext does not exist)

2017-04-06 Thread Alvaro Herrera
mark wrote: > m=# create table mytable (myid serial, mytext text); > CREATE TABLE > m=# \d mytable > ERROR: relation "pg_catalog.pg_statistic_ext" does not exist > LINE 8: FROM pg_catalog.pg_statistic_ext stat WHERE starelid = '163... > ^ Ah, what happens is you're using a new psql

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-06 Thread Heikki Linnakangas
Another version attached. I think this is now in committable state, but there's been a lot of small changes here and there, so please have one more look over it if you have a chance. I'm planning to push this tomorrow, after sleeping on it. The code-organization issue with the utf8 functions,

[HACKERS] Re: Letting the client choose the protocol to use during a SASL exchange

2017-04-06 Thread Heikki Linnakangas
On 04/06/2017 08:13 AM, Noah Misch wrote: If any SCRAM open item is a beta blocker, it's this one. (But SASLprep is also in or near that status.) Post-beta wire protocol changes are bad, considering beta is normally the time for projects like pgjdbc and npgsql to start adapting to such changes.

Re: [HACKERS] Re: Letting the client choose the protocol to use during a SASL exchange

2017-04-06 Thread Álvaro Hernández Tortosa
On 06/04/17 19:05, Heikki Linnakangas wrote: On 04/06/2017 08:13 AM, Noah Misch wrote: If any SCRAM open item is a beta blocker, it's this one. (But SASLprep is also in or near that status.) Post-beta wire protocol changes are bad, considering beta is normally the time for projects like pgj

Re: [HACKERS] Postgresql10 Bug report. (pg_catalog.pg_statistic_ext does not exist)

2017-04-06 Thread Tom Lane
Alvaro Herrera writes: > mark wrote: >> m=# create table mytable (myid serial, mytext text); >> CREATE TABLE >> m=# \d mytable >> ERROR: relation "pg_catalog.pg_statistic_ext" does not exist > Ah, what happens is you're using a new psql with a pre-10 server. Yeah, > this is a bug since psql is

Re: [HACKERS] logical replication access control patches

2017-04-06 Thread Peter Eisentraut
On 3/29/17 19:01, Petr Jelinek wrote: >> So this CREATE SUBSCRIPTION priv actually gives you the power to cause >> the system to open network connections to the outside world. It's not >> something you give freely to random strangers -- should be guarded >> moderately tight, because it could be us

Re: [HACKERS] logical replication apply to run with sync commit off by default

2017-04-06 Thread Peter Eisentraut
On 3/24/17 10:49, Petr Jelinek wrote: > On 07/03/17 06:23, Petr Jelinek wrote: >> there has been discussion at the logical replication initial copy thread >> [1] about making apply work with sync commit off by default for >> performance reasons and adding option to change that per s

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-06 Thread Heikki Linnakangas
(sorry, I didn't notice your email until after I just sent version 4!) On 04/06/2017 10:32 AM, Michael Paquier wrote: On Thu, Apr 6, 2017 at 1:33 AM, Heikki Linnakangas wrote: Attached is a new version. Notable changes since yesterday: * Implemented the rest of the SASLPrep, mapping some char

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-06 Thread Heikki Linnakangas
On 04/06/2017 08:42 PM, Heikki Linnakangas wrote: D'oh. Here's a new version, with saslprep.h included. And here it is for real. Sigh. There is for example this portion in the new tables: +static const Codepoint prohibited_output_chars[] = +{ + 0xD800, 0xF8FF, /* C.3, C.5 */

Re: [HACKERS] Postgresql10 Bug report. (pg_catalog.pg_statistic_ext does not exist)

2017-04-06 Thread mark
it would appear that it didn't restart when I thought it had with the service command. apologies, I'm not able to reproduce anymore after restarting things. On Thu, Apr 6, 2017 at 11:27 AM, Tom Lane wrote: > Alvaro Herrera writes: > > mark wrote: > >> m=# create table mytable (myid serial, my

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-06 Thread Joe Conway
On 04/06/2017 08:29 AM, Tom Lane wrote: > Joe Conway writes: >> I'm going to push the attached in a few hours unless there is any >> additional discussion. As stated above we'll do the regression changes >> in a separate patch once that is sorted. I used Tom's approach and >> comment wording for 0

Re: [HACKERS] Re: new set of psql patches for loading (saving) data from (to) text, binary files

2017-04-06 Thread Pavel Stehule
2017-04-06 14:47 GMT+02:00 Stephen Frost : > Greetings, > > * Pavel Stehule (pavel.steh...@gmail.com) wrote: > > 2017-04-06 3:34 GMT+02:00 Stephen Frost : > > > Having the template not require the row/column place-holders included > > > strikes me as more likely to be confusing. My initial thinki

Re: [HACKERS] [GENERAL] C++ port of Postgres

2017-04-06 Thread Peter Eisentraut
On 4/5/17 19:14, Andres Freund wrote: > Hi Peter, > > On 2017-02-28 22:30:16 -0800, Andres Freund wrote: >> On 2017-02-28 23:42:45 -0500, Peter Eisentraut wrote: >>> On 1/26/17 22:46, Andres Freund wrote: On 2016-09-30 15:24:09 -0400, Peter Eisentraut wrote: > Yeah, I have committed a few

Re: [HACKERS] Functions Immutable but not parallel safe?

2017-04-06 Thread Peter Eisentraut
On 4/5/17 12:26, Robert Haas wrote: > On Wed, Apr 5, 2017 at 12:18 PM, Peter Eisentraut > wrote: >> On 4/5/17 09:58, Robert Haas wrote: - Maybe add a check to opr_sanity to make sure the default set of functions is configured the way we want? >>> That seems like a good idea. >> >> patch

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-04-06 Thread Pavel Stehule
2017-04-06 12:30 GMT+02:00 Andrew Dunstan : > > > On 04/05/2017 05:41 PM, Andres Freund wrote: > > On 2017-04-05 17:22:34 -0400, Tom Lane wrote: > >> Andres Freund writes: > >>> I'd like some input from other committers whether we want this. I'm > >>> somewhat doubtful, but don't have particular

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread David Steele
On 4/5/17 7:29 AM, Simon Riggs wrote: > On 5 April 2017 at 06:04, Beena Emerson wrote: >> >> The WALfilename - LSN mapping disruption for higher values you mean? Is >> there anything else I have missed? > > I see various issues raised but not properly addressed > > 1. we would need to drop supp

Re: [HACKERS] Logical Replication and Character encoding

2017-04-06 Thread Peter Eisentraut
On 4/6/17 11:47, Peter Eisentraut wrote: > On 4/5/17 21:32, Kyotaro HORIGUCHI wrote: >> At Wed, 5 Apr 2017 11:33:51 -0400, Peter Eisentraut >> wrote in >> <5401fef6-c0c0-7e8a-d8b1-169e30cbd...@2ndquadrant.com> >>> After further thinking, I prefer the alternative approach of using >>> pq_sendcoun

Re: [HACKERS] SCRAM authentication, take three

2017-04-06 Thread Heikki Linnakangas
On 04/06/2017 08:36 AM, Noah Misch wrote: On Tue, Mar 07, 2017 at 02:36:13PM +0200, Heikki Linnakangas wrote: I didn't include the last-minute changes to the way you specify this in pg_hba.conf. So it's still just "scram". I agree in general that we should think about how to extend that too, but

Re: [HACKERS] Fast Default WIP patch for discussion

2017-04-06 Thread Andres Freund
Hi, On 2017-04-05 22:31:15 -0700, Serge Rielau wrote: > Andres, > Yes, I still want to push this in. However I have not had time to get back to > it. I’m embarrassed to say that I don’t even know where the comments that > were issued occurred. > Cheers Serge You mean https://www.postgresql.org/

Re: [HACKERS] CONNECTION LIMIT and Parallel Query don't play well together

2017-04-06 Thread Peter Eisentraut
On 2/15/17 11:19, Peter Eisentraut wrote: > So I would like to have a background worker limit per user, as you > allude to. Attached is a patch that implements a GUC setting > max_worker_processes_per_user. > > Besides the uses for background sessions, but it can also be useful for > parallel wor

Re: [HACKERS] Performance improvement for joins where outer side is unique

2017-04-06 Thread Tom Lane
David Rowley writes: > On 2 April 2017 at 21:21, David Rowley wrote: >> I've attached an updated patch which updates the regression test output of >> a recent commit to include the "Unique Inner" in the expected results. > The patch must've fallen off. Attempt number 2 at attaching. I'm looking

Re: [pgsql-students] [HACKERS] [GSoC] Push-based query executor discussion

2017-04-06 Thread Simon Riggs
On 22 March 2017 at 14:58, Oleg Bartunov wrote: > Should we reject this interesting project, which based on several years of > research work of academician group in the institute ? May be better help him > to reformulate the scope of project and let him work ? I don't know exactly > if the result

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-06 Thread Tom Lane
Joe Conway writes: > Any thoughts on whether 0001a and 0001b ought to be backpatched? I'm > thinking not given the lack of past complaints but it might make sense > to do. I think 0001a absolutely needs to be, because it is fixing what is really an ABI violation: sepgsql_needs_fmgr_hook is suppos

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-06 Thread Simon Riggs
On 4 April 2017 at 02:02, Michael Paquier wrote: > Hi all, > > There is still one open item pending for SCRAM that has not been > treated which is mentioned here: > https://www.postgresql.org/message-id/b081887e-1712-3aa4-7dbe-e012333d5...@iki.fi > > When doing an authentication with SASL, the ser

[HACKERS] Row Level Security UPDATE Confusion

2017-04-06 Thread Rod Taylor
I'm a little confused on why a SELECT policy fires against the NEW record for an UPDATE when using multiple FOR policies. The ALL policy doesn't seem to have that restriction. DROP TABLE IF EXISTS t; CREATE USER simple; CREATE USER split; CREATE TABLE t(value int); grant select, update on table

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-06 Thread Tom Lane
Simon Riggs writes: > How would we provide the list of protocols? Surely the protocol is > defined by pg_hba.conf, which makes it dependent upon username, > database and ip range. If the list were accurate, it would allow an > attacker to discover how best to attack. If the list were inaccurate >

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-06 Thread Álvaro Hernández Tortosa
On 06/04/17 22:05, Tom Lane wrote: Simon Riggs writes: How would we provide the list of protocols? Surely the protocol is defined by pg_hba.conf, which makes it dependent upon username, database and ip range. If the list were accurate, it would allow an attacker to discover how best to attack

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-06 Thread Simon Riggs
On 6 April 2017 at 16:05, Tom Lane wrote: > Perhaps we could turn this around: have the client send (in the connection > request packet) a list of auth protocols it thinks it is able to handle. > (I'm envisioning this as being more or less fixed for any one version of > any one client, since it w

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-04-06 Thread Tomas Vondra
Hi, I've been looking at this issue today, and so far I don't think it's a bug in the foreign key estimation. It seems mostly that the 9.5 estimates were hopelessly bad, and the join estimation changes simply pushed it a tiny bit the wrong direction. Although maybe there is a bug (or at leas

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-06 Thread Andres Freund
On 2017-04-06 10:00:32 +0530, Dilip Kumar wrote: > On Tue, Apr 4, 2017 at 5:51 AM, Dilip Kumar wrote: > > Sure I can do that, In attached patch, I only fixed the problem of not > > executing the bitmap test. Now, I will add few cases to cover other > > parts especially rescan and prefetching logi

Re: [HACKERS] parallel explain analyze support not exercised

2017-04-06 Thread Andres Freund
Hi, On 2017-04-03 17:11:33 -0400, Robert Haas wrote: > On Mon, Apr 3, 2017 at 3:31 PM, Andres Freund wrote: > >> If this is 'make check', then we should have 8 parallel workers > >> allowed, so if we only do one of these at a time, then I think we're > >> OK. But if somebody changes that configu

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2017-04-06 Thread Andres Freund
On 2017-04-04 13:49:11 -0700, Peter Geoghegan wrote: > On Tue, Apr 4, 2017 at 1:32 PM, Andres Freund wrote: > >> static bool > >> tuplesort_gettuple_common(Tuplesortstate *state, bool forward, > >> @@ -2091,12 +2092,15 @@ tuplesort_gettuple_common(Tuplesortstate *state, > >> bool forward, > >>

Re: [HACKERS] increasing the default WAL segment size

2017-04-06 Thread Tomas Vondra
On 04/06/2017 08:33 PM, David Steele wrote: On 4/5/17 7:29 AM, Simon Riggs wrote: On 5 April 2017 at 06:04, Beena Emerson wrote: The WALfilename - LSN mapping disruption for higher values you mean? Is there anything else I have missed? I see various issues raised but not properly addressed

Re: [HACKERS] Making clausesel.c Smarter

2017-04-06 Thread Claudio Freire
On Tue, Apr 4, 2017 at 1:00 PM, Claudio Freire wrote: >>> If you ask me, I'd just leave: >>> >>> + if (rqlist->hibound == DEFAULT_INEQ_SEL || rqlist->lobound == >>> DEFAULT_INEQ_SEL) >>> + { >>> + /* No point in checking null selectivity, DEFAULT_INEQ_SEL >>> implies we have no stats */ >>> +

  1   2   >