Re: [HACKERS] Notice lock waits

2016-09-28 Thread Haribabu Kommi
On Sat, Aug 6, 2016 at 3:00 AM, Jeff Janes wrote: > One time too many, I ran some minor change using psql on a production > server and was wondering why it was taking so much longer than it did > on the test server. Only to discover, after messing around with > opening new windows and running qu

Re: [HACKERS] Showing parallel status in \df+

2016-09-28 Thread Rushabh Lathia
On Thu, Sep 29, 2016 at 12:07 AM, Pavel Stehule wrote: > Hi > > 2016-09-28 18:57 GMT+02:00 Tom Lane : > >> Pavel Stehule writes: >> > 2016-09-28 16:03 GMT+02:00 Tom Lane : >> >> I propose to push my current patch (ie, move PL function >> >> source code to \df+ footers), and we can use it in HEAD

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2016-09-28 Thread Kyotaro HORIGUCHI
Hello, At Wed, 28 Sep 2016 12:49:25 -0400, Robert Haas wrote in > This patch hasn't been updated in over a week and we're just about out > of time for this CommitFest, so I've marked it "Returned with > Feedback" for now. If it gets updated, it can be resubmitted for the > next CommitFest. Th

Re: [HACKERS] asynchronous execution

2016-09-28 Thread Kyotaro HORIGUCHI
Hello, thank you for the comment. At Wed, 28 Sep 2016 10:00:08 +0530, Amit Khandekar wrote in > On 24 September 2016 at 06:39, Robert Haas wrote: > > Since Kyotaro Horiguchi found that my previous design had a > > system-wide performance impact due to the ExecProcNode changes, I > > decided to

Re: [HACKERS] Sample configuration files

2016-09-28 Thread Vik Fearing
On 09/29/2016 05:55 AM, Michael Paquier wrote: > On Thu, Sep 29, 2016 at 2:25 AM, Robert Haas wrote: >> So, anyone else have an opinion, pro or con? > > Going through this thread, I'd vote -1. This is a documentation effort > mainly, and installing those files has zero effect if they are not > lo

Re: [HACKERS] New SQL counter statistics view (pg_stat_sql)

2016-09-28 Thread Haribabu Kommi
On Thu, Sep 22, 2016 at 3:05 AM, Alvaro Herrera wrote: > Peter Eisentraut wrote: > > > How about having the tag not be a column name but a row entry. So you'd > > do something like > > > > SELECT * FROM pg_stat_sql WHERE tag = 'ALTER VIEW'; > > > > That way, we don't have to keep updating (and r

Re: [HACKERS] asynchronous execution

2016-09-28 Thread Kyotaro HORIGUCHI
Sorry for delayed response, I'll have enough time from now and address this. At Fri, 23 Sep 2016 21:09:03 -0400, Robert Haas wrote in > Well, I promised to post this, so here it is. It's not really working > all that well at this point, and it's definitely not doing anything > that interesting

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-28 Thread David Fetter
On Thu, Sep 29, 2016 at 11:12:11AM +1300, Thomas Munro wrote: > On Mon, Sep 26, 2016 at 5:11 PM, Thomas Munro > wrote: > > On Mon, Sep 26, 2016 at 1:18 PM, Thomas Munro > > wrote: > >> > >> On Mon, Sep 19, 2016 at 4:02 PM, David Fetter wrote: > >> > > >> > [training_wheels_004.patch] > >> > >> [

Re: [HACKERS] Handling dropped attributes in pglogical_proto

2016-09-28 Thread Petr Jelinek
On 29/09/16 05:33, Michael Paquier wrote: > On Wed, Sep 28, 2016 at 11:25 PM, Konstantin Knizhnik > wrote: >> But if table was just altered and some attribute was removed from the table, >> then rel->natts can be greater than natts. > > This is part of pglogical, so you may want to reply on the d

Re: [HACKERS] Sample configuration files

2016-09-28 Thread Michael Paquier
On Thu, Sep 29, 2016 at 2:25 AM, Robert Haas wrote: > So, anyone else have an opinion, pro or con? Going through this thread, I'd vote -1. This is a documentation effort mainly, and installing those files has zero effect if they are not loaded via include_if_exists or include in postgresql.conf.

Re: [HACKERS] WAL consistency check facility

2016-09-28 Thread Michael Paquier
On Fri, Sep 16, 2016 at 10:36 PM, Michael Paquier wrote: > On Fri, Sep 16, 2016 at 10:30 PM, Robert Haas wrote: >> I don't think you have the right to tell Kuntal that he has to move >> the patch to the next CommitFest because there are unspecified things >> about the current version you don't li

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-09-28 Thread Michael Paquier
On Wed, Sep 28, 2016 at 8:55 PM, Michael Paquier wrote: >> Our b64_encode routine does use whitespace, so we can't use it as is for >> SCRAM. As the patch stands, we might never output anything long enough to >> create linefeeds, but let's be tidy. The base64 implementation is about 100 >> lines o

Re: [HACKERS] Handling dropped attributes in pglogical_proto

2016-09-28 Thread Michael Paquier
On Wed, Sep 28, 2016 at 11:25 PM, Konstantin Knizhnik wrote: > But if table was just altered and some attribute was removed from the table, > then rel->natts can be greater than natts. This is part of pglogical, so you may want to reply on the dedicated thread or send directly a patch to them. By

Re: [HACKERS] Add support for restrictive RLS policies

2016-09-28 Thread Craig Ringer
On 27 September 2016 at 15:15, Jeevan Chalke wrote: > Hello Stephen, > > On Tue, Sep 27, 2016 at 12:57 AM, Stephen Frost wrote: >> >> Jeevan, >> >> * Jeevan Chalke (jeevan.cha...@enterprisedb.com) wrote: >> > I have started reviewing this patch and here are couple of points I have >> > observed s

Re: [HACKERS] [GENERAL] C++ port of Postgres

2016-09-28 Thread Thomas Munro
On Mon, Sep 26, 2016 at 10:57 AM, Thomas Munro wrote: > On Thu, Sep 1, 2016 at 1:41 AM, Peter Eisentraut > wrote: >> >> [trimmed cc list because of big attachments] >> >> On 8/16/16 4:22 PM, Jim Nasby wrote: >> > Joy, do you have an idea what a *minimally invasive* patch for C++ >> > support woul

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Tom Lane
Peter Eisentraut writes: > On 9/28/16 6:13 PM, Robert Haas wrote: >> Christoph/Debian: >> log_line_prefix = '%t [%p-%l] %q%u@%d ' >> Peter: >> log_line_prefix = '%t [%p]: [%l] %qapp=%a ' > ... > I don't know why it wants that "-1" there, and I'm actually not sure > what the point of %l is in prac

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-09-28 Thread Michael Paquier
On Thu, Sep 29, 2016 at 7:45 AM, David Steele wrote: > OK, I've done functional testing and this patch seems to work as > specified (including the caveat noted above). Some comments: Thanks! > * [PATCH 1/3] hs-checkpoints-v12-1 > > +++ b/src/backend/access/transam/xlog.c > +* Taking a l

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Peter Eisentraut
On 9/28/16 6:13 PM, Robert Haas wrote: > Christoph/Debian: > log_line_prefix = '%t [%p-%l] %q%u@%d ' > Peter: > log_line_prefix = '%t [%p]: [%l] %qapp=%a ' I'm aware of two existing guidelines on log line formats: syslog and pgbadger. Syslog output looks like this: Sep 28 00:58:56 hostna

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Peter Eisentraut
On 9/28/16 6:07 PM, Alvaro Herrera wrote: > Adopting a default prefix is a different question. A default prefix would require different settings for syslog, plain text, and possibly some of the other variants. I'm all in favor of figuring that out, but it needs more work. -- Peter Eisentraut

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-28 Thread Peter Eisentraut
On 9/28/16 2:45 AM, Michael Paquier wrote: > After all that fixed, I have moved the patch to "Ready for Committer". > Please use the updated patch though. Committed after some cosmetic changes. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Rem

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

2016-09-28 Thread Peter Eisentraut
On 9/25/16 8:06 AM, Ashutosh Sharma wrote: > Hi Peter, > >> I just wanted to update you, I have taken this commit fest entry patch >> to review because I think it will be addresses as part of "Exclude >> additional directories in pg_basebackup", which I'm also reviewing. >> Therefore, I'm not actu

Re: [HACKERS] Tracking wait event for latches

2016-09-28 Thread Michael Paquier
On Wed, Sep 28, 2016 at 9:45 PM, Robert Haas wrote: > On Wed, Sep 28, 2016 at 8:38 AM, Michael Paquier > wrote: >> So should I change back the patch to have only one argument for the >> eventId, and guess classId from it? > > Why would you need to guess? Incorrect wording from me perhaps? i just

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-28 Thread Tomas Vondra
On 09/29/2016 01:59 AM, Robert Haas wrote: On Wed, Sep 28, 2016 at 6:45 PM, Tomas Vondra wrote: So, is 300 too little? I don't think so, because Dilip saw some benefit from that. Or what scale factor do we think is needed to reproduce the benefit? My machine has 256GB of ram, so I can easily go

Re: [HACKERS] Hash Indexes

2016-09-28 Thread Robert Haas
On Wed, Sep 28, 2016 at 3:04 PM, Robert Haas wrote: > I'll write another email with my thoughts about the rest of the patch. I think that the README changes for this patch need a fairly large amount of additional work. Here are a few things I notice: - The confusion between buckets and pages ha

Re: [HACKERS] Transaction user id through logical decoding

2016-09-28 Thread Craig Ringer
On 28 Sep. 2016 17:50, "valeriof" wrote: > > Hi all, > I'm developing a custom plugin to stream Postgres CDC changes to my client > application. One of the info the application needs is the user id of the > user who executed a certain transaction. I can see we have access to other > transaction in

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-28 Thread Robert Haas
On Wed, Sep 28, 2016 at 6:45 PM, Tomas Vondra wrote: > So, is 300 too little? I don't think so, because Dilip saw some benefit from > that. Or what scale factor do we think is needed to reproduce the benefit? > My machine has 256GB of ram, so I can easily go up to 15000 and still keep > everything

Re: [HACKERS] less expensive pg_buffercache on big shmem

2016-09-28 Thread Ivan Kartyshov
Hello everyone, patch was rebased. Thank you Tomas for your reviewing this patch and for your valuable comments. From the very beginning we had the misunderstanding with the naming of meethods. > It'd be really useful if you could provide actual numbers, explain what > metrics you compare a

Re: [HACKERS] Better tracking of free space during SP-GiST index build

2016-09-28 Thread Jeff Janes
On Wed, Sep 28, 2016 at 10:48 AM, Robert Haas wrote: > On Sun, Sep 25, 2016 at 3:28 PM, Tomas Vondra > wrote: > > Sure, that would be useful. > > > > I think it would be useful to make repository of such data sets, so that > > patch authors & reviewers can get a reasonable collection of data set

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-28 Thread Tomas Vondra
On 09/28/2016 05:39 PM, Robert Haas wrote: On Tue, Sep 27, 2016 at 5:15 PM, Tomas Vondra wrote: So, I got the results from 3.10.101 (only the pgbench data), and it looks like this: 3.10.101 1 8 16 32 64128192 -

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-09-28 Thread David Steele
On 9/28/16 3:35 AM, Michael Paquier wrote: > On Wed, Sep 28, 2016 at 6:12 AM, David Steele wrote: >> I tried the attached patch set and noticed an interesting behavior. With >> archive_timeout=5 whenever I made a change I would get a WAL segment within >> a few seconds as expected then another one

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Perhaps we should first try to get a consensus on the regression test >> use-case. > I thought Peter's suggestion for regression test drivers was a good one > and I see no reason to block that. Why do you (Tom) object so strongly > against having a dif

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-28 Thread David Fetter
On Thu, Sep 29, 2016 at 11:12:11AM +1300, Thomas Munro wrote: > On Mon, Sep 26, 2016 at 5:11 PM, Thomas Munro > wrote: > > On Mon, Sep 26, 2016 at 1:18 PM, Thomas Munro > > wrote: > >> > >> On Mon, Sep 19, 2016 at 4:02 PM, David Fetter wrote: > >> > > >> > [training_wheels_004.patch] > >> > >> [

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-28 Thread Thomas Munro
On Mon, Sep 26, 2016 at 5:11 PM, Thomas Munro wrote: > On Mon, Sep 26, 2016 at 1:18 PM, Thomas Munro > wrote: >> >> On Mon, Sep 19, 2016 at 4:02 PM, David Fetter wrote: >> > >> > [training_wheels_004.patch] >> >> [review] Ping. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Robert Haas
On Wed, Sep 28, 2016 at 6:07 PM, Alvaro Herrera wrote: > I thought Peter's suggestion for regression test drivers was a good one > and I see no reason to block that. Why do you (Tom) object so strongly > against having a different one on buildfarm than elsewhere? I'd rather > have buildfarm adop

Re: [HACKERS] compiler warning read_objtype_from_string()

2016-09-28 Thread Alvaro Herrera
Tom Lane wrote: > I do not think you should assume that the compiler is smart enough to > deduce that, nor that all compilers even know ereport(ERROR) doesn't > return. Personally I don't see the point of the "type" variable at > all, anyway. I would have written this as > > [code] Makes sense

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Alvaro Herrera
Tom Lane wrote: > Robert Haas writes: > > On Thu, Sep 15, 2016 at 5:18 PM, Robert Haas wrote: > >> On Sat, Aug 27, 2016 at 3:59 PM, Tom Lane wrote: > >>> I think the odds of getting to something that everyone would agree on > >>> are nil, so I'm not excited about getting into that particular > >

Re: [HACKERS] compiler warning read_objtype_from_string()

2016-09-28 Thread Tom Lane
Alvaro Herrera writes: > Peter Eisentraut wrote: >> I'm getting the following compiler warning (using nondefault >> optimization options): >> objectaddress.c: In function 'read_objtype_from_string': >> objectaddress.c:2309:9: error: 'type' may be used uninitialized in this >> function [-Werror=may

Re: [HACKERS] Showing parallel status in \df+

2016-09-28 Thread Tom Lane
Pavel Stehule writes: > We are in cycle because prosrc field is used for two independent features - > and then it can be hard to find a agreement. I thought pretty much everyone was on board with the idea of keeping prosrc in \df+ for internal/C-language functions (and then probably renaming the

Re: [HACKERS] kqueue

2016-09-28 Thread Thomas Munro
On Thu, Sep 29, 2016 at 9:09 AM, Keith Fiske wrote: > On Thu, Sep 15, 2016 at 11:11 PM, Thomas Munro > wrote: >> Ok, here's a version tweaked to use EVFILT_PROC for postmaster death >> detection instead of the pipe, as Tom Lane suggested in another >> thread[1]. >> >> [...] > > Ran benchmarks on

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 15, 2016 at 5:18 PM, Robert Haas wrote: >> On Sat, Aug 27, 2016 at 3:59 PM, Tom Lane wrote: >>> I think the odds of getting to something that everyone would agree on >>> are nil, so I'm not excited about getting into that particular >>> bikeshed-painting discuss

Re: [HACKERS] Bug in to_timestamp().

2016-09-28 Thread Tom Lane
Artur Zakirov writes: > - now DCH_cache_getnew() is called after parse_format(). Because now > parse_format() can raise an error and in the next attempt > DCH_cache_search() could return broken cache entry. I started looking at your 0001-to-timestamp-format-checking-v4.patch and this point imme

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2016-09-28 Thread Kevin Grittner
On Wed, Sep 28, 2016 at 2:04 PM, Kevin Grittner wrote: > Will take a look and post again. I am moving this patch to the next CF. You'll be hearing from me sometime after this CF is closed. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql

Re: [HACKERS] kqueue

2016-09-28 Thread Keith Fiske
On Thu, Sep 15, 2016 at 11:11 PM, Thomas Munro < thomas.mu...@enterprisedb.com> wrote: > On Thu, Sep 15, 2016 at 11:04 AM, Thomas Munro > wrote: > > On Thu, Sep 15, 2016 at 10:48 AM, Keith Fiske wrote: > >> Thomas Munro brought up in #postgresql on freenode needing someone to > test a > >> patch

Re: [HACKERS] Showing parallel status in \df+

2016-09-28 Thread Pavel Stehule
2016-09-28 21:59 GMT+02:00 Alvaro Herrera : > Pavel Stehule wrote: > > > I am sorry, I disagree. Proposed form is hard readable. Is not possible > to > > simply copy/paste. > > Why do you care? You can use \sf if you want to copy&paste the > function code. > I know so I can use \sf. But I don't

Re: [HACKERS] ICU integration

2016-09-28 Thread Thomas Munro
On Fri, Sep 23, 2016 at 6:27 PM, Thomas Munro wrote: > On Wed, Aug 31, 2016 at 2:46 PM, Peter Eisentraut > wrote: >> Here is a patch I've been working on to allow the use of ICU for sorting >> and other locale things. > > This is very interesting work, and it's great to see some development > in

Re: [HACKERS] Showing parallel status in \df+

2016-09-28 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Pavel Stehule wrote: > > I cannot to imagine any use case for proposed format. > > My vote (which was not counted by Stephen) was to remove it from \df+ Oh, sorry about that, not sure how I missed it. :/ > altogether. I stand by that. People

Re: [HACKERS] Showing parallel status in \df+

2016-09-28 Thread Alvaro Herrera
Pavel Stehule wrote: > I am sorry, I disagree. Proposed form is hard readable. Is not possible to > simply copy/paste. Why do you care? You can use \sf if you want to copy&paste the function code. > I cannot to imagine any use case for proposed format. My vote (which was not counted by Stephen

Re: [HACKERS] Add support for restrictive RLS policies

2016-09-28 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen, the typo "awseome" in the tests is a bit distracting. Can you > please fix it? Done. > I think you should use braces here, not parens: Fixed. > I don't think this paragraph is right -- you should call out each of the > values PERMIS

Re: [HACKERS] Hash Indexes

2016-09-28 Thread Robert Haas
On Wed, Sep 28, 2016 at 3:06 PM, Andres Freund wrote: > On 2016-09-28 15:04:30 -0400, Robert Haas wrote: >> Andres already >> stated that he things working on btree-over-hash would be more >> beneficial than fixing hash, but at this point it seems like he's the >> only one who takes that position.

Re: [HACKERS] Hash Indexes

2016-09-28 Thread Andres Freund
On 2016-09-28 15:04:30 -0400, Robert Haas wrote: > Andres already > stated that he things working on btree-over-hash would be more > beneficial than fixing hash, but at this point it seems like he's the > only one who takes that position. Note that I did *NOT* take that position. I was saying that

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-09-28 Thread Stephen Frost
Heikki, Michael, Magnus, * Michael Paquier (michael.paqu...@gmail.com) wrote: > On Tue, Sep 27, 2016 at 10:42 PM, Heikki Linnakangas wrote: > > The libpq-side is not. Just calling random() won't do. We haven't needed for > > random numbers in libpq before, but now we do. Is the pgcrypto solution

Re: [HACKERS] Hash Indexes

2016-09-28 Thread Robert Haas
On Tue, Sep 27, 2016 at 3:06 PM, Jesper Pedersen wrote: > I have been running various tests, and applications with this patch together > with the WAL v5 patch [1]. > > As I havn't seen any failures and doesn't currently have additional feedback > I'm moving this patch to "Ready for Committer" for

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2016-09-28 Thread Kevin Grittner
On Wed, Sep 28, 2016 at 12:02 PM, Emre Hasegeli wrote: >> `make check` finds differences per the attached. Please >> investigate why the regression tests are failing and what the >> appropriate response is. > > I fixed the first one and workaround the second with COLLATE "C". I > have how my ch

Re: [HACKERS] Binary I/O for isn extension

2016-09-28 Thread Robert Haas
On Wed, Sep 28, 2016 at 2:05 PM, Shay Rojansky wrote: > Sorry about this, I just haven't had a free moment (and it's definitely not > very high priority...) No issues, just cleaning house. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via

Re: [HACKERS] Better tracking of free space during SP-GiST index build

2016-09-28 Thread Robert Haas
On Wed, Sep 28, 2016 at 2:11 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Sep 22, 2016 at 1:37 PM, Tom Lane wrote: >>> OK, I'll think about how to do that more efficiently. The smaller >>> incremental improvement isn't surprising, because in this example the >>> index would still be 90-

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Christoph Berg
Re: Robert Haas 2016-09-28 > > Well, practically anything that includes a PID and the timestamp is > > going to be an improvement over the status quo. Just because we can't > > all agree on what would be perfect does not mean that we can't do > > better than what we've got now. +1 for trying. >

Re: [HACKERS] Showing parallel status in \df+

2016-09-28 Thread Pavel Stehule
Hi 2016-09-28 18:57 GMT+02:00 Tom Lane : > Pavel Stehule writes: > > 2016-09-28 16:03 GMT+02:00 Tom Lane : > >> I propose to push my current patch (ie, move PL function > >> source code to \df+ footers), and we can use it in HEAD for awhile > >> and see what we think. We can alway improve or re

Re: [HACKERS] should xlog_outdesc modify its argument?

2016-09-28 Thread Mark Dilger
> On Sep 27, 2016, at 11:25 PM, Heikki Linnakangas wrote: > > On 09/28/2016 02:35 AM, Mark Dilger wrote: >> The function >> >> static void xlog_outdesc(StringInfo buf, XLogReaderState *record); >> >> in src/backend/access/transam/xlog.c is called by XLogInsertRecord, >> and after returning a

Re: [HACKERS] psql casts aspersions on server reliability

2016-09-28 Thread Petr Jelinek
On 28/09/16 17:13, David Steele wrote: > On 9/28/16 10:22 AM, Robert Haas wrote: >> On Wed, Sep 28, 2016 at 9:14 AM, Tom Lane wrote: >>> Robert Haas writes: psql tends to do things like this: rhaas=# select * from pg_stat_activity; FATAL: terminating connection due to administrato

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-28 Thread Heikki Linnakangas
On 09/28/2016 07:11 PM, Peter Geoghegan wrote: On Wed, Sep 28, 2016 at 5:04 PM, Heikki Linnakangas wrote: Not sure that I understand. I agree that each merge pass tends to use roughly the same number of tapes, but the distribution of real runs on tapes is quite unbalanced in earlier merge passe

Re: [HACKERS] Better tracking of free space during SP-GiST index build

2016-09-28 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 22, 2016 at 1:37 PM, Tom Lane wrote: >> OK, I'll think about how to do that more efficiently. The smaller >> incremental improvement isn't surprising, because in this example the >> index would still be 90-something MB if it had no free space at all, >> so there

Re: [HACKERS] Binary I/O for isn extension

2016-09-28 Thread Shay Rojansky
Sorry about this, I just haven't had a free moment (and it's definitely not very high priority...) On Wed, Sep 28, 2016 at 5:04 PM, Robert Haas wrote: > On Mon, Aug 22, 2016 at 8:14 AM, Fabien COELHO > wrote: > > Hello Shay, > >> Attached is a new version of the patch, adding an upgrade script

[HACKERS] Batches, error handling and transaction in the protocol

2016-09-28 Thread Shay Rojansky
Hi everyone, I'd appreciate some guidance on an issue that's been raised with Npgsql, input from other driver writers would be especially helpful. Npgsql currently supports batching (or pipelining) to avoid roundtrips, and sends a Sync message only at the end of the batch (so Parse1/Bind1/Describe

[HACKERS] CommitFest wrap-up

2016-09-28 Thread Robert Haas
As some of you probably noticed, I just made a sweep through everything that was marked "Waiting on Author" in the CommitFest and hadn't been updated in the last couple of days. Most of those I marked as "Returned with Feedback", but some of them got some other status, one I committed, and a few I

Re: [HACKERS] Better tracking of free space during SP-GiST index build

2016-09-28 Thread Robert Haas
On Thu, Sep 22, 2016 at 1:37 PM, Tom Lane wrote: > OK, I'll think about how to do that more efficiently. The smaller > incremental improvement isn't surprising, because in this example the > index would still be 90-something MB if it had no free space at all, > so there's going to be decreasing r

Re: [HACKERS] Better tracking of free space during SP-GiST index build

2016-09-28 Thread Robert Haas
On Sun, Sep 25, 2016 at 3:28 PM, Tomas Vondra wrote: > Sure, that would be useful. > > I think it would be useful to make repository of such data sets, so that > patch authors & reviewers can get a reasonable collection of data sets if > needed, instead of scrambling every time. Opinions? In theo

Re: [HACKERS] less expensive pg_buffercache on big shmem

2016-09-28 Thread Robert Haas
On Tue, Sep 20, 2016 at 7:43 PM, Tomas Vondra wrote: > On 09/02/2016 11:01 AM, Robert Haas wrote: >> >> On Fri, Sep 2, 2016 at 8:49 AM, Andres Freund wrote: >>> >>> On 2016-09-02 08:31:42 +0530, Robert Haas wrote: I wonder whether we ought to just switch from the consistent method to >>

Re: [HACKERS] Set log_line_prefix and application name in test drivers

2016-09-28 Thread Robert Haas
On Thu, Sep 15, 2016 at 5:18 PM, Robert Haas wrote: > On Sat, Aug 27, 2016 at 3:59 PM, Tom Lane wrote: >> Christoph Berg writes: >>> I've always been wondering why we don't set a log_line_prefix by >>> default. >> >> I think the odds of getting to something that everyone would agree on >> are ni

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

2016-09-28 Thread Robert Haas
On Tue, Sep 13, 2016 at 9:07 PM, Kouhei Kaigai wrote: > It is because of just my time pressure around the patch submission days. > I'll try to enhance postgres_fdw as a usage of this run-time optimization. Time has (pretty much) expired for this CommitFest. In any case, this will amount to a who

Re: [HACKERS] identity columns

2016-09-28 Thread Robert Haas
On Mon, Sep 12, 2016 at 5:02 PM, Peter Eisentraut wrote: > Thank you for this extensive testing. I will work on getting the bugs > fixed. It looks like the patch has not been updated; since the CommitFest is (hopefully) wrapping up, I am marking this "Returned with Feedback" for now. -- Robert

Re: [HACKERS] pgbench more operators & functions

2016-09-28 Thread Jeevan Ladhe
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed The patch looks good to me now. Passing this to committer. The n

Re: [HACKERS] Index Onlys Scan for expressions

2016-09-28 Thread Robert Haas
On Thu, Sep 8, 2016 at 2:58 PM, Vladimir Sitnikov wrote: > Ildar> Could you please try the patch and tell if it works for you? > > I've tested patch6 against recent head. The patch applies with no problems. > > The previous case (filter on top of i-o-s) is fixed. Great work. > > Here are the test

Re: [HACKERS] Change error code for hstore syntax error

2016-09-28 Thread Sherrylyn Branchaw
Since no revised patch has been forthcoming and the CommitFest is due to end shortly, I've marked this "Returned with Feedback". Sherrylyn, please feel free to update the patch and resubmit to the next CommitFest. Will do, Robert, and many thanks to Marko for the feedback. I apologize for the del

Re: [HACKERS] Sample configuration files

2016-09-28 Thread Robert Haas
On Thu, Sep 8, 2016 at 8:52 AM, Tom Lane wrote: > Vik Fearing writes: >> I noticed that this patch has been marked Waiting on Author with no >> comment. Peter, what more should I be doing right now while waiting for >> Martín's review? > > FWIW, I agree with the upthread misgivings about whether

Re: [HACKERS] Truncating/vacuuming relations on full tablespaces

2016-09-28 Thread Robert Haas
On Thu, Sep 8, 2016 at 2:46 AM, Haribabu Kommi wrote: > Patch needs rebase, it is failing to apply on latest master. > And also there are some pending comment fix from Robert. It's been almost three weeks and this hasn't been updated, so I think it's pretty clear that it should be marked "Returne

Re: [HACKERS] [PATCH] add option to pg_dumpall to exclude tables from the dump

2016-09-28 Thread Robert Haas
On Tue, Sep 6, 2016 at 9:37 PM, Gerdan Rezende dos Santos wrote: > After review, I realized that there is a call to the function: > doShellQuoting (pgdumpopts, OPTARG), which no longer seems to exist ... > After understand the code, I saw that the call is appendShellString > (pgdumpopts, OPTARG).

Re: [HACKERS] pg_basebackup stream xlog to tar

2016-09-28 Thread Magnus Hagander
On Sep 28, 2016 19:11, "Robert Haas" wrote: > > On Mon, Sep 5, 2016 at 4:01 AM, Michael Paquier > wrote: > > [ review comments ] > > This thread has been sitting idle for more than 3 weeks, so I'm > marking it "Returned with Feedback" in the CommitFest application. > Magnus, Michael's latest roun

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-28 Thread Robert Haas
On Tue, Sep 6, 2016 at 10:11 AM, David Steele wrote: > On 9/6/16 8:07 AM, Robert Haas wrote: >> On Wed, Aug 31, 2016 at 9:45 PM, Simon Riggs >> wrote: >>> Related cleanup >>> * Promotion signal file is now called "promote.trigger" rather than >>> just "promote" >>> * Remove user configurable "tri

Re: [HACKERS] pg_basebackup stream xlog to tar

2016-09-28 Thread Robert Haas
On Mon, Sep 5, 2016 at 4:01 AM, Michael Paquier wrote: > [ review comments ] This thread has been sitting idle for more than 3 weeks, so I'm marking it "Returned with Feedback" in the CommitFest application. Magnus, Michael's latest round of comments seem pretty trivial, so perhaps you want to ju

Re: [HACKERS] Change error code for hstore syntax error

2016-09-28 Thread Robert Haas
On Sun, Sep 4, 2016 at 7:15 PM, Marko Tiikkaja wrote: > On 2016-05-09 19:42, Sherrylyn Branchaw wrote: >> >> I'm attaching a revised patch; please let me know if there are any other >> issues before I submit to the commitfest. > > I think this is mostly good, but these two should be changed: > >

Re: [HACKERS] Binary I/O for isn extension

2016-09-28 Thread Robert Haas
On Mon, Aug 22, 2016 at 8:14 AM, Fabien COELHO wrote: > Hello Shay, >> Attached is a new version of the patch, adding an upgrade script and the >> rest of it. Note that because, as Fabien noted, there's doesn't seem to be >> a way to add send/receive functions with ALTER TYPE, I did that by >> upd

Re: [HACKERS] [PATCH] SortSupport for macaddr type

2016-09-28 Thread Robert Haas
On Wed, Sep 14, 2016 at 6:14 AM, Julien Rouhaud wrote: > On 26/08/2016 19:44, Brandur wrote: >> Hello, > Hello, > >> I've attached a patch to add SortSupport for Postgres' macaddr which has the >> effect of improving the performance of sorting operations for the type. The >> strategy that I employ

Re: [HACKERS] hstore: add hstore_length function

2016-09-28 Thread Robert Haas
On Wed, Jun 8, 2016 at 10:44 AM, Robert Haas wrote: > On Mon, Jun 6, 2016 at 7:57 PM, Korbin Hoffman wrote: >> With regards to your second point- I've been maintaining consistency >> with the rest of the hstore module. Hstore's _size is internally >> stored as a uint, but all uses of HS_COUNT acr

Re: [HACKERS] Showing parallel status in \df+

2016-09-28 Thread Tom Lane
Pavel Stehule writes: > 2016-09-28 16:03 GMT+02:00 Tom Lane : >> I propose to push my current patch (ie, move PL function >> source code to \df+ footers), and we can use it in HEAD for awhile >> and see what we think. We can alway improve or revert it later. > I had some objection to format of s

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2016-09-28 Thread Robert Haas
On Tue, Sep 20, 2016 at 3:50 AM, Michael Paquier wrote: > On Mon, Sep 19, 2016 at 6:11 PM, Pavel Stehule > wrote: >> I am thinking so commit's description should be inside README > > Horiguchi-san, your patch has some whitespace issues, you may want to > get a run with git diff --check. Here are

Re: [HACKERS] Issue with bgworker, SPI and pgstat_report_stat

2016-09-28 Thread Robert Haas
On Sat, Sep 3, 2016 at 12:29 AM, Michael Paquier wrote: > On Sat, Sep 3, 2016 at 10:02 AM, Tomas Vondra > wrote: >> In any case, I think adding the pgstat_report_stat() into worker_spi seems >> like a reasonable (and backpatchable) fix. > > Doing just that sounds reasonable seen from here. I am w

Re: [HACKERS] [COMMITTERS] pgsql: pg_ctl: Detect current standby state from pg_control

2016-09-28 Thread Peter Eisentraut
On 9/28/16 12:44 AM, Michael Paquier wrote: > On Tue, Sep 27, 2016 at 9:55 AM, Michael Paquier > wrote: >> > Seems overcomplicated to me. How about returning the control file all >> > the time and let the caller pfree the result? You could then use >> > crc_ok in pg_ctl.c's get_control_dbstate() t

Re: [HACKERS] compiler warning read_objtype_from_string()

2016-09-28 Thread Alvaro Herrera
Peter Eisentraut wrote: > I'm getting the following compiler warning (using nondefault > optimization options): > > objectaddress.c: In function 'read_objtype_from_string': > objectaddress.c:2309:9: error: 'type' may be used uninitialized in this > function [-Werror=maybe-uninitialized] > return

Re: [HACKERS] "Some tests to cover hash_index"

2016-09-28 Thread Robert Haas
On Tue, Sep 20, 2016 at 2:26 PM, Alvaro Herrera wrote: > Why not use generate_series() queries to insert the appropriate number > of tuples, instead of a handful of INSERT lines each time? Since each > insert is a separate transaction, that would probably be faster. > > Why do you have a plpgsql

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-28 Thread Peter Geoghegan
On Wed, Sep 28, 2016 at 5:11 PM, Peter Geoghegan wrote: > This is why I never pursued batch memory for non-final merges. Isn't > that what you're doing here? You're pretty much always setting > "state->batchUsed = true". Wait. I guess you feel you have to, since it wouldn't be okay to use almost

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-28 Thread Peter Geoghegan
On Wed, Sep 28, 2016 at 5:04 PM, Heikki Linnakangas wrote: >> Not sure that I understand. I agree that each merge pass tends to use >> roughly the same number of tapes, but the distribution of real runs on >> tapes is quite unbalanced in earlier merge passes (due to dummy runs). >> It looks like y

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-28 Thread Heikki Linnakangas
On 09/28/2016 06:05 PM, Peter Geoghegan wrote: On Thu, Sep 15, 2016 at 9:51 PM, Heikki Linnakangas wrote: I don't think it makes much difference in practice, because most merge passes use all, or almost all, of the available tapes. BTW, I think the polyphase algorithm prefers to do all the merg

Re: [HACKERS] Cache Hash Index meta page.

2016-09-28 Thread Mithun Cy
On Tue, Sep 27, 2016 at 1:53 AM, Jeff Janes wrote: > I think that this needs to be updated again for v8 of concurrent and v5 of wal Adding the rebased patch over [1] + [2] [1] Concurrent Hash index.

[HACKERS] compiler warning read_objtype_from_string()

2016-09-28 Thread Peter Eisentraut
I'm getting the following compiler warning (using nondefault optimization options): objectaddress.c: In function 'read_objtype_from_string': objectaddress.c:2309:9: error: 'type' may be used uninitialized in this function [-Werror=maybe-uninitialized] return type; The comment for the function s

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-28 Thread Robert Haas
On Tue, Sep 27, 2016 at 5:15 PM, Tomas Vondra wrote: > So, I got the results from 3.10.101 (only the pgbench data), and it looks > like this: > > 3.10.101 1 8 16 32 64128192 > > granular-l

Re: [HACKERS] Remove superuser() checks from pgstattuple

2016-09-28 Thread Stephen Frost
Michael, * Michael Paquier (michael.paqu...@gmail.com) wrote: > On Tue, Sep 27, 2016 at 4:43 AM, Stephen Frost wrote: > > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > >> This is now being obsoleted by the later idea of allowing base installs > >> from a chain of upgrade scripts.

Re: [HACKERS] assert violation in logical messages serialization

2016-09-28 Thread Robert Haas
On Tue, Sep 27, 2016 at 10:45 AM, Stas Kelvich wrote: > During processing of logical messages in ReorderBufferSerializeChange() > pointer to ondisk structure isn’t updated after possible reorder buffer > realloc by > ReorderBufferSerializeReserve(). > > Actual reason spotted by Konstantin Knizhni

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-28 Thread David Steele
On 9/28/16 2:45 AM, Michael Paquier wrote: > On Tue, Sep 27, 2016 at 11:27 PM, David Steele wrote: >> On 9/26/16 2:36 AM, Michael Paquier wrote: >> >>> Just a nit: >>> >>> - postmaster.pid >>> + postmaster.pid and postmaster.opts >>> >>> Having one block for ea

Re: [HACKERS] Tuplesort merge pre-reading

2016-09-28 Thread Peter Geoghegan
On Thu, Sep 15, 2016 at 9:51 PM, Heikki Linnakangas wrote: >> I still don't get why you're doing all of this within mergeruns() (the >> beginning of when we start merging -- we merge all quicksorted runs), >> rather than within beginmerge() (the beginning of one particular merge >> pass, of which

Re: [HACKERS] psql casts aspersions on server reliability

2016-09-28 Thread David Steele
On 9/28/16 10:22 AM, Robert Haas wrote: > On Wed, Sep 28, 2016 at 9:14 AM, Tom Lane wrote: >> Robert Haas writes: >>> psql tends to do things like this: >>> rhaas=# select * from pg_stat_activity; >>> FATAL: terminating connection due to administrator command >>> server closed the connection une

  1   2   >