Re: New GUC to sample log queries

2018-11-18 Thread Adrien Nayrat
On 11/18/18 12:47 AM, Dmitry Dolgov wrote: > This patch went through last few commitfests without any activity, but cfbot > says it still applies and doesn't break any tests. From what I see there was > some agreement about naming, so the patch is indeed needs more review. Could > anyone from the r

Re: zheap: a new storage format for PostgreSQL

2018-11-18 Thread Komяpa
On Sat, Nov 17, 2018 at 8:51 AM Adam Brusselback wrote: > > I don't know how much what I write on this thread is read by others or > how useful this is for others who are following this work > > I've been following this thread and many others like it, silently soaking > it up, because I don't fe

Re: [HACKERS] pgbench - allow to store select results into variables

2018-11-18 Thread Fabien COELHO
Hello Alvaro, I think this patch's Command->lines would benefit from using PQExpBuffer (or maybe StringInfo?) for the command string instead of open-coding string manipulation and allocation. [...] Ok. I'm not sure that Command->first_line is really all that useful. It seems we go to a l

Re: New GUC to sample log queries

2018-11-18 Thread Dmitry Dolgov
> On Sun, 18 Nov 2018 at 10:52, Adrien Nayrat > wrote: > > For me, the question is how to name this GUC? Is "log_sample_rate" is enough? > I am not sure because it is only related to queries logged by > log_min_duration_statements. Since it's hard to come up with a concise name that will mention

[PATCH] Allow UNLISTEN during recovery

2018-11-18 Thread Shay Rojansky
Hi all, Here is a tiny patch removing PreventCommandDuringRecovery() for UNLISTEN. See previous discussion in https://www.postgresql.org/message-id/CADT4RqBweu7QKRYAYzeRW77b%2BMhJdUikNe45m%2BfL4GJSq_u2Fg%40mail.gmail.com . In a nutshell, this prevents an error being raised when UNLISTEN is issued

Re: _isnan() on Windows

2018-11-18 Thread Andrew Dunstan
On 11/17/18 2:46 PM, Andres Freund wrote: Hi, On 2018-07-12 11:28:46 -0400, Andrew Dunstan wrote: On 07/12/2018 10:38 AM, Tom Lane wrote: Andrew Dunstan writes: On 07/12/2018 10:20 AM, Tom Lane wrote: bowerbird and hamerkop have some gripes like this: bowerbird | c:\perl64\lib\core\w

Re: pg_dumpall --exclude-database option

2018-11-18 Thread Andrew Dunstan
On 11/17/18 9:55 AM, Alvaro Herrera wrote: The comment in expand_dbname_patterns is ungrammatical and mentions "OID" rather than "name", so I suggest /* * The loop below might sometimes result in duplicate entries in the * output name list, but we don't care.

Re: doc - improve description of default privileges

2018-11-18 Thread Fabien COELHO
Hello Tom, Thanks for this precise feedback. Progress on this patch seems to be blocked on the question of whether we want to keep enlarging the amount of psql-specific information in the GRANT reference page, or move that all somewhere else. Yep. FWIW, I think I agree with Peter's positio

Re: _isnan() on Windows

2018-11-18 Thread Andres Freund
Hi, On 2018-11-18 12:46:14 -0500, Andrew Dunstan wrote: > On 11/17/18 2:46 PM, Andres Freund wrote: > > On 2018-07-12 11:28:46 -0400, Andrew Dunstan wrote: > > > By inspection the perl header is just defining it to _isnan, for every MSC > > > version. > > Let's try undefining it before plperl.h th

docs should mention that max_wal_size default depends on WAL segment size

2018-11-18 Thread Tomas Vondra
Hi, while investigating something on a cluster with a non-default WAL segment (say 256MB), I've noticed a somewhat surprising behavior of max_wal_size default. While the docs claim the default is 1GB, the actual default depends on the WAL segment size. For example with the 256MB WAL segments, you

Re: Fixing AC_CHECK_DECLS to do the right thing with clang

2018-11-18 Thread Andres Freund
Hi, On 2018-11-17 23:32:47 -0500, Tom Lane wrote: > We've seen repeated complaints about bogus build warnings when using > "clang": it complains that strlcpy and some related library functions > haven't been declared. Several of the buildfarm animals exhibit such > warnings, for instance. That's

Re: docs should mention that max_wal_size default depends on WAL segment size

2018-11-18 Thread Andres Freund
Hi, On 2018-11-18 22:16:12 +0100, Tomas Vondra wrote: > while investigating something on a cluster with a non-default WAL > segment (say 256MB), I've noticed a somewhat surprising behavior of > max_wal_size default. While the docs claim the default is 1GB, the > actual default depends on the WAL s

Re: [HACKERS] WIP: Aggregation push-down

2018-11-18 Thread Tom Lane
Antonin Houska writes: > [ agg_pushdown_v8.tgz ] I spent a few hours looking at this today. It seems to me that at no point has there been a clear explanation of what the patch is trying to accomplish, so let me see whether I've got it straight or not. (I suggest that something like this ought

logical decoding vs. VACUUM FULL / CLUSTER on table with TOAST-ed data

2018-11-18 Thread Tomas Vondra
Hi, It seems we have pretty annoying problem with logical decoding when performing VACUUM FULL / CLUSTER on a table with toast-ed data. The trouble is that the rewritten heap is WAL-logged using XLOG/FPI records, the TOAST data is logged as regular INSERT records. XLOG/FPI is ignored in logical d

Re: Fixing AC_CHECK_DECLS to do the right thing with clang

2018-11-18 Thread Tom Lane
Andres Freund writes: > Seems like a good plan. The problem doesn't reproduce for me on debian > (using any version of clang), so all I can report is that at patched > build still works as it should. Interesting. It's hardly surprising that the problem would occur only on some platforms, since i

Re: docs should mention that max_wal_size default depends on WAL segment size

2018-11-18 Thread Tomas Vondra
On 11/18/18 10:22 PM, Andres Freund wrote: > Hi, > > On 2018-11-18 22:16:12 +0100, Tomas Vondra wrote: >> while investigating something on a cluster with a non-default WAL >> segment (say 256MB), I've noticed a somewhat surprising behavior of >> max_wal_size default. While the docs claim the de

Re: has_table_privilege for a table in unprivileged schema causes an error

2018-11-18 Thread Tom Lane
"David G. Johnston" writes: > On Sat, Aug 25, 2018 at 8:29 PM, Tom Lane wrote: >> There's a backwards-compatibility argument for not changing this behavior, >> sure, but I don't buy the other arguments you made here. And I don't >> think there's all that much to the backwards-compatibility argum

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-11-18 Thread Haribabu Kommi
On Sun, Nov 18, 2018 at 1:11 AM Alvaro Herrera wrote: > On 2018-Nov-17, Amit Kapila wrote: > > > On Sat, Nov 17, 2018 at 4:46 PM Alvaro Herrera > wrote: > > > > Uh, ouch! Seems to me that this is a high-caliber foot-gun, and will > > > cause nasty suprises when production stats data disappear >

Re: spurious(?) warnings in archive recovery

2018-11-18 Thread Vik Fearing
On 13/11/2018 16:34, Andrew Gierth wrote: > So while investigating a case of this warning (in > UpdateMinRecoveryPoint): > > "xlog min recovery request %X/%X is past current point %X/%X" > > I noticed that it is issued even in cases where we know that > minRecoveryPoint is not yet valid, for exam

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-11-18 Thread Michael Paquier
On Fri, Nov 16, 2018 at 03:23:55PM +0900, Michael Paquier wrote: > I was looking at this patch, and shouldn't we worry about compatibility > with plugins or utilities which look directly at what's in readRecordBuf > for the record contents? Let's not forget that the contents of > XLogReaderState a

Re: [HACKERS] PoC: full merge join on comparison clause

2018-11-18 Thread Tom Lane
Alexander Kuzmenkov writes: > [ Inequality-merge-join-v10.patch ] Just thinking about this patch a bit ... I wonder why you were so quick to reject the UNION approach at the outset. This patch is pretty messy, and it complicates a lot of stuff that is quite fundamental to the planner, and you st

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-11-18 Thread Michael Paquier
On Mon, Nov 19, 2018 at 10:41:22AM +1100, Haribabu Kommi wrote: > So 6 new functions needs to be added to cover all the above cases, > IMO, we may need functions for all combinations, because I feel some > user may have the requirement of left out combination, in case if we choose > only some combi

Re: New GUC to sample log queries

2018-11-18 Thread Michael Paquier
On Sun, Nov 18, 2018 at 12:18:33PM +0100, Dmitry Dolgov wrote: > Since it's hard to come up with a concise name that will mention sampling rate > in the context of min_duration_statement, I think it's fine to name this > configuration "log_sample_rate", as long as it's dependency from > log_min_dur

Re: WIP: Avoid creation of the free space map for small tables

2018-11-18 Thread John Naylor
On 11/16/18, Amit Kapila wrote: > +/* Status codes for the local map. */ > +#define FSM_LOCAL_ZERO 0x00 /* Beyond the end of the relation */ > +#define FSM_LOCAL_AVAIL 0x01 /* Available to try */ > +#define FSM_LOCAL_TRIED 0x02 /* Already tried, not enough space */ > > Instead of maintaining three

Re: ToDo: show size of partitioned table

2018-11-18 Thread Alvaro Herrera
On 2018-Jul-23, Pavel Stehule wrote: > p.s. Another patch can be replacement of relation type from "table" to > "partitioned table" > > postgres=# \dt+ > List of relations > +++---+---+-+-+ > | Schema |Na

Re: Psql patch to show access methods info

2018-11-18 Thread Michael Paquier
On Sat, Nov 17, 2018 at 11:20:50PM -0300, Alvaro Herrera wrote: > Here's a rebased version, fixing the rejects, pgindenting, and fixing > some "git show --check" whitespace issues. Haven't reviewed any further > than that. Schema qualifications are missing in many places, and they are added somet

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-11-18 Thread Alvaro Herrera
On 2018-Nov-19, Michael Paquier wrote: > On Mon, Nov 19, 2018 at 10:41:22AM +1100, Haribabu Kommi wrote: > > So 6 new functions needs to be added to cover all the above cases, > > IMO, we may need functions for all combinations, because I feel some > > user may have the requirement of left out com

Re: ToDo: show size of partitioned table

2018-11-18 Thread Amit Langote
On 2018/11/19 11:17, Alvaro Herrera wrote: > On 2018-Jul-23, Pavel Stehule wrote: > >> p.s. Another patch can be replacement of relation type from "table" to >> "partitioned table" >> >> postgres=# \dt+ >> List of relations >> +++---

Re: Psql patch to show access methods info

2018-11-18 Thread Alvaro Herrera
On 2018-Nov-19, Michael Paquier wrote: > On Sat, Nov 17, 2018 at 11:20:50PM -0300, Alvaro Herrera wrote: > > Here's a rebased version, fixing the rejects, pgindenting, and fixing > > some "git show --check" whitespace issues. Haven't reviewed any further > > than that. > > Schema qualifications

Re: ToDo: show size of partitioned table

2018-11-18 Thread Michael Paquier
On Sun, Nov 18, 2018 at 11:17:37PM -0300, Alvaro Herrera wrote: > To be certain I'm not going against some old decision, I digged up > Amit's old patches. Turns out he submitted psql's describe.c using the > term "partitioned table" on August 10th [1] and then based on a > discussion where Robert

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2018-11-18 Thread Amit Langote
On 2018/11/17 9:06, Michael Paquier wrote: > On Fri, Nov 16, 2018 at 09:38:40AM -0500, Robert Haas wrote: >> OK, but it seems to me that your version of my patch rearranges the >> code more than necessary. >> >> How about the attached? > > What you are proposing here looks good to me. Thanks! Me

Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

2018-11-18 Thread Kyotaro HORIGUCHI
At Sat, 17 Nov 2018 11:14:49 -0500, Tom Lane wrote in <23489.1542471...@sss.pgh.pa.us> > I wrote: > > After still further thought, it seems like "if (bytesread == 0)" > > is the right way to proceed. That protects us against incorrectly > > setting reached_eof if the pipe is being run in unbuffe

Re: Pull up sublink of type 'NOT NOT (expr)'

2018-11-18 Thread Richard Guo
Hi Tom, Thanks for your input. On Fri, Nov 16, 2018 at 6:06 AM, Tom Lane wrote: > Richard Guo writes: > > On Tue, Nov 13, 2018 at 10:05 AM, Tom Lane wrote: > >> What is the argument that this occurs often enough to be worth expending > >> extra cycles and code space on? > > > I am not using a

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-11-18 Thread Kyotaro HORIGUCHI
Thank you for taking this. At Mon, 19 Nov 2018 10:27:29 +0900, Michael Paquier wrote in <20181119012728.ga1...@paquier.xyz> > On Fri, Nov 16, 2018 at 03:23:55PM +0900, Michael Paquier wrote: > > I was looking at this patch, and shouldn't we worry about compatibility > > with plugins or utilities

Re: Early WIP/PoC for inlining CTEs

2018-11-18 Thread Craig Ringer
On Sat, 17 Nov 2018 at 10:12, Stephen Frost wrote: > Greetings, > > * Andrew Gierth (and...@tao11.riddles.org.uk) wrote: > > > "Tom" == Tom Lane writes: > > > > >> [ inlining-ctes-v5.patch ] > > > > Tom> I took a little bit of a look through this. Some thoughts: > > > > Tom> * I think it

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-11-18 Thread Amit Kapila
On Sat, Nov 17, 2018 at 7:41 PM Alvaro Herrera wrote: > > On 2018-Nov-17, Amit Kapila wrote: > > > On Sat, Nov 17, 2018 at 4:46 PM Alvaro Herrera > > wrote: > > > > Uh, ouch! Seems to me that this is a high-caliber foot-gun, and will > > > cause nasty suprises when production stats data disappe

Re: zheap: a new storage format for PostgreSQL

2018-11-18 Thread Amit Kapila
On Sun, Nov 18, 2018 at 3:42 PM Darafei "Komяpa" Praliaskouski wrote: > > On Sat, Nov 17, 2018 at 8:51 AM Adam Brusselback > wrote: >> >> > I don't know how much what I write on this thread is read by others or >> how useful this is for others who are following this work >> >> I've been followi

Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

2018-11-18 Thread Etsuro Fujita
(2018/11/17 8:10), Tom Lane wrote: I wrote: I'm not quite sure whether the reached_eof test should be "if (bytesread == 0)" or "if (bytesread< maxread)". The former seems more certain to indicate real EOF; are there other cases where the fread might return a short read? On the other hand, if

Re: Function to promote standby servers

2018-11-18 Thread Michael Paquier
On Fri, Oct 26, 2018 at 01:51:24PM +0900, Michael Paquier wrote: > Or we could use "the function returns true immediately after initiating > the promotion by sending the promotion signal to the postmaster"? As a > native speaker which one feels more natural to you? Sorry for the time it took. Af

Re: Accounting of zero-filled buffers in EXPLAIN (BUFFERS)

2018-11-18 Thread Kyotaro HORIGUCHI
At Sat, 17 Nov 2018 11:15:54 -0300, Alvaro Herrera wrote in <20181117141554.4dkx2u4j6md3bqdh@alvherre.pgsql> > Is this patch committable now? I don't think so. We should make a decision on a point. I was a bit confused (sorry) but IIUIC Haribabu suggested that the RBM_ZERO_ON_ERROR case should

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-11-18 Thread Michael Paquier
On Mon, Nov 19, 2018 at 12:28:06PM +0900, Kyotaro HORIGUCHI wrote: > Yeah, it is incorrect in some sense, but the comment was > suggesting the lifetime of the pointed record. So I'd like to > have a comment like this instead. I think that we can still live without as it is not the business of this

Re: [HACKERS] Restricting maximum keep segments by repslots

2018-11-18 Thread Michael Paquier
On Fri, Oct 26, 2018 at 11:26:36AM +0900, Kyotaro HORIGUCHI wrote: > The reason for doing that in the fucntion is it can happen also > for physical replication when walsender is active but far > behind. The removed(renamed)-but-still-open segment may be > recycled and can be overwritten while readi

Re: Postgres, fsync, and OSs (specifically linux)

2018-11-18 Thread Thomas Munro
On Fri, Nov 9, 2018 at 9:03 AM Thomas Munro wrote: > On Fri, Nov 9, 2018 at 7:07 AM Robert Haas wrote: > > On Wed, Nov 7, 2018 at 9:41 PM Thomas Munro > > wrote: > > > My plan is do a round of testing and review of this stuff next week > > > once the dust is settled on the current minor releases

Re: WIP: Avoid creation of the free space map for small tables

2018-11-18 Thread Amit Kapila
On Mon, Nov 19, 2018 at 7:30 AM John Naylor wrote: > > On 11/16/18, Amit Kapila wrote: > > +/* Status codes for the local map. */ > > +#define FSM_LOCAL_ZERO 0x00 /* Beyond the end of the relation */ > > +#define FSM_LOCAL_AVAIL 0x01 /* Available to try */ > > +#define FSM_LOCAL_TRIED 0x02 /* Alr

Re: New function pg_stat_statements_reset_query() to reset statistics of a specific query

2018-11-18 Thread Haribabu Kommi
On Mon, Nov 19, 2018 at 1:37 PM Alvaro Herrera wrote: > On 2018-Nov-19, Michael Paquier wrote: > > > On Mon, Nov 19, 2018 at 10:41:22AM +1100, Haribabu Kommi wrote: > > > So 6 new functions needs to be added to cover all the above cases, > > > IMO, we may need functions for all combinations, beca

Re: [PATCH] XLogReadRecord returns pointer to currently read page

2018-11-18 Thread Andrey Lepikhov
On 16.11.2018 11:23, Michael Paquier wrote: On Thu, Nov 15, 2018 at 06:12:38PM +0900, Kyotaro HORIGUCHI wrote: This patch eliminates unnecessary copying that was done for non-continued records. Now the return value of XLogReadRecord directly points into page buffer holded in XLogReaderStats. I

Re: In-place updates and serializable transactions

2018-11-18 Thread Amit Kapila
On Fri, Nov 16, 2018 at 4:07 AM Kevin Grittner wrote: > > On Thu, Nov 15, 2018 at 3:03 AM Kuntal Ghosh > wrote: > > > The test multiple-row-versions is failing because of the > > above-discussed scenario. I've attached the regression diff file and > > the result output file for the same. Here is

Re: In-place updates and serializable transactions

2018-11-18 Thread Kuntal Ghosh
On Fri, Nov 16, 2018 at 4:07 AM Kevin Grittner wrote: > > I would say this should be considered a resounding success. We should > probably add an alternative result file to cover this case, but > otherwise I don't see anything which requires action. > As of now, we've added an extra expected file

Re: NOTIFY and pg_notify performance when deduplicating notifications

2018-11-18 Thread Julien Demoor
On 10/10/2018 19:42, Catalin Iacob wrote: On Tue, Oct 9, 2018 at 2:17 PM wrote: I just caught an error in my patch, it's fixed in the attachment. The 'never' and 'maybe' collapse modes were mixed up in one location. Here's a partial review of this version, did not read the doc part very caref