Re: Have an encrypted pgpass file

2018-08-01 Thread Kyotaro HORIGUCHI
Hello. I have had complaints several times on lack of this kind of feature. At Wed, 1 Aug 2018 17:33:39 +0200, Marco van Eck wrote in > After explaining the patch to a college we identified potentially execution > of another user when it is defined in as a command parameter. To protect > again

RE: [HACKERS] Cached plans and statement generalization

2018-08-01 Thread Yamaji, Ryo
> -Original Message- > From: Konstantin Knizhnik [mailto:k.knizh...@postgrespro.ru] > Sent: Wednesday, August 1, 2018 4:53 PM > To: Yamaji, Ryo/山地 亮 > Cc: PostgreSQL mailing lists > Subject: Re: [HACKERS] Cached plans and statement generalization > > I failed to reproduce the problem. >

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-08-01 Thread Peter Geoghegan
On Wed, Aug 1, 2018 at 9:48 PM, Andrey Lepikhov wrote: > I use v3 version of the patch for a Retail Indextuple Deletion and from time > to time i catch regression test error (see attachment). > As i see in regression.diff, the problem is instability order of DROP ... > CASCADE deletions. > Most fr

Re: Explain buffers wrong counter with parallel plans

2018-08-01 Thread Amit Kapila
On Thu, Aug 2, 2018 at 8:38 AM, Andres Freund wrote: > Hi, > > On 2018-08-02 08:21:58 +0530, Amit Kapila wrote: >> I think something on the lines what Tom and you are suggesting can be >> done with the help of EXEC_FLAG_BACKWARD, but I don't see the need to >> do anything for this patch. The chan

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2018-08-01 Thread Andrey Lepikhov
I use v3 version of the patch for a Retail Indextuple Deletion and from time to time i catch regression test error (see attachment). As i see in regression.diff, the problem is instability order of DROP ... CASCADE deletions. Most frequently i get error on a test called 'updatable views'. I chec

Re: insert on conflict on updatable views

2018-08-01 Thread Amit Langote
On 2018/08/02 11:24, Peter Geoghegan wrote: > On Wed, Aug 1, 2018 at 7:15 PM, Amit Langote > wrote: >> create table foo (a int unique, b text); >> create view foo_view as select b, a+1 as c, a from foo; >> >> explain insert into foo_view (a, b) select 1, 2 on conflict (a) do update >> set b = excl

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-08-01 Thread David Rowley
On 31 July 2018 at 19:03, David Rowley wrote: > I've attached a complete v4 patch. I've attached v5 of the patch which is based on top of today's master (@ 579b985b22) A couple of recent patches conflict with v4. I've also made another tidy up pass, which was mostly just rewording comments in ex

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Thomas Munro
On Thu, Aug 2, 2018 at 1:20 PM, Alvaro Herrera wrote: > On 2018-Aug-02, Thomas Munro wrote: >> PostgreSQL only requires atomic writes of 512 bytes (see >> PG_CONTROL_MAX_SAFE_SIZE), the traditional sector size for disks made >> approximately 1980-2010, though as far as I know spinning disks made >

Re: Explain buffers wrong counter with parallel plans

2018-08-01 Thread Andres Freund
Hi, On 2018-08-02 08:21:58 +0530, Amit Kapila wrote: > I think something on the lines what Tom and you are suggesting can be > done with the help of EXEC_FLAG_BACKWARD, but I don't see the need to > do anything for this patch. The change in nodeLimit.c is any way for > forward scans, so there sho

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andres Freund
On 2018-08-01 21:20:22 -0400, Alvaro Herrera wrote: > On 2018-Aug-02, Thomas Munro wrote: > > > PostgreSQL only requires atomic writes of 512 bytes (see > > PG_CONTROL_MAX_SAFE_SIZE), the traditional sector size for disks made > > approximately 1980-2010, though as far as I know spinning disks mad

Re: Explain buffers wrong counter with parallel plans

2018-08-01 Thread Amit Kapila
On Wed, Aug 1, 2018 at 7:51 PM, Robert Haas wrote: > On Sat, Jul 28, 2018 at 2:14 AM, Amit Kapila wrote: >> We have done verification that the approach works and fixes the bug in >> all known cases. Do you see any problem with this approach? > > Regarding the change to execParallel.c, I think if

Re: insert on conflict on updatable views

2018-08-01 Thread Peter Geoghegan
On Wed, Aug 1, 2018 at 7:15 PM, Amit Langote wrote: > create table foo (a int unique, b text); > create view foo_view as select b, a+1 as c, a from foo; > > explain insert into foo_view (a, b) select 1, 2 on conflict (a) do update > set b = excluded.b where excluded.c > 0; > IOW, the EXCLUDED pse

insert on conflict on updatable views

2018-08-01 Thread Amit Langote
Hi. While working on updating the patch for the bug reported below: https://www.postgresql.org/message-id/flat/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg%40mail.gmail.com I noticed that the EXCLUDED pseudo-relation allows accessing columns that, ISTM, should rather be inaccessible. Exa

Re: Problems with plan estimates in postgres_fdw

2018-08-01 Thread Kyotaro HORIGUCHI
Hello. At Thu, 02 Aug 2018 01:06:41 +0100, Andrew Gierth wrote in <87pnz1aby9@news-spur.riddles.org.uk> > This analysis comes from investigating a report from an IRC user. A > summary of the initial report is: > > Using PG 9.6.9 and postgres_fdw, a query of the form "select * from > for

Re: [report] memory leaks in COPY FROM on partitioned table

2018-08-01 Thread Alvaro Herrera
On 2018-Aug-02, Amit Langote wrote: > On 2018/08/02 6:03, Alvaro Herrera wrote: > > Hmm, variable shadowing ... > > Crap, sorry about forgetting to remove that and thanks for taking care of > that. I think it was my bug actually, while rebasing to branch master after renaming the variable. --

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Alvaro Herrera
On 2018-Aug-02, Thomas Munro wrote: > PostgreSQL only requires atomic writes of 512 bytes (see > PG_CONTROL_MAX_SAFE_SIZE), the traditional sector size for disks made > approximately 1980-2010, though as far as I know spinning disks made > this decade use 4KB sectors, and for SSDs there is more va

Re: [report] memory leaks in COPY FROM on partitioned table

2018-08-01 Thread Amit Langote
On 2018/08/02 6:03, Alvaro Herrera wrote: > On 2018-Aug-01, Andres Freund wrote: > >> On 2018-08-01 16:38:11 -0400, Alvaro Herrera wrote: >>> On 2018-Jul-24, Amit Langote wrote: >>> Your patch takes care of allocation happening inside get_partition_for_tuple, but as you mention there mig

Re: [report] memory leaks in COPY FROM on partitioned table

2018-08-01 Thread Amit Langote
On 2018/08/02 5:38, Alvaro Herrera wrote: > On 2018-Jul-24, Amit Langote wrote: > >> Your patch takes care of allocation happening inside >> get_partition_for_tuple, but as you mention there might be others in its >> caller ExecFindPartition. So, I think we should switch to the per-tuple >> conte

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-08-01 Thread David Rowley
On 2 August 2018 at 11:48, Tom Lane wrote: > I've pushed the v3 patch with a lot of editorial work (e.g. cleaning > up comments you hadn't). Thanks for doing that. > >> In [1] I mentioned that I think that bug should be fixed as part of >> this bug fix too. > > I didn't include this change becau

[Patch] Create a new session in postmaster by calling setsid()

2018-08-01 Thread Paul Guo
Hello, Recently I encountered an issue during testing and rootcaused it as the title mentioned. postmaster children have done this (creating a new session) by calling InitPostmasterChild(), but postmaster itself does not. This could lead to some issues (e..g signal handling). The test script belo

Problems with plan estimates in postgres_fdw

2018-08-01 Thread Andrew Gierth
This analysis comes from investigating a report from an IRC user. A summary of the initial report is: Using PG 9.6.9 and postgres_fdw, a query of the form "select * from foreign_table order by col limit 1" is getting a local Sort plan, not pushing the ORDER BY to the remote. Turning off use_

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-08-01 Thread Tom Lane
David Rowley writes: > On 20 July 2018 at 01:03, David Rowley wrote: >> I've attached a patch intended for master which is just v2 based on >> post 5220bb7533. I've pushed the v3 patch with a lot of editorial work (e.g. cleaning up comments you hadn't). I still want to think about getting rid o

Re: Making "COPY partitioned_table FROM" faster

2018-08-01 Thread David Rowley
On 31 July 2018 at 11:51, David Rowley wrote: > The attached v6 delta replaces the v5 delta and should be applied on > top of the full v4 patch. (now committed) Many thanks for committing this Peter and many thanks to Melanie and Karen for reviewing it! -- David Rowley http:

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Thomas Munro
On Wed, Aug 1, 2018 at 11:06 PM, Andrey Borodin wrote: >> 1 авг. 2018 г., в 13:49, Thomas Munro >> написал(а): >> Hmm. This proposal doesn't seem to deal with torn writes. > > That's true, but it's a bit orthogonal to problem solved with checksums. > Checksums provide way to avoid reading bad p

Re: patch to ensure logical decoding errors early

2018-08-01 Thread Alvaro Herrera
On 2018-Aug-01, Dave Cramer wrote: > See attached patch which fixes it, and adds a test for it. Pushed, thanks. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Thomas Munro
On Thu, Aug 2, 2018 at 6:17 AM, Andres Freund wrote: > On 2018-08-01 11:14:18 -0700, Shawn Debnath wrote: >> On Wed, Aug 01, 2018 at 11:00:46AM -0700, Andres Freund wrote: >> > I believe you also planned to work on that, do I remember that >> > correctly, or is that just wishful thinking? >> >> Ye

Re: [report] memory leaks in COPY FROM on partitioned table

2018-08-01 Thread Alvaro Herrera
On 2018-Aug-01, Andres Freund wrote: > On 2018-08-01 16:38:11 -0400, Alvaro Herrera wrote: > > On 2018-Jul-24, Amit Langote wrote: > > > > > Your patch takes care of allocation happening inside > > > get_partition_for_tuple, but as you mention there might be others in its > > > caller ExecFindPar

Re: [report] memory leaks in COPY FROM on partitioned table

2018-08-01 Thread Andres Freund
On 2018-08-01 16:38:11 -0400, Alvaro Herrera wrote: > On 2018-Jul-24, Amit Langote wrote: > > > Your patch takes care of allocation happening inside > > get_partition_for_tuple, but as you mention there might be others in its > > caller ExecFindPartition. So, I think we should switch to the per-t

Re: [report] memory leaks in COPY FROM on partitioned table

2018-08-01 Thread Alvaro Herrera
On 2018-Aug-01, Alvaro Herrera wrote: > Right, makes sense. Pushed that way. KaiGai, if you can please confirm that the pushed change fixes your test case, I'd appreciate it. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Trainin

Re: [report] memory leaks in COPY FROM on partitioned table

2018-08-01 Thread Alvaro Herrera
On 2018-Jul-24, Amit Langote wrote: > Your patch takes care of allocation happening inside > get_partition_for_tuple, but as you mention there might be others in its > caller ExecFindPartition. So, I think we should switch to the per-tuple > context in ExecFindPartition. Right, makes sense. Pus

Re: [HACKERS] logical decoding of two-phase transactions

2018-08-01 Thread Andres Freund
On 2018-08-01 21:55:18 +0200, Petr Jelinek wrote: > On 01/08/18 16:00, Nikhil Sontakke wrote: > > > >> I was wondering if anything else would be needed for user-defined > >> catalog tables.. > >> > > > > We don't need to do anything else for user-defined catalog tables > > since they will also ge

Re: [HACKERS] logical decoding of two-phase transactions

2018-08-01 Thread Petr Jelinek
On 01/08/18 16:00, Nikhil Sontakke wrote: > >> I was wondering if anything else would be needed for user-defined >> catalog tables.. >> > > We don't need to do anything else for user-defined catalog tables > since they will also get accessed via the systable_* scan APIs. > They can be, but curr

Re: Alter index rename concurrently to

2018-08-01 Thread Andres Freund
On 2018-08-01 15:33:09 -0400, Robert Haas wrote: > On Wed, Aug 1, 2018 at 3:04 AM, Peter Eisentraut > wrote: > > On 31/07/2018 23:25, Tom Lane wrote: > >> Peter Eisentraut writes: > >>> On 27/07/2018 16:16, Robert Haas wrote: > I also suspect that an appropriate fix might be to ensure that >

Re: Alter index rename concurrently to

2018-08-01 Thread Robert Haas
On Wed, Aug 1, 2018 at 3:04 AM, Peter Eisentraut wrote: > On 31/07/2018 23:25, Tom Lane wrote: >> Peter Eisentraut writes: >>> On 27/07/2018 16:16, Robert Haas wrote: I also suspect that an appropriate fix might be to ensure that AcceptInvalidationMessages() is run at least once at the

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-01 Thread Robert Haas
On Wed, Aug 1, 2018 at 7:44 AM, Etsuro Fujita wrote: > I updated the patch that way. Updated patch attached. I fixed a bug and > did a bit of cleanups as well. Looking this over from a technical point of view, I think it's better than what you proposed before but I still don't agree with the ap

Re: Online enabling of checksums

2018-08-01 Thread Sergei Kornilov
Hi > This doesn't test the consequences of the restart being skipped, nor > does it review on a code level the correctness. I check not only one stuff during review. I look code too: bgworker checksumhelper.c registered with: > bgw.bgw_start_time = BgWorkerStart_RecoveryFinished; And then process

Re: Allow COPY's 'text' format to output a header

2018-08-01 Thread Simon Muller
On 1 August 2018 at 17:18, Cynthia Shang wrote: > > > On Aug 1, 2018, at 10:20 AM, Daniel Verite > wrote: > > > > /* Check header */ > > - if (!cstate->csv_mode && cstate->header_line) > > + if (cstate->binary && cstate->header_line) > > ereport(ERROR, > > - (errcode(ERRCODE_FEATURE_NOT

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-08-01 Thread Tom Lane
[ getting back to this thread at last ] Amit Langote writes: > On 2018/07/21 0:17, David Rowley wrote: >> You could work around that by having some array that points to the >> target partitioned table of each hierarchy, but I don't see why that's >> better than having the additional struct. > Or

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andres Freund
On 2018-08-01 11:14:18 -0700, Shawn Debnath wrote: > On Wed, Aug 01, 2018 at 11:00:46AM -0700, Andres Freund wrote: > > I believe you also planned to work on that, do I remember that > > correctly, or is that just wishful thinking? > > Yep - I am actively working on this at the moment, planning on

Re: Online enabling of checksums

2018-08-01 Thread Andres Freund
Hi, On 2018-08-01 21:03:22 +0300, Sergei Kornilov wrote: > > They fail over to a secondary to do maintenance on a primary. > But this is not problem even in current patch state. We can restart replica > before failover and it works. I tested this behavior during my review. > We can: > - call pg_e

Re: Online enabling of checksums

2018-08-01 Thread Sergei Kornilov
Hi > They fail over to a secondary to do maintenance on a primary. But this is not problem even in current patch state. We can restart replica before failover and it works. I tested this behavior during my review. We can: - call pg_enable_data_checksums() on master - wait change data_checksums to

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andres Freund
Hi, On 2018-08-01 10:58:03 -0700, Shawn Debnath wrote: > On Wed, Aug 01, 2018 at 02:06:44PM +0300, Andrey Borodin wrote: > > But adding LSNs, and whole regular PageHeader is quite easy in this > > patch. Do you think we should really go that way? > > > > Putting SLRUs into usual shared buffers a

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-01 Thread Robert Haas
On Wed, Aug 1, 2018 at 7:46 AM, Etsuro Fujita wrote: > I posted the updated patch [1]. Etsuro-san: I really think we should just go with what Ashutosh had proposed. Tom, Ashutosh, and I all seem to agree that we shouldn't try to re-jigger things at create-plan time. I think that a 3-1 consensus

Re: Online enabling of checksums

2018-08-01 Thread Andres Freund
Hi, On 2018-08-01 10:34:55 -0700, Joshua D. Drake wrote: > Lastly, I think Alvaro has a point with the incremental development and I > also think some others on this thread need to, "show me the patch" instead > of being armchair directors of development. Oh, FFS. I pointed out the issue that led

Re: Memory leak with CALL to Procedure with COMMIT.

2018-08-01 Thread Jonathan S. Katz
> On Jul 23, 2018, at 3:06 AM, Michael Paquier wrote: > > On Mon, Jul 23, 2018 at 12:19:12PM +0530, Prabhat Sahu wrote: >> While testing with PG procedure, I found a memory leak on HEAD, with below >> steps: >> >> postgres=# CREATE OR REPLACE PROCEDURE proc1(v1 INOUT INT) >> AS $$ >> BEGIN >> c

Re: Online enabling of checksums

2018-08-01 Thread Joshua D. Drake
On 08/01/2018 09:20 AM, Alvaro Herrera wrote: my problem is that I think the "restart" approach is just using the entirely wrong hammer to solve the problem at hand. At the very least it's very problematic in respect to replicas, which need to know about the setting too, and can have similar p

RE: GSOC 2018 Project - A New Sorting Routine

2018-08-01 Thread Kefan Yang
Thanks for your time! From: Tomas Vondra Sent: August 1, 2018 6:30 AM To: Kefan Yang Cc: Andrey Borodin; Peter Geoghegan; alvhe...@2ndquadrant.com; PostgreSQL Hackers Subject: Re: GSOC 2018 Project - A New Sorting Routine On 07/30/2018 11:21 PM, Kefan Yang wrote: > Hey Tomas! > > Sorry to both

Re: Online enabling of checksums

2018-08-01 Thread Andres Freund
Hi, On 2018-08-01 12:45:27 -0400, Bruce Momjian wrote: > On Wed, Aug 1, 2018 at 09:39:43AM -0700, Andres Freund wrote: > > On 2018-08-01 12:36:13 -0400, Bruce Momjian wrote: > > > This patchset is weird because it is perhaps our first case of trying to > > > change the state of the server while i

Re: Online enabling of checksums

2018-08-01 Thread Bruce Momjian
On Wed, Aug 1, 2018 at 09:39:43AM -0700, Andres Freund wrote: > On 2018-08-01 12:36:13 -0400, Bruce Momjian wrote: > > This patchset is weird because it is perhaps our first case of trying to > > change the state of the server while it is running. We just don't have > > an established protocol fo

Re: Online enabling of checksums

2018-08-01 Thread Andres Freund
On 2018-08-01 12:36:13 -0400, Bruce Momjian wrote: > This patchset is weird because it is perhaps our first case of trying to > change the state of the server while it is running. We just don't have > an established protocol for how to orchestrate that, so we are limping > along toward a solution.

Re: Online enabling of checksums

2018-08-01 Thread Bruce Momjian
On Tue, Jul 31, 2018 at 04:05:23PM -0700, Andres Freund wrote: > Hi, > > On 2018-07-31 18:56:29 -0400, Alvaro Herrera wrote: > > In the spirit of supporting incremental development, I think it's quite > > sensible to get the current thing done, then see what it takes to get > > the next thing done

Re: Online enabling of checksums

2018-08-01 Thread Andres Freund
Hi, On 2018-08-01 18:25:48 +0200, Tomas Vondra wrote: > Sure, if there are issues with this approach, that would make it > unacceptable. I'm not sure why would it be an issue for replicas (which is > what you mention elsewhere), considering those don't write data and so can't > fail to update a ch

Re: Online enabling of checksums

2018-08-01 Thread Andres Freund
On 2018-08-01 12:20:12 -0400, Alvaro Herrera wrote: > Hello > > On 2018-Aug-01, Andres Freund wrote: > > > My problem isn't just that I shouldn't think this should be committed > > without at least a firm committement to do better, > > I take "I think this shouldn't be committed" is what you mea

Re: Online enabling of checksums

2018-08-01 Thread Tomas Vondra
On 08/01/2018 05:58 PM, Andres Freund wrote: Hi, On 2018-08-01 11:15:38 +0200, Tomas Vondra wrote: On 08/01/2018 10:40 AM, Michael Banck wrote: If this was one week before feature freeze, I would agree with you that it makes sense to ship it with the restart requirement rather than not shippin

Ideas for a relcache test mode about missing invalidations

2018-08-01 Thread Andres Freund
Hi, The issue at [1] is caused by missing invalidations, and [2] seems like a likely candidate too. I wonder if it'd be good to have a relcache test mode akin to CLOBBER_CACHE_ALWAYS and RELCACHE_FORCE_RELEASE, that tries to ensure that we've done sufficiently to ensure the right invalidations are

Re: Online enabling of checksums

2018-08-01 Thread Alvaro Herrera
Hello On 2018-Aug-01, Andres Freund wrote: > My problem isn't just that I shouldn't think this should be committed > without at least a firm committement to do better, I take "I think this shouldn't be committed" is what you meant. I'm not sure I agree with this line of argument. The reality i

Re: Online enabling of checksums

2018-08-01 Thread Andres Freund
Hi, On 2018-08-01 11:15:38 +0200, Tomas Vondra wrote: > On 08/01/2018 10:40 AM, Michael Banck wrote: > > If this was one week before feature freeze, I would agree with you > > that it makes sense to ship it with the restart requirement rather > > than not shipping it at all. But we're several co

Re: Online enabling of checksums

2018-08-01 Thread Andres Freund
Hi, On 2018-08-01 10:40:24 +0200, Michael Banck wrote: > If this was one week before feature freeze, I would agree with you that > it makes sense to ship it with the restart requirement rather than not > shipping it at all. But we're several commitfests away from v12, so > making an effort to havi

Re: Have an encrypted pgpass file

2018-08-01 Thread Marco van Eck
After explaining the patch to a college we identified potentially execution of another user when it is defined in as a command parameter. To protect agains it I've removed the possibility to pass the 'passcommand'. With the result libpq only allows the PGPASSCOMMAND environment variable, which can

Re: Explain buffers wrong counter with parallel plans

2018-08-01 Thread Tom Lane
Robert Haas writes: > Looking at it, this sounds suspiciously something where we could just > test EXEC_FLAG_BACKWARD, based on the comments for that flag, but I > wouldn't be willing to bet very much on me being right about that. > Can somebody try to fetch backward even if this flag wasn't provi

Re: Allow COPY's 'text' format to output a header

2018-08-01 Thread Cynthia Shang
> On Aug 1, 2018, at 10:20 AM, Daniel Verite wrote: > > /* Check header */ > - if (!cstate->csv_mode && cstate->header_line) > + if (cstate->binary && cstate->header_line) > ereport(ERROR, > - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), > -errmsg("COPY HEADER available only in CSV mod

Re: patch to ensure logical decoding errors early

2018-08-01 Thread Dave Cramer
On 1 August 2018 at 10:13, Petr Jelinek wrote: > Hi, > > On 31/07/18 20:58, Andres Freund wrote> > >> diff --git a/src/backend/replication/logical/logical.c > b/src/backend/replication/logical/logical.c > >> index 3cd4eef..9f883b9 100644 > >> --- a/src/backend/replication/logical/logical.c > >> +

Re: Explain buffers wrong counter with parallel plans

2018-08-01 Thread Robert Haas
On Wed, Aug 1, 2018 at 10:27 AM, Tom Lane wrote: > Robert Haas writes: >> ... But scanning backwards is >> a problem. I'm not exactly sure what the best way of handling that >> is, but one thing I think might work is to save ExecutePlan's >> execute_once flag in the EState and then make the call

Re: Internal error XX000 with enable_partition_pruning=on, pg 11 beta1 on Debian

2018-08-01 Thread David Rowley
On 20 July 2018 at 01:03, David Rowley wrote: > I've attached a patch intended for master which is just v2 based on > post 5220bb7533. In [1] I mentioned that I think that bug should be fixed as part of this bug fix too. It just seems a little strange to fix that one separately when without the v

Re: FailedAssertion on partprune

2018-08-01 Thread David Rowley
On 31 July 2018 at 11:25, Alvaro Herrera wrote: > Not looking at the code right now either, but removing that assert and > then removing the TABLESAMPLE clause, the query returns identical > results with and without pruning, so maybe you're right. No time for > further looking now. I thought abo

Re: Explain buffers wrong counter with parallel plans

2018-08-01 Thread Tom Lane
Robert Haas writes: > ... But scanning backwards is > a problem. I'm not exactly sure what the best way of handling that > is, but one thing I think might work is to save ExecutePlan's > execute_once flag in the EState and then make the call in nodeLimit.c > and the one in ExecutePlan itself cond

Re: Explain buffers wrong counter with parallel plans

2018-08-01 Thread Robert Haas
On Sat, Jul 28, 2018 at 2:14 AM, Amit Kapila wrote: > We have done verification that the approach works and fixes the bug in > all known cases. Do you see any problem with this approach? Regarding the change to execParallel.c, I think if you're going to move that code you should add a comment ex

Re: Allow COPY's 'text' format to output a header

2018-08-01 Thread Daniel Verite
Simon Muller wrote: > I've incorporated both your suggestions and included the patch you provided > in the attached patch. Hope it's as expected. Still unconvinced about the use case, since COPY's text format is only meant to be consumed by Postgres, and the only way that Postgres will consume

Re: patch to ensure logical decoding errors early

2018-08-01 Thread Petr Jelinek
Hi, On 31/07/18 20:58, Andres Freund wrote> >> diff --git a/src/backend/replication/logical/logical.c >> b/src/backend/replication/logical/logical.c >> index 3cd4eef..9f883b9 100644 >> --- a/src/backend/replication/logical/logical.c >> +++ b/src/backend/replication/logical/logical.c >> @@ -143,8

Re: patch to ensure logical decoding errors early

2018-08-01 Thread Dave Cramer
On 31 July 2018 at 14:58, Andres Freund wrote: > Hi, > > On 2018-07-31 14:51:12 -0400, Dave Cramer wrote: > > This patch does 2 things > > > > 1) Ensure that when the slot is created > > with pg_create_physical_replication_slot if the output plugin does not > > exist it will error. > > *logical,

Re: New Defects reported by Coverity Scan for PostgreSQL

2018-08-01 Thread Emre Hasegeli
> I think there are three different things that need to be addressed: > > * Underspecified comments. I agree. My patch added comments, the next one with actual fixes adds more. I just didn't want to invest even more time on them while the code is its current shape. > * The function names and ar

Re: New Defects reported by Coverity Scan for PostgreSQL

2018-08-01 Thread Tom Lane
Tomas Vondra writes: > On 08/01/2018 11:55 AM, Emre Hasegeli wrote: >> Consistency. I organized all xxx_closept_yyy(Point *result, xxx *l1, >> yyy *l2) functions in a way that they find the find the point on "l1". > IMHO the main issue here is that the rule is not obvious / documented > anywher

Re: [HACKERS] logical decoding of two-phase transactions

2018-08-01 Thread Nikhil Sontakke
Hi, PFA, latest patchset which incorporates the additional feedback. >>> There's an additional test case in >>> 0005-Additional-test-case-to-demonstrate-decoding-rollbac.patch which >>> uses a sleep in the "change" plugin API to allow a concurrent rollback >>> on the 2PC being currently decoded.

Re: doc - add missing documentation for "acldefault"

2018-08-01 Thread Pavel Luzanov
Fabien, On 01.08.2018 15:28, Fabien COELHO wrote: I couldn't find the documentation. Attached patch adds one. Probably this function should have been named pg_*. Too late. I think this is intentionally hidden function, like others started with acl*. postgres=# \df acl* List of fun    S

Re: [HACKERS] Can ICU be used for a database's default sort order?

2018-08-01 Thread Daniel Verite
Andrey Borodin wrote: > Overall patch looks solid and thoughtful work and adds important > functionality. I tried the patch, with some minor changes to build with HEAD. I was surprised by the interface, that is, the fact that a user is not allowed to freely choose the ICU collation of a

Re: New Defects reported by Coverity Scan for PostgreSQL

2018-08-01 Thread Tomas Vondra
On 08/01/2018 11:55 AM, Emre Hasegeli wrote: Or perhaps I have it backwards and "l1" and "l2" need to be swapped in that description. But the mere fact that there is any question about that means that the function is poorly documented and perhaps poorly named as well. For that matter, is the

Re: partition tree inspection functions

2018-08-01 Thread Robert Haas
On Thu, Jul 26, 2018 at 4:47 AM, Amit Langote wrote: > Alright, I have replaced pg_partition_tree_tables with > pg_partition_children with an 'include_all' argument, as you suggested, > but I implemented it as an optional argument. So, one would use that > argument only if need to get *all* parti

Re: [HACKERS] Parallel Append implementation

2018-08-01 Thread Robert Haas
On Mon, Jul 30, 2018 at 8:02 PM, Thomas Munro wrote: > On Tue, Jul 31, 2018 at 5:05 AM, Robert Haas wrote: >> New version attached. > > Looks good to me. Committed to master and v11. Thanks for the review. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Comp

Re: Usability fail with psql's \dp command

2018-08-01 Thread Pavel Luzanov
Fabien, On 31.07.2018 22:02, Fabien COELHO wrote: Indeed, all \d* which display perms have the empty/default confusion:   \dp \ddp \des \dew \l \dn \db \df \dT \dD \dL I fixed them all to display the default acl in the patch I just sent. I also noticed that although large objects have permiss

Connection slots reserved for replication

2018-08-01 Thread Alexander Kukushkin
Hello hackers, at the moment it is possible to reserve some amount of connection slots for superusers and this behavior is controlled by superuser_reserved_connections configuration parameter with the default value = 3. In case if all non-reserved connection slots are busy, replica fails to open

doc - add missing documentation for "acldefault"

2018-08-01 Thread Fabien COELHO
I couldn't find the documentation. Attached patch adds one. Probably this function should have been named pg_*. Too late. -- Fabien.diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index edc9be92a6..0e5f8b914b 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -1639

Re: Problem while updating a foreign table pointing to a partitioned table on foreign server

2018-08-01 Thread Etsuro Fujita
(2018/06/12 12:19), Kyotaro HORIGUCHI wrote: I have demonstrated and actually shown a problem of the PARAM_EXEC case. A. Just detecting and reporting/erroring the problematic case. B. Giving to Sort-like nodes an ability to convert PARAMS into junk columns. C. Adding a space for 64bit tu

Re: [HACKERS] Bug in to_timestamp().

2018-08-01 Thread Arthur Zakirov
Hello, Alexander, On Mon, Jul 23, 2018 at 05:21:43PM +0300, Alexander Korotkov wrote: > Thank you, Arthur. These examples shows downside of this patch, where > users may be faced with incompatibility. But it's good that this > situation can be handled by altering format string. I think these >

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-01 Thread Etsuro Fujita
(2018/08/01 5:31), Alvaro Herrera wrote: On 2018-Jul-31, Etsuro Fujita wrote: (2018/07/31 4:06), Andres Freund wrote: On 2018-07-20 08:38:09 -0400, Robert Haas wrote: I'm going to study this some more now, but I really think this is going in the wrong direction. We're going to have to get so

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-08-01 Thread Etsuro Fujita
(2018/07/26 21:11), Etsuro Fujita wrote: (2018/07/26 5:27), Robert Haas wrote: Well, I could have the wrong idea here, but I tend to think allowing for ConvertRowTypeExpr elsewhere won't be that bad. I still don't like that because in my opinion, changes needed for that would not be localized,

Re: [PATCH] Improve geometric types

2018-08-01 Thread Emre Hasegeli
> Hmmm. It'll be difficult to review such patch without access to a platform > exhibiting such behavior ... IIRC IBM offers free access to open-source > devs, I wonder if that would be a way. I don't have access to such platform either, and I don't know too much about this business. I left this p

Re: Doc patch: add RECURSIVE to bookindex

2018-08-01 Thread Fabien COELHO
Hello Daniel, Patch applies cleanly, doc build ok, works for me. I have added it to the next CF and marked it as ready. -- Fabien.

Re: Standby trying "restore_command" before local WAL

2018-08-01 Thread Emre Hasegeli
> There's still a question here, at least from my perspective, as to which > is actually going to be faster to perform recovery based off of. A good > restore command, which pre-fetches the WAL in parallel and gets it local > and on the same filesystem, meaning that the restore_command only has to

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Andrey Borodin
Hi! > 1 авг. 2018 г., в 13:49, Thomas Munro > написал(а): > > Hmm. This proposal doesn't seem to deal with torn writes. That's true, but it's a bit orthogonal to problem solved with checksums. Checksums provide way to avoid reading bad page, torn pages - is about preventing writing bad writes

Re: BUG #15182: Canceling authentication due to timeout aka Denial of Service Attack

2018-08-01 Thread ahsan hadi
Hi Michael, Can you rebase the reindex-priv-93.patch, it is getting some hunk failures. patching file doc/src/sgml/ref/reindex.sgml Hunk #1 succeeded at 227 (offset 13 lines). patching file src/backend/commands/indexcmds.c Hunk #1 FAILED at 1873. 1 out of 1 hunk FAILED -- saving rejects to file

Re: [Patch] Checksums for SLRU files

2018-08-01 Thread Thomas Munro
On Wed, Jul 18, 2018 at 5:54 PM, Thomas Munro wrote: > On Wed, Jul 18, 2018 at 5:41 PM, Andrey Borodin wrote: >>> I think we'd want pg_upgrade tests showing an example of each SLRU >>> growing past one segment, and then being upgraded, and then being >>> accessed in various different pages and se

Re: Online enabling of checksums

2018-08-01 Thread Sergei Kornilov
Hello I think one restart is acceptable for such feature. I doubt user want often disable-enable checksums. In many cases checksums will be enabled one time during all cluster life. We need more downtimes for minor updates (4/year) and changes in config PGC_POSTMASTER (max_connections or autovac

Re: New Defects reported by Coverity Scan for PostgreSQL

2018-08-01 Thread Emre Hasegeli
> Or perhaps I have it backwards and "l1" and "l2" need to be swapped in > that description. But the mere fact that there is any question about > that means that the function is poorly documented and perhaps poorly > named as well. For that matter, is there a good reason why l1/l2 > have those ro

Re: New Defects reported by Coverity Scan for PostgreSQL

2018-08-01 Thread Tomas Vondra
On 08/01/2018 04:22 AM, Tom Lane wrote: > Ning Yu writes: >> From my point of view it's better to also put some comments for humans to >> understand why we are passing l1 and l2 in reverse order. > > The header comment for lseg_closept_lseg() is pretty far from adequate > as well. After studying

Re: Online enabling of checksums

2018-08-01 Thread Tomas Vondra
On 08/01/2018 10:40 AM, Michael Banck wrote: > Hi, > > Am Dienstag, den 31.07.2018, 18:56 -0400 schrieb Alvaro Herrera: >> The ability to get checksums enabled is a killer feature; the >> ability to do it with no restart ... okay, it's better than >> requiring a restart, but it's not *that* big a

Re: Fw: Windows 10 got stuck with PostgreSQL at starting up. Adding delay lets it avoid.

2018-08-01 Thread Yugo Nagata
On Fri, 20 Jul 2018 10:48:15 -0400 Tom Lane wrote: > Yugo Nagata writes: > > Recently, one of our clients reported a problem that Windows 10 sometime > > (approximately once in 300 tries) hung up at OS starting up while PostgreSQL > > 9.3.x service is starting up. My co-worker analyzed this and

Re: Online enabling of checksums

2018-08-01 Thread Michael Banck
Hi, Am Dienstag, den 31.07.2018, 18:56 -0400 schrieb Alvaro Herrera: > The ability to get checksums enabled is a killer feature; the ability to > do it with no restart ... okay, it's better than requiring a restart, > but it's not *that* big a deal. Well, it's a downtime and service interruption

Re: [HACKERS] Cached plans and statement generalization

2018-08-01 Thread Konstantin Knizhnik
On 31.07.2018 12:12, Yamaji, Ryo wrote: 3. I confirmed the transition of the amount of the memory when it tried to prepare query of the number that exceeded the value specified for autoprepare_limit. [autoprepare_limit=1 and execute 10 different queries] plan cache context: 1032 used

Re: [HACKERS] Cached plans and statement generalization

2018-08-01 Thread Konstantin Knizhnik
On 01.08.2018 00:30, Konstantin Knizhnik wrote: Hi Yamaji, On 31.07.2018 12:12, Yamaji, Ryo wrote: -Original Message- From: Konstantin Knizhnik [mailto:k.knizh...@postgrespro.ru] Sent: Friday, January 12, 2018 9:53 PM To: Thomas Munro ; Stephen Frost Cc: Michael Paquier ; PostgreSQL

Re: Problem while setting the fpw with SIGHUP

2018-08-01 Thread Kyotaro HORIGUCHI
Thank you, Amit, Michael. At Sun, 29 Jul 2018 08:19:11 +0900, Michael Paquier wrote in <20180728231911.gb1...@paquier.xyz> > On Sat, Jul 28, 2018 at 07:10:24PM +0530, Amit Kapila wrote: > > I have just responded to your first patch (0001). Can you once again > > summarize what the 0002 exactly

  1   2   >