Re: Postgres 11 release notes

2018-09-29 Thread Amit Langote
On Fri, Sep 28, 2018 at 7:24 PM Bruce Momjian wrote: > On Fri, Sep 28, 2018 at 10:21:16AM +0900, Amit Langote wrote: > > On 2018/09/27 23:24, Alvaro Herrera wrote: > > > On 2018-Sep-27, Amit Langote wrote: > > > > > >> Sorry I couldn't reply sooner, but the following of your proposed text > > >> n

Re: Optional message to user when terminating/cancelling backend

2018-09-29 Thread Pavel Stehule
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed I tested this patch, and all looks well and functional. I rer

Re: Progress reporting for pg_verify_checksums

2018-09-29 Thread Fabien COELHO
The time() granularity to the second makes the result awkward on small tests: 8/1554552 kB (0%, 8 kB/s) 1040856/1554552 kB (66%, 1040856 kB/s) 1554552/1554552 kB (100%, 1554552 kB/s) I'd suggest to reuse the "portability/instr_time.h" stuff which allows a much better precision. I still thin

Re: [HACKERS] kqueue

2018-09-29 Thread Matteo Beccati
On 28/09/2018 14:19, Thomas Munro wrote: > On Fri, Sep 28, 2018 at 11:09 AM Andres Freund wrote: >> On 2018-09-28 10:55:13 +1200, Thomas Munro wrote: >>> Matteo Beccati reported a 5-10% performance drop on a >>> low-end Celeron NetBSD box which we have no explanation for, and we >>> have no report

Re: doc - improve description of default privileges

2018-09-29 Thread Fabien COELHO
The Owner column is redundant, because it's always all applicable privileges. (Having this column would give the impression that it's not always all privileges, so it would be confusing.) The reason I put the owner column is to show (1) the privileges that apply to the objects (i.e. what is

Re: Use durable_unlink for .ready and .done files for WAL segment removal

2018-09-29 Thread Michael Paquier
On Fri, Sep 28, 2018 at 07:16:25PM -0400, Stephen Frost wrote: > An alternative would be to go through the .ready files on crash-recovery > and remove any .ready files that don't have corresponding WAL files, or > if we felt that it was necessary, we could do that on every restart but > do we reall

Re: Online verification of checksums

2018-09-29 Thread Tomas Vondra
Hi, On 09/26/2018 05:15 PM, Michael Banck wrote: > ... > > New version 5 attached. > I've looked at v5, and the retry/recheck logic seems OK to me - I'd still vote to keep it consistent with what pg_basebackup does (i.e. doing the LSN check first, before looking at the checksum), but I don't th

Re: [HACKERS] proposal: schema variables

2018-09-29 Thread Pavel Stehule
so 22. 9. 2018 v 8:00 odesílatel Pavel Stehule napsal: > Hi > > rebased against yesterday changes in tab-complete.c > rebased against last changes in master Regards Pavel > Regards > > Pavel > schema-variables-20180929-01.patch.gz Description: application/gzip

Re: Online enabling of checksums

2018-09-29 Thread Tomas Vondra
Hi, While looking at the online checksum verification patch (which I guess will get committed before this one), it occurred to me that disabling checksums may need to be more elaborate, to protect against someone using the stale flag value (instead of simply switching to "off" assuming that's fine

Re: Online verification of checksums

2018-09-29 Thread Tomas Vondra
Hi, One more thought - when running similar tools on a live system, it's usually a good idea to limit the impact by throttling the throughput. As the verification runs in an independent process it can't reuse the vacuum-like cost limit directly, but perhaps it could do something similar? Like, lim

Re: Online verification of checksums

2018-09-29 Thread Michael Paquier
On Sat, Sep 29, 2018 at 10:51:23AM +0200, Tomas Vondra wrote: > One more thought - when running similar tools on a live system, it's > usually a good idea to limit the impact by throttling the throughput. As > the verification runs in an independent process it can't reuse the > vacuum-like cost lim

Re: Online verification of checksums

2018-09-29 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Sat, Sep 29, 2018 at 10:51:23AM +0200, Tomas Vondra wrote: > > One more thought - when running similar tools on a live system, it's > > usually a good idea to limit the impact by throttling the throughput. As > > the verification runs

Re: Online enabling of checksums

2018-09-29 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > While looking at the online checksum verification patch (which I guess > will get committed before this one), it occurred to me that disabling > checksums may need to be more elaborate, to protect against someone > using the stale f

Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread David Hedberg
Hello, I recently wanted a way to encrypt/decrypt backups while still utilizing the parallel dump/restore functionality. I couldn't see a way to do this so I experimented a bit with the directory backup format. If there's in fact already a way to do this, please tell me now :-) The idea is to add

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-09-29 Thread Andrew Dunstan
On 09/29/2018 01:36 AM, Andrew Gierth wrote: Mark> What should I try next? What is the size of a C "int" on this platform? 4. cheers andrew -- Andrew Dunstanhttps://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: speeding up planning with partitions

2018-09-29 Thread Dilip Kumar
On Fri, Sep 14, 2018 at 3:58 PM, Amit Langote wrote: > Thanks a lot for your detailed review. > I was going through your patch (v3-0002) and I have some suggestions 1. - if (nparts > 0) + /* + * For partitioned tables, we just allocate space for RelOptInfo's. + * pointers for all partitions and c

Re: Postgres 11 release notes

2018-09-29 Thread Bruce Momjian
On Sat, Sep 29, 2018 at 04:28:13PM +0900, Amit Langote wrote: > > The default partition can store rows that don't match any of the > > other defined partitions, and is searched accordingly. > > I was commenting on the Jonathan's patch upthread [1] whereby he > proposed to add a lin

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread Stephen Frost
Greetings, * David Hedberg (david.hedb...@gmail.com) wrote: > I recently wanted a way to encrypt/decrypt backups while still > utilizing the parallel dump/restore functionality. I couldn't see a > way to do this so I experimented a bit with the directory backup > format. If there's in fact already

Cygwin linking rules

2018-09-29 Thread Tom Lane
Most of the buildfarm is now happy with the changes I made to have libpq + ecpg get src/port and src/common files via libraries ... but lorikeet isn't. It gets through the core regression tests fine (so libpq, per se, works), but contrib/dblink fails: ! ERROR: could not establish connection ! DE

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread Tom Lane
Stephen Frost writes: > * David Hedberg (david.hedb...@gmail.com) wrote: >> The idea is to add a --pipe option to pg_dump / pg_restore where you >> can specify a custom shell command that is used to write / read each >> .dat-file. Usage examples include encryption with pgp and/or custom >> compres

Re: Online verification of checksums

2018-09-29 Thread Tomas Vondra
On 09/29/2018 02:14 PM, Stephen Frost wrote: > Greetings, > > * Michael Paquier (mich...@paquier.xyz) wrote: >> On Sat, Sep 29, 2018 at 10:51:23AM +0200, Tomas Vondra wrote: >>> One more thought - when running similar tools on a live system, it's >>> usually a good idea to limit the impact by t

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread David Fetter
On Sat, Sep 29, 2018 at 11:42:40AM -0400, Tom Lane wrote: > Stephen Frost writes: > > * David Hedberg (david.hedb...@gmail.com) wrote: > >> The idea is to add a --pipe option to pg_dump / pg_restore where > >> you can specify a custom shell command that is used to write / > >> read each .dat-file.

Re: Online enabling of checksums

2018-09-29 Thread Tomas Vondra
On 09/29/2018 02:19 PM, Stephen Frost wrote: > Greetings, > > * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: >> While looking at the online checksum verification patch (which I guess >> will get committed before this one), it occurred to me that disabling >> checksums may need to be more ela

Re: Cygwin linking rules

2018-09-29 Thread Andrew Dunstan
On 09/29/2018 11:35 AM, Tom Lane wrote: Most of the buildfarm is now happy with the changes I made to have libpq + ecpg get src/port and src/common files via libraries ... but lorikeet isn't. It gets through the core regression tests fine (so libpq, per se, works), but contrib/dblink fails:

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread David Hedberg
On Sat, Sep 29, 2018 at 5:56 PM, David Fetter wrote: > On Sat, Sep 29, 2018 at 11:42:40AM -0400, Tom Lane wrote: >> Stephen Frost writes: >> > * David Hedberg (david.hedb...@gmail.com) wrote: >> >> The idea is to add a --pipe option to pg_dump / pg_restore where >> >> you can specify a custom she

Re: Implementing SQL ASSERTION

2018-09-29 Thread Joe Wildish
On 26 Sep 2018, at 12:36, Peter Eisentraut wrote: > > On 25/09/2018 01:04, Joe Wildish wrote: >> Having said all that: there are obviously going to be some expressions >> that cannot be proven to have no potential for invalidating the assertion >> truth. I guess this is the prime concern from a

Re: Implementing SQL ASSERTION

2018-09-29 Thread Joe Wildish
Hi Andrew, On 25 Sep 2018, at 01:51, Andrew Gierth wrote: > I haven't looked at the background of this, but if what you want to know > is whether the aggregate function has the semantics of min() or max() > (and if so, which) then the place to look is pg_aggregate.aggsortop. Thanks for the point

Re: Online enabling of checksums

2018-09-29 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > On 09/29/2018 02:19 PM, Stephen Frost wrote: > > * Tomas Vondra (tomas.von...@2ndquadrant.com) wrote: > >> While looking at the online checksum verification patch (which I guess > >> will get committed before this one), it occurred

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread Stephen Frost
Greetings, * David Hedberg (david.hedb...@gmail.com) wrote: > On Sat, Sep 29, 2018 at 5:56 PM, David Fetter wrote: > > On Sat, Sep 29, 2018 at 11:42:40AM -0400, Tom Lane wrote: > > As I understand it, those are the options for providing secrets in > > general. At least in the case of encryption,

Re: Cygwin linking rules

2018-09-29 Thread Andrew Dunstan
On 09/29/2018 12:09 PM, Andrew Dunstan wrote: On 09/29/2018 11:35 AM, Tom Lane wrote: Most of the buildfarm is now happy with the changes I made to have libpq + ecpg get src/port and src/common files via libraries ... but lorikeet isn't.  It gets through the core regression tests fine (so l

Re: Implementing SQL ASSERTION

2018-09-29 Thread Joe Wildish
Hi David, > On 26 Sep 2018, at 19:47, David Fetter wrote: > >> Invalidating operations are "INSERT(t) and UPDATE(t.b, t.n)". > > So would DELETE(t), assuming n can be negative. Oops, right you are. Bug in my implementation :-) > Is there some interesting and fairly easily documented subset o

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread David Hedberg
Hi, On Sat, Sep 29, 2018 at 7:01 PM, Stephen Frost wrote: > Greetings, > > * David Hedberg (david.hedb...@gmail.com) wrote: >> On Sat, Sep 29, 2018 at 5:56 PM, David Fetter wrote: >> > On Sat, Sep 29, 2018 at 11:42:40AM -0400, Tom Lane wrote: >> > As I understand it, those are the options for pr

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread Andres Freund
Hi, On 2018-09-29 14:51:33 +0200, David Hedberg wrote: > I recently wanted a way to encrypt/decrypt backups while still > utilizing the parallel dump/restore functionality. I couldn't see a > way to do this so I experimented a bit with the directory backup > format. If there's in fact already a wa

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread Stephen Frost
Greetings, * David Hedberg (david.hedb...@gmail.com) wrote: > On Sat, Sep 29, 2018 at 7:01 PM, Stephen Frost wrote: > > * David Hedberg (david.hedb...@gmail.com) wrote: > >> On Sat, Sep 29, 2018 at 5:03 PM, Stephen Frost wrote: > >> Generally, my thinking is that this can be pretty useful in gen

Re: Cygwin linking rules

2018-09-29 Thread Marco Atzeri
Am 29.09.2018 um 19:03 schrieb Andrew Dunstan: On 09/29/2018 12:09 PM, Andrew Dunstan wrote: On 09/29/2018 11:35 AM, Tom Lane wrote: Most of the buildfarm is now happy with the changes I made to have libpq + ecpg get src/port and src/common files via libraries ... but lorikeet isn't.  It ge

Re: Cygwin linking rules

2018-09-29 Thread Andrew Dunstan
On 09/29/2018 01:03 PM, Andrew Dunstan wrote: On 09/29/2018 12:09 PM, Andrew Dunstan wrote: On 09/29/2018 11:35 AM, Tom Lane wrote: Most of the buildfarm is now happy with the changes I made to have libpq + ecpg get src/port and src/common files via libraries ... but lorikeet isn't.  It

Re: Adding pipe support to pg_dump and pg_restore

2018-09-29 Thread David Hedberg
Hi, On Sat, Sep 29, 2018 at 8:03 PM, Stephen Frost wrote: > Greetings, > > * David Hedberg (david.hedb...@gmail.com) wrote: >> On Sat, Sep 29, 2018 at 7:01 PM, Stephen Frost wrote: >> > * David Hedberg (david.hedb...@gmail.com) wrote: >> >> On Sat, Sep 29, 2018 at 5:03 PM, Stephen Frost wrote:

Re: executor relation handling

2018-09-29 Thread Tom Lane
David Rowley writes: > I've attached v10 which fixes this and aligns the WARNING text in > ExecInitRangeTable() and addRangeTableEntryForRelation(). I started poking at this. I thought that it would be a good cross-check to apply just the "front half" of 0001 (i.e., creation and population of th

Re: Cygwin linking rules

2018-09-29 Thread Tom Lane
Andrew Dunstan writes: > Not sure where to go from here. What would happen if we stopped building libpq.a, so that the linker didn't have any choice about what to do? regards, tom lane

Re: MERGE SQL statement for PG12

2018-09-29 Thread Jaime Casanova
On Mon, 24 Sep 2018 at 05:15, Pavan Deolasee wrote: > > A new version rebased against the current master is attached. > Hi Pavan, A day after you posted this patch commit 29c94e03c7d05d2b29afa1de32795ce178531246 removed ExecStoreTuple. I'm right in believe that the change in src/backend/executor

Re: Continue work on changes to recovery.conf API

2018-09-29 Thread Sergei Kornilov
Hello thank you for review! >>  - if present both standby.signal and recovery.signal - we use standby mode >>  (this design from previous thread) > > Didn't find the discussion on that and don't understand the reasons, but > seems OK and easy to change if we don't like it. I meant this was implem

Re: libpq host/hostaddr/conninfo inconsistencies

2018-09-29 Thread Arthur Zakirov
Hello, On Fri, Aug 24, 2018 at 11:22:47AM +0200, Fabien COELHO wrote: > Attached is a rebase after 5ca00774. I looked a little bit the patch. And I have a few notes. > However, the actual capability is slightly different: specifying an ip > address to "host" does work, without ensuing any name o

Re: executor relation handling

2018-09-29 Thread Tom Lane
I wrote: > I started poking at this. I thought that it would be a good cross-check > to apply just the "front half" of 0001 (i.e., creation and population of > the RTE lockmode field), and then to insert checks in each of the > "back half" places (executor, plancache, etc) that the lockmodes they

Re: [HACKERS] proposal: schema variables

2018-09-29 Thread Pavel Stehule
using content of schema variable for estimation + subtransaction support I hope so now, there are almost complete functionality. Please, check it. Regards Pavel > > Regards > > Pavel > > > >> Regards >> >> Pavel >> > schema-variables-20180929-02.patch.gz Description: application/gzip

Re: Cygwin linking rules

2018-09-29 Thread Andrew Dunstan
On 09/29/2018 04:06 PM, Tom Lane wrote: Andrew Dunstan writes: Not sure where to go from here. What would happen if we stopped building libpq.a, so that the linker didn't have any choice about what to do? I will test Marco's patch, which I think does that, tomor

Re: Implementing SQL ASSERTION

2018-09-29 Thread Andrew Gierth
> "Joe" == Joe Wildish writes: >> I haven't looked at the background of this, but if what you want to >> know is whether the aggregate function has the semantics of min() or >> max() (and if so, which) then the place to look is >> pg_aggregate.aggsortop. Joe> Thanks for the pointer. I'v

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-09-29 Thread Andrew Gierth
> "Andrew" == Andrew Dunstan writes: >> What is the size of a C "int" on this platform? Andrew> 4. Hmm. Because int being more than 32 bits is the simplest explanation for this difference. How about the output of this query: with d(a) as (values ('----':

Re: libpq host/hostaddr/conninfo inconsistencies

2018-09-29 Thread Thomas Munro
On Sat, Aug 25, 2018 at 7:25 AM Tom Lane wrote: > Fabien COELHO writes: > > Attached is a rebase after 5ca00774. > > I notice that the cfbot thinks that *none* of your pending patches apply > successfully. I tried this one locally and what I get is > > $ patch -p1 <~/libpq-host-ip-2.patch > (Str

Re: [HACKERS] kqueue

2018-09-29 Thread Thomas Munro
On Sat, Sep 29, 2018 at 7:51 PM Matteo Beccati wrote: > On 28/09/2018 14:19, Thomas Munro wrote: > > On Fri, Sep 28, 2018 at 11:09 AM Andres Freund wrote: > >> On 2018-09-28 10:55:13 +1200, Thomas Munro wrote: > >>> Matteo Beccati reported a 5-10% performance drop on a > >>> low-end Celeron NetBS

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-09-29 Thread Tom Lane
Andrew Gierth writes: > Because int being more than 32 bits is the simplest explanation for this > difference. Curious to hear your reasoning behind that statement? I hadn't gotten further than "memcmp is broken" ... and neither of those theories is tenable, because if they were true then a lot

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-09-29 Thread Andrew Gierth
> "Tom" == Tom Lane writes: > Andrew Gierth writes: >> Because int being more than 32 bits is the simplest explanation for >> this difference. Tom> Curious to hear your reasoning behind that statement? I hadn't Tom> gotten further than "memcmp is broken" ... and neither of those Tom>

Re: SerializeParamList vs machines with strict alignment

2018-09-29 Thread Amit Kapila
On Mon, Sep 10, 2018 at 7:05 PM Tom Lane wrote: > > Amit Kapila writes: > > On Mon, Sep 10, 2018 at 8:58 AM Tom Lane wrote: > >> In particular, SerializeParamList does this: > >> > >> /* Write flags. */ > >> memcpy(*start_address, &prm->pflags, sizeof(uint16)); > >> *start_address += sizeof(uint

Re: [HACKERS] [PATCH] Generic type subscripting

2018-09-29 Thread Pavel Stehule
ne 30. 9. 2018 v 0:21 odesílatel Dmitry Dolgov <9erthali...@gmail.com> napsal: > > On Fri, 20 Jul 2018 at 23:32, Dmitry Dolgov <9erthali...@gmail.com> > wrote: > > > > > On Thu, 26 Apr 2018 at 16:44, Dmitry Dolgov <9erthali...@gmail.com> > wrote: > > > > > > > On 22 March 2018 at 23:25, Dmitry Dol