Re: [PATCH v1] Add \echo_stderr to psql

2019-04-22 Thread Fabien COELHO
Hello Corey, \warn ... \warning ... These two seem about the best to me, drawing from the perl warn command. Yep, I was thinking of perl & gmake. Maybe the 4 letter option is better because its the same length as "echo". I suppose we could go the bash &2 route here, but I don't w

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Kyotaro HORIGUCHI
Hello. At Mon, 22 Apr 2019 12:36:43 +0800, Paul Guo wrote in > Please see my replies inline. Thanks. > > On Fri, Apr 19, 2019 at 12:38 PM Asim R P wrote: > > > On Wed, Apr 17, 2019 at 1:27 PM Paul Guo wrote: > > > > > > create db with tablespace > > > drop database > > > drop tablespace. >

Re: block-level incremental backup

2019-04-22 Thread Konstantin Knizhnik
On 22.04.2019 2:02, Robert Haas wrote: On Sat, Apr 20, 2019 at 4:32 PM Stephen Frost wrote: Having been around for a while working on backup-related things, if I was to implement the protocol for pg_basebackup today, I'd definitely implement "give me a list" and "give me this file" rather th

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Kyotaro HORIGUCHI
Oops! The comment in the previous patch is wrong. At Mon, 22 Apr 2019 16:15:13 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20190422.161513.258021727.horiguchi.kyot...@lab.ntt.co.jp> > At Mon, 22 Apr 2019 12:36:43 +0800, Paul Guo wrote in > > > Please see my replies inline. Thank

display of variables in EXPLAIN VERBOSE

2019-04-22 Thread Amit Langote
Hi, ISTM show_plan_tlist()'s rule of whether to the show range table prefix with displayed variables contradicts the description of the VERBOSE option in EXPLAIN documentation, which is as follows: === VERBOSE Display additional information regarding the plan. Specifically, include the outpu

Allow any[] as input arguments for sql/plpgsql functions to mimic format()

2019-04-22 Thread phoe
Hey everyone, I am writing a plpgsql function that (to greatly simplify) raises an exception with a formatted* message. Ideally, I should be able to call it with raise_exception('The person %I has only %I bananas.', 'Fred', 8), which mimics the format(text, any[]) calling convention. Here is wher

Re: Allow any[] as input arguments for sql/plpgsql functions to mimic format()

2019-04-22 Thread Pavel Stehule
Hi po 22. 4. 2019 v 11:27 odesílatel Michał "phoe" Herda napsal: > Hey everyone, > > I am writing a plpgsql function that (to greatly simplify) raises an > exception with a formatted* message. Ideally, I should be able to call > it with raise_exception('The person %I has only %I bananas.', 'Fred

Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Kuntal Ghosh
Hello hackers, With a master-standby setup configured on the same machine, I'm getting a panic in tablespace test while running make installcheck. 1. CREATE TABLESPACE regress_tblspacewith LOCATION 'blah'; 2. DROP TABLESPACE regress_tblspacewith; 3. CREATE TABLESPACE regress_tblspace LOCATION 'bl

Re: bug in update tuple routing with foreign partitions

2019-04-22 Thread Etsuro Fujita
(2019/04/19 14:39), Etsuro Fujita wrote: (2019/04/19 13:00), Amit Langote wrote: On 2019/04/18 22:10, Etsuro Fujita wrote: On 2019/04/18 14:06, Amit Langote wrote: On 2019/04/17 21:49, Etsuro Fujita wrote: So what I'm thinking is to throw an error for cases like this. (Though, I think we shou

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-04-22 Thread Kyotaro HORIGUCHI
At Mon, 22 Apr 2019 16:40:27 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20190422.164027.33866403.horiguchi.kyot...@lab.ntt.co.jp> > At Mon, 22 Apr 2019 16:15:13 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI > wrote in > <20190422.161513.258021727.horiguchi.kyot...@lab.ntt.co.jp

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Kyotaro HORIGUCHI
Hello. At Mon, 22 Apr 2019 15:52:59 +0530, Kuntal Ghosh wrote in > Hello hackers, > > With a master-standby setup configured on the same machine, I'm > getting a panic in tablespace test while running make installcheck. > > 1. CREATE TABLESPACE regress_tblspacewith LOCATION 'blah'; > 2. DROP

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Etsuro Fujita
(2019/04/20 20:53), Laurenz Albe wrote: On Fri, 2018-04-06 at 23:24 +, Robert Haas wrote: Allow insert and update tuple routing and COPY for foreign tables. Also enable this for postgres_fdw. Etsuro Fujita, based on an earlier patch by Amit Langote. The larger patch series of which this is

Re: Regression test PANICs with master-standby setup on same machine

2019-04-22 Thread Kuntal Ghosh
On Mon, Apr 22, 2019 at 6:07 PM Kyotaro HORIGUCHI wrote: > If you don't have a problem using TAP test suite, tablespace is > allowed with a bit restricted steps using the first patch in my > just posted patchset[1]. This will work for you if you are okay > with creating a standby after creating a

Re: Runtime pruning problem

2019-04-22 Thread David Rowley
On Fri, 19 Apr 2019 at 05:25, Tom Lane wrote: > So what I'm thinking is that I made a bad decision in 1cc29fe7c, > which did this: > > ... In passing, simplify the EXPLAIN code by > having it deal primarily in the PlanState tree rather than separately > searching Plan and PlanState tre

Re: Unhappy about API changes in the no-fsm-for-small-rels patch

2019-04-22 Thread Amit Kapila
On Fri, Apr 19, 2019 at 2:46 PM Amit Kapila wrote: > > > > > I am thinking that we should at least give it a try to move the map to > > > rel cache level to see how easy or difficult it is and also let's wait > > > for a day or two to see if Andres/Tom has to say anything about this > > > or on th

Re: display of variables in EXPLAIN VERBOSE

2019-04-22 Thread David Rowley
On Mon, 22 Apr 2019 at 19:49, Amit Langote wrote: > Seeing that "Sort Key" is always displayed with the range table alias, I > checked explain.c to see why the discrepancy exists and it seems that > show_plan_tlist() (and show_tablesample()) use the following condition for > whether or not to use

Re: Do CustomScan as not projection capable node

2019-04-22 Thread Robert Haas
On Fri, Apr 19, 2019 at 12:45 AM Tom Lane wrote: > I don't buy this for a minute. Where do you think projection is > going to happen? There isn't any existing node type that *couldn't* > support projection if we insisted that that be done across-the-board. > I think it's mostly just a legacy thi

Re: [PATCH v1] Add \echo_stderr to psql

2019-04-22 Thread David Fetter
On Mon, Apr 22, 2019 at 09:04:08AM +0200, Fabien COELHO wrote: > > Hello Corey, > > > >\warn ... > > >\warning ... > > > > These two seem about the best to me, drawing from the perl warn command. > > Yep, I was thinking of perl & gmake. Maybe the 4 letter option is better > because its

Re: proposal: psql PSQL_TABULAR_PAGER variable

2019-04-22 Thread Robert Haas
On Thu, Apr 18, 2019 at 11:46 AM Pavel Stehule wrote: > My idea is following - pseudocode > > if view is a table > { > if is_defined PSQL_TABULAR_PAGER > { > pager = PSQL_TABULAR_PAGER > } > else if is_defined PSQL_PAGER > { > pager = PSQL_PAGER > } > else > { > pager =

Re: proposal: psql PSQL_TABULAR_PAGER variable

2019-04-22 Thread Tom Lane
Robert Haas writes: > Seems like pspg could just hand off to the regular pager if it > discovers that the input is not in a format it finds suitable. It might be slightly tricky to do that after having already consumed some of the input :-(. Still, I've got to say that I find this proposal prett

Re: proposal: psql PSQL_TABULAR_PAGER variable

2019-04-22 Thread Pavel Stehule
po 22. 4. 2019 v 15:46 odesílatel Robert Haas napsal: > On Thu, Apr 18, 2019 at 11:46 AM Pavel Stehule > wrote: > > My idea is following - pseudocode > > > > if view is a table > > { > > if is_defined PSQL_TABULAR_PAGER > > { > > pager = PSQL_TABULAR_PAGER > > } > > else if is_define

Re: proposal: psql PSQL_TABULAR_PAGER variable

2019-04-22 Thread Pavel Stehule
po 22. 4. 2019 v 16:21 odesílatel Tom Lane napsal: > Robert Haas writes: > > Seems like pspg could just hand off to the regular pager if it > > discovers that the input is not in a format it finds suitable. > > It might be slightly tricky to do that after having already consumed > some of the in

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Robert Haas
On Wed, Apr 17, 2019 at 6:06 PM Alvaro Herrera wrote: > > > 3. Partitioned relations can have the database tablespace in > > >pg_class.reltablespace, as opposed to storage-bearing relations which > > >cannot. This is useful to be able to put partitions in the database > > >tablespace

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > Andres has suggested that I work on teaching nbtree to accommodate > variable-width, logical table identifiers, such as those required for > indirect indexes, or clustered indexes, where secondary indexes must > use a logical primary key value i

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Sun, Apr 21, 2019 at 06:24:50PM -0400, Robert Haas wrote: > On Sat, Apr 20, 2019 at 5:54 PM Bruce Momjian wrote: > > Good point. You mentioned: > > > > It seems better to me to give the files names like > > ${TLI}.${STARTLSN}.${ENDLSN}.modblock, e.g. > > 0001.00

make \d pg_toast.foo show its indices

2019-04-22 Thread Justin Pryzby
It's deliberate that \dt doesn't show toast tables. \d shows them, but doesn't show their indices. It seems to me that their indices should be shown, without having to think and know to query pg_index. postgres=# \d pg_toast.pg_toast_2600 TOAST table "pg_toast.pg_toast_2600" Column | Type

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2019 at 11:20:43AM +0900, Michael Paquier wrote: > On Sat, Apr 20, 2019 at 12:21:36AM -0400, Robert Haas wrote: > > The segment size doesn't have much to do with it. If you make > > segments bigger, you'll have to scan fewer larger ones; if you make > > them smaller, you'll have mo

Re: display of variables in EXPLAIN VERBOSE

2019-04-22 Thread Tom Lane
David Rowley writes: > I'd vote to make the code match the documentation, but probably > implement it by adding a new field to ExplainState and just calculate > what to do once in ExplainQuery() instead of calculating what to do in > various random places. Yeah, this is none too consistent: $ gr

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Robert Haas
On Sun, Apr 21, 2019 at 10:21 PM Michael Paquier wrote: > If you create the extra file when a segment is finished and we switch > to a new one, then the extra work would happen for a random backend, > and it is going to be more costly to scan a 1GB segment than a 16MB > segment as a one-time opera

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 14:48:26 +0900, Michael Paquier wrote: > On Fri, Apr 19, 2019 at 09:43:01AM -0500, Justin Pryzby wrote: > > Thanks for committing those portions. > > I have done an extra pass on your patch set to make sure that I am > missing nothing, and the last two remaining places which nee

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 11:48 AM Bruce Momjian wrote: > My point is that most WAL archive tools will order and remove files > based on their lexical ordering, so if you put the start first, the file > will normally be removed when it should be kept, e.g., if you have WAL > files like: > >

Re: clean up docs for v12

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Andres Freund wrote: > On 2019-04-22 14:48:26 +0900, Michael Paquier wrote: > > /* > > -* Check if's guaranteed the all the desired attributes are available in > > -* tuple. If so, we can start deforming. If not, need to make sure to > > -* fetch the missing column

Re: clean up docs for v12

2019-04-22 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Apr-22, Andres Freund wrote: >> On 2019-04-22 14:48:26 +0900, Michael Paquier wrote: >>> /* >>> -* Check if's guaranteed the all the desired attributes are available in >>> -* tuple. If so, we can start deforming. If not, need to make sure to >>> -* fet

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Andres Freund
Hi, On 2019-04-21 17:46:09 -0700, Peter Geoghegan wrote: > Andres has suggested that I work on teaching nbtree to accommodate > variable-width, logical table identifiers, such as those required for > indirect indexes, or clustered indexes, where secondary indexes must > use a logical primary key v

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2019 at 12:15:32PM -0400, Robert Haas wrote: > On Mon, Apr 22, 2019 at 11:48 AM Bruce Momjian wrote: > > My point is that you would normally only remove the modblock file when 4 > > is removed because this modblock files is useful for incremental backups > > from base backups that

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 12:33:24 -0400, Tom Lane wrote: > ISTM that Michael's proposed wording change shows that the existing > comment is easily misinterpreted. I don't think these minor grammatical > fixes will avoid the misinterpretation problem, and so some more-extensive > rewording is called for.

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 09:43:56 -0700, Andres Freund wrote: > On 2019-04-22 12:33:24 -0400, Tom Lane wrote: > > ISTM that Michael's proposed wording change shows that the existing > > comment is easily misinterpreted. I don't think these minor grammatical > > fixes will avoid the misinterpretation pro

Optimizer items in the release notes

2019-04-22 Thread Bruce Momjian
We had a discussion in October about adding more optimizer items to the release notes: https://www.postgresql.org/message-id/flat/20181010220601.GA7807%40momjian.us#11d805ea0b0fcd0552dfa99251417cc1 There was no agreement on a change, but if people want to propose a change, please post he

Trouble with FETCH_COUNT and combined queries in psql

2019-04-22 Thread Daniel Verite
Hi, When FETCH_COUNT is set, queries combined in a single request don't work as expected: \set FETCH_COUNT 10 select pg_sleep(2) \; select 1; No result is displayed, the pg_sleep(2) is not run, and no error is shown. That's disconcerting. The sequence that is sent under the hood is: #1 BEG

Re: Unhappy about API changes in the no-fsm-for-small-rels patch

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 18:49:44 +0530, Amit Kapila wrote: > Attached is a hacky and work-in-progress patch to move fsm map to > relcache. This will give you some idea. I think we need to see if > there is a need to invalidate the relcache due to this patch. I think > some other comments of Andres al

Re: block-level incremental backup

2019-04-22 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Sat, Apr 20, 2019 at 4:32 PM Stephen Frost wrote: > > Having been around for a while working on backup-related things, if I > > was to implement the protocol for pg_basebackup today, I'd definitely > > implement "give me a list" and "gi

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 12:35 PM Bruce Momjian wrote: > I assumed the modblock files would be stored in the WAL archive so some > external tools could generate incremental backups using just the WAL > files. I assumed they would also be sent to standby servers so > incremental backups could be do

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Mon, Apr 22, 2019 at 01:11:22PM -0400, Robert Haas wrote: > On Mon, Apr 22, 2019 at 12:35 PM Bruce Momjian wrote: > > I assumed the modblock files would be stored in the WAL archive so some > > external tools could generate incremental backups using just the WAL > > files. I assumed they would

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 8:36 AM Stephen Frost wrote: > This seems like it would be helpful for global indexes as well, wouldn't > it? Yes, though that should probably work by reusing what we already do with heap TID (use standard IndexTuple fields on the leaf level for heap TID), plus an addition

Re: clean up docs for v12

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 12:33:24 -0400, Tom Lane wrote: >> But TBH, now that I look at the code, I think the entire optimization >> is a bad idea and should be removed. Am I right in thinking that the >> presence of a wrong attnotnull marker could cause the generated code to >> actu

Re: Do CustomScan as not projection capable node

2019-04-22 Thread Andrey Lepikhov
On 22/04/2019 18:40, Robert Haas wrote: On Fri, Apr 19, 2019 at 12:45 AM Tom Lane wrote: I don't buy this for a minute. Where do you think projection is going to happen? There isn't any existing node type that *couldn't* support projection if we insisted that that be done across-the-board.

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 9:35 AM Andres Freund wrote: > I, more generally, wonder if there's not a case to squeeze out more > padding than "just" what you describe (since we IIRC don't frequently > keep pointers into such tuples anyway, and definitely don't for byval > attrs). But that's very likel

Re: clean up docs for v12

2019-04-22 Thread Tom Lane
Andres Freund writes: > The computation of that variable above has: >* If the column is possibly missing, we can't rely on its (or >* subsequent) NOT NULL constraints to indicate minimum > attributes in >* the tuple, so stop here. >

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Stephen Frost
Greetings, * Peter Geoghegan (p...@bowt.ie) wrote: > On Mon, Apr 22, 2019 at 8:36 AM Stephen Frost wrote: > > This seems like it would be helpful for global indexes as well, wouldn't > > it? > > Yes, though that should probably work by reusing what we already do > with heap TID (use standard Ind

Re: block-level incremental backup

2019-04-22 Thread Andres Freund
Hi, On 2019-04-19 20:04:41 -0400, Stephen Frost wrote: > I agree that we don't want another implementation and that there's a lot > that we want to do to improve replay performance. We've already got > frontend tools which work with multiple execution threads, so I'm not > sure I get the "not eas

Re: Trouble with FETCH_COUNT and combined queries in psql

2019-04-22 Thread Fabien COELHO
Bonjour Daniel, When FETCH_COUNT is set, queries combined in a single request don't work as expected: \set FETCH_COUNT 10 select pg_sleep(2) \; select 1; No result is displayed, the pg_sleep(2) is not run, and no error is shown. That's disconcerting. Indeed. Does anyone have thoughts abo

Re: block-level incremental backup

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 1:08 PM Stephen Frost wrote: > > I think we're getting closer to a meeting of the minds here, but I > > don't think it's intrinsically necessary to rewrite the whole method > > of operation of pg_basebackup to implement incremental backup in a > > sensible way. > > It wasn'

Re: Thoughts on nbtree with logical/varwidth table identifiers, v12 on-disk representation

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 10:32 AM Stephen Frost wrote: > Yes, global indexes for partitioned tables could potentially be simpler > than the logical row identifiers, but maybe it'd be useful to just have > one implementation based around logical row identifiers which ends up > working for global ind

Re: Allow any[] as input arguments for sql/plpgsql functions to mimic format()

2019-04-22 Thread phoe
Hey! OK - thank you for the update and the explanation. My reasoning in this case is - if we allow the any[] type to only be passed to other functions that accept any[], and disallow any kind of other operations on this array (such as retrieving its elements or modifying it), I do not yet see any

Re: [PATCH v1] Add \echo_stderr to psql

2019-04-22 Thread Fabien COELHO
\warn ... \warning ... These two seem about the best to me, drawing from the perl warn command. Yep, I was thinking of perl & gmake. Maybe the 4 letter option is better because its the same length as "echo". I suppose we could go the bash &2 route here, but I don't want to. I agre

Re: Allow any[] as input arguments for sql/plpgsql functions to mimic format()

2019-04-22 Thread Pavel Stehule
Hi po 22. 4. 2019 v 19:20 odesílatel Michał "phoe" Herda napsal: > Hey! > > OK - thank you for the update and the explanation. > > My reasoning in this case is - if we allow the any[] type to only be > passed to other functions that accept any[], and disallow any kind of other > operations on th

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 13:27:17 -0400, Tom Lane wrote: > Andres Freund writes: > > The computation of that variable above has: > > > * If the column is possibly missing, we can't rely on its (or > > * subsequent) NOT NULL constraints to indicate minimum > > attributes in >

Re: Patch: doc for pg_logical_emit_message()

2019-04-22 Thread Fujii Masao
On Fri, Apr 19, 2019 at 3:21 PM Matsumura, Ryo wrote: > > Hi, Hackers > > pg_logical_emit_message() can be used by any user, > but the following document says that it can be used by only superuser. > > > Table 9.88. Replication SQL Functions > > Use of these functions is restricted to superusers.

Re: block-level incremental backup

2019-04-22 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-04-19 20:04:41 -0400, Stephen Frost wrote: > > I agree that we don't want another implementation and that there's a lot > > that we want to do to improve replay performance. We've already got > > frontend tools which work with multi

Re: Do CustomScan as not projection capable node

2019-04-22 Thread Tom Lane
Andrey Lepikhov writes: > It is possible that custom_scan_tlist is designed too nontrivially, and > it is possible that it needs some comments describing in more detail how > to use it. I totally buy the argument that the custom scan stuff is underdocumented :-(. FWIW, if we did have a use-cas

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Robert Haas wrote: > PostgreSQL has historically and very deliberately *not made a > distinction* between "this object is in the default tablespace" and > "this object is in tablespace X which happens to be the default." I > think that it's too late to invent such a distinction fo

Re: clean up docs for v12

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 13:27:17 -0400, Tom Lane wrote: >> I wonder >> also if it wouldn't be smart to explicitly check that the "guaranteeing" >> column is not attisdropped. > Yea, that probably would be smart. I don't think there's an active > problem, because we remove NOT NULL

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 14:16:28 -0400, Alvaro Herrera wrote: > On 2019-Apr-22, Robert Haas wrote: > > > PostgreSQL has historically and very deliberately *not made a > > distinction* between "this object is in the default tablespace" and > > "this object is in tablespace X which happens to be the defa

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 14:17:48 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-04-22 13:27:17 -0400, Tom Lane wrote: > >> I wonder > >> also if it wouldn't be smart to explicitly check that the "guaranteeing" > >> column is not attisdropped. > > > Yea, that probably would be smart. I don

Re: clean up docs for v12

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 13:18:18 -0400, Tom Lane wrote: > Andres Freund writes: > >> (Possibly I'd not think this if I weren't fresh off a couple of days > >> with my nose in the ALTER TABLE SET NOT NULL code. But right now, > >> I think that believing that that code does not and never will have > >>

Re: block-level incremental backup

2019-04-22 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Apr 22, 2019 at 1:08 PM Stephen Frost wrote: > > > One could instead just do a straightforward extension > > > to the existing BASE_BACKUP command to enable incremental backup. > > > > Ok, how do you envision that? As I mentioned

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Tom Lane
Andres Freund writes: > On 2019-04-22 14:16:28 -0400, Alvaro Herrera wrote: >> I think we can get out of this whole class of problems by forbidding the >> TABLESPACE clause for partitioned rels from mentioning the database >> tablespace -- that is, users either mention some *other* tablespace, or

Re: block-level incremental backup

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 14:26:40 -0400, Stephen Frost wrote: > I'm disappointed that the concerns about the trouble that end users are > likely to have with this didn't garner more discussion. My impression is that endusers are having a lot more trouble due to important backup/restore features not bein

Re: TM format can mix encodings in to_char()

2019-04-22 Thread Peter Geoghegan
On Sun, Apr 21, 2019 at 6:26 AM Andrew Dunstan wrote: > How does one do that? Just set a Turkish locale? tr_TR is, in a sense, special among locales: http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html The Turkish dotless i has apparently been implicated in all kinds of bu

Re: Allow any[] as input arguments for sql/plpgsql functions to mimic format()

2019-04-22 Thread Tom Lane
=?UTF-8?Q?Micha=c5=82_=22phoe=22_Herda?= writes: > My reasoning in this case is - if we allow the any[] type to only be > passed to other functions that accept any[], and disallow any kind of > other operations on this array (such as retrieving its elements or > modifying it), I do not yet see any

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Andres Freund wrote: > Why is the obvious answer is to not just remove the whole tablespace > inheritance behaviour? Because it was requested by many, and there were plenty of people surprised that things didn't work that way. -- Álvaro Herrerahttps://www.2ndQuad

Re: block-level incremental backup

2019-04-22 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-04-22 14:26:40 -0400, Stephen Frost wrote: > > I'm disappointed that the concerns about the trouble that end users are > > likely to have with this didn't garner more discussion. > > My impression is that endusers are having a lot m

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Tom Lane wrote: > Yeah, that's where I'm at as well. Alvaro's proposal could be made > to work perhaps, but I think it would still end up with some odd > corner-case behaviors. One example is that "TABLESPACE X" would > be allowed if the database's default tablespace is Y, but if

Re: TM format can mix encodings in to_char()

2019-04-22 Thread Tom Lane
Peter Geoghegan writes: > On Sun, Apr 21, 2019 at 6:26 AM Andrew Dunstan > wrote: >> How does one do that? Just set a Turkish locale? > tr_TR is, in a sense, special among locales: > http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html > The Turkish dotless i has apparently

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Apr-22, Andres Freund wrote: >> Why is the obvious answer is to not just remove the whole tablespace >> inheritance behaviour? > Because it was requested by many, and there were plenty of people > surprised that things didn't work that way. There are lots of thin

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Laurenz Albe
On Mon, 2019-04-22 at 21:45 +0900, Etsuro Fujita wrote: Thanks for looking into this! > (2019/04/20 20:53), Laurenz Albe wrote: > > On Fri, 2018-04-06 at 23:24 +, Robert Haas wrote: > > > Allow insert and update tuple routing and COPY for foreign tables. > > > > > > Also enable this for post

Re: block-level incremental backup

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 2:26 PM Stephen Frost wrote: > There was basically zero discussion about what things would look like at > a protocol level (I went back and skimmed over the thread before sending > my last email to specifically see if I was going to get this response > back..). I get the i

translatability tweaks

2019-04-22 Thread Alvaro Herrera
Here's a small number of translatability tweaks to the error messages in the backend. I found these while updating the Spanish translation over the past weekend, a task I had neglected for two or three years, so they might involve some older messages. However, I won't backpatch this -- only apply

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2019-04-22 Thread Alexander Korotkov
On Thu, Nov 29, 2018 at 3:44 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Fri, Aug 24, 2018 at 5:53 PM Alexander Korotkov > > wrote: > > > > Given I've no feedback on this idea yet, I'll try to implement a PoC > > patch for that. It doesn't look to be difficult. And we'll see how >

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 3:08 PM Alvaro Herrera wrote: > On 2019-Apr-22, Andres Freund wrote: > > Why is the obvious answer is to not just remove the whole tablespace > > inheritance behaviour? > > Because it was requested by many, and there were plenty of people > surprised that things didn't work

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 3:37 PM Laurenz Albe wrote: > Sure, it is not hard to modify a FDW to continue working with v11. > > My point is that this should not be necessary. I'm not sure whether this proposal is a good idea or a bad idea, but I think that it's inevitable that FDWs are going to need

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 21:37:25 +0200, Laurenz Albe wrote: > Subject: [PATCH] Foreign table COPY FROM and tuple routing requires > BeginForeignInsert > > Commit 3d956d956a introduced support for foreign tables as partitions > and COPY FROM on foreign tables. > > If a foreign data wrapper supports da

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Apr-22, Robert Haas wrote: > On Mon, Apr 22, 2019 at 3:08 PM Alvaro Herrera > wrote: > > On 2019-Apr-22, Andres Freund wrote: > > > Why is the obvious answer is to not just remove the whole tablespace > > > inheritance behaviour? > > > > Because it was requested by many, and there were p

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Laurenz Albe
On Mon, 2019-04-22 at 16:24 -0400, Robert Haas wrote: > On Mon, Apr 22, 2019 at 3:37 PM Laurenz Albe wrote: > > Sure, it is not hard to modify a FDW to continue working with v11. > > > > My point is that this should not be necessary. > > I'm not sure whether this proposal is a good idea or a bad

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Laurenz Albe
On Mon, 2019-04-22 at 13:27 -0700, Andres Freund wrote: > On 2019-04-22 21:37:25 +0200, Laurenz Albe wrote: > > Commit 3d956d956a introduced support for foreign tables as partitions > > and COPY FROM on foreign tables. > > > > If a foreign data wrapper supports data modifications, but either has >

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 22:56:20 +0200, Laurenz Albe wrote: > On Mon, 2019-04-22 at 13:27 -0700, Andres Freund wrote: > > On 2019-04-22 21:37:25 +0200, Laurenz Albe wrote: > > > Commit 3d956d956a introduced support for foreign tables as partitions > > > and COPY FROM on foreign tables. > > > > > > If a

Re: TM format can mix encodings in to_char()

2019-04-22 Thread Juan José Santamaría Flecha
Actually, I tried to show my findings with the tr_TR regression test, but you can reproduce the same issue with other locales and non-ASCII characters, as Tom has pointed out. For exampe: de_DE ISO-8859-1: March es_ES ISO-8859-1: Wednesday fr_FR ISO-8859-1: February Regards, Juan José Santamarí

Re: speeding up planning with partitions

2019-04-22 Thread Tom Lane
Amit Langote writes: > On 2019/04/02 2:34, Tom Lane wrote: >> I'm not at all clear on what we think the interaction between >> enable_partition_pruning and constraint_exclusion ought to be, >> so I'm not sure what the appropriate resolution is here. Thoughts? > Prior to 428b260f87 (that is, in P

Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-22 Thread Melanie Plageman
Hi, PrepareTempTablespaces is called by most callers of BufFileCreateTemp, so I was wondering if there is a reason not to call it inside BufFileCreateTemp. As a developer using BufFileCreateTemp to write code that will create spill files, it was easy to forget the extra step of checking the temp_t

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Robert Haas
On Mon, Apr 22, 2019 at 4:43 PM Alvaro Herrera wrote: > Well, frequently when people discuss ideas on this list, others discuss > and provide further ideas to try help to find a working solution, rather > than throw every roadblock they can think of (though roadblocks are > indeed thrown now and t

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-22 Thread Peter Geoghegan
On Mon, Apr 22, 2019 at 3:12 PM Melanie Plageman wrote: > PrepareTempTablespaces is called by most callers of BufFileCreateTemp, so I > was > wondering if there is a reason not to call it inside BufFileCreateTemp. The best answer I can think of is that a BufFileCreateTemp() caller might not want

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Alvaro Herrera
On 2019-Mar-06, Tom Lane wrote: > David Rowley writes: > > As far as I can see, the biggest fundamental difference with doing > > things this way will be that the column order of partitions will be > > preserved, where before it would inherit the order of the partitioned > > table. I'm a little

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Tomas Vondra
On Thu, Apr 18, 2019 at 04:25:24PM -0400, Robert Haas wrote: On Thu, Apr 18, 2019 at 3:51 PM Bruce Momjian wrote: How would you choose the STARTLSN/ENDLSN? If you could do it per checkpoint, rather than per-WAL, I think that would be great. I thought of that too. It seems appealing, because

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Tomas Vondra
On Mon, Apr 22, 2019 at 01:15:49PM -0400, Bruce Momjian wrote: On Mon, Apr 22, 2019 at 01:11:22PM -0400, Robert Haas wrote: On Mon, Apr 22, 2019 at 12:35 PM Bruce Momjian wrote: > I assumed the modblock files would be stored in the WAL archive so some > external tools could generate incremental

Re: Calling PrepareTempTablespaces in BufFileCreateTemp

2019-04-22 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Apr 22, 2019 at 3:12 PM Melanie Plageman > wrote: >> PrepareTempTablespaces is called by most callers of BufFileCreateTemp, so I >> was >> wondering if there is a reason not to call it inside BufFileCreateTemp. > The best answer I can think of is that a BufFile

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Tom Lane
Alvaro Herrera writes: > Now that I re-read this complaint once again, I wonder if a mismatching > column order in partitions isn't a thing we ought to preserve anyhow. > Robert, Amit -- is it by design that pg_dump loses the original column > order for partitions, when not in binary-upgrade mode?

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Tomas Vondra
On Sat, Apr 20, 2019 at 04:21:52PM -0400, Robert Haas wrote: On Sat, Apr 20, 2019 at 12:42 AM Stephen Frost wrote: > Oh. Well, I already explained my algorithm for doing that upthread, > which I believe would be quite cheap. > > 1. When you generate the .modblock files, stick all the block > r

Re: finding changed blocks using WAL scanning

2019-04-22 Thread Bruce Momjian
On Tue, Apr 23, 2019 at 01:21:27AM +0200, Tomas Vondra wrote: > On Sat, Apr 20, 2019 at 04:21:52PM -0400, Robert Haas wrote: > > On Sat, Apr 20, 2019 at 12:42 AM Stephen Frost wrote: > > > > Oh. Well, I already explained my algorithm for doing that upthread, > > > > which I believe would be quite

memory leak checking

2019-04-22 Thread Mikhail Bautin
Hello PostgreSQL Hackers, What is the standard memory leak checking policy for the PostgreSQL codebase? I know there is some support for valgrind -- is the test suite being run continuously with valgrind on the build farm? Is there any plan to support clang's AddressSanitizer? I've seen a thread

Re: memory leak checking

2019-04-22 Thread Andres Freund
Hi, On 2019-04-22 16:50:25 -0700, Mikhail Bautin wrote: > What is the standard memory leak checking policy for the PostgreSQL > codebase? I know there is some support for valgrind -- is the test suite > being run continuously with valgrind on the build farm? There's continuous use of valgrind on

  1   2   >