Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Magnus Hagander
On Mon, Feb 25, 2019 at 10:14 PM Andres Freund wrote: > On 2019-02-25 08:42:02 -0500, Stephen Frost wrote: > > Greetings, > > > > * Andres Freund (and...@anarazel.de) wrote: > > > On 2019-02-24 17:19:22 -0500, Stephen Frost wrote: > > > > You say above that the new interface is unquestionably an

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Andres Freund
Hi, On 2019-02-25 09:24:32 -0800, Christophe Pettus wrote: > But the resistance to major version upgrades is *huge*, and I'm > strongly biased against anything that will make that harder. I'm not > sure I'm communicating how big a problem telling many large > installations, "If you move to v12/13

Re: POC: converting Lists into arrays

2019-02-25 Thread Andres Freund
Hi, On 2019-02-25 13:41:48 -0800, Peter Geoghegan wrote: > On Mon, Feb 25, 2019 at 1:31 PM Andres Freund wrote: > > > Andres said that he doesn't like the pg_list.h API. It's not pretty, > > > but is it really that bad? > > > > Yes. The function names alone confound anybody new to postgres, we te

Re: POC: converting Lists into arrays

2019-02-25 Thread Peter Geoghegan
On Mon, Feb 25, 2019 at 1:31 PM Andres Freund wrote: > > Andres said that he doesn't like the pg_list.h API. It's not pretty, > > but is it really that bad? > > Yes. The function names alone confound anybody new to postgres, we tend > to forget that after a few years. A lot of the function return

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread David Steele
On 2/25/19 11:38 PM, Andres Freund wrote: Hi, On 2019-02-25 09:24:32 -0800, Christophe Pettus wrote: But the resistance to major version upgrades is *huge*, and I'm strongly biased against anything that will make that harder. I'm not sure I'm communicating how big a problem telling many large

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Christophe Pettus
> On Feb 25, 2019, at 13:38, Andres Freund wrote: > > I think you might be right about this specific issue. But to me it > sounds like you also don't appreciate that development resources are > really constrained too, and providing endless backward compatibility for > everything is going to us

Re: POC: converting Lists into arrays

2019-02-25 Thread Peter Geoghegan
On Mon, Feb 25, 2019 at 1:51 PM Andres Freund wrote: > Those could trivially support distinguisiong at least between lists > containing pointer, int, oid, or node. But even optionally doing more > than that would be fairly easy. It's not those modules don't currently > know the types of elements t

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-02-25 08:14:16 -0500, Stephen Frost wrote: > > > It will be annoying if after this removal, companies must change their > > > backup strategy by using specific postgres tools (pgbackrest, barman). > > > > You don't write your own da

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread David Steele
On 2/25/19 11:59 PM, Christophe Pettus wrote: On Feb 25, 2019, at 13:38, Andres Freund wrote: I think you might be right about this specific issue. But to me it sounds like you also don't appreciate that development resources are really constrained too, and providing endless backward compati

Re: Auxiliary Processes and MyAuxProc

2019-02-25 Thread Mike Palmiotto
On Mon, Feb 25, 2019 at 1:41 PM Mike Palmiotto wrote: > > > > > > If memory serves, StartChildProcess already was an attempt to unify > > the treatment of postmaster children. It's possible that another > > round of unification would be productive, but I think you'll find > > that there are rand

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Paul Ramsey
On Mon, Jan 28, 2019 at 9:51 AM Tom Lane wrote: > is people like PostGIS, who already cleared that bar. I hope that > we'll soon have a bunch of examples, like those in the 0004 patch, > that people can look at to see how to do things in this area. I see > no reason to believe it'll be all that

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-02-25 08:42:02 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > On 2019-02-24 17:19:22 -0500, Stephen Frost wrote: > > > > You say above that the new interface is unquestionably an improvement > > >

Re: POC: converting Lists into arrays

2019-02-25 Thread Andres Freund
Hi, On 2019-02-23 21:24:40 -0500, Tom Lane wrote: > For quite some years now there's been dissatisfaction with our List > data structure implementation. Because it separately palloc's each > list cell, it chews up lots of memory, and it's none too cache-friendly > because the cells aren't necessa

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Tom Lane
Paul Ramsey writes: > So... trying to figure out how to use SupportRequestIndexCondition to > convert a call to Intersects() in to a call that also has the operator > && as well. OK. > Looking at the examples, they are making use of the opfamily that > comes in SupportRequestIndexCondition.opfam

Re: POC: converting Lists into arrays

2019-02-25 Thread Tom Lane
Andres Freund writes: > Btw, should we remove the ENABLE_LIST_COMPAT stuff independent of this > discussion? Seems like we could even just do that for 12. +1. I took it out in the POC patch, but I see no very good reason not to do it sooner than that. >> The most critical thing that we lose by

Re: NOT IN subquery optimization

2019-02-25 Thread David Rowley
On Tue, 26 Feb 2019 at 11:51, Li, Zheng wrote: > Resend the patch with a whitespace removed so that "git apply patch" works > directly. I had a quick look at this and it seems to be broken for the empty table case I mentioned up thread. Quick example: Setup: create table t1 (a int); create ta

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Paul Ramsey
On Mon, Feb 25, 2019 at 3:01 PM Tom Lane wrote: > > Looking at the examples, they are making use of the opfamily that > > comes in SupportRequestIndexCondition.opfamily. > > That opfamily Oid is the first one in the IndexOptInfo.opfamily array. > > Here's where my thread of understanding fails to

Re: POC: converting Lists into arrays

2019-02-25 Thread Tom Lane
I wrote: > Andres Freund writes: >>> 1. This still involves at least two palloc's for every nonempty List, >>> because I kept the header and the data array separate. Perhaps it's >>> worth allocating those in one palloc. >> Hm, I think if we force external code to audit their code, we better >>

Re: Allowing extensions to supply operator-/function-specific info

2019-02-25 Thread Tom Lane
Paul Ramsey writes: > On Mon, Feb 25, 2019 at 3:01 PM Tom Lane wrote: >> It's whichever one the index column's opclass belongs to. Basically what >> you're trying to do here is verify whether the index will support the >> optimization you want to perform. > * If I have tbl1.geom > * and I have

RE: Timeout parameters

2019-02-25 Thread Jamison, Kirk
On Monday, February 25, 2019 1:49 PM (GMT+9), Nagaura, Ryohei wrote: > Thank you for discussion. > I made documentation about socket_timeout and reflected Tsunakawa-san's > comment in the new patch. > # Mainly only fixing documentation... > The current documentation of socket_timeout is as follows

Re: current_logfiles not following group access and instead follows log_file_mode permissions

2019-02-25 Thread Haribabu Kommi
On Mon, Feb 4, 2019 at 12:16 PM Haribabu Kommi wrote: > > And regarding current_logfiles permissions, I feel this file should have > permissions of data directory files as it is present in the data directory > whether it stores the information of log file, until this file is > completely > remove

crosstab/repivot...any interest?

2019-02-25 Thread Merlin Moncure
On Fri, Jan 25, 2019 at 9:14 PM Morris de Oryx wrote: > > Hello, I'm not a C coder and can't helpbut I love cross-tab/pivot tables. They're the best, and just fantastic for preparing data to feed into various analysis tools. The tablefunc module is helpful, but a bit awkward to use (as I remem

RE: Timeout parameters

2019-02-25 Thread Nagaura, Ryohei
Hi, kirk-san. > From: Jamison, Kirk > $ git apply ../socket_timeout_v5.patch > fatal: corrupt patch at line 24 > --> l24: diff --git a/src/interfaces/libpq/fe-connect.c > --> b/src/interfaces/libpq/fe-connect.c How about applying by "patch -p1"? I have confirmed that my patch could be applied in

Re: POC: converting Lists into arrays

2019-02-25 Thread Andres Freund
Hi, On 2019-02-25 18:41:17 -0500, Tom Lane wrote: > I wrote: > > Andres Freund writes: > >>> 1. This still involves at least two palloc's for every nonempty List, > >>> because I kept the header and the data array separate. Perhaps it's > >>> worth allocating those in one palloc. > > >> Hm, I t

Re: pgsql: Avoid creation of the free space map for small heap relations, t

2019-02-25 Thread Amit Kapila
On Mon, Feb 25, 2019 at 10:32 PM Tom Lane wrote: > > Amit Kapila writes: > > Avoid creation of the free space map for small heap relations, take 2. > > I think this patch still has some issues. > I will look into it today and respond back with my findings. John, see if you also get time to inve

Re: POC: converting Lists into arrays

2019-02-25 Thread Andres Freund
Hi, On 2019-02-25 17:55:46 -0800, Andres Freund wrote: > Hm, I wonder if that's necessary / whether we can just work around user > visible breakage at a small cost. I think I'm mostly concerned with two > allocations for the very common case of small (1-3 entries) lists. We > could just allocate t

Re: JIT on FreeBSD ARMv7

2019-02-25 Thread Andrew Gierth
> "Andrew" == Andrew Gierth writes: Andrew> This seems to be the cleanest way to get a JIT build working on Andrew> FreeBSD on the armv7 platform. Without this, it fails because Andrew> the ABI functions __aeabi_ldivmod and so on are not found by Andrew> the symbol lookup for JITted code.

RE: SQL statement PREPARE does not work in ECPG

2019-02-25 Thread Takahashi, Ryohei
Hi Matsumura-san, Thank you for your explaining. > An important point of the route is that it calls PQprepare() and PQprepare() > needs type-Oid list. (Idea-1) If we fix for Prepare statement with AS clause > and > with parameter list to walk through the route, preprocessor must parse the > pa

Re: POC: converting Lists into arrays

2019-02-25 Thread David Rowley
On Sun, 24 Feb 2019 at 15:24, Tom Lane wrote: > I haven't attempted any performance measurements on this, but at > least in principle it should speed things up a bit, especially > for complex test cases involving longer Lists. I don't have any > very suitable test cases at hand, anyway. I've not

Re: pgsql: Avoid creation of the free space map for small heap relations, t

2019-02-25 Thread John Naylor
On Tue, Feb 26, 2019 at 8:59 AM Amit Kapila wrote: > I will look into it today and respond back with my findings. John, > see if you also get time to investigate this. Will do, but it will likely be tomorrow -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x

Re: pgsql: Avoid creation of the free space map for small heap relations, t

2019-02-25 Thread Amit Kapila
On Mon, Feb 25, 2019 at 11:14 PM Tom Lane wrote: > > I wrote: > > Amit Kapila writes: > >> Avoid creation of the free space map for small heap relations, take 2. > > > I think this patch still has some issues. > > Just out of curiosity ... how can it possibly be even a little bit sane > that fsm_

Re: pgsql: Avoid creation of the free space map for small heap relations, t

2019-02-25 Thread Amit Kapila
On Tue, Feb 26, 2019 at 8:38 AM John Naylor wrote: > > On Tue, Feb 26, 2019 at 8:59 AM Amit Kapila wrote: > > I will look into it today and respond back with my findings. John, > > see if you also get time to investigate this. > > Will do, but it will likely be tomorrow > No problem, thanks! -

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Robert Haas
On Mon, Feb 25, 2019 at 4:38 PM David Steele wrote: > > FWIW, if you weren't selling backrest quite so hard everywhere backups > > are mentioned, I'd find this thread a lot more convicing. > > pgBackRest has not used exclusive backups since the new API was > introduced in 9.6 so this is not an iss

Re: Pluggable Storage - Andres's take

2019-02-25 Thread Amit Khandekar
On Sat, 23 Feb 2019 at 01:22, Robert Haas wrote: > > On Fri, Feb 22, 2019 at 11:19 AM Amit Khandekar > wrote: > > Thanks for the review. Attached v2. > > Thanks. I took this, combined it with Andres's > v12-0040-WIP-Move-xid-horizon-computation-for-page-level-.patch, did > some polishing of the

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > Hmm, so what you're saying is that you'd like to disable an API that > some non-backrest users are relying upon but which no backrest users > are relying upon. And you don't understand why some non-backrest > users are opposed to that plan.

Re: [HACKERS] Block level parallel vacuum

2019-02-25 Thread Haribabu Kommi
On Thu, Feb 14, 2019 at 9:17 PM Masahiko Sawada wrote: > Thank you. Attached the rebased patch. > I ran some performance tests to compare the parallelism benefits, but I got some strange results of performance overhead, may be it is because, I tested it on my laptop. FYI, Table schema: create

Re: Segfault when restoring -Fd dump on current HEAD

2019-02-25 Thread Michael Paquier
On Mon, Feb 25, 2019 at 11:20:14AM -0500, Tom Lane wrote: > It appears to me that f831d4acc required a good deal more adult > supervision than it actually got. That was alleged to be a small > notational refactoring, not a redefinition of what gets put into > dump files. How much consistent do we

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Mark Kirkwood
On 26/02/19 4:53 PM, Robert Haas wrote: On Mon, Feb 25, 2019 at 4:38 PM David Steele wrote: FWIW, if you weren't selling backrest quite so hard everywhere backups are mentioned, I'd find this thread a lot more convicing. pgBackRest has not used exclusive backups since the new API was introdu

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Stephen Frost
Greetings Mark, * Mark Kirkwood (mark.kirkw...@catalyst.net.nz) wrote: > ISTM that the onus should be on the patch submitter to provide additions to > pg_basebackup that make it as painless as possible for those people *not* > using pgBackRest to continue making backups. Breaking this is just not

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Michael Paquier
On Mon, Feb 25, 2019 at 08:17:27PM +0200, David Steele wrote: > Here's the really obvious bad thing: if users do not update their procedures > and we ignore backup_label.pending on startup then they will end up with a > corrupt database because it will not replay from the correct checkpoint. If >

Re: [HACKERS] generated columns

2019-02-25 Thread Michael Paquier
On Mon, Feb 25, 2019 at 09:51:52PM +0100, Peter Eisentraut wrote: > On 2019-01-15 08:13, Michael Paquier wrote: >> + boolhas_generated_stored; >> + boolhas_generated_virtual; >> } TupleConstr; >> Could have been more simple to use a char as representation here. > > Seems confu

Re: Segfault when restoring -Fd dump on current HEAD

2019-02-25 Thread Tom Lane
Michael Paquier writes: > On Mon, Feb 25, 2019 at 11:20:14AM -0500, Tom Lane wrote: >> It appears to me that f831d4acc required a good deal more adult >> supervision than it actually got. That was alleged to be a small >> notational refactoring, not a redefinition of what gets put into >> dump fi

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Mark Kirkwood
On 26/02/19 5:41 PM, Stephen Frost wrote: Greetings Mark, * Mark Kirkwood (mark.kirkw...@catalyst.net.nz) wrote: ISTM that the onus should be on the patch submitter to provide additions to pg_basebackup that make it as painless as possible for those people *not* using pgBackRest to continue m

Re: [HACKERS] generated columns

2019-02-25 Thread Michael Paquier
On Mon, Feb 25, 2019 at 09:46:35PM +0100, Peter Eisentraut wrote: > The virtual generated column part is still a bit iffy. I'm still > finding places here and there where virtual columns are not being > expanded correctly. Maybe it needs more refactoring. One big unsolved > issue is how the stor

Re: POC: converting Lists into arrays

2019-02-25 Thread Tom Lane
David Rowley writes: > Using the attached patch (as text file so as not to upset the CFbot), > which basically just measures and logs the time taken to run > pg_plan_query. ... > Surprisingly it took 1.13% longer. I did these tests on an AWS > md5.large instance. Interesting. Seems to suggest t

Re: Segfault when restoring -Fd dump on current HEAD

2019-02-25 Thread Michael Paquier
On Tue, Feb 26, 2019 at 12:16:35AM -0500, Tom Lane wrote: > Well, if we didn't want to fix this, a reasonable way to go about > it would be to bump the archive version number in pg_dump output, > so that old versions would issue a useful complaint instead of crashing. > However, I repeat that this

Re: No-rewrite timestamp<->timestamptz conversions

2019-02-25 Thread Noah Misch
On Thu, Feb 05, 2015 at 08:36:18PM -0500, Noah Misch wrote: > On Tue, Nov 05, 2013 at 05:02:58PM -0800, Josh Berkus wrote: > > I'd also love some way of doing a no-rewrite conversion between > > timestamp and timestamptz, based on the assumption that the original > > values are UTC time. That's on

RE: reloption to prevent VACUUM from truncating empty pages at the end of relation

2019-02-25 Thread Tsunakawa, Takayuki
From: Masahiko Sawada [mailto:sawada.m...@gmail.com] > This test expects that the inserted tuple is always reclaimed by > subsequent vacuum, but it's not always true if there are concurrent > transactions. So size of the reloptions_test table will not be 0 if > the tuple is not vacuumed. In my envi

Re: Reaping Temp tables to avoid XID wraparound

2019-02-25 Thread Michael Paquier
On Fri, Feb 22, 2019 at 04:01:02PM +0100, Magnus Hagander wrote: > I did the "insert column in the middle of pg_stat_get_activity", I'm not > sure that is right -- how do we treate that one? Do we just append at the > end because people are expected to use the pg_stat_activity view? It's a > nontri

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread David Steele
On 2/26/19 6:51 AM, Michael Paquier wrote: On Mon, Feb 25, 2019 at 08:17:27PM +0200, David Steele wrote: Here's the really obvious bad thing: if users do not update their procedures and we ignore backup_label.pending on startup then they will end up with a corrupt database because it will not re

RE: [RFC] [PATCH] Flexible "partition pruning" hook

2019-02-25 Thread Tsunakawa, Takayuki
From: Mike Palmiotto [mailto:mike.palmio...@crunchydata.com] > Attached is a patch which attempts to solve a few problems: > > 1) Filtering out partitions flexibly based on the results of an external > function call (supplied by an extension). > 2) Filtering out partitions from pg_inherits based o

inted RE: Timeout parameters

2019-02-25 Thread Nagaura, Ryohei
Hi, kirk-san. > From: Nagaura, Ryohei > This is my bad. I'll remake it. > Very sorry for the same mistake. I remade the patches and attached in this mail. In socket_timeout patch, I replaced "atoi" to "parse_int_param" and inserted spaces just after some comma. There are a few changes about docu

Re: Remove Deprecated Exclusive Backup Mode

2019-02-25 Thread Laurenz Albe
Fujii Masao wrote: > Another idea is to improve an exclusive backup method so that it will never > cause such issue. What about changing an exclusive backup mode of > pg_start_backup() so that it creates something like backup_label.pending file > instead of backup_label? Then if the database cluste

Re: Unused parameters & co in code

2019-02-25 Thread Michael Paquier
On Thu, Jan 31, 2019 at 03:47:59PM +0900, Michael Paquier wrote: > - 0001 cleans up port in SendAuthRequest. This one is disappointing, > so I am fine to discard it. > - 0002 works on _bt_relbuf, whose callers don't actually benefit from > the cleanup as the relation worked on is always used for d

Re: [RFC] [PATCH] Flexible "partition pruning" hook

2019-02-25 Thread Michael Paquier
On Tue, Feb 26, 2019 at 06:55:30AM +, Tsunakawa, Takayuki wrote: > What concrete problems would you expect this patch to solve? What > kind of extensions do you imagine? I'd like to hear about the > examples. For example, "PostgreSQL 12 will not be able to filter > out enough partitions when

<    1   2