Re: [HACKERS] Block level parallel vacuum

2019-12-02 Thread tushar
On 11/27/19 11:13 PM, Masahiko Sawada wrote: Thank you for reviewing this patch. All changes you made looks good to me. I thought I already have posted all v34 patches but didn't, sorry. So I've attached v35 patch set that incorporated your changes and it includes Dilip's patch for gist index (0

Re: Using XLogFileNameP in critical section

2019-12-02 Thread Michael Paquier
On Mon, Dec 02, 2019 at 10:14:58PM +0100, Masahiko Sawada wrote: > Agreed. I've attached the updated version patch. Please review it. Thanks, applied on HEAD after a few edits. gettext() does not set errno, so the new style of issue_xlog_fsync() is actually fine. Please note that there was one mi

Re: Implementing Incremental View Maintenance

2019-12-02 Thread Yugo Nagata
On Mon, 2 Dec 2019 13:48:40 -0300 Alvaro Herrera wrote: > On 2019-Dec-02, Yugo Nagata wrote: > > > On Mon, 02 Dec 2019 10:36:36 +0900 (JST) > > Tatsuo Ishii wrote: > > > > > >> One thing pending in this development line is how to catalogue > > > >> aggregate > > > >> functions that can be use

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-12-02 Thread Amit Khandekar
On Wed, 27 Nov 2019 at 14:16, Amit Khandekar wrote: > What I found was : We do attempt to close the opened vfds in the > PG_CATCH block. In ReorderBufferCommit(), ReorderBufferIterTXNFinish > is called both in PG_TRY and PG_CATCH. This closes all the opened > vfds. But the issue is : if the erepor

Re: Simplify passing of configure arguments to pg_config

2019-12-02 Thread Tom Lane
Peter Eisentraut writes: > Currently, configure puts the configure args into the makefiles and > then have the makefiles pass them to the build of pg_config. That looks > like an unnecessary redirection, and indeed that method was > put in place when pg_config was a shell script. We can simplif

Re: Bogus EXPLAIN results with column aliases for mismatched partitions

2019-12-02 Thread Etsuro Fujita
On Tue, Dec 3, 2019 at 6:45 AM Tom Lane wrote: > I wrote: > > Etsuro Fujita writes: > >> On Mon, Dec 2, 2019 at 6:34 AM Tom Lane wrote: > >>> BTW, the existing code always schema-qualifies the relation names, > >>> on the rather lame grounds that it's producing the string without > >>> knowing w

Re: Failure in TAP tests of pg_ctl on Windows with parallel instance set

2019-12-02 Thread Michael Paquier
On Mon, Dec 02, 2019 at 07:57:31AM -0500, Andrew Dunstan wrote: > Looks reasonable. Thanks, committed and back-patched down to 11, which is where we have PostgresNode::get_free_port. This could go further down with more refactoring of PostgresNode.pm but as it took a long time to find this issue

Re: Update minimum SSL version

2019-12-02 Thread Michael Paquier
On Mon, Dec 02, 2019 at 12:51:26PM -0500, Tom Lane wrote: > Yah. Although, looking at the code in be-secure-openssl.c, > it doesn't look that hard to do in an extensible way. > Something like (untested) While we are on the topic... Here is another wild idea. We discussed not so long ago about r

Re: Update minimum SSL version

2019-12-02 Thread Michael Paquier
On Mon, Dec 02, 2019 at 09:59:44AM -0500, Tom Lane wrote: > Is there a simple way to ask the library itself for version info? > It might be worth the cycles to have configure run a small test > program to extract and print that data (not on cross-compile > builds, of course). SSLeay_version(): htt

add age for databases list in psql

2019-12-02 Thread Ioseph Kim
Hi. I need to check database's age in psql, so I want to append that. like this, (12) [postgres@ioseph-centos7 ~]$ psql -E psql (12.0) Type "help" for help. (12.0) postgres@postgres=# \l * QUERY ** SELECT d.datname as "Name",    pg_catalog.pg_get_userbyid(d.datdba) as "Ow

Re: pgbench -i progress output on terminal

2019-12-02 Thread Amit Langote
Thanks for the review. On Mon, Dec 2, 2019 at 3:28 PM Michael Paquier wrote: > On Mon, Dec 02, 2019 at 02:30:47PM +0900, Amit Langote wrote: > > On Sun, Dec 1, 2019 at 4:33 AM Fabien COELHO wrote: > >> Patch applies, compiles, works for me. No further comments. > >> > >> I switched the patch as

Re: [PATCH] kNN for btree

2019-12-02 Thread Alexander Korotkov
On Mon, Sep 9, 2019 at 11:28 PM Alexander Korotkov wrote: > On Sun, Sep 8, 2019 at 11:35 PM Alexander Korotkov > wrote: > > I'm going to push 0001 changing "attno >= 1" to assert. > > Pushed. Rebased patchset is attached. I propose to limit > consideration during this commitfest to this set of

Re: surprisingly expensive join planning query

2019-12-02 Thread Tom Lane
Tomas Vondra writes: >> (Speaking of which, I don't quite see why this would have been a problem >> once you got past geqo_threshold; the context resets that GEQO does >> should've kept things under control.) > Not sure I follow. geqo_threshold is 12 by default, and the OOM issues > are hapenning

Re: surprisingly expensive join planning query

2019-12-02 Thread Tomas Vondra
On Mon, Dec 02, 2019 at 03:21:47PM -0500, Tom Lane wrote: Tomas Vondra writes: On Sun, Dec 01, 2019 at 02:17:15PM -0500, Tom Lane wrote: LGTM. Thanks. Do you think this is backpatch-worthy? I'm leaning to yes, but maybe tweaking this just in master is fine. The query is somewhat artificial

Re: Bogus EXPLAIN results with column aliases for mismatched partitions

2019-12-02 Thread Tom Lane
I wrote: > Etsuro Fujita writes: >> On Mon, Dec 2, 2019 at 6:34 AM Tom Lane wrote: >>> BTW, the existing code always schema-qualifies the relation names, >>> on the rather lame grounds that it's producing the string without >>> knowing whether EXPLAIN VERBOSE will be specified. In this code, >>>

Re: Using XLogFileNameP in critical section

2019-12-02 Thread Masahiko Sawada
On Mon, 2 Dec 2019 at 16:56, Alvaro Herrera wrote: > > On 2019-Dec-02, Michael Paquier wrote: > > > I think that we had better just get rid of XLogFileNameP() (on HEAD) > > and just change those code paths so as they use a buffer of size > > MAXFNAMELEN, with XLogFileName() generating the file nam

Re: surprisingly expensive join planning query

2019-12-02 Thread Tom Lane
Tomas Vondra writes: > On Sun, Dec 01, 2019 at 02:17:15PM -0500, Tom Lane wrote: >> LGTM. > Thanks. Do you think this is backpatch-worthy? I'm leaning to yes, but > maybe tweaking this just in master is fine. The query is somewhat > artificial and there are probably ways to rewrite it. I don't o

Re: Windows buildfarm members vs. new async-notify isolation test

2019-12-02 Thread Andrew Dunstan
On 12/2/19 11:23 AM, Tom Lane wrote: > I see from the buildfarm status page that since commits 6b802cfc7 > et al went in a week ago, frogmouth and currawong have failed that > new test case every time, with the symptom > > == pgsql.build/src/test/isolation/regression.diffs >

Re: Binary support for pgoutput plugin

2019-12-02 Thread Dave Cramer
Rebased against head Dave Cramer On Sat, 30 Nov 2019 at 20:48, Michael Paquier wrote: > Hi, > > On Mon, Nov 11, 2019 at 03:24:59PM -0500, Dave Cramer wrote: > > Attached, > > The latest patch set does not apply correctly. Could you send a > rebase please? I am moving the patch to next CF, wa

Re: [HACKERS] Block level parallel vacuum

2019-12-02 Thread Masahiko Sawada
On Sun, 1 Dec 2019 at 18:31, Sergei Kornilov wrote: > > Hi > > > I think I got your point. Your proposal is that it's more efficient if > > we make the leader process vacuum the index that can be processed only > > the leader process (i.e. indexes not supporting parallel index vacuum) > > while wo

Re: surprisingly expensive join planning query

2019-12-02 Thread Tomas Vondra
On Sun, Dec 01, 2019 at 02:17:15PM -0500, Tom Lane wrote: Tomas Vondra writes: On Sun, Dec 01, 2019 at 01:27:04PM -0500, Tom Lane wrote: Alternatively, it'd be possible to get rid of the separate List altogether, and just add the rinfo's to "mergeclauses" immediately. The functionality of the

Re: global / super barriers (for checksums)

2019-12-02 Thread Robert Haas
On Sun, Nov 17, 2019 at 8:38 AM Magnus Hagander wrote: > I'm definitely happy to work with it, but I did not and do not feel I have > the skills for doing the "proper review" needed for it. So I am also very > happy for you to pick it up and run with it. OK, here's what I came up with. 0001 is

Re: Should we add xid_current() or a int8->xid cast?

2019-12-02 Thread Mark Dilger
On 11/30/19 5:14 PM, Thomas Munro wrote: On Sun, Dec 1, 2019 at 12:22 PM Mark Dilger wrote: These two patches (v3) no longer apply cleanly. Could you please rebase? Hi Mark, Thanks. Here's v4. Thanks, Thomas. The new patches apply cleanly and pass 'installcheck'. -- Mark Dilger

Re: Update minimum SSL version

2019-12-02 Thread Tom Lane
Robert Haas writes: > On Mon, Dec 2, 2019 at 11:39 AM Tom Lane wrote: >> Maybe it'd be worth extending that to show the max supported >> version, with some rats-nest of #ifdefs, but I'm not sure if >> it's worth the trouble. > Especially if we mess up the #ifdefs. :-) Yah. Although, looking at

Re: Update minimum SSL version

2019-12-02 Thread Robert Haas
On Mon, Dec 2, 2019 at 11:39 AM Tom Lane wrote: > That's an excellent point, but it looks like we're pretty good > already. I tried the patch with openssl 0.9.8x, and got this > failure at server start: > > FATAL: ssl_min_protocol_version setting TLSv1.2 not supported by this build Oh, that's p

Re: Implementing Incremental View Maintenance

2019-12-02 Thread Alvaro Herrera
On 2019-Dec-02, Yugo Nagata wrote: > On Mon, 02 Dec 2019 10:36:36 +0900 (JST) > Tatsuo Ishii wrote: > > > >> One thing pending in this development line is how to catalogue aggregate > > >> functions that can be used in incrementally-maintainable views. > > >> I saw a brief mention somewhere that

Re: Update minimum SSL version

2019-12-02 Thread Tom Lane
Robert Haas writes: > ... However, it would be worth putting in some > effort to make sure that we give a good error message if this happens. That's an excellent point, but it looks like we're pretty good already. I tried the patch with openssl 0.9.8x, and got this failure at server start: FATA

Windows buildfarm members vs. new async-notify isolation test

2019-12-02 Thread Tom Lane
I see from the buildfarm status page that since commits 6b802cfc7 et al went in a week ago, frogmouth and currawong have failed that new test case every time, with the symptom == pgsql.build/src/test/isolation/regression.diffs === *** c:/prog/bf/root/REL_10_STABLE

Re: Protocol problem with GSSAPI encryption?

2019-12-02 Thread Peter Eisentraut
On 2019-12-01 02:13, Andrew Gierth wrote: But ProcessStartupPacket assumes that the packet after a failed negotiation of either kind will be the actual startup packet, so the SSL connection request is rejected with "unsupported version 1234.5679". I'm guessing this usually goes unnoticed because

Re: Using XLogFileNameP in critical section

2019-12-02 Thread Alvaro Herrera
On 2019-Dec-02, Michael Paquier wrote: > I think that we had better just get rid of XLogFileNameP() (on HEAD) > and just change those code paths so as they use a buffer of size > MAXFNAMELEN, with XLogFileName() generating the file name. This leads > actually to some simplifications, see for exam

Re: Proposal: Add more compile-time asserts to expose inconsistencies.

2019-12-02 Thread Andres Freund
Hi, On 2019-11-29 11:11:25 +0900, Michael Paquier wrote: > On Wed, Nov 27, 2019 at 12:23:33PM +, Smith, Peter wrote: > > * That is beyond the scope for what I wanted my patch to achieve; my > > * use-cases are C code only. I really don't think that's justification enough for having diverging

Re: Update minimum SSL version

2019-12-02 Thread Robert Haas
On Fri, Nov 29, 2019 at 9:44 PM Michael Paquier wrote: > Actually, no, what I am writing here is incorrect. We should make > sure of that the default configuration is correct at initdb time, and > the patch does not do that. I think that would be overkill. There shouldn't be many people who are

Re: Update minimum SSL version

2019-12-02 Thread Daniel Gustafsson
> On 2 Dec 2019, at 15:59, Tom Lane wrote: > Is there a simple way to ask the library itself for version info? > It might be worth the cycles to have configure run a small test > program to extract and print that data (not on cross-compile > builds, of course). Asking the lib is easy, making tha

Adding a Partition in Painful

2019-12-02 Thread David Fetter
Folks, While it would be lovely if we could all plan all table partitioning in advance of inserting the first row, that's not the reality, and has little prospect of becoming the reality. One reality I'm dealing with is where people discover the list partitions they need to add based on the conten

Re: Update minimum SSL version

2019-12-02 Thread Tom Lane
Peter Eisentraut writes: > On 2019-11-30 04:06, Tom Lane wrote: >> I think the real question we have to answer is this: are we intent on >> making people upgrade ancient openssl installations? > The trade-off is that this makes the defaults better for the vast > majority of users and gives users

Re: Undo logs

2019-12-02 Thread Robert Haas
On Sat, Nov 30, 2019 at 9:25 PM Michael Paquier wrote: > On Thu, Feb 07, 2019 at 07:35:31AM +0530, Andres Freund wrote: > > It was JUST added ... :) thought I saw you reply on the other thread > > about it, but I was wrong... > > Six months later without any activity, I am marking this entry as >

Re: Update minimum SSL version

2019-12-02 Thread Tom Lane
Daniel Gustafsson writes: > On 30 Nov 2019, at 03:43, Michael Paquier wrote: >> The buildfarm logs don't directly report the version of OpenSSL used >> as far as I recalled, and a quick lookup shows that.. > Not explicitly, but it would be a nice if it did. Since the version depends > on > the

Re: Bogus EXPLAIN results with column aliases for mismatched partitions

2019-12-02 Thread Tom Lane
Etsuro Fujita writes: > I think "names of relation" should be "names of relations", so how > about fixing that as well? Ah, missed that. > On Mon, Dec 2, 2019 at 6:34 AM Tom Lane wrote: >> BTW, the existing code always schema-qualifies the relation names, >> on the rather lame grounds that it's

Re: pgbench -i progress output on terminal

2019-12-02 Thread Fabien COELHO
Another question I have is why doing only that for the data initialization phase? Wouldn't it make sense to be consistent with the other tools having --progress and do the same dance in pgbench's printProgressReport()? I thought of it but did not suggest it. When running a bench I like se

Re: Update minimum SSL version

2019-12-02 Thread Daniel Gustafsson
> On 30 Nov 2019, at 03:43, Michael Paquier wrote: > > On Fri, Nov 29, 2019 at 10:30:47AM -0500, Tom Lane wrote: >> What's the impact going to be on buildfarm members with older openssl >> installations? Perhaps "none", if they aren't running the ssl test >> suite, but we should be clear about i

Re: Collation versioning

2019-12-02 Thread Robert Haas
On Thu, Nov 28, 2019 at 8:08 AM Julien Rouhaud wrote: > What we could do is storing an empty string if the compatibility is > unknown, and detect it in index_check_collation_version() to report a > slightly different message. I'm assuming that not knowing the > compatibility would be system-wide

Re: Failure in TAP tests of pg_ctl on Windows with parallel instance set

2019-12-02 Thread Andrew Dunstan
On 12/1/19 10:14 PM, Michael Paquier wrote: > Hi all, > > I have run the TAP tests with an instance of Postgres locally set at > port 5432 on Windows, to notice that 001_start_stop.pl fails various > tests because the test tries to use the default port for the node > initialized with pg_ctl. The

Simplify passing of configure arguments to pg_config

2019-12-02 Thread Peter Eisentraut
Currently, configure puts the configure args into the makefiles and then have the makefiles pass them to the build of pg_config. That looks like an unnecessary redirection, and indeed that method was put in place when pg_config was a shell script. We can simplify that by having configure put t

Re: Bogus EXPLAIN results with column aliases for mismatched partitions

2019-12-02 Thread Etsuro Fujita
On Mon, Dec 2, 2019 at 6:34 AM Tom Lane wrote: > I wrote: > > Now, there is another thing that set_rtable_names() is doing that > > postgres_fdw doesn't do, which is to unique-ify the chosen alias > > names by adding "_NN" if the querytree contains multiple uses of > > the same table alias. I don

Re: Update minimum SSL version

2019-12-02 Thread Peter Eisentraut
On 2019-11-30 04:06, Tom Lane wrote: I think the real question we have to answer is this: are we intent on making people upgrade ancient openssl installations? If so, shouldn't we be doing something even more aggressive than this? If not, wouldn't the patch need to try to autoconfigure the mini

Re: Update minimum SSL version

2019-12-02 Thread Peter Eisentraut
On 2019-11-29 16:30, Tom Lane wrote: Michael Paquier writes: On Fri, Nov 29, 2019 at 01:40:48PM +0100, Magnus Hagander wrote: +1. As long as we still have support to change it down if needed, it's a good thing to ship with a proper default. +1. What's the impact going to be on buildfarm m

Re: psql - add SHOW_ALL_RESULTS option

2019-12-02 Thread Fabien COELHO
The below addition can be removed, it seems to be duplicate: Indeed. I'm unsure how this got into the patch, probably some rebase mix-up. Attached v7 removes the duplicates. Attached patch v8 is a rebase. -- Fabien.diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/

Re: [HACKERS] Cached plans and statement generalization

2019-12-02 Thread Konstantin Knizhnik
On 01.12.2019 6:26, Michael Paquier wrote: On Thu, Sep 26, 2019 at 10:23:38AM +0300, Konstantin Knizhnik wrote: Sorry, New version of the patch with corrected expected output for rules test is attached. It looks like the documentation is failing to build. Could you fix that? There may be ot

Re: Global temporary tables

2019-12-02 Thread Konstantin Knizhnik
On 01.12.2019 4:56, Michael Paquier wrote: On Wed, Nov 20, 2019 at 07:32:14PM +0300, Konstantin Knizhnik wrote: Now pg_gtt_statistic view is provided for global temp tables. Latest patch fails to apply, per Mr Robot's report. Could you please rebase and send an updated version? For now I ha

Re: Autovacuum on partitioned table

2019-12-02 Thread yuzuko
Hi Laurenz, Thanks for the comments. On Mon, Dec 2, 2019 at 6:19 PM Laurenz Albe wrote: > > On Mon, 2019-12-02 at 18:02 +0900, yuzuko wrote: > > Greg reported in [1] before, autovacuum ignores partitioned tables. > > That is, even if individual partitions’ statistics are updated, its parent's >

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-12-02 Thread Alexey Kondratov
On 02.12.2019 11:21, Michael Paquier wrote: On Wed, Nov 27, 2019 at 08:47:06PM +0300, Alexey Kondratov wrote: The only difference is that point 3) and tablespace part of 5) were missing in RelationSetNewRelfilenode, so I added them, and I do 4) after 6) in REINDEX. Thus, it seems that in my impl

Re: Partitioning versus autovacuum

2019-12-02 Thread yuzuko
Hello Greg, > At the risk of forking this thread... I think there's actually a > planner estimation bug here too. > I think that is not a bug. The estimation error occurred there were no parent's statistics. We should run analyze on *partitioned table*. Here is your test case: create table p (i

Re: Autovacuum on partitioned table

2019-12-02 Thread Laurenz Albe
On Mon, 2019-12-02 at 18:02 +0900, yuzuko wrote: > Greg reported in [1] before, autovacuum ignores partitioned tables. > That is, even if individual partitions’ statistics are updated, its parent's > statistics are not updated. This is TODO for declarative partitioning. > As Amit mentioned in [2],

Autovacuum on partitioned table

2019-12-02 Thread yuzuko
Hello, Greg reported in [1] before, autovacuum ignores partitioned tables. That is, even if individual partitions’ statistics are updated, its parent's statistics are not updated. This is TODO for declarative partitioning. As Amit mentioned in [2], a way to make parent's statistics from partition

Re: log bind parameter values on error

2019-12-02 Thread Alexey Bashtanov
Patch file name is not great, changing it. On 30/11/2019 12:35, Alexey Bashtanov wrote: I've implemented it using error contexts, please could you have a look at the patch attached? diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 4ec13f3311..b3a0d27861 100644 --- a/doc/s

Re: [HACKERS] Block level parallel vacuum

2019-12-02 Thread Amit Kapila
On Sun, Dec 1, 2019 at 11:01 PM Sergei Kornilov wrote: > Hi > > > I think I got your point. Your proposal is that it's more efficient if > > we make the leader process vacuum the index that can be processed only > > the leader process (i.e. indexes not supporting parallel index vacuum) > > while

Re: [HACKERS] Block level parallel vacuum

2019-12-02 Thread Dilip Kumar
On Sun, Dec 1, 2019 at 11:01 PM Sergei Kornilov wrote: > > Hi > > > I think I got your point. Your proposal is that it's more efficient if > > we make the leader process vacuum the index that can be processed only > > the leader process (i.e. indexes not supporting parallel index vacuum) > > while

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2019-12-02 Thread Michael Paquier
On Wed, Nov 27, 2019 at 08:47:06PM +0300, Alexey Kondratov wrote: > The only difference is that point 3) and tablespace part of 5) were missing > in RelationSetNewRelfilenode, so I added them, and I do 4) after 6) in > REINDEX. Thus, it seems that in my implementation of tablespace change in > REIN