Re: Performance Evaluation of Result Cache by using TPC-DS

2021-04-21 Thread David Rowley
On Tue, 20 Apr 2021 at 16:43, Yuya Watari wrote: > I listed all indexes on my machine by executing your query. I attached > the result to this e-mail. I hope it will help you. Thanks for sending that. I've now run some benchmarks of TPC-DS both with enable_resultcache on and off. I think I've u

Re: Table refer leak in logical replication

2021-04-21 Thread Amit Langote
On Wed, Apr 21, 2021 at 11:13 AM Amit Langote wrote: > On Wed, Apr 21, 2021 at 9:31 AM Michael Paquier wrote: > > On Tue, Apr 20, 2021 at 06:20:03PM +0530, Amit Kapila wrote: > > > +1. I think it makes sense to add a test case especially because we > > > don't have any existing test in this area.

Re: Synchronous commit behavior during network outage

2021-04-21 Thread Aleksander Alekseev
Hi Timas, > > Thanks for the report. It seems to be a clear violation of what is > > promised in the docs. Although it's unlikely that someone implemented > > an application which deals with important data and "pressed Ctr+C" as > > it's done in psql. So this might be not such a critical issue aft

Re: Replication slot stats misgivings

2021-04-21 Thread Dilip Kumar
On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > > I've attached the patch. In addition to the test Vignesh prepared, I > added one test for the message for creating a slot that checks if the > statistics are initialized after re-creating the same name slot. > Please review it. Overall t

Re: Synchronous commit behavior during network outage

2021-04-21 Thread Laurenz Albe
On Tue, 2021-04-20 at 18:49 +0100, Ondřej Žižka wrote: > tecmint=# select * from a; --> LAN on sync replica is OK > id > >1 > (1 row) > > tecmint=# insert into a values (2); ---> LAN on sync replica is DOWN and > insert is waiting. During this time kill the background process on the >

Re: Synchronous commit behavior during network outage

2021-04-21 Thread Pavel Stehule
st 21. 4. 2021 v 9:51 odesílatel Laurenz Albe napsal: > On Tue, 2021-04-20 at 18:49 +0100, Ondřej Žižka wrote: > > tecmint=# select * from a; --> LAN on sync replica is OK > > id > > > >1 > > (1 row) > > > > tecmint=# insert into a values (2); ---> LAN on sync replica is DOWN and > > i

RE: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-21 Thread houzj.f...@fujitsu.com
> I think we've found a few existing problems with handling the parallel safety > of > functions while doing an experiment. Could I hear your opinions on what we > should do? I'd be willing to create and submit a patch to fix them. > > The experiment is to add a parallel safety check in Functio

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Dave Page
Hi On Tue, Apr 20, 2021 at 8:44 PM Daniel Carter < danielchriscarter+postg...@gmail.com> wrote: > Hi Stephen, > > On 20/04/2021 20:01, Stephen Frost wrote: > > I'm not necessarily against this, but typically the GSSAPI library > > provides a way for you to control this using, eg, the KRB5_CCACHE

Re: Synchronous commit behavior during network outage

2021-04-21 Thread SATYANARAYANA NARLAPURAM
> > This can be an option for us in our case. But there also needs to be a > process how to detect these "stuck commits" and how to invalidate/remove > them, because in reality, if the app/user would not see the change in the > database, it/he/she will try to insert/delete it again. If it just stuc

Re: prerequisites of pull_up_sublinks

2021-04-21 Thread David Rowley
On Wed, 21 Apr 2021 at 14:55, Andy Fan wrote: > * However, this optimization *only* > * works at the top level of WHERE or a JOIN/ON clause, because we cannot > * distinguish whether the ANY ought to return FALSE or NULL in cases > * involving NULL inputs. Also, in an outer join's ON clause we

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Stefan Keller
Di., 20. Apr. 2021 23:50 Tom Lane wrote: > There's enough support these days that you can build a new index > type as an extension, without touching the core code at all. Thanks. I'm ramping up knowledge about extending PG with C++. I'm still interested to understand in principle what an index h

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 12:50 PM Amit Kapila wrote: > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > > > > I have one question: > > + /* > + * Create the replication slot stats hash table if we don't have > + * it already. > + */ > + if (replSlotStats == NULL) > { > - if (namestrcmp

Re: libpq compression

2021-04-21 Thread Ian Zagorskikh
Hi all! I took a look at proposed patches and found several typos/mistakes. Where should I send my comments? In this thread or directly to the authors? Thanks! On Wed, Apr 21, 2021 at 9:03 AM Daniil Zakhlystov wrote: > Hi, thanks for your review! > > > On Mar 19, 2021, at 11:28 AM, Justin Pry

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 3:09 PM Amit Kapila wrote: > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > > > > > > I've attached the patch. In addition to the test Vignesh prepared, I > > added one test for the message for creating a slot that checks if the > > statistics are initialized a

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Oleg Bartunov
On Tue, Apr 20, 2021 at 8:56 PM Stefan Keller wrote: > Dear Olegs, dear Nikolay, dear all > > Allow me to revive this thread: > > Are there any advances in a learned index for PostgreSQL? > > Background: I'm trying to benchmark those experimental indices. For > this I did some bibliography work (

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 2:37 PM Masahiko Sawada wrote: > > On Wed, Apr 21, 2021 at 12:50 PM Amit Kapila wrote: > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > wrote: > > > > > > > I have one question: > > > > + /* > > + * Create the replication slot stats hash table if we don't have

Re: wal stats questions

2021-04-21 Thread Masahiro Ikeda
On 2021/04/21 15:08, torikoshia wrote: > On 2021-04-16 10:27, Masahiro Ikeda wrote: >> On 2021/04/13 9:33, Fujii Masao wrote: >>> >>> >>> On 2021/03/30 20:37, Masahiro Ikeda wrote: OK, I added the condition to the fast-return check. I noticed that I misunderstood that the purpose is

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 2:46 PM Masahiko Sawada wrote: > > On Wed, Apr 21, 2021 at 3:09 PM Amit Kapila wrote: > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > wrote: > > > > > > > > > I've attached the patch. In addition to the test Vignesh prepared, I > > > added one test for the me

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 6:20 PM Amit Kapila wrote: > > On Wed, Apr 21, 2021 at 2:37 PM Masahiko Sawada wrote: > > > > On Wed, Apr 21, 2021 at 12:50 PM Amit Kapila > > wrote: > > > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > > wrote: > > > > > > > > > > I have one question: > > >

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 6:36 PM Amit Kapila wrote: > > On Wed, Apr 21, 2021 at 2:46 PM Masahiko Sawada wrote: > > > > On Wed, Apr 21, 2021 at 3:09 PM Amit Kapila wrote: > > > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > > wrote: > > > > > > > > > > > > I've attached the patch. In

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 8:12 AM tsunakawa.ta...@fujitsu.com wrote: > > From: Tom Lane > > [ raised eyebrow... ] I find it very hard to understand why that would > > be necessary, or even a good idea. Not least because there's no spare > > room there; you'd have to incur a substantial enlargemen

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 3:39 PM Masahiko Sawada wrote: > > > > > The test is not waiting for a new slot creation message to reach the > > stats collector. So, if the old slot data still exists in the file and > > now when we read stats via backend, then won't there exists a chance > > that old slo

Re: Table refer leak in logical replication

2021-04-21 Thread Michael Paquier
On Wed, Apr 21, 2021 at 04:21:52PM +0900, Amit Langote wrote: > So I had started last night by adding some tests for this in > 003_constraints.pl because there are already some replica BR trigger > tests there. I like your suggestion to have some tests around > partitions, so added some in 013_par

Re: libpq compression

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 2:38 PM Ian Zagorskikh wrote: > > Hi all! > > I took a look at proposed patches and found several typos/mistakes. Where > should I send my comments? In this thread or directly to the authors? > I feel it is good to send comments here. This is what we normally do for all t

Re: libpq compression

2021-04-21 Thread Michael Paquier
On Wed, Apr 21, 2021 at 09:08:09AM +, Ian Zagorskikh wrote: > I took a look at proposed patches and found several typos/mistakes. Where > should I send my comments? In this thread or directly to the authors? Patch reviews had better be posted on the community lists. This way, if the patch is

INT64_FORMAT in translatable strings

2021-04-21 Thread Kyotaro Horiguchi
Hello. I found the following lines in xlogprefetch.c. >ereport(LOG, >(errmsg("recovery finished prefetching at %X/%X; " >"prefetch = " UINT64_FORMAT ", " >"skip_hit = " UINT64_FORMAT ", " ... It is found in ja.po as "recovery finished pref

Re: prerequisites of pull_up_sublinks

2021-04-21 Thread Andy Fan
On Wed, Apr 21, 2021 at 4:37 PM David Rowley wrote: > On Wed, 21 Apr 2021 at 14:55, Andy Fan wrote: > > * However, this optimization *only* > > * works at the top level of WHERE or a JOIN/ON clause, because we cannot > > * distinguish whether the ANY ought to return FALSE or NULL in cases > >

回复: Partitioned table permission question

2021-04-21 Thread Junfeng Yang
I see. Thanks for your explanation!

Re: RFE: Make statistics robust for unplanned events

2021-04-21 Thread Magnus Hagander
On Tue, Apr 20, 2021 at 2:00 PM Patrik Novotny wrote: > > Hello PostgreSQL Hackers, > > is it possible to preserve the PostgreSQL statistics on a server crash? > > Steps to reproduce the behaviour: > 1) Observe the statistics counters, take note > 2) Crash the machine, e.g. with sysrq; perhaps kil

Re: Table refer leak in logical replication

2021-04-21 Thread Amit Langote
On Wed, Apr 21, 2021 at 7:38 PM Michael Paquier wrote: > On Wed, Apr 21, 2021 at 04:21:52PM +0900, Amit Langote wrote: > > So I had started last night by adding some tests for this in > > 003_constraints.pl because there are already some replica BR trigger > > tests there. I like your suggestion

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-21 Thread Tom Lane
Amit Kapila writes: > On Wed, Apr 21, 2021 at 8:12 AM tsunakawa.ta...@fujitsu.com > wrote: >> From: Tom Lane >>> [ raised eyebrow... ] I find it very hard to understand why that would >>> be necessary, or even a good idea. > IIUC, the idea here is to check for parallel safety of functions at >

Re: Do we work with LLVM 12 on s390x?

2021-04-21 Thread Honza Horak
On 3/19/21 8:15 PM, Tom Lane wrote: Andres Freund writes: I think the error above comes from a "mismatch" between the clang used to compile bitcode, and the LLVM version linked to. Normally we're somewhat tolerant of differences between the two, but there was an ABI change at some point, leadin

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Bruce Momjian
On Wed, Apr 21, 2021 at 10:52:19AM +0200, Stefan Keller wrote: > Di., 20. Apr. 2021 23:50 Tom Lane wrote: > > There's enough support these days that you can build a new index > > type as an extension, without touching the core code at all. > > Thanks. I'm ramping up knowledge about extending PG w

Re: multi-install PostgresNode fails with older postgres versions

2021-04-21 Thread Andrew Dunstan
On 4/21/21 1:13 AM, Michael Paquier wrote: > On Tue, Apr 20, 2021 at 01:11:59PM -0400, Andrew Dunstan wrote: >> Here's the patch for that. > Thanks. > >> +# Accept standard formats, in case caller has handed us the output of a >> +# postgres command line tool >> +$arg = $1 >> +

Re: Stale description for pg_basebackup

2021-04-21 Thread Fujii Masao
On 2021/04/21 11:09, Kyotaro Horiguchi wrote: At Wed, 21 Apr 2021 10:43:30 +0900 (JST), Kyotaro Horiguchi wrote in At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi wrote in Hello. It seems to me that there's a stale description in the documentation of pg_basebackup. I think

Re: Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-21 Thread Andrew Dunstan
On 4/20/21 6:49 PM, Alexey Kondratov wrote: > On 2021-04-20 18:03, Tom Lane wrote: >> Andrew Dunstan writes: >>> On 4/19/21 7:22 PM, Tom Lane wrote: I wonder whether we could get away with just replacing the $use_tcp test with $TestLib::windows_os.  It's not really apparent to me

Re: track_planning causing performance regression

2021-04-21 Thread Fujii Masao
On 2021/04/19 23:55, Justin Pryzby wrote: What does "kind" mean ? I think it means a "normalized" query or a "query structure". "a fewer kinds" is wrong, so I think the docs should say "a small number of queries" or maybe: Okay, I agree to update the description. Enabling this para

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Alvaro Herrera
On 2021-Apr-20, Daniel Carter wrote: > +#ifdef ENABLE_GSS > + {"ccache_name", NULL, NULL, NULL, > + "Credential-cache-name", "", 64, > + offsetof(struct pg_conn, ccache_name)}, > +#endif I think it would be better that this option name includes "gss" somewhere, and perhaps eve

Re: TRUNCATE on foreign table

2021-04-21 Thread Fujii Masao
On 2021/04/16 14:20, Kyotaro Horiguchi wrote: At Fri, 16 Apr 2021 11:54:16 +0900, Fujii Masao wrote in On 2021/04/16 9:15, Bharath Rupireddy wrote: On Thu, Apr 15, 2021 at 8:19 PM Fujii Masao wrote: On 2021/04/14 12:54, Bharath Rupireddy wrote: IMHO, we can push all the TRUNCATE options

Re: track_planning causing performance regression

2021-04-21 Thread Justin Pryzby
On Wed, Apr 21, 2021 at 11:38:52PM +0900, Fujii Masao wrote: > On 2021/04/19 23:55, Justin Pryzby wrote: > > What does "kind" mean ? I think it means a "normalized" query or a "query > > structure". > > > > "a fewer kinds" is wrong, so I think the docs should say "a small number of > > queries" o

Re: TRUNCATE on foreign table

2021-04-21 Thread Fujii Masao
On 2021/04/16 15:13, Bharath Rupireddy wrote: On Fri, Apr 16, 2021 at 8:24 AM Fujii Masao wrote: We are still discussing whether RESTRICT option should be pushed down to a foreign data wrapper. But ISTM at least we could reach the consensus about the drop of extra information for each foreign

Re: RFE: Make statistics robust for unplanned events

2021-04-21 Thread Tomas Vondra
On 4/21/21 2:38 PM, Magnus Hagander wrote: > On Tue, Apr 20, 2021 at 2:00 PM Patrik Novotny wrote: >> >> Hello PostgreSQL Hackers, >> >> is it possible to preserve the PostgreSQL statistics on a server crash? >> >> Steps to reproduce the behaviour: >> 1) Observe the statistics counters, take no

Re: RFE: Make statistics robust for unplanned events

2021-04-21 Thread Magnus Hagander
On Wed, Apr 21, 2021 at 5:02 PM Tomas Vondra wrote: > > > > On 4/21/21 2:38 PM, Magnus Hagander wrote: > > On Tue, Apr 20, 2021 at 2:00 PM Patrik Novotny wrote: > >> > >> Hello PostgreSQL Hackers, > >> > >> is it possible to preserve the PostgreSQL statistics on a server crash? > >> > >> Steps to

Re: track_planning causing performance regression

2021-04-21 Thread Fujii Masao
On 2021/04/21 23:53, Justin Pryzby wrote: Or: Enabling this parameter may incur a noticeable performance penalty, especially similar queries are executed by many concurrent connections and compete to update a small number of pg_stat_statements entries. I prefer this.

decoupling table and index vacuum

2021-04-21 Thread Robert Haas
Hi, We are used to thinking about table vacuum and index vacuum as parts of a single, indivisible operation. You vacuum the table -- among other things by performing HOT pruning and remembering dead TIDs -- and then you vacuum the indexes -- removing the remembered TIDs from the index -- and then

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-21 Thread Fujii Masao
On 2021/04/21 1:22, Bruce Momjian wrote: On Wed, Apr 14, 2021 at 02:33:26PM -0400, Bruce Momjian wrote: On Tue, Apr 13, 2021 at 01:30:16PM -0400, Álvaro Herrera wrote: On 2021-Apr-12, Bruce Momjian wrote: OK, the attached patch renames pg_stat_activity.queryid to 'query_id'. I have not cha

Re: libpq compression

2021-04-21 Thread Ian Zagorskikh
All, thanks! On Wed, Apr 21, 2021 at 10:47 AM Michael Paquier wrote: > Patch reviews had better be posted on the community lists. This way, > if the patch is left dead by the authors (things happen in life), then > somebody else could move on with the patch without having to worry > about this

Re: track_planning causing performance regression

2021-04-21 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 12:13:17AM +0900, Fujii Masao wrote: > On 2021/04/21 23:53, Justin Pryzby wrote: > > Or: > > > > Enabling this parameter may incur a noticeable performance penalty, > > especially similar queries are executed by many concurrent > > connections > > a

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Stefan Keller
Mi., 21. Apr. 2021, 11:16 Uhr, Oleg Bartunov wrote: > Have you seen recent paper "Benchmarking Learned Indexes" ? Yes. I skipped it after that this benchmark "just" compares the algorithm implementations. What's needed - and what many here as well as the "ML-In-Databases" paper from Kraska et al

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Tom Lane
Thomas Munro writes: > Yeah, it would have been nice to include that but it'll have to be for > v15 due to lack of time to convince myself that it was correct. I do > intend to look into more concurrency of that kind for v15. I have > pushed these patches, updated to be disabled by default. I h

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Stephen Frost
Greetings, * Daniel Carter (danielchriscarter+postg...@gmail.com) wrote: > On 20/04/2021 20:01, Stephen Frost wrote: > >I'm not necessarily against this, but typically the GSSAPI library > >provides a way for you to control this using, eg, the KRB5_CCACHE > >environment variable. Is there some re

Re: when the startup process doesn't

2021-04-21 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-04-20 14:56:58 -0400, Tom Lane wrote: > > I wonder though whether we really need authentication here. pg_ping > > already exposes whether the database is up, to anyone who can reach the > > postmaster port at all. Would it be so ho

Re: when the startup process doesn't

2021-04-21 Thread Tom Lane
Stephen Frost writes: > * Andres Freund (and...@anarazel.de) wrote: >> On 2021-04-20 14:56:58 -0400, Tom Lane wrote: >>> I wonder though whether we really need authentication here. pg_ping >>> already exposes whether the database is up, to anyone who can reach the >>> postmaster port at all. Wou

Re: Synchronous commit behavior during network outage

2021-04-21 Thread Ondřej Žižka
Hello, > You can monitor the pg_stat_activity for the SYNC_REP_WAIT_FLUSH wait types to detect this. I tried to see this this wait_event_type Client or IPC and wait_event Client_Read or SyncRep. In which situation I can see the SYNC_REP_WAIT_FLUSH value? > You should consider these as in d

Re: when the startup process doesn't

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 14:36:24 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > Unfortunately I think something like a percentage is hard to calculate > > right now. Even just looking at crash recovery (vs replication or > > PITR), we don't currently know where the WAL en

Re: when the startup process doesn't

2021-04-21 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-04-21 14:36:24 -0400, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > Unfortunately I think something like a percentage is hard to calculate > > > right now. Even just looking at crash recovery (vs replica

Re: when the startup process doesn't

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 15:51:38 -0400, Stephen Frost wrote: > It does seem like we have some trade-offs here to weigh, but > pg_control is indeed quite small.. What do you mean by that? That the overhead of writing it out more frequently wouldn't be too bad? Or that we shouldn't "unnecessarily" add mo

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Tomas Vondra
On 4/21/21 6:30 PM, Tom Lane wrote: > Thomas Munro writes: >> Yeah, it would have been nice to include that but it'll have to be for >> v15 due to lack of time to convince myself that it was correct. I do >> intend to look into more concurrency of that kind for v15. I have >> pushed these patche

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Thomas Munro
On Thu, Apr 22, 2021 at 8:07 AM Tomas Vondra wrote: > On 4/21/21 6:30 PM, Tom Lane wrote: > > Thomas Munro writes: > >> Yeah, it would have been nice to include that but it'll have to be for > >> v15 due to lack of time to convince myself that it was correct. I do > >> intend to look into more c

Re: Privilege boundary between sysadmin and database superuser [Was: Re: pg_amcheck option to install extension]

2021-04-21 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Apr 20, 2021, at 3:19 PM, Tom Lane wrote: > > The rest of your analysis seems a bit off-point to me, which is what > > makes me think that one of us is confused. If Alice is storing her > > data in a Postgres database, she had

Re: Bogus collation version recording in recordMultipleDependencies

2021-04-21 Thread Andres Freund
Hi, On 2021-04-20 12:05:27 +1200, Thomas Munro wrote: > I'll hold off reverting for a few more days to see if anyone has any > other thoughts on that, because there doesn't seem to be any advantage > in being too hasty about it. I'm not really convinced that this is warranted, and that it isn't b

Re: when the startup process doesn't

2021-04-21 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-04-21 15:51:38 -0400, Stephen Frost wrote: > > It does seem like we have some trade-offs here to weigh, but > > pg_control is indeed quite small.. > > What do you mean by that? That the overhead of writing it out more > frequently w

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-21 Thread Alvaro Herrera
While the approach in the previous email does pass the tests, I think (but couldn't find a test case to prove) it does so coincidentally, not because it is correct. If I make the test for "detached exist" use the cached omits-partitions-partdesc, it does fail, because we had previously cached one

Re: when the startup process doesn't

2021-04-21 Thread Tom Lane
Stephen Frost writes: > * Andres Freund (and...@anarazel.de) wrote: >> What do you mean by that? That the overhead of writing it out more >> frequently wouldn't be too bad? Or that we shouldn't "unnecessarily" add >> more fields to it? > Mostly just that the added overhead in writing it out more

multirange constructor strictness

2021-04-21 Thread Peter Eisentraut
The multirange constructors created in makeMultirangeConstructors() are: multirange_constructor0 -> not strict multirange_constructor1 -> strict multirange_constructor2 -> not strict And both multirange_constructor1 and multirange_constructor2 contain code like /* * These checks should be gu

Re: when the startup process doesn't

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 16:28:26 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2021-04-21 15:51:38 -0400, Stephen Frost wrote: > > > It does seem like we have some trade-offs here to weigh, but > > > pg_control is indeed quite small.. > > > > What do you mean by that?

Re: when the startup process doesn't

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 16:55:28 -0400, Tom Lane wrote: > My concern about it was not at all about performance, but that every time > you write it is a new opportunity for the filesystem to lose or corrupt > the data. We already do, sometimes very frequent, control file updates on standbys to update mi

Re: when the startup process doesn't

2021-04-21 Thread Stephen Frost
Greetings, On Wed, Apr 21, 2021 at 17:01 Andres Freund wrote: > On 2021-04-21 16:55:28 -0400, Tom Lane wrote: > > My concern about it was not at all about performance, but that every time > > you write it is a new opportunity for the filesystem to lose or corrupt > > the data. > > We already do,

proposal for PostgreSQL program

2021-04-21 Thread Khaled Anas
Dear team , hi, I am sending this email to propose to join PostgreSQl program to enhance my skills and to keep up with market needs so kindly accept my proposal . thanks & regards khaled

Re: decoupling table and index vacuum

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 11:21:31 -0400, Robert Haas wrote: > Opportunistic index cleanup strategies like > kill_prior_tuple and bottom-up deletion may work much better for some > indexes than others, meaning that you could have some indexes that > badly need to be vacuumed because they are full of garba

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-21 Thread Alvaro Herrera
On 2021-Apr-10, Justin Pryzby wrote: > If it *implies* the partition constraint, then it's at least as tight (and > maybe tighter), yes ? > > I think you're concerned with the case that someone has a partition with > "tight" bounds like (a>=200 and a<300) and a check constraint that's "less > tig

Re: proposal for PostgreSQL program

2021-04-21 Thread Bruce Momjian
On Tue, Apr 20, 2021 at 03:17:08AM +0300, Khaled Anas wrote: > Dear team , > > hi, I am sending this email to propose to join PostgreSQl program to enhance > my > skills and to keep up with market needs  so kindly accept my proposal . Uh, there isn't an official joining process. You should read

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-21 Thread Alvaro Herrera
On 2021-Apr-08, Tom Lane wrote: > BTW, another thing that looks like a race condition is the > extract_autovac_opts() call that is done a little bit earlier, > also without lock. I think this is actually safe, but it's ONLY > safe because we resisted the calls by certain people to add a > toast t

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-21 Thread Alvaro Herrera
On 2021-Apr-09, Robert Haas wrote: > Does this need to worry about new partitions getting attached to a > partitioned table, or old ones getting detached? (Maybe it does > already, not sure.) I was pinged because this is listed as an open item. I don't think it is one. Handling ATTACH/DETACH/DR

Re: when the startup process doesn't

2021-04-21 Thread Jehan-Guillaume de Rorthais
On Wed, 21 Apr 2021 12:36:05 -0700 Andres Freund wrote: > [...] > > I don't think that concern equally applies for what I am proposing > here. For one, we already have minRecoveryPoint in ControlData, and we > already use it for the purpose of determining where we need to recover > to, albeit

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Thomas Munro
On Thu, Apr 22, 2021 at 8:16 AM Thomas Munro wrote: > That wasn't my plan, but I admit that the timing was non-ideal. In > any case, I'll dig into these failures and then consider options. > More soon. Yeah, this clearly needs more work. xlogreader.c is difficult to work with and I think we nee

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Stephen Frost
Greetings, On Wed, Apr 21, 2021 at 19:17 Thomas Munro wrote: > On Thu, Apr 22, 2021 at 8:16 AM Thomas Munro > wrote: > > That wasn't my plan, but I admit that the timing was non-ideal. In > > any case, I'll dig into these failures and then consider options. > > More soon. > > Yeah, this clearl

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-21 Thread yuzuko
Hi, Thank you for discussing this item. > I think we should treat ATTACH/ > DETACH/DROP handling as a further feature to be added in a future > release, not an open item to be fixed in the current one. > I agree with your opinion. > Now, if somebody sees a very trivial way to handle it, let's di

Re: decoupling table and index vacuum

2021-04-21 Thread Peter Geoghegan
On Wed, Apr 21, 2021 at 8:21 AM Robert Haas wrote: > We are used to thinking about table vacuum and index vacuum as parts > of a single, indivisible operation. You vacuum the table -- among > other things by performing HOT pruning and remembering dead TIDs -- > and then you vacuum the indexes -- r

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Daniel Carter
Hi Stephen, On 21/04/2021 18:40, Stephen Frost wrote: I surely hope that the intent here is to use Negotiate / SPNEGO to authenticate the user who is connecting to the webserver and then have credentials delegated (ideally through constrained credential delegation..) to the web server by the use

Re: Stale description for pg_basebackup

2021-04-21 Thread Kyotaro Horiguchi
Ugg. I was confused. At Wed, 21 Apr 2021 23:06:56 +0900, Fujii Masao wrote in > > Hmm. Some words need to be qualified. Attached. > > + If you are using -X none, there is no guarantee > on > + the primary that all WAL files required for the backup are archived > at > + the end of backup

Re: problem with RETURNING and update row movement

2021-04-21 Thread Tom Lane
Amit Langote writes: > FWIW, I think we should go ahead and apply the patches for the bug > reported here. Anyone who tries to project an updated tuple's system > columns using RETURNING are likely to face problems one way or > another, especially if they have partitioned tables containing > part

Re: TRUNCATE on foreign table

2021-04-21 Thread Bharath Rupireddy
On Wed, Apr 21, 2021 at 8:31 PM Fujii Masao wrote: > Applied. Attached is the updated version of the patch > (truncate_foreign_table_dont_pass_only_clause_v2.patch). > This patch includes the patch that Horiguchi-san posted upthead. > I'm thinking to commit this patch at first. +1. > > 2) Instea

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Stephen Frost
Greetings, * Daniel Carter (danielchriscarter+postg...@gmail.com) wrote: > On 21/04/2021 18:40, Stephen Frost wrote: > >I surely hope that the intent here is to use Negotiate / SPNEGO to > >authenticate the user who is connecting to the webserver and then have > >credentials delegated (ideally thr

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Tom Lane
Stephen Frost writes: > On Wed, Apr 21, 2021 at 19:17 Thomas Munro wrote: >> ... Personally I think the right thing to do now is to revert it >> and re-propose for 15 early in the cycle, supported with some better >> testing infrastructure. > I tend to agree with the idea to revert it, perhaps a

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 21:21:05 -0400, Tom Lane wrote: > What I'm doing is running the core regression tests with a single > standby (on the same machine) and wal_consistency_checking = all. Do you run them over replication, or sequentially by storing data into an archive? Just curious, because its so

Re: Stale description for pg_basebackup

2021-04-21 Thread Fujii Masao
On 2021/04/22 9:25, Kyotaro Horiguchi wrote: What about the following description? --- When you are using -X none, if write activity on the primary is low, pg_basebackup may need to wait a long time for all WAL files required for the backup to be archived. It may be useful to r

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Thomas Munro
On Thu, Apr 22, 2021 at 1:21 PM Tom Lane wrote: > I've also tried to reproduce on 32-bit and 64-bit Intel, without > success. So if this is real, maybe it's related to being big-endian > hardware? But it's also quite sensitive to $dunno-what, maybe the > history of WAL records that have already

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Tom Lane
Andres Freund writes: > On 2021-04-21 21:21:05 -0400, Tom Lane wrote: >> What I'm doing is running the core regression tests with a single >> standby (on the same machine) and wal_consistency_checking = all. > Do you run them over replication, or sequentially by storing data into > an archive? Ju

Re: Stale description for pg_basebackup

2021-04-21 Thread Kyotaro Horiguchi
At Thu, 22 Apr 2021 10:56:10 +0900, Fujii Masao wrote in > > > On 2021/04/22 9:25, Kyotaro Horiguchi wrote: > >> What about the following description? > >> > >> --- > >> When you are using -X none, if write activity on the primary is low, > >> pg_basebackup may need to wait a l

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 4:44 PM Dilip Kumar wrote: > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > > > > > I've attached the patch. In addition to the test Vignesh prepared, I > > added one test for the message for creating a slot that checks if the > > statistics are initialized aft

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 7:11 PM Amit Kapila wrote: > > On Wed, Apr 21, 2021 at 3:39 PM Masahiko Sawada wrote: > > > > > > > > The test is not waiting for a new slot creation message to reach the > > > stats collector. So, if the old slot data still exists in the file and > > > now when we read st

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-21 Thread Etsuro Fujita
On Wed, Mar 31, 2021 at 2:12 PM Etsuro Fujita wrote: > On Wed, Mar 31, 2021 at 10:11 AM Kyotaro Horiguchi > wrote: > > +* We'll prefer to consider this join async-capable if any > > table from > > +* either side of the join is considered async-capable. > > +

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Andrey Borodin
> 21 апр. 2021 г., в 21:01, Stefan Keller написал(а): > > What's needed - and what many here as well as the "ML-In-Databases" > paper from Kraska et al. (2021) are saying - is, that a new index > (like a learned index) should be implemented as a PostgreSQL > extension. BTW, you don't have to

Re: Stale description for pg_basebackup

2021-04-21 Thread Fujii Masao
On 2021/04/22 11:19, Kyotaro Horiguchi wrote: At Thu, 22 Apr 2021 10:56:10 +0900, Fujii Masao wrote in On 2021/04/22 9:25, Kyotaro Horiguchi wrote: What about the following description? --- When you are using -X none, if write activity on the primary is low, pg_basebacku

Re: Replication slot stats misgivings

2021-04-21 Thread Dilip Kumar
On Thu, Apr 22, 2021 at 7:52 AM Masahiko Sawada wrote: > > On Wed, Apr 21, 2021 at 4:44 PM Dilip Kumar wrote: > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > wrote: > > > > > > > > I've attached the patch. In addition to the test Vignesh prepared, I > > > added one test for the mess

Re: Stale description for pg_basebackup

2021-04-21 Thread Kyotaro Horiguchi
At Thu, 22 Apr 2021 13:06:50 +0900, Fujii Masao wrote in > Either works for me. I didn't add "()" because I just used the same > description > as that in func.sgml. > > it may be useful to run pg_switch_wal on the > primary in order to trigger an immediate segment switch.) .. >

Re: Table refer leak in logical replication

2021-04-21 Thread Michael Paquier
On Wed, Apr 21, 2021 at 09:58:10PM +0900, Amit Langote wrote: > Okay, done. So, I have been working on that today, and tried to apply the full set before realizing when writing the commit message that this was a set of bullet points, and that this was too much for a single commit. The tests are a

Re: ERROR: "ft1" is of the wrong type.

2021-04-21 Thread Michael Paquier
Hi Horiguchi-san, On Fri, Feb 19, 2021 at 05:30:39PM +0900, Kyotaro Horiguchi wrote: > The attached is that. > > ATT_VIEW is used for "CREATE OR REPLACE view" and checked against > earlier in DefineVirtualRelation. But we can add a test to make sure > that is checked anywhere. My apologies for n

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Thu, Apr 22, 2021 at 8:26 AM Masahiko Sawada wrote: > Few comments: 1. I think we want stats collector to not use pgStatLocalContext unless it has read the stats file similar to other cases. So probably, we should allocate it in pgStatLocalContext when we read 'R' message in pgstat_read_statsf

  1   2   >