Re: multi-install PostgresNode fails with older postgres versions

2021-04-21 Thread Michael Paquier
On Wed, Apr 21, 2021 at 10:04:40AM -0400, Andrew Dunstan wrote: > Here's a patch with these things attended to. Thanks. Reading through it, that seems pretty much fine to me. I have not spent time checking _version_cmp in details though :) -- Michael signature.asc Description: PGP signature

RE: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-21 Thread tsunakawa.ta...@fujitsu.com
From: Tom Lane > Amit Kapila writes: > > IIUC, the idea here is to check for parallel safety of functions at > > someplace in the code during function invocation so that if we execute > > any parallel unsafe/restricted function via parallel worker then we > > error out. If so, isn't it possible t

Re: TRUNCATE on foreign table

2021-04-21 Thread Fujii Masao
On 2021/04/22 9:39, Bharath Rupireddy wrote: One comment on truncate_foreign_table_docs_v1.patch: 1) I think it is "to be truncated" + rels is a list of Relation + data structures for each foreign table to truncated. Fixed. Thanks! How about a slightly changed phrasing like below? +

Re: PageGetItemIdCareful - should we MAXALIGN sizeof(BTPageOpaqueData)?

2021-04-21 Thread Dilip Kumar
On Thu, Apr 22, 2021 at 10:40 AM Bharath Rupireddy wrote: > > Hi, > > In the PageGetItemIdCareful() introduced by commit a9ce839a, it seems > like we are using btree page pd_special structure BTPageOpaqueData for > error case without max aligning it. > if (ItemIdGetOffset(itemid) + ItemIdGetLe

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Thu, Apr 22, 2021 at 10:39 AM Masahiko Sawada wrote: > > On Thu, Apr 22, 2021 at 1:50 PM Amit Kapila wrote: > > > > On Thu, Apr 22, 2021 at 8:26 AM Masahiko Sawada > > wrote: > > > > > > 2. > > + if (replSlotStatHash != NULL) > > + (void) hash_search(replSlotStatHash, > > +(void *) &(msg

Re: Procedure of modification of autoconf scripts

2021-04-21 Thread Ian Zagorskikh
Tom, Thank you now it's clear! On Thu, Apr 22, 2021 at 5:58 AM Tom Lane wrote: > If you see massive changes in the configure script after a localized > change in configure.ac, it probably means that you're not using the > right autoconf version. > > Our project convention is to use exactly the

Re: ERROR: "ft1" is of the wrong type.

2021-04-21 Thread Kyotaro Horiguchi
At Thu, 22 Apr 2021 13:48:45 +0900, Michael Paquier wrote in > Hi Horiguchi-san, > > On Fri, Feb 19, 2021 at 05:30:39PM +0900, Kyotaro Horiguchi wrote: > > The attached is that. > > > > ATT_VIEW is used for "CREATE OR REPLACE view" and checked against > > earlier in DefineVirtualRelation. But

Re: Procedure of modification of autoconf scripts

2021-04-21 Thread Tom Lane
Ian Zagorskikh writes: > Can you please explain the process of adding new items into autoconf > scripts? Specifically into configure.ac. For example, if I want to add a > new --with-foo argument, let's say a new 3rd party library. What should I > do after proper configure.ac modification? Should I

Procedure of modification of autoconf scripts

2021-04-21 Thread Ian Zagorskikh
Hi all, Can you please explain the process of adding new items into autoconf scripts? Specifically into configure.ac. For example, if I want to add a new --with-foo argument, let's say a new 3rd party library. What should I do after proper configure.ac modification? Should I also re-generate confi

PageGetItemIdCareful - should we MAXALIGN sizeof(BTPageOpaqueData)?

2021-04-21 Thread Bharath Rupireddy
Hi, In the PageGetItemIdCareful() introduced by commit a9ce839a, it seems like we are using btree page pd_special structure BTPageOpaqueData for error case without max aligning it. if (ItemIdGetOffset(itemid) + ItemIdGetLength(itemid) > BLCKSZ - sizeof(BTPageOpaqueData)) erepor

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Thu, Apr 22, 2021 at 1:50 PM Amit Kapila wrote: > > On Thu, Apr 22, 2021 at 8:26 AM Masahiko Sawada wrote: > > > > Few comments: > 1. > I think we want stats collector to not use pgStatLocalContext unless > it has read the stats file similar to other cases. So probably, we > should allocate it

Re: Synchronous commit behavior during network outage

2021-04-21 Thread Andrey Borodin
Hi Ondrej! > 19 апр. 2021 г., в 22:19, Ondřej Žižka написал(а): > > Do you think, that would be possible to implement a process that would solve > this use case? > Thank you > Ondrej > Feel free to review patch fixing this at [0]. It's classified as "Server Features", but I'm sure it's a bug

Re: problem with RETURNING and update row movement

2021-04-21 Thread Amit Langote
On Thu, Apr 22, 2021 at 9:37 AM Tom Lane wrote: > Amit Langote writes: > > FWIW, I think we should go ahead and apply the patches for the bug > > reported here. Anyone who tries to project an updated tuple's system > > columns using RETURNING are likely to face problems one way or > > another, e

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Thu, Apr 22, 2021 at 8:26 AM Masahiko Sawada wrote: > Few comments: 1. I think we want stats collector to not use pgStatLocalContext unless it has read the stats file similar to other cases. So probably, we should allocate it in pgStatLocalContext when we read 'R' message in pgstat_read_statsf

Re: ERROR: "ft1" is of the wrong type.

2021-04-21 Thread Michael Paquier
Hi Horiguchi-san, On Fri, Feb 19, 2021 at 05:30:39PM +0900, Kyotaro Horiguchi wrote: > The attached is that. > > ATT_VIEW is used for "CREATE OR REPLACE view" and checked against > earlier in DefineVirtualRelation. But we can add a test to make sure > that is checked anywhere. My apologies for n

Re: Table refer leak in logical replication

2021-04-21 Thread Michael Paquier
On Wed, Apr 21, 2021 at 09:58:10PM +0900, Amit Langote wrote: > Okay, done. So, I have been working on that today, and tried to apply the full set before realizing when writing the commit message that this was a set of bullet points, and that this was too much for a single commit. The tests are a

Re: Stale description for pg_basebackup

2021-04-21 Thread Kyotaro Horiguchi
At Thu, 22 Apr 2021 13:06:50 +0900, Fujii Masao wrote in > Either works for me. I didn't add "()" because I just used the same > description > as that in func.sgml. > > it may be useful to run pg_switch_wal on the > primary in order to trigger an immediate segment switch.) .. >

Re: Replication slot stats misgivings

2021-04-21 Thread Dilip Kumar
On Thu, Apr 22, 2021 at 7:52 AM Masahiko Sawada wrote: > > On Wed, Apr 21, 2021 at 4:44 PM Dilip Kumar wrote: > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > wrote: > > > > > > > > I've attached the patch. In addition to the test Vignesh prepared, I > > > added one test for the mess

Re: Stale description for pg_basebackup

2021-04-21 Thread Fujii Masao
On 2021/04/22 11:19, Kyotaro Horiguchi wrote: At Thu, 22 Apr 2021 10:56:10 +0900, Fujii Masao wrote in On 2021/04/22 9:25, Kyotaro Horiguchi wrote: What about the following description? --- When you are using -X none, if write activity on the primary is low, pg_basebacku

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Andrey Borodin
> 21 апр. 2021 г., в 21:01, Stefan Keller написал(а): > > What's needed - and what many here as well as the "ML-In-Databases" > paper from Kraska et al. (2021) are saying - is, that a new index > (like a learned index) should be implemented as a PostgreSQL > extension. BTW, you don't have to

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-21 Thread Etsuro Fujita
On Wed, Mar 31, 2021 at 2:12 PM Etsuro Fujita wrote: > On Wed, Mar 31, 2021 at 10:11 AM Kyotaro Horiguchi > wrote: > > +* We'll prefer to consider this join async-capable if any > > table from > > +* either side of the join is considered async-capable. > > +

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 7:11 PM Amit Kapila wrote: > > On Wed, Apr 21, 2021 at 3:39 PM Masahiko Sawada wrote: > > > > > > > > The test is not waiting for a new slot creation message to reach the > > > stats collector. So, if the old slot data still exists in the file and > > > now when we read st

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 4:44 PM Dilip Kumar wrote: > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > > > > > I've attached the patch. In addition to the test Vignesh prepared, I > > added one test for the message for creating a slot that checks if the > > statistics are initialized aft

Re: Stale description for pg_basebackup

2021-04-21 Thread Kyotaro Horiguchi
At Thu, 22 Apr 2021 10:56:10 +0900, Fujii Masao wrote in > > > On 2021/04/22 9:25, Kyotaro Horiguchi wrote: > >> What about the following description? > >> > >> --- > >> When you are using -X none, if write activity on the primary is low, > >> pg_basebackup may need to wait a l

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Tom Lane
Andres Freund writes: > On 2021-04-21 21:21:05 -0400, Tom Lane wrote: >> What I'm doing is running the core regression tests with a single >> standby (on the same machine) and wal_consistency_checking = all. > Do you run them over replication, or sequentially by storing data into > an archive? Ju

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Thomas Munro
On Thu, Apr 22, 2021 at 1:21 PM Tom Lane wrote: > I've also tried to reproduce on 32-bit and 64-bit Intel, without > success. So if this is real, maybe it's related to being big-endian > hardware? But it's also quite sensitive to $dunno-what, maybe the > history of WAL records that have already

Re: Stale description for pg_basebackup

2021-04-21 Thread Fujii Masao
On 2021/04/22 9:25, Kyotaro Horiguchi wrote: What about the following description? --- When you are using -X none, if write activity on the primary is low, pg_basebackup may need to wait a long time for all WAL files required for the backup to be archived. It may be useful to r

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 21:21:05 -0400, Tom Lane wrote: > What I'm doing is running the core regression tests with a single > standby (on the same machine) and wal_consistency_checking = all. Do you run them over replication, or sequentially by storing data into an archive? Just curious, because its so

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Tom Lane
Stephen Frost writes: > On Wed, Apr 21, 2021 at 19:17 Thomas Munro wrote: >> ... Personally I think the right thing to do now is to revert it >> and re-propose for 15 early in the cycle, supported with some better >> testing infrastructure. > I tend to agree with the idea to revert it, perhaps a

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Stephen Frost
Greetings, * Daniel Carter (danielchriscarter+postg...@gmail.com) wrote: > On 21/04/2021 18:40, Stephen Frost wrote: > >I surely hope that the intent here is to use Negotiate / SPNEGO to > >authenticate the user who is connecting to the webserver and then have > >credentials delegated (ideally thr

Re: TRUNCATE on foreign table

2021-04-21 Thread Bharath Rupireddy
On Wed, Apr 21, 2021 at 8:31 PM Fujii Masao wrote: > Applied. Attached is the updated version of the patch > (truncate_foreign_table_dont_pass_only_clause_v2.patch). > This patch includes the patch that Horiguchi-san posted upthead. > I'm thinking to commit this patch at first. +1. > > 2) Instea

Re: problem with RETURNING and update row movement

2021-04-21 Thread Tom Lane
Amit Langote writes: > FWIW, I think we should go ahead and apply the patches for the bug > reported here. Anyone who tries to project an updated tuple's system > columns using RETURNING are likely to face problems one way or > another, especially if they have partitioned tables containing > part

Re: Stale description for pg_basebackup

2021-04-21 Thread Kyotaro Horiguchi
Ugg. I was confused. At Wed, 21 Apr 2021 23:06:56 +0900, Fujii Masao wrote in > > Hmm. Some words need to be qualified. Attached. > > + If you are using -X none, there is no guarantee > on > + the primary that all WAL files required for the backup are archived > at > + the end of backup

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Daniel Carter
Hi Stephen, On 21/04/2021 18:40, Stephen Frost wrote: I surely hope that the intent here is to use Negotiate / SPNEGO to authenticate the user who is connecting to the webserver and then have credentials delegated (ideally through constrained credential delegation..) to the web server by the use

Re: decoupling table and index vacuum

2021-04-21 Thread Peter Geoghegan
On Wed, Apr 21, 2021 at 8:21 AM Robert Haas wrote: > We are used to thinking about table vacuum and index vacuum as parts > of a single, indivisible operation. You vacuum the table -- among > other things by performing HOT pruning and remembering dead TIDs -- > and then you vacuum the indexes -- r

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-21 Thread yuzuko
Hi, Thank you for discussing this item. > I think we should treat ATTACH/ > DETACH/DROP handling as a further feature to be added in a future > release, not an open item to be fixed in the current one. > I agree with your opinion. > Now, if somebody sees a very trivial way to handle it, let's di

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Stephen Frost
Greetings, On Wed, Apr 21, 2021 at 19:17 Thomas Munro wrote: > On Thu, Apr 22, 2021 at 8:16 AM Thomas Munro > wrote: > > That wasn't my plan, but I admit that the timing was non-ideal. In > > any case, I'll dig into these failures and then consider options. > > More soon. > > Yeah, this clearl

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Thomas Munro
On Thu, Apr 22, 2021 at 8:16 AM Thomas Munro wrote: > That wasn't my plan, but I admit that the timing was non-ideal. In > any case, I'll dig into these failures and then consider options. > More soon. Yeah, this clearly needs more work. xlogreader.c is difficult to work with and I think we nee

Re: when the startup process doesn't

2021-04-21 Thread Jehan-Guillaume de Rorthais
On Wed, 21 Apr 2021 12:36:05 -0700 Andres Freund wrote: > [...] > > I don't think that concern equally applies for what I am proposing > here. For one, we already have minRecoveryPoint in ControlData, and we > already use it for the purpose of determining where we need to recover > to, albeit

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-21 Thread Alvaro Herrera
On 2021-Apr-09, Robert Haas wrote: > Does this need to worry about new partitions getting attached to a > partitioned table, or old ones getting detached? (Maybe it does > already, not sure.) I was pinged because this is listed as an open item. I don't think it is one. Handling ATTACH/DETACH/DR

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-21 Thread Alvaro Herrera
On 2021-Apr-08, Tom Lane wrote: > BTW, another thing that looks like a race condition is the > extract_autovac_opts() call that is done a little bit earlier, > also without lock. I think this is actually safe, but it's ONLY > safe because we resisted the calls by certain people to add a > toast t

Re: proposal for PostgreSQL program

2021-04-21 Thread Bruce Momjian
On Tue, Apr 20, 2021 at 03:17:08AM +0300, Khaled Anas wrote: > Dear team , > > hi, I am sending this email to propose to join PostgreSQl program to enhance > my > skills and to keep up with market needs  so kindly accept my proposal . Uh, there isn't an official joining process. You should read

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-21 Thread Alvaro Herrera
On 2021-Apr-10, Justin Pryzby wrote: > If it *implies* the partition constraint, then it's at least as tight (and > maybe tighter), yes ? > > I think you're concerned with the case that someone has a partition with > "tight" bounds like (a>=200 and a<300) and a check constraint that's "less > tig

Re: decoupling table and index vacuum

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 11:21:31 -0400, Robert Haas wrote: > Opportunistic index cleanup strategies like > kill_prior_tuple and bottom-up deletion may work much better for some > indexes than others, meaning that you could have some indexes that > badly need to be vacuumed because they are full of garba

proposal for PostgreSQL program

2021-04-21 Thread Khaled Anas
Dear team , hi, I am sending this email to propose to join PostgreSQl program to enhance my skills and to keep up with market needs so kindly accept my proposal . thanks & regards khaled

Re: when the startup process doesn't

2021-04-21 Thread Stephen Frost
Greetings, On Wed, Apr 21, 2021 at 17:01 Andres Freund wrote: > On 2021-04-21 16:55:28 -0400, Tom Lane wrote: > > My concern about it was not at all about performance, but that every time > > you write it is a new opportunity for the filesystem to lose or corrupt > > the data. > > We already do,

Re: when the startup process doesn't

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 16:55:28 -0400, Tom Lane wrote: > My concern about it was not at all about performance, but that every time > you write it is a new opportunity for the filesystem to lose or corrupt > the data. We already do, sometimes very frequent, control file updates on standbys to update mi

Re: when the startup process doesn't

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 16:28:26 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > On 2021-04-21 15:51:38 -0400, Stephen Frost wrote: > > > It does seem like we have some trade-offs here to weigh, but > > > pg_control is indeed quite small.. > > > > What do you mean by that?

multirange constructor strictness

2021-04-21 Thread Peter Eisentraut
The multirange constructors created in makeMultirangeConstructors() are: multirange_constructor0 -> not strict multirange_constructor1 -> strict multirange_constructor2 -> not strict And both multirange_constructor1 and multirange_constructor2 contain code like /* * These checks should be gu

Re: when the startup process doesn't

2021-04-21 Thread Tom Lane
Stephen Frost writes: > * Andres Freund (and...@anarazel.de) wrote: >> What do you mean by that? That the overhead of writing it out more >> frequently wouldn't be too bad? Or that we shouldn't "unnecessarily" add >> more fields to it? > Mostly just that the added overhead in writing it out more

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-21 Thread Alvaro Herrera
While the approach in the previous email does pass the tests, I think (but couldn't find a test case to prove) it does so coincidentally, not because it is correct. If I make the test for "detached exist" use the cached omits-partitions-partdesc, it does fail, because we had previously cached one

Re: when the startup process doesn't

2021-04-21 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-04-21 15:51:38 -0400, Stephen Frost wrote: > > It does seem like we have some trade-offs here to weigh, but > > pg_control is indeed quite small.. > > What do you mean by that? That the overhead of writing it out more > frequently w

Re: Bogus collation version recording in recordMultipleDependencies

2021-04-21 Thread Andres Freund
Hi, On 2021-04-20 12:05:27 +1200, Thomas Munro wrote: > I'll hold off reverting for a few more days to see if anyone has any > other thoughts on that, because there doesn't seem to be any advantage > in being too hasty about it. I'm not really convinced that this is warranted, and that it isn't b

Re: Privilege boundary between sysadmin and database superuser [Was: Re: pg_amcheck option to install extension]

2021-04-21 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Apr 20, 2021, at 3:19 PM, Tom Lane wrote: > > The rest of your analysis seems a bit off-point to me, which is what > > makes me think that one of us is confused. If Alice is storing her > > data in a Postgres database, she had

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Thomas Munro
On Thu, Apr 22, 2021 at 8:07 AM Tomas Vondra wrote: > On 4/21/21 6:30 PM, Tom Lane wrote: > > Thomas Munro writes: > >> Yeah, it would have been nice to include that but it'll have to be for > >> v15 due to lack of time to convince myself that it was correct. I do > >> intend to look into more c

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Tomas Vondra
On 4/21/21 6:30 PM, Tom Lane wrote: > Thomas Munro writes: >> Yeah, it would have been nice to include that but it'll have to be for >> v15 due to lack of time to convince myself that it was correct. I do >> intend to look into more concurrency of that kind for v15. I have >> pushed these patche

Re: when the startup process doesn't

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 15:51:38 -0400, Stephen Frost wrote: > It does seem like we have some trade-offs here to weigh, but > pg_control is indeed quite small.. What do you mean by that? That the overhead of writing it out more frequently wouldn't be too bad? Or that we shouldn't "unnecessarily" add mo

Re: when the startup process doesn't

2021-04-21 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-04-21 14:36:24 -0400, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > Unfortunately I think something like a percentage is hard to calculate > > > right now. Even just looking at crash recovery (vs replica

Re: when the startup process doesn't

2021-04-21 Thread Andres Freund
Hi, On 2021-04-21 14:36:24 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > Unfortunately I think something like a percentage is hard to calculate > > right now. Even just looking at crash recovery (vs replication or > > PITR), we don't currently know where the WAL en

Re: Synchronous commit behavior during network outage

2021-04-21 Thread Ondřej Žižka
Hello, > You can monitor the pg_stat_activity for the SYNC_REP_WAIT_FLUSH wait types to detect this. I tried to see this this wait_event_type Client or IPC and wait_event Client_Read or SyncRep. In which situation I can see the SYNC_REP_WAIT_FLUSH value? > You should consider these as in d

Re: when the startup process doesn't

2021-04-21 Thread Tom Lane
Stephen Frost writes: > * Andres Freund (and...@anarazel.de) wrote: >> On 2021-04-20 14:56:58 -0400, Tom Lane wrote: >>> I wonder though whether we really need authentication here. pg_ping >>> already exposes whether the database is up, to anyone who can reach the >>> postmaster port at all. Wou

Re: when the startup process doesn't

2021-04-21 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2021-04-20 14:56:58 -0400, Tom Lane wrote: > > I wonder though whether we really need authentication here. pg_ping > > already exposes whether the database is up, to anyone who can reach the > > postmaster port at all. Would it be so ho

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Stephen Frost
Greetings, * Daniel Carter (danielchriscarter+postg...@gmail.com) wrote: > On 20/04/2021 20:01, Stephen Frost wrote: > >I'm not necessarily against this, but typically the GSSAPI library > >provides a way for you to control this using, eg, the KRB5_CCACHE > >environment variable. Is there some re

Re: WIP: WAL prefetch (another approach)

2021-04-21 Thread Tom Lane
Thomas Munro writes: > Yeah, it would have been nice to include that but it'll have to be for > v15 due to lack of time to convince myself that it was correct. I do > intend to look into more concurrency of that kind for v15. I have > pushed these patches, updated to be disabled by default. I h

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Stefan Keller
Mi., 21. Apr. 2021, 11:16 Uhr, Oleg Bartunov wrote: > Have you seen recent paper "Benchmarking Learned Indexes" ? Yes. I skipped it after that this benchmark "just" compares the algorithm implementations. What's needed - and what many here as well as the "ML-In-Databases" paper from Kraska et al

Re: track_planning causing performance regression

2021-04-21 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 12:13:17AM +0900, Fujii Masao wrote: > On 2021/04/21 23:53, Justin Pryzby wrote: > > Or: > > > > Enabling this parameter may incur a noticeable performance penalty, > > especially similar queries are executed by many concurrent > > connections > > a

Re: libpq compression

2021-04-21 Thread Ian Zagorskikh
All, thanks! On Wed, Apr 21, 2021 at 10:47 AM Michael Paquier wrote: > Patch reviews had better be posted on the community lists. This way, > if the patch is left dead by the authors (things happen in life), then > somebody else could move on with the patch without having to worry > about this

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-21 Thread Fujii Masao
On 2021/04/21 1:22, Bruce Momjian wrote: On Wed, Apr 14, 2021 at 02:33:26PM -0400, Bruce Momjian wrote: On Tue, Apr 13, 2021 at 01:30:16PM -0400, Álvaro Herrera wrote: On 2021-Apr-12, Bruce Momjian wrote: OK, the attached patch renames pg_stat_activity.queryid to 'query_id'. I have not cha

decoupling table and index vacuum

2021-04-21 Thread Robert Haas
Hi, We are used to thinking about table vacuum and index vacuum as parts of a single, indivisible operation. You vacuum the table -- among other things by performing HOT pruning and remembering dead TIDs -- and then you vacuum the indexes -- removing the remembered TIDs from the index -- and then

Re: track_planning causing performance regression

2021-04-21 Thread Fujii Masao
On 2021/04/21 23:53, Justin Pryzby wrote: Or: Enabling this parameter may incur a noticeable performance penalty, especially similar queries are executed by many concurrent connections and compete to update a small number of pg_stat_statements entries. I prefer this.

Re: RFE: Make statistics robust for unplanned events

2021-04-21 Thread Magnus Hagander
On Wed, Apr 21, 2021 at 5:02 PM Tomas Vondra wrote: > > > > On 4/21/21 2:38 PM, Magnus Hagander wrote: > > On Tue, Apr 20, 2021 at 2:00 PM Patrik Novotny wrote: > >> > >> Hello PostgreSQL Hackers, > >> > >> is it possible to preserve the PostgreSQL statistics on a server crash? > >> > >> Steps to

Re: RFE: Make statistics robust for unplanned events

2021-04-21 Thread Tomas Vondra
On 4/21/21 2:38 PM, Magnus Hagander wrote: > On Tue, Apr 20, 2021 at 2:00 PM Patrik Novotny wrote: >> >> Hello PostgreSQL Hackers, >> >> is it possible to preserve the PostgreSQL statistics on a server crash? >> >> Steps to reproduce the behaviour: >> 1) Observe the statistics counters, take no

Re: TRUNCATE on foreign table

2021-04-21 Thread Fujii Masao
On 2021/04/16 15:13, Bharath Rupireddy wrote: On Fri, Apr 16, 2021 at 8:24 AM Fujii Masao wrote: We are still discussing whether RESTRICT option should be pushed down to a foreign data wrapper. But ISTM at least we could reach the consensus about the drop of extra information for each foreign

Re: track_planning causing performance regression

2021-04-21 Thread Justin Pryzby
On Wed, Apr 21, 2021 at 11:38:52PM +0900, Fujii Masao wrote: > On 2021/04/19 23:55, Justin Pryzby wrote: > > What does "kind" mean ? I think it means a "normalized" query or a "query > > structure". > > > > "a fewer kinds" is wrong, so I think the docs should say "a small number of > > queries" o

Re: TRUNCATE on foreign table

2021-04-21 Thread Fujii Masao
On 2021/04/16 14:20, Kyotaro Horiguchi wrote: At Fri, 16 Apr 2021 11:54:16 +0900, Fujii Masao wrote in On 2021/04/16 9:15, Bharath Rupireddy wrote: On Thu, Apr 15, 2021 at 8:19 PM Fujii Masao wrote: On 2021/04/14 12:54, Bharath Rupireddy wrote: IMHO, we can push all the TRUNCATE options

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-21 Thread Alvaro Herrera
On 2021-Apr-20, Daniel Carter wrote: > +#ifdef ENABLE_GSS > + {"ccache_name", NULL, NULL, NULL, > + "Credential-cache-name", "", 64, > + offsetof(struct pg_conn, ccache_name)}, > +#endif I think it would be better that this option name includes "gss" somewhere, and perhaps eve

Re: track_planning causing performance regression

2021-04-21 Thread Fujii Masao
On 2021/04/19 23:55, Justin Pryzby wrote: What does "kind" mean ? I think it means a "normalized" query or a "query structure". "a fewer kinds" is wrong, so I think the docs should say "a small number of queries" or maybe: Okay, I agree to update the description. Enabling this para

Re: Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-21 Thread Andrew Dunstan
On 4/20/21 6:49 PM, Alexey Kondratov wrote: > On 2021-04-20 18:03, Tom Lane wrote: >> Andrew Dunstan writes: >>> On 4/19/21 7:22 PM, Tom Lane wrote: I wonder whether we could get away with just replacing the $use_tcp test with $TestLib::windows_os.  It's not really apparent to me

Re: Stale description for pg_basebackup

2021-04-21 Thread Fujii Masao
On 2021/04/21 11:09, Kyotaro Horiguchi wrote: At Wed, 21 Apr 2021 10:43:30 +0900 (JST), Kyotaro Horiguchi wrote in At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi wrote in Hello. It seems to me that there's a stale description in the documentation of pg_basebackup. I think

Re: multi-install PostgresNode fails with older postgres versions

2021-04-21 Thread Andrew Dunstan
On 4/21/21 1:13 AM, Michael Paquier wrote: > On Tue, Apr 20, 2021 at 01:11:59PM -0400, Andrew Dunstan wrote: >> Here's the patch for that. > Thanks. > >> +# Accept standard formats, in case caller has handed us the output of a >> +# postgres command line tool >> +$arg = $1 >> +

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Bruce Momjian
On Wed, Apr 21, 2021 at 10:52:19AM +0200, Stefan Keller wrote: > Di., 20. Apr. 2021 23:50 Tom Lane wrote: > > There's enough support these days that you can build a new index > > type as an extension, without touching the core code at all. > > Thanks. I'm ramping up knowledge about extending PG w

Re: Do we work with LLVM 12 on s390x?

2021-04-21 Thread Honza Horak
On 3/19/21 8:15 PM, Tom Lane wrote: Andres Freund writes: I think the error above comes from a "mismatch" between the clang used to compile bitcode, and the LLVM version linked to. Normally we're somewhat tolerant of differences between the two, but there was an ABI change at some point, leadin

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-21 Thread Tom Lane
Amit Kapila writes: > On Wed, Apr 21, 2021 at 8:12 AM tsunakawa.ta...@fujitsu.com > wrote: >> From: Tom Lane >>> [ raised eyebrow... ] I find it very hard to understand why that would >>> be necessary, or even a good idea. > IIUC, the idea here is to check for parallel safety of functions at >

Re: Table refer leak in logical replication

2021-04-21 Thread Amit Langote
On Wed, Apr 21, 2021 at 7:38 PM Michael Paquier wrote: > On Wed, Apr 21, 2021 at 04:21:52PM +0900, Amit Langote wrote: > > So I had started last night by adding some tests for this in > > 003_constraints.pl because there are already some replica BR trigger > > tests there. I like your suggestion

Re: RFE: Make statistics robust for unplanned events

2021-04-21 Thread Magnus Hagander
On Tue, Apr 20, 2021 at 2:00 PM Patrik Novotny wrote: > > Hello PostgreSQL Hackers, > > is it possible to preserve the PostgreSQL statistics on a server crash? > > Steps to reproduce the behaviour: > 1) Observe the statistics counters, take note > 2) Crash the machine, e.g. with sysrq; perhaps kil

回复: Partitioned table permission question

2021-04-21 Thread Junfeng Yang
I see. Thanks for your explanation!

Re: prerequisites of pull_up_sublinks

2021-04-21 Thread Andy Fan
On Wed, Apr 21, 2021 at 4:37 PM David Rowley wrote: > On Wed, 21 Apr 2021 at 14:55, Andy Fan wrote: > > * However, this optimization *only* > > * works at the top level of WHERE or a JOIN/ON clause, because we cannot > > * distinguish whether the ANY ought to return FALSE or NULL in cases > >

INT64_FORMAT in translatable strings

2021-04-21 Thread Kyotaro Horiguchi
Hello. I found the following lines in xlogprefetch.c. >ereport(LOG, >(errmsg("recovery finished prefetching at %X/%X; " >"prefetch = " UINT64_FORMAT ", " >"skip_hit = " UINT64_FORMAT ", " ... It is found in ja.po as "recovery finished pref

Re: libpq compression

2021-04-21 Thread Michael Paquier
On Wed, Apr 21, 2021 at 09:08:09AM +, Ian Zagorskikh wrote: > I took a look at proposed patches and found several typos/mistakes. Where > should I send my comments? In this thread or directly to the authors? Patch reviews had better be posted on the community lists. This way, if the patch is

Re: libpq compression

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 2:38 PM Ian Zagorskikh wrote: > > Hi all! > > I took a look at proposed patches and found several typos/mistakes. Where > should I send my comments? In this thread or directly to the authors? > I feel it is good to send comments here. This is what we normally do for all t

Re: Table refer leak in logical replication

2021-04-21 Thread Michael Paquier
On Wed, Apr 21, 2021 at 04:21:52PM +0900, Amit Langote wrote: > So I had started last night by adding some tests for this in > 003_constraints.pl because there are already some replica BR trigger > tests there. I like your suggestion to have some tests around > partitions, so added some in 013_par

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 3:39 PM Masahiko Sawada wrote: > > > > > The test is not waiting for a new slot creation message to reach the > > stats collector. So, if the old slot data still exists in the file and > > now when we read stats via backend, then won't there exists a chance > > that old slo

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 8:12 AM tsunakawa.ta...@fujitsu.com wrote: > > From: Tom Lane > > [ raised eyebrow... ] I find it very hard to understand why that would > > be necessary, or even a good idea. Not least because there's no spare > > room there; you'd have to incur a substantial enlargemen

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 6:36 PM Amit Kapila wrote: > > On Wed, Apr 21, 2021 at 2:46 PM Masahiko Sawada wrote: > > > > On Wed, Apr 21, 2021 at 3:09 PM Amit Kapila wrote: > > > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > > wrote: > > > > > > > > > > > > I've attached the patch. In

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 6:20 PM Amit Kapila wrote: > > On Wed, Apr 21, 2021 at 2:37 PM Masahiko Sawada wrote: > > > > On Wed, Apr 21, 2021 at 12:50 PM Amit Kapila > > wrote: > > > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > > wrote: > > > > > > > > > > I have one question: > > >

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 2:46 PM Masahiko Sawada wrote: > > On Wed, Apr 21, 2021 at 3:09 PM Amit Kapila wrote: > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > wrote: > > > > > > > > > I've attached the patch. In addition to the test Vignesh prepared, I > > > added one test for the me

Re: wal stats questions

2021-04-21 Thread Masahiro Ikeda
On 2021/04/21 15:08, torikoshia wrote: > On 2021-04-16 10:27, Masahiro Ikeda wrote: >> On 2021/04/13 9:33, Fujii Masao wrote: >>> >>> >>> On 2021/03/30 20:37, Masahiro Ikeda wrote: OK, I added the condition to the fast-return check. I noticed that I misunderstood that the purpose is

Re: Replication slot stats misgivings

2021-04-21 Thread Amit Kapila
On Wed, Apr 21, 2021 at 2:37 PM Masahiko Sawada wrote: > > On Wed, Apr 21, 2021 at 12:50 PM Amit Kapila wrote: > > > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada > > wrote: > > > > > > > I have one question: > > > > + /* > > + * Create the replication slot stats hash table if we don't have

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-21 Thread Oleg Bartunov
On Tue, Apr 20, 2021 at 8:56 PM Stefan Keller wrote: > Dear Olegs, dear Nikolay, dear all > > Allow me to revive this thread: > > Are there any advances in a learned index for PostgreSQL? > > Background: I'm trying to benchmark those experimental indices. For > this I did some bibliography work (

Re: Replication slot stats misgivings

2021-04-21 Thread Masahiko Sawada
On Wed, Apr 21, 2021 at 3:09 PM Amit Kapila wrote: > > On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > > > > > > I've attached the patch. In addition to the test Vignesh prepared, I > > added one test for the message for creating a slot that checks if the > > statistics are initialized a

  1   2   >