Re: Separate out FileSet from SharedFileSet (was Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o)

2021-08-23 Thread Amit Kapila
On Mon, Aug 23, 2021 at 3:13 PM Dilip Kumar wrote: > > On Mon, Aug 23, 2021 at 1:43 PM Amit Kapila wrote: > > Note: merge comments from multiple mails > > > > I think we should handle that in worker.c itself, by adding a > > > before_dsm_detach function before_shmem_exit right? > > > > > > > Yeah

Re: [PROPOSAL] new diagnostic items for the dynamic sql

2021-08-23 Thread Pavel Stehule
út 24. 8. 2021 v 8:16 odesílatel Dinesh Chemuduru napsal: > Hi Pavel, > > On Tue, 24 Aug 2021 at 00:19, Pavel Stehule > wrote: > >> Hi >> >> pá 20. 8. 2021 v 10:24 odesílatel Dinesh Chemuduru < >> dinesh.ku...@migops.com> napsal: >> >>> On Sun, 25 Jul 2021 at 16:34, Pavel Stehule >>> wrote: >>>

Re: [PROPOSAL] new diagnostic items for the dynamic sql

2021-08-23 Thread Dinesh Chemuduru
Hi Pavel, On Tue, 24 Aug 2021 at 00:19, Pavel Stehule wrote: > Hi > > pá 20. 8. 2021 v 10:24 odesílatel Dinesh Chemuduru < > dinesh.ku...@migops.com> napsal: > >> On Sun, 25 Jul 2021 at 16:34, Pavel Stehule >> wrote: >> > > please, can you register this patch to commitfest app > > https://commi

RE: Skipping logical replication transactions on subscriber side

2021-08-23 Thread tanghy.f...@fujitsu.com
On Monday, August 23, 2021 11:09 AM Masahiko Sawada wrote: > > I've attached updated patches. Please review them. > I tested v10-0001 patch in both streaming and no-streaming more. All tests works well. I also tried two-phase commit feature, the error context was set as expected, but please

Re: pgstat_send_connstats() introduces unnecessary timestamp and UDP overhead

2021-08-23 Thread Kyotaro Horiguchi
At Wed, 18 Aug 2021 05:16:38 +0200, Laurenz Albe wrote in > On Tue, 2021-08-17 at 02:14 -0700, Andres Freund wrote: > > On 2021-08-17 10:44:51 +0200, Laurenz Albe wrote: > > > On Sun, 2021-08-01 at 13:55 -0700, Andres Freund wrote: > > > > We maintain last_report as GetCurrentTransactionStopTime

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2021-08-23 Thread Masahiko Sawada
On Mon, Aug 23, 2021 at 10:46 AM Masahiko Sawada wrote: > > On Thu, Aug 19, 2021 at 10:52 PM Ranier Vilela wrote: > > > > Em qui., 19 de ago. de 2021 às 09:21, Masahiko Sawada > > escreveu: > >> > >> Hi all , > >> > >> It's reported on pgsql-bugs[1] that I/O timings in EXPLAIN don't show > >> t

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Craig Ringer
On Mon, 23 Aug 2021 at 22:45, Julien Rouhaud wrote: > On Mon, Aug 23, 2021 at 10:22 PM Tom Lane wrote: > > > > Bruce Momjian writes: > > > On Mon, Aug 23, 2021 at 10:15:04AM -0400, Tom Lane wrote: > > >> By that argument, *every* globally-visible variable should be marked > > >> PGDLLIMPORT. B

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Craig Ringer
On Tue, 24 Aug 2021 at 05:08, Chapman Flack wrote: > On 08/23/21 14:30, Robert Haas wrote: > > it seems likely that this proposed > > API would have the exact same problem, because it would let people do > > exactly the same thing. And, going through this proposed API would > > still be significa

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Craig Ringer
On Tue, 24 Aug 2021 at 13:21, Craig Ringer wrote: > > There is not even the thinnest pretense of Pg having a dedicated extension > API or any sort of internal vs public API separation. > Oh, and if we do want such a separation, we'll need to introduce a MUCH lower-pain-and-overhead way to get re

Re: Failure of subscription tests with topminnow

2021-08-23 Thread Ajin Cherian
On Mon, Aug 16, 2021 at 6:33 PM Amit Kapila wrote: > The "ALTER SUBSCRIPTION tap_sub CONNECTION" would lead to restart the > walsender process. Now, here the problem seems to be that the previous > walsender process (16336) didn't exit and the new process started to > use the slot which leads to

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Craig Ringer
On Tue, 24 Aug 2021 at 02:31, Robert Haas wrote: > On Mon, Aug 23, 2021 at 11:40 AM Alvaro Herrera > wrote: > > In that case, why not improve the API with functions that return the > > values in some native datatype? For scalars with native C types (int, > > floats, Boolean etc) this is easy en

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Craig Ringer
On Mon, 23 Aug 2021 at 22:15, Tom Lane wrote: > Bruce Momjian writes: > > On Sun, Aug 22, 2021 at 09:29:01PM +0800, Julien Rouhaud wrote: > >> Then shouldn't we try to prevent direct access on all platforms rather > than > >> only one? > > > Agreed. If Julian says 99% of the non-export problems

Re: Queries that should be canceled will get stuck on secure_write function

2021-08-23 Thread Fujii Masao
On 2021/08/24 0:26, Alvaro Herrera wrote: Aren't we talking about query cancellations that occur in response to a standby delay limit? Those aren't in response to user action. What I mean is that if the standby delay limit is exceeded, then we send a query cancel; we expect the standby to can

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Craig Ringer
On Sun, 22 Aug 2021 at 21:29, Julien Rouhaud wrote: > On Sun, Aug 22, 2021 at 09:19:42AM -0400, Tom Lane wrote: > > > > Uh, no, it's exactly *not* clear. There are a lot of GUCs that are only > > of interest to particular subsystems. I do not see why being a GUC makes > > something automaticall

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Julien Rouhaud
On Tue, Aug 24, 2021 at 12:36 PM Pavel Stehule wrote: > > There are few GUC variables, where we need extra fast access - work_mem, > encoding settings, and maybe an application name. For others I hadn't needed > to access it for over 20 years. But I understand that more complex extensions > lik

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Pavel Stehule
po 23. 8. 2021 v 23:08 odesílatel Chapman Flack napsal: > On 08/23/21 14:30, Robert Haas wrote: > > it seems likely that this proposed > > API would have the exact same problem, because it would let people do > > exactly the same thing. And, going through this proposed API would > > still be sign

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-08-23 Thread Greg Nancarrow
On Tue, Aug 24, 2021 at 5:00 AM Robert Haas wrote: > > > I think this looks pretty good. I am not sure I see any reason to > introduce a new function RestoreTxnSnapshotAndSetAsActive. Couldn't we > just use RestoreTransactionSnapshot() and then call > PushActiveSnapshot() from parallel.c? That see

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

2021-08-23 Thread Mahendra Singh Thalor
On Sun, 22 Aug 2021 at 22:53, Sadhuprasad Patro wrote: > > > On 2021/08/20 11:07, Mahendra Singh Thalor wrote: > > > 1) > > > Resets statistics for a single table or index in the current database > > > -to zero. > > > +to zero. The input can be a shared table also > > > >

Re: Allow batched insert during cross-partition updates

2021-08-23 Thread Amit Langote
On Tue, Jul 27, 2021 at 11:32 AM Amit Langote wrote: > On Thu, Jul 22, 2021 at 2:18 PM vignesh C wrote: > > One of the test postgres_fdw has failed, can you please post an > > updated patch for the fix: > > test postgres_fdw ... FAILED (test process exited with > > exit code 2)

Re: prevent immature WAL streaming

2021-08-23 Thread Kyotaro Horiguchi
At Mon, 23 Aug 2021 18:52:17 -0400, Alvaro Herrera wrote in > Included 蔡梦娟 and Jakub Wartak because they've expressed interest on > this topic -- notably [2] ("Bug on update timing of walrcv->flushedUpto > variable"). > > As mentioned in the course of thread [1], we're missing a fix for > strea

Re: .ready and .done files considered harmful

2021-08-23 Thread Kyotaro Horiguchi
(sigh..) At Tue, 24 Aug 2021 11:35:06 +0900 (JST), Kyotaro Horiguchi wrote in > > IIUC partial WAL files are handled because the next file in the > > sequence with the given TimeLineID won't be there, so we will fall > > back to a directory scan and pick it up. Timeline history files are > > h

Re: .ready and .done files considered harmful

2021-08-23 Thread Kyotaro Horiguchi
At Tue, 24 Aug 2021 00:03:37 +, "Bossart, Nathan" wrote in > On 8/23/21, 10:49 AM, "Robert Haas" wrote: > > On Mon, Aug 23, 2021 at 11:50 AM Bossart, Nathan > > wrote: > >> To handle a "cheating" archive command, I'd probably need to add a > >> stat() for every time pgarch_readyXLog() ret

Re: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2021-08-23 Thread Masahiko Sawada
On Tue, Aug 24, 2021 at 10:01 AM houzj.f...@fujitsu.com wrote: > > > > > -Original Message- > > From: Masahiko Sawada > > Sent: Tuesday, August 24, 2021 8:52 AM > > > > Thanks. The patch for HEAD looks good to me. Regarding the patch for PG14, I > > think we need to update the comment as

RE: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2021-08-23 Thread houzj.f...@fujitsu.com
> -Original Message- > From: Masahiko Sawada > Sent: Tuesday, August 24, 2021 8:52 AM > > Thanks. The patch for HEAD looks good to me. Regarding the patch for PG14, I > think we need to update the comment as well: > > @@ -987,7 +986,7 @@ AlterSubscription(AlterSubscriptionStmt *stmt, b

Re: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2021-08-23 Thread Masahiko Sawada
On Mon, Aug 23, 2021 at 11:05 PM houzj.f...@fujitsu.com wrote: > > On Mon, Aug 23, 2021 8:01 PM Amit Kapila wrote: > > On Mon, Aug 23, 2021 at 2:45 PM > > houzj.f...@fujitsu.com wrote: > > > > > > On Mon, Aug 23, 2021 12:59 PM Amit Kapila wrote: > > > > > > > > On Sat, Aug 7, 2021 at 6:53 PM ho

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-23 Thread Bruce Momjian
On Mon, Aug 23, 2021 at 04:57:31PM -0400, Robert Haas wrote: > On Fri, Aug 20, 2021 at 1:36 PM Shruthi Gowda wrote: > > Thanks Robert for your comments. > > I have split the patch into two portions. One that handles DB OID and > > the other that > > handles tablespace OID and relfilenode OID. > >

Re: .ready and .done files considered harmful

2021-08-23 Thread Bossart, Nathan
On 8/23/21, 10:49 AM, "Robert Haas" wrote: > On Mon, Aug 23, 2021 at 11:50 AM Bossart, Nathan wrote: >> To handle a "cheating" archive command, I'd probably need to add a >> stat() for every time pgarch_readyXLog() returned something from >> arch_files. I suspect something similar might be neede

prevent immature WAL streaming

2021-08-23 Thread Alvaro Herrera
Included 蔡梦娟 and Jakub Wartak because they've expressed interest on this topic -- notably [2] ("Bug on update timing of walrcv->flushedUpto variable"). As mentioned in the course of thread [1], we're missing a fix for streaming replication to avoid sending records that the primary hasn't fully flu

Re: The Free Space Map: Problems and Opportunities

2021-08-23 Thread Peter Geoghegan
On Fri, Aug 20, 2021 at 1:30 PM Robert Haas wrote: > On Fri, Aug 20, 2021 at 3:40 PM Peter Geoghegan wrote: > > My concern here is really the data structure and its overall > > complexity. > I don't think I understand the data structure that you have in mind > well enough to comment intelligentl

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-23 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Aug 20, 2021 at 1:36 PM Shruthi Gowda wrote: > > Thanks Robert for your comments. > > I have split the patch into two portions. One that handles DB OID and > > the other that > > handles tablespace OID and relfilenode OID. > > It'

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Chapman Flack
On 08/23/21 14:30, Robert Haas wrote: > it seems likely that this proposed > API would have the exact same problem, because it would let people do > exactly the same thing. And, going through this proposed API would > still be significantly more expensive than just accessing the bare > variables, b

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-08-23 Thread Robert Haas
On Fri, Aug 20, 2021 at 1:36 PM Shruthi Gowda wrote: > Thanks Robert for your comments. > I have split the patch into two portions. One that handles DB OID and > the other that > handles tablespace OID and relfilenode OID. It's pretty clear from the discussion, I think, that the database OID one

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-08-23 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Aug 23, 2021, at 12:51 PM, Stephen Frost wrote: > > Simply using superuser_arg() isn't sufficient is exactly the point that > > I'm trying to make. As a 'landlord', I might very well want to have > > some kind of 'landlord' ro

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-08-23 Thread Mark Dilger
> On Aug 23, 2021, at 12:51 PM, Stephen Frost wrote: > > Simply using superuser_arg() isn't sufficient is exactly the point that > I'm trying to make. As a 'landlord', I might very well want to have > some kind of 'landlord' role that isn't directly a superuser but which > could *become* a su

Re: archive status ".ready" files may be created too early

2021-08-23 Thread alvhe...@alvh.no-ip.org
On 2021-Aug-23, Bossart, Nathan wrote: > On 8/23/21, 12:55 PM, "alvhe...@alvh.no-ip.org" > wrote: > > Thanks. I've pushed this now to all live branches. > > Thank you! I appreciate the thorough reviews. Should we make a new > thread for the streaming replication fix? Yeah, this one is long

Re: archive status ".ready" files may be created too early

2021-08-23 Thread Bossart, Nathan
On 8/23/21, 12:55 PM, "alvhe...@alvh.no-ip.org" wrote: > Thanks. I've pushed this now to all live branches. Thank you! I appreciate the thorough reviews. Should we make a new thread for the streaming replication fix? Nathan

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread David Fetter
On Mon, Aug 23, 2021 at 10:56:52AM -0400, Robert Haas wrote: > On Mon, Aug 23, 2021 at 10:15 AM Tom Lane wrote: > > And yes, I absolutely would prohibit extensions from accessing many > > of them, if there were a reasonable way to do it. It would be a good > > start towards establishing a defined

Re: archive status ".ready" files may be created too early

2021-08-23 Thread alvhe...@alvh.no-ip.org
On 2021-Aug-23, Bossart, Nathan wrote: > Ah, okay. BTW the other changes you mentioned made sense to me. Thanks. I've pushed this now to all live branches. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ It does it in a really, really complicated way why doe

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-08-23 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Aug 23, 2021, at 11:13 AM, Stephen Frost wrote: > > This I have to object to pretty strongly- we have got to get away from > > the idea that just because X isn't a superuser or isn't owned by a > > superuser that it's fine to a

Re: Postgres perl module namespace

2021-08-23 Thread Robert Haas
On Mon, Aug 23, 2021 at 3:03 PM Andrew Dunstan wrote: > OK, I count 3 in favor of changing to PgTest::Cluster, 1 against, > remainder don't care. I'd have gone with something starting with Postgres:: ... but I don't care much. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-08-23 Thread Mark Dilger
> On Aug 23, 2021, at 11:13 AM, Stephen Frost wrote: > > This I have to object to pretty strongly- we have got to get away from > the idea that just because X isn't a superuser or isn't owned by a > superuser that it's fine to allow some non-superuser to mess with it. I thought we were trying

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-08-23 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Aug 23, 2021 at 2:13 PM Stephen Frost > wrote:> This I have to object to pretty strongly- we have got to get > away from > > the idea that just because X isn't a superuser or isn't owned by a > > superuser that it's fine to allow s

Re: Postgres perl module namespace

2021-08-23 Thread Andrew Dunstan
On 8/11/21 9:30 AM, Tom Lane wrote: > Alvaro Herrera writes: >> I'll recast my vote to make this line be >>     my $node = PgTest::Cluster->new('nodename'); >> which seems succint enough. I could get behind PgTest::PgCluster too, >> but having a second Pg there seems unnecessary. > Either of t

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-08-23 Thread Robert Haas
On Wed, Aug 18, 2021 at 9:28 AM Greg Nancarrow wrote: > Yes, I think I agree on that. > I've updated the patch to restore the actual transaction snapshot in > the IsolationUsesXactSnapshot() case, otherwise the active snapshot is > installed as the transaction snapshot. > I've tested the patch for

Re: [PROPOSAL] new diagnostic items for the dynamic sql

2021-08-23 Thread Pavel Stehule
Hi pá 20. 8. 2021 v 10:24 odesílatel Dinesh Chemuduru napsal: > On Sun, 25 Jul 2021 at 16:34, Pavel Stehule > wrote: > please, can you register this patch to commitfest app https://commitfest.postgresql.org/34/ Regards Pavel > >> >> ne 25. 7. 2021 v 12:52 odesílatel Dinesh Chemuduru < >> d

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Peter Geoghegan
On Mon, Aug 23, 2021 at 7:57 AM Robert Haas wrote: > In short, +1 from me for the original proposal of marking all GUCs as > PGDLLIMPORT. +1 > And, heck, +1 for marking all the other global variables > that way, too. We're not solving any problem here. We're just annoying > people, mostly people

Re: very long record lines in expanded psql output

2021-08-23 Thread Platon Pronko
Hi! Done, here's the link: https://commitfest.postgresql.org/34/3295/ Best regards, Platon Pronko On 2021-08-23 21:14, Andrew Dunstan wrote: On 8/23/21 2:00 PM, Platon Pronko wrote: Hi! Apparently I did forget something, and that's the patch itself :) Thanks for Justin Pryzby for pointing t

Re: Queries that should be canceled will get stuck on secure_write function

2021-08-23 Thread Robert Haas
On Mon, Aug 23, 2021 at 11:26 AM Alvaro Herrera wrote: > Aren't we talking about query cancellations that occur in response to a > standby delay limit? Those aren't in response to user action. Oh, you're right. But I guess a similar problem could also occur in response to pg_terminate_backend(),

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-08-23 Thread Robert Haas
On Mon, Aug 23, 2021 at 2:13 PM Stephen Frost wrote:> This I have to object to pretty strongly- we have got to get away from > the idea that just because X isn't a superuser or isn't owned by a > superuser that it's fine to allow some non-superuser to mess with it. > In particlar, just because a r

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Robert Haas
On Mon, Aug 23, 2021 at 11:40 AM Alvaro Herrera wrote: > In that case, why not improve the API with functions that return the > values in some native datatype? For scalars with native C types (int, > floats, Boolean etc) this is easy enough; I bet it'll solve 99% of the > problems or more. Sure,

Re: very long record lines in expanded psql output

2021-08-23 Thread Andrew Dunstan
On 8/23/21 2:00 PM, Platon Pronko wrote: > Hi! > > Apparently I did forget something, and that's the patch itself :) > Thanks for Justin Pryzby for pointing this out. > > Attaching the patch now. > > Please add it to the commitfest, the next one starts in about a week. cheers andrew -- Andr

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-08-23 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Jul 22, 2021, at 1:01 PM, Robert Haas wrote: > > It's awkward. I think that we can't afford to create a separate > > predefined role for every single thing that someone could > > theoretically want to sever, because then we'll

Re: pretty slow merge-join due rescan?

2021-08-23 Thread Pavel Stehule
po 23. 8. 2021 v 18:44 odesílatel Pavel Stehule napsal: > Hi > > The customer reports a very slow query. I have a reproducer script. The > dataset is not too big > > postgres=# \dt+ > List of relations > ┌┬───┬───┬───┬─┬┬───

Re: very long record lines in expanded psql output

2021-08-23 Thread Platon Pronko
Hi! Apparently I did forget something, and that's the patch itself :) Thanks for Justin Pryzby for pointing this out. Attaching the patch now. Best regards, Platon Pronko On 2021-08-23 20:51, Platon Pronko wrote: Hi! Please find attached the patch implementing the proposed changes. I hope I

Re: very long record lines in expanded psql output

2021-08-23 Thread Platon Pronko
Hi! Please find attached the patch implementing the proposed changes. I hope I didn't forget anything (docs, tab completion, comments, etc), if I did forget something please tell me and I'll fix it. Best regards, Platon Pronko On 2021-08-08 01:18, Andrew Dunstan wrote: On 8/7/21 10:56 AM, Pla

Re: .ready and .done files considered harmful

2021-08-23 Thread Robert Haas
On Mon, Aug 23, 2021 at 11:50 AM Bossart, Nathan wrote: > To handle a "cheating" archive command, I'd probably need to add a > stat() for every time pgarch_readyXLog() returned something from > arch_files. I suspect something similar might be needed in Dipesh's > patch to handle backup history fi

Re: archive status ".ready" files may be created too early

2021-08-23 Thread Bossart, Nathan
On 8/23/21, 10:31 AM, "alvhe...@alvh.no-ip.org" wrote: > On 2021-Aug-23, alvhe...@alvh.no-ip.org wrote: > >> The only way .ready files are created is that XLogNotifyWrite() is >> called. For regular WAL files during regular operation, that only >> happens in XLogNotifyWriteSeg(). That, in turn,

Re: archive status ".ready" files may be created too early

2021-08-23 Thread alvhe...@alvh.no-ip.org
On 2021-Aug-23, alvhe...@alvh.no-ip.org wrote: > The only way .ready files are created is that XLogNotifyWrite() is > called. For regular WAL files during regular operation, that only > happens in XLogNotifyWriteSeg(). That, in turn, only happens in > NotifySegmentsReadyForArchive(). But if the

Re: archive status ".ready" files may be created too early

2021-08-23 Thread alvhe...@alvh.no-ip.org
On 2021-Aug-23, Bossart, Nathan wrote: > Sorry, I'm still not following this one. If we skipped creating > .ready segments due to a crash, we rely on RemoveOldXlogFiles() to > create them as needed in the end-of-recovery checkpoint. If a record > fits perfectly in the end of a segment, we'll sti

Re: archive status ".ready" files may be created too early

2021-08-23 Thread Bossart, Nathan
On 8/23/21, 9:33 AM, "alvhe...@alvh.no-ip.org" wrote: > On 2021-Aug-23, Bossart, Nathan wrote: > >> Hm. My expectation would be that if there are no registrations, we >> cannot create .ready files for the flushed segments. The scenario >> where I can see that happening is when a record gets flus

pretty slow merge-join due rescan?

2021-08-23 Thread Pavel Stehule
Hi The customer reports a very slow query. I have a reproducer script. The dataset is not too big postgres=# \dt+ List of relations ┌┬───┬───┬───┬─┬┬─┐ │ Schema │ Name │ Type │ Owner │ Persistence │Size

Re: [Patch] change the default value of shared_buffers in postgresql.conf.sample

2021-08-23 Thread Bruce Momjian
On Wed, Aug 18, 2021 at 08:27:19PM +0200, Magnus Hagander wrote: > On Wed, Aug 18, 2021 at 8:16 PM Bruce Momjian wrote: > > > > On Wed, Aug 18, 2021 at 02:03:56PM -0400, Tom Lane wrote: > > > Bruce Momjian writes: > > > > I think the only question is whether this is a PG 15-only patch or a > > >

Re: archive status ".ready" files may be created too early

2021-08-23 Thread alvhe...@alvh.no-ip.org
On 2021-Aug-23, Bossart, Nathan wrote: > Hm. My expectation would be that if there are no registrations, we > cannot create .ready files for the flushed segments. The scenario > where I can see that happening is when a record gets flushed to disk > prior to registration. In that case, we'll sti

Re: archive status ".ready" files may be created too early

2021-08-23 Thread Bossart, Nathan
On 8/23/21, 8:50 AM, "alvhe...@alvh.no-ip.org" wrote: > ... while reading the resulting code after backpatching to all branches, > I realized that if there are no registrations whatsoever, then archiving > won't do anything, which surely is the wrong thing to do. The correct > behavior should be

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Chapman Flack
On 08/23/21 10:57, Chapman Flack wrote: > Maybe those cases aren't very numerous ... and maybe they're distinctive > enough to recognize when creating one ("hmm, I am creating a check or > assign hook that does significant work here, will it be worth exposing > a getter API for the product of the w

Re: .ready and .done files considered harmful

2021-08-23 Thread Bossart, Nathan
On 8/23/21, 6:42 AM, "Robert Haas" wrote: > On Sun, Aug 22, 2021 at 10:31 PM Bossart, Nathan wrote: >> I ran this again on a bigger machine with 200K WAL files pending >> archive. The v9 patch took ~5.5 minutes, the patch I sent took ~8 >> minutes, and the existing logic took just under 3 hour

Re: archive status ".ready" files may be created too early

2021-08-23 Thread alvhe...@alvh.no-ip.org
On 2021-Aug-21, Bossart, Nathan wrote: > > Well, the thing I realized is that these three helper functions have > > exactly one caller each. I think the compiler is going to inline them, > > so there isn't going to be a function call in the assembly. I haven't > > verified this, though. > > Goo

Regexp: observable bug from careless usage of zaptreesubs

2021-08-23 Thread Tom Lane
While looking at the regexp code, I started to get uncomfortable about the implications of commit 0c3405cf1: it means that not only the cdissect() phase, but also the preceding DFA-check phase (longest()/shortest()) rely on saved subexpression match positions to be valid for the match we're current

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Alvaro Herrera
On 2021-Aug-23, Robert Haas wrote: > It's also a bit unfair to say, well we have APIs for accessing GUC > values. It's true that we do. But if the GUC variable is, say, a > Boolean, you do not want your extension to call some function that > does a bunch of shenanigans and returns a string so that

Re: Queries that should be canceled will get stuck on secure_write function

2021-08-23 Thread Alvaro Herrera
On 2021-Aug-23, Robert Haas wrote: > On Mon, Aug 23, 2021 at 10:45 AM Alvaro Herrera > wrote: > > Do we actually need new GUCs, though? I think we should never let an > > unresponsive client dictate what the server does, because that opens the > > door for uncooperative or malicious clients to

Re: Queries that should be canceled will get stuck on secure_write function

2021-08-23 Thread Robert Haas
On Mon, Aug 23, 2021 at 10:45 AM Alvaro Herrera wrote: > Do we actually need new GUCs, though? I think we should never let an > unresponsive client dictate what the server does, because that opens the > door for uncooperative or malicious clients to wreak serious havoc. I > think the implementat

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Chapman Flack
On 08/23/21 10:36, Bruce Momjian wrote: > So the problem is that extensions only _need_ to use that API on > Windows, so many initially don't, or that the API is too limited? I think there can be cases where it's too limited, such as when significant computation or validation is needed between th

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Robert Haas
On Mon, Aug 23, 2021 at 10:15 AM Tom Lane wrote: > And yes, I absolutely would prohibit extensions from accessing many > of them, if there were a reasonable way to do it. It would be a good > start towards establishing a defined API for extensions. Mostly, it would make extension development mor

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Julien Rouhaud
On Mon, Aug 23, 2021 at 10:15 PM Tom Lane wrote: > > And yes, I absolutely would prohibit extensions from accessing many > of them, if there were a reasonable way to do it. It would be a good > start towards establishing a defined API for extensions. The v2 patch I sent does that, at least when

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Julien Rouhaud
On Mon, Aug 23, 2021 at 10:36 PM Bruce Momjian wrote: > > So the problem is that extensions only _need_ to use that API on > Windows, so many initially don't, or that the API is too limited? The inconvenience with that API is that it's only returning c strings, so you gave to convert it back to t

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Julien Rouhaud
On Mon, Aug 23, 2021 at 10:22 PM Tom Lane wrote: > > Bruce Momjian writes: > > On Mon, Aug 23, 2021 at 10:15:04AM -0400, Tom Lane wrote: > >> By that argument, *every* globally-visible variable should be marked > >> PGDLLIMPORT. But the mere fact that two backend .c files need to access > > > No

Re: Queries that should be canceled will get stuck on secure_write function

2021-08-23 Thread Alvaro Herrera
On 2021-Aug-23, Robert Haas wrote: > On Mon, Aug 23, 2021 at 4:15 AM 蔡梦娟(玊于) wrote: > > I want to know why the interrupt is only handled when ProcDiePending > > is true, I think query which is supposed to be canceled also should > > respond to the signal. Yeah, I agree. > Well, if we're halfway

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Bruce Momjian
On Mon, Aug 23, 2021 at 10:22:51AM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Mon, Aug 23, 2021 at 10:15:04AM -0400, Tom Lane wrote: > >> By that argument, *every* globally-visible variable should be marked > >> PGDLLIMPORT. But the mere fact that two backend .c files need to access >

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Tom Lane
Bruce Momjian writes: > On Mon, Aug 23, 2021 at 10:15:04AM -0400, Tom Lane wrote: >> By that argument, *every* globally-visible variable should be marked >> PGDLLIMPORT. But the mere fact that two backend .c files need to access > No, Julien says 99% need only the GUCs, so that is not the argume

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Bruce Momjian
On Mon, Aug 23, 2021 at 10:15:04AM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Sun, Aug 22, 2021 at 09:29:01PM +0800, Julien Rouhaud wrote: > >> Then shouldn't we try to prevent direct access on all platforms rather than > >> only one? > > > Agreed. If Julian says 99% of the non-export

Re: Added schema level support for publication.

2021-08-23 Thread vignesh C
On Tue, Aug 17, 2021 at 6:55 PM Tom Lane wrote: > > Amit Kapila writes: > > On Tue, Aug 17, 2021 at 6:40 AM Peter Smith wrote: > >> On Mon, Aug 16, 2021 at 11:31 PM Tom Lane wrote: > >>> Abstractly it'd be > >>> > >>> createpub := CREATE PUBLICATION pubname FOR cpitem [, ... ] [ WITH ... ] > >>

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Tom Lane
Bruce Momjian writes: > On Sun, Aug 22, 2021 at 09:29:01PM +0800, Julien Rouhaud wrote: >> Then shouldn't we try to prevent direct access on all platforms rather than >> only one? > Agreed. If Julian says 99% of the non-export problems are GUCs, and we > can just export them all, why not do it?

Re: Queries that should be canceled will get stuck on secure_write function

2021-08-23 Thread Robert Haas
On Mon, Aug 23, 2021 at 4:15 AM 蔡梦娟(玊于) wrote: > I want to know why the interrupt is only handled when ProcDiePending is true, > I think query which is supposed to be canceled also should respond to the > signal. Well, if we're halfway through sending a message to the client and we abort the wr

Re: Mark all GUC variable as PGDLLIMPORT

2021-08-23 Thread Bruce Momjian
On Sun, Aug 22, 2021 at 09:29:01PM +0800, Julien Rouhaud wrote: > On Sun, Aug 22, 2021 at 09:19:42AM -0400, Tom Lane wrote: > > > > Uh, no, it's exactly *not* clear. There are a lot of GUCs that are only > > of interest to particular subsystems. I do not see why being a GUC makes > > something a

RE: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2021-08-23 Thread houzj.f...@fujitsu.com
On Mon, Aug 23, 2021 8:01 PM Amit Kapila wrote: > On Mon, Aug 23, 2021 at 2:45 PM > houzj.f...@fujitsu.com wrote: > > > > On Mon, Aug 23, 2021 12:59 PM Amit Kapila wrote: > > > > > > On Sat, Aug 7, 2021 at 6:53 PM houzj.f...@fujitsu.com > > > wrote: > > > > > > > > Personally, I also think it

Re: .ready and .done files considered harmful

2021-08-23 Thread Robert Haas
On Sun, Aug 22, 2021 at 10:31 PM Bossart, Nathan wrote: > I ran this again on a bigger machine with 200K WAL files pending > archive. The v9 patch took ~5.5 minutes, the patch I sent took ~8 > minutes, and the existing logic took just under 3 hours. Hmm. On the one hand, 8 minutes > 5.5 minutes,

Re: replay of CREATE TABLESPACE eats data at wal_level=minimal

2021-08-23 Thread Robert Haas
On Sun, Aug 22, 2021 at 6:59 PM Noah Misch wrote: > Here's what I plan to push. Besides adding a test, I modified things so > CREATE TABLESPACE redo continues to report an error if a non-directory exists > under the name we seek to create. One could argue against covering that > corner case, but

Re: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2021-08-23 Thread Amit Kapila
On Mon, Aug 23, 2021 at 2:45 PM houzj.f...@fujitsu.com wrote: > > On Mon, Aug 23, 2021 12:59 PM Amit Kapila wrote: > > > > On Sat, Aug 7, 2021 at 6:53 PM houzj.f...@fujitsu.com > > wrote: > > > > > > Personally, I also think it will be better to make the behavior > > > consistent. > > > Attach

Re: replay of CREATE TABLESPACE eats data at wal_level=minimal

2021-08-23 Thread Prabhat Sahu
On Mon, Aug 23, 2021 at 4:29 AM Noah Misch wrote: > On Wed, Aug 18, 2021 at 10:32:10PM -0700, Noah Misch wrote: > > On Wed, Aug 18, 2021 at 10:47:24AM -0400, Robert Haas wrote: > > > On Tue, Aug 10, 2021 at 9:35 AM Robert Haas > wrote: > > > > Oh, yeah, I think that works, actually. I was imagin

Re: [bug] Logical Decoding of relation rewrite with toast does not reset toast_hash

2021-08-23 Thread Amit Kapila
On Wed, Aug 18, 2021 at 8:09 PM Drouvot, Bertrand wrote: > > Hi, > > On 8/18/21 12:01 PM, Amit Kapila wrote: > > On Wed, Aug 18, 2021 at 1:27 PM Drouvot, Bertrand > > wrote: > >> Hi, > > I've updated the comment and prepared the back patch versions: > I have verified and all your patches look g

Re: Separate out FileSet from SharedFileSet (was Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o)

2021-08-23 Thread Dilip Kumar
On Mon, Aug 23, 2021 at 1:43 PM Amit Kapila wrote: Note: merge comments from multiple mails > > I think we should handle that in worker.c itself, by adding a > > before_dsm_detach function before_shmem_exit right? > > > > Yeah, I thought of handling it in worker.c similar to what you've in 0002

Re: Proposal: More structured logging

2021-08-23 Thread Magnus Hagander
On Sat, Aug 21, 2021 at 2:37 AM Michael Paquier wrote: > > On Fri, Aug 20, 2021 at 11:35:29AM +0200, Ronan Dunklau wrote: > > Michael, your jsonlog module already fullfills this need. Is it something > > that > > should be merged into our tree ? > > Yes, there is nothing technically preventing to

Re: Defer selection of asynchronous subplans until the executor initialization stage

2021-08-23 Thread Etsuro Fujita
On Wed, Jun 30, 2021 at 1:50 PM Andrey Lepikhov wrote: > I have completely rewritten this patch. > > Main idea: > > The async_capable field of a plan node inform us that this node could > work in async mode. Each node sets this field based on its own logic. > The actual mode of a node is defined b

RE: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2021-08-23 Thread houzj.f...@fujitsu.com
On Mon, Aug 23, 2021 12:59 PM Amit Kapila wrote: > > On Sat, Aug 7, 2021 at 6:53 PM houzj.f...@fujitsu.com > wrote: > > > > Personally, I also think it will be better to make the behavior consistent. > > Attach the new version patch make both ADD and DROP behave the same as > > SET PUBLICATION

RE: [BUG] wrong refresh when ALTER SUBSCRIPTION ADD/DROP PUBLICATION

2021-08-23 Thread houzj.f...@fujitsu.com
On Mon, Aug 23, 2021 1:18 PM Masahiko Sawada wrote: > On Mon, Aug 23, 2021 at 1:59 PM Amit Kapila > wrote: > > > > On Sat, Aug 7, 2021 at 6:53 PM houzj.f...@fujitsu.com > > wrote: > > > > > > Personally, I also think it will be better to make the behavior > > > consistent. > > > Attach the new

Re: pg_dump handling of ALTER DEFAULT PRIVILEGES IN SCHEMA

2021-08-23 Thread Muhammad Usama
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested The patch looks fine and successfully fixes the issue of missing

Queries that should be canceled will get stuck on secure_write function

2021-08-23 Thread 蔡梦娟(玊于)
Hi, all Recently, I got a problem that the startup process of standby is stuck and keeps in a waiting state. The backtrace of startup process shows that it is waiting for a backend process which conflicts with recovery processing to exit, the guc parameters max_standby_streaming_delay and max_

Re: Separate out FileSet from SharedFileSet (was Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o)

2021-08-23 Thread Amit Kapila
On Mon, Aug 23, 2021 at 12:52 PM Dilip Kumar wrote: > > On Mon, Aug 23, 2021 at 11:43 AM Amit Kapila wrote: > > > > On Mon, Aug 23, 2021 at 9:53 AM Dilip Kumar wrote: > > > > > > On Mon, Aug 23, 2021 at 9:11 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > 4) > > > > -extern File SharedFile

Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o

2021-08-23 Thread Masahiko Sawada
On Sat, Aug 21, 2021 at 9:38 PM Dilip Kumar wrote: > > On Wed, Aug 18, 2021 at 3:45 PM Dilip Kumar wrote: > > > > On Wed, Aug 18, 2021 at 11:24 AM Amit Kapila > > wrote: > > > > > > On Tue, Aug 17, 2021 at 4:34 PM Andres Freund wrote: > > > > > > > > On 2021-08-17 10:54:30 +0530, Amit Kapila w

Re: Minor pg_amcheck fixes spotted while reading code

2021-08-23 Thread Daniel Gustafsson
> On 21 Aug 2021, at 02:43, Michael Paquier wrote: > > On Fri, Aug 20, 2021 at 11:42:09AM -0700, Mark Dilger wrote: >> These look correct. > > static void > -help(const char *progname) > +help(const char *program_name) > These were discussed not long ago, and I recall that they were in the > we-

  1   2   >