Re: Remove a unused argument from qual_is_pushdown_safe

2022-11-28 Thread Richard Guo
On Mon, Nov 28, 2022 at 3:40 PM Michael Paquier wrote: > > On Fri, Nov 25, 2022 at 04:05:13PM +0800, Richard Guo wrote: > >> I wonder if we need to revise the comment atop qual_is_pushdown_safe() > >> too which says > >> > >> * rinfo is a restriction clause applying to the given subquery (whose

RE: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-28 Thread shiy.f...@fujitsu.com
On Sun, Nov 27, 2022 1:33 PM Dilip Kumar wrote: > > On Sat, Nov 26, 2022 at 12:15 PM Amit Kapila > wrote: > > > > On Fri, Nov 25, 2022 at 5:38 PM Amit Kapila > wrote: > > > > > > On Fri, Nov 25, 2022 at 1:35 PM Dilip Kumar > wrote: > > > > > > > > During DecodeCommit() for skipping a transacti

Re: Logical Replication Custom Column Expression

2022-11-28 Thread Peter Smith
On Fri, Nov 25, 2022 at 9:43 PM Stavros Koureas wrote: > > Yes, if the property is on the subscription side then it should be applied > for all the tables that the connected publication is exposing. > So if the property is enabled you should be sure that this origin column > exists to all of the

Failed Assert while pgstat_unlink_relation

2022-11-28 Thread vignesh C
Hi, While reviewing/testing one of the patches I found the following Assert: #0 0x55c6312ba639 in pgstat_unlink_relation (rel=0x7fb73bcbac58) at pgstat_relation.c:161 #1 0x55c6312bbb5a in pgstat_relation_delete_pending_cb (entry_ref=0x55c6335563a8) at pgstat_relation.c:839 #2 0x55c6

Re: Understanding, testing and improving our Windows filesystem code

2022-11-28 Thread Thomas Munro
On Mon, Nov 28, 2022 at 8:58 PM Ian Lawrence Barwick wrote: > For my understanding, does this entry supersede the proposal in > https://commitfest.postgresql.org/40/3347/ ? I think so (Victor hasn't commented). Patch 0004 derives from Victor's patch and has him as primary author still, but I mad

Re: Report roles in pg_upgrade pg_ prefix check

2022-11-28 Thread Daniel Gustafsson
> On 28 Nov 2022, at 02:18, Michael Paquier wrote: > > On Thu, Nov 24, 2022 at 12:31:09PM +0100, Daniel Gustafsson wrote: >> Looking at a recent pg_upgrade thread I happened to notice that the check for >> roles with a pg_ prefix only reports the error, not the roles it found. >> Other >> simil

Failed Assert in pgstat_assoc_relation

2022-11-28 Thread vignesh C
Hi, While reviewing/testing one of the patches I found the following Assert: #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=139624429171648) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=139624429171648) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_ki

Re: Avoid streaming the transaction which are skipped (in corner cases)

2022-11-28 Thread Dilip Kumar
On Mon, Nov 28, 2022 at 1:46 PM shiy.f...@fujitsu.com wrote: > > Thanks for your patch. > > I saw that the patch added a check when selecting largest transaction, but in > addition to ReorderBufferCheckMemoryLimit(), the transaction can also be > streamed in ReorderBufferProcessPartialChange(). Sh

Re: Bug in row_number() optimization

2022-11-28 Thread Sergey Shinderuk
On 28.11.2022 03:23, David Rowley wrote: On Sat, 26 Nov 2022 at 05:19, Tom Lane wrote: Sergey Shinderuk writes: What about user-defined operators? I created my own <= operator for int8 which returns true on null input, and put it in a btree operator class. Admittedly, it's weird that (null <

[PATCH] Check snapshot argument of index_beginscan and family

2022-11-28 Thread Aleksander Alekseev
Hi hackers, A colleague of mine (cc'ed) reported that he was able to pass a NULL snapshot to index_beginscan() and it even worked to a certain degree. I took my toy extension [1] and replaced the argument with NULL as an experiment: ``` eax=# CREATE EXTENSION experiment; CREATE EXTENSION eax=# S

Re: [PATCH] Check snapshot argument of index_beginscan and family

2022-11-28 Thread Pavel Borisov
Hi, Alexander! > A colleague of mine (cc'ed) reported that he was able to pass a NULL > snapshot to index_beginscan() and it even worked to a certain degree. > > I took my toy extension [1] and replaced the argument with NULL as an > experiment: > > ``` > eax=# CREATE EXTENSION experiment; > CREATE

Re: Failed Assert in pgstat_assoc_relation

2022-11-28 Thread Ajin Cherian
On Mon, Nov 28, 2022 at 8:10 PM vignesh C wrote: > > Hi, > > While reviewing/testing one of the patches I found the following Assert: > #0 __pthread_kill_implementation (no_tid=0, signo=6, > threadid=139624429171648) at ./nptl/pthread_kill.c:44 > #1 __pthread_kill_internal (signo=6, threadid=139

Re: Another multi-row VALUES bug

2022-11-28 Thread Dean Rasheed
On Wed, 23 Nov 2022 at 18:56, Tom Lane wrote: > > I wonder if somehow we could just make one pass over > all the VALUES RTEs, and process each one as needed? The problem > is to identify the relevant target relation, I guess. > I have been thinking about that some more, but I think it would be p

Re: [PATCH] Check snapshot argument of index_beginscan and family

2022-11-28 Thread Aleksander Alekseev
Hi Pavel, Thanks for the feedback! > I think it's a nice catch and worth fixing. The one thing I don't > agree with is using asserts for handling the error that can appear > because most probably the server is built with assertions off and in > this case, there still will be a crash in this case.

Re: Introduce a new view for checkpointer related stats

2022-11-28 Thread Bharath Rupireddy
On Sat, Nov 26, 2022 at 4:32 AM Andres Freund wrote: > Thanks Andres for reviewing. > > May I know what it means to deprecate pg_stat_bgwriter columns? > > Add a note to the docs saying that the columns will be removed. Done. > > Are > > you suggesting to add deprecation warnings to correspond

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-28 Thread Amit Kapila
On Sun, Nov 27, 2022 at 9:43 AM houzj.f...@fujitsu.com wrote: > > Attach the new version patch which addressed all comments so far. > Few comments on v52-0001* 1. pa_free_worker() { ... + /* Free the worker information if the worker exited cleanly. */ + if (!winfo->error_

Re: Logical Replication Custom Column Expression

2022-11-28 Thread Stavros Koureas
Sure I understand and neither do I have good knowledge of what else could be influenced by such a change. If the value of the column is the subscriber name has no benefit to this idea of merging multiple upstreams with same primary keys, later you describe the "connection dbname", yes this could be

Re: Reducing power consumption on idle servers

2022-11-28 Thread Robert Haas
On Sun, Nov 27, 2022 at 6:57 PM Thomas Munro wrote: > The main documentation of pg_promote() etc now has "The parameter > promote_trigger_file has been removed" in the > places where the GUC was previously mentioned. Perhaps we should just > remove the mentions completely (it's somehow either too

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-28 Thread Simon Riggs
On Sun, 27 Nov 2022 at 23:34, Nathan Bossart wrote: > > Thanks for taking a look! > > On Thu, Nov 24, 2022 at 05:31:02PM +, Simon Riggs wrote: > > * not sure I believe that everything it does can always be aborted out > > of and shutdown - to achieve that you will need a > > CHECK_FOR_INTERRUP

Re: Perform streaming logical transactions by background workers and parallel apply

2022-11-28 Thread Amit Kapila
On Mon, Nov 28, 2022 at 12:49 PM Peter Smith wrote: > ... > > 17. > @@ -388,10 +401,9 @@ static inline void cleanup_subxact_info(void); > /* > * Serialize and deserialize changes for a toplevel transaction. > */ > -static void stream_cleanup_files(Oid subid, TransactionId xid); > static void

Re: TAP output format in pg_regress

2022-11-28 Thread Daniel Gustafsson
> On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > Gustafsson написал: > I wold suggest to use word immediate instead of noatexit. This will do the > code much more sensible for me. I think noatexit is clearer since t

Re: Non-decimal integer literals

2022-11-28 Thread Peter Eisentraut
On 23.11.22 17:25, Dean Rasheed wrote: Taking a quick look, I noticed that there are no tests for negative values handled in the parser. Giving that a spin shows that make_const() fails to correctly identify the base of negative non-decimal integers in the T_Float case, causing it to fall throug

Re: Reducing power consumption on idle servers

2022-11-28 Thread Tom Lane
Robert Haas writes: > I think we should remove those mentions. Otherwise the documentation > just collects mentions of an increasing number of things that are no > longer relevant. Yeah, I think the same. There will be a release-note entry, and I don't object to having something about it in appe

Re: Remove a unused argument from qual_is_pushdown_safe

2022-11-28 Thread Tom Lane
Michael Paquier writes: > On Mon, Nov 28, 2022 at 11:54:45AM +0900, Michael Paquier wrote: >> On Fri, Nov 25, 2022 at 04:05:13PM +0800, Richard Guo wrote: > I wonder if we need to revise the comment atop qual_is_pushdown_safe() > too which says > > * rinfo is a restriction clause applying to the

Re: Remove a unused argument from qual_is_pushdown_safe

2022-11-28 Thread Daniel Gustafsson
> On 28 Nov 2022, at 15:15, Tom Lane wrote: > My viewpoint is that this change is misguided. Even if the current > coding of qual_is_pushdown_safe doesn't happen to reference the > subquery, it might need to in future. If I understand the code correctly the variable has some value in terms of "

Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Simon Riggs
Issue in current XactLockTableWait, starting with 3c27944fb2141, discovered while reviewing https://commitfest.postgresql.org/40/3806/ Test demonstrating the problem is 001-isotest-tuplelock-subxact.v1.patch A narrative description of the issue follows: session1 - requests multiple nested subtran

[PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Nikita Malakhov
Hi hackers! While working on Pluggable TOAST we've detected a defective behavior on tables with large amounts of TOASTed data - queries freeze and DB stalls. Further investigation led us to the loop with GetNewOidWithIndex function call - when all available Oid already exist in the related TOAST t

Re: fixing CREATEROLE

2022-11-28 Thread Robert Haas
On Thu, Nov 24, 2022 at 2:41 AM wrote: > INHERITCREATEDROLES and SETCREATEDROLES behave much like DEFAULT > PRIVILEGES. What about something like: > > ALTER DEFAULT PRIVILEGES FOR alice > GRANT TO alice WITH INHERIT FALSE, SET TRUE, ADMIN TRUE > > The "abbreviated grant" is very much abbreviated,

Re: Removing another gen_node_support.pl special case

2022-11-28 Thread Peter Eisentraut
On 27.11.22 02:39, Tom Lane wrote: I got confused about how we were managing EquivalenceClass pointers in the copy/equal infrastructure, and it took me awhile to remember that the reason it works is that gen_node_support.pl has hard-wired knowledge about that. I think that's something we'd be be

Re: Removing another gen_node_support.pl special case

2022-11-28 Thread Tom Lane
Peter Eisentraut writes: > On 27.11.22 02:39, Tom Lane wrote: >> I got confused about how we were managing EquivalenceClass pointers >> in the copy/equal infrastructure, and it took me awhile to remember >> that the reason it works is that gen_node_support.pl has hard-wired >> knowledge about that

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Robert Haas
On Sat, Nov 26, 2022 at 4:08 AM Chris Travers wrote: > I didn't see any changes to pg_upgrade to make this change possible on > upgrade. Is that also outside of the scope of your patch set? If so how is > that continuity supposed to be ensured? The scheme is documented in their 0006 patch, in

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-28 Thread Andrew Dunstan
On 2022-11-23 We 18:54, Nathan Bossart wrote: > On Wed, Nov 23, 2022 at 02:56:28PM -0500, Andrew Dunstan wrote: >> I have committed the first couple of these to get them out of the way. > Thanks! > >> But I think we need a bit of cleanup in the next patch. >> vacuum_is_relation_owner() looks like

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 10:28 AM Simon Riggs wrote: > So we have these options > > 1. Removing the XactLockTableDelete() call in CommitSubTransaction(). > That releases lock waiters earlier than expected, which requires > pushups in XactLockTableWait() to cope with that (which are currently > brok

Re: Introduce a new view for checkpointer related stats

2022-11-28 Thread Robert Haas
On Tue, Nov 22, 2022 at 3:53 PM Andres Freund wrote: > I think we should consider deprecating the pg_stat_bgwriter columns but > leaving them in place for a few years. New stuff should only be added to > pg_stat_checkpointer, but we don't need to break old monitoring queries. I vote to just remov

Re: Decouple last important WAL record LSN from WAL insert locks

2022-11-28 Thread Andres Freund
Hi, On 2022-11-28 11:42:19 +0530, Bharath Rupireddy wrote: > On Sun, Nov 27, 2022 at 2:43 AM Andres Freund wrote: > > On 2022-11-23 19:12:07 +0530, Bharath Rupireddy wrote: > > > While working on something else, I noticed that each WAL insert lock > > > tracks its own last important WAL record's

Re: TAP output format in pg_regress

2022-11-28 Thread Andres Freund
On 2022-11-28 14:13:16 +0100, Daniel Gustafsson wrote: > > On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > > Gustafsson написал: > > > I wold suggest to use word immediate instead of noatexit. This will do the > > code

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-28 Thread Andres Freund
On 2022-11-28 13:08:57 +, Simon Riggs wrote: > On Sun, 27 Nov 2022 at 23:34, Nathan Bossart wrote: > > > Rather than explicitly use DEBUG1 everywhere I would have an > > > #define CUSTODIAN_LOG_LEVEL LOG > > > so we can run with it in LOG mode and then set it to DEBUG1 with a one > > > lin

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 1:31 PM Andres Freund wrote: > On 2022-11-28 13:08:57 +, Simon Riggs wrote: > > On Sun, 27 Nov 2022 at 23:34, Nathan Bossart > > wrote: > > > > Rather than explicitly use DEBUG1 everywhere I would have an > > > > #define CUSTODIAN_LOG_LEVEL LOG > > > > so we can r

Re: Failed Assert in pgstat_assoc_relation

2022-11-28 Thread Tom Lane
vignesh C writes: > I could reproduce this issue with the following steps: > create table t1(c int); > BEGIN; > CREATE TABLE t (c int); > SAVEPOINT q; > CREATE RULE "_RETURN" AS ON SELECT TO t DO INSTEAD SELECT * FROM t1; > select * from t; > ROLLBACK TO q; > CREATE RULE "_RETURN" AS ON SELECT TO

Re: Failed Assert in pgstat_assoc_relation

2022-11-28 Thread Andres Freund
Hi, On 2022-11-28 13:37:16 -0500, Tom Lane wrote: > vignesh C writes: > > I could reproduce this issue with the following steps: > > create table t1(c int); > > BEGIN; > > CREATE TABLE t (c int); > > SAVEPOINT q; > > CREATE RULE "_RETURN" AS ON SELECT TO t DO INSTEAD SELECT * FROM t1; > > select

Re: Another multi-row VALUES bug

2022-11-28 Thread Tom Lane
Dean Rasheed writes: > A different way to do this, without relying on the contents of the > targetlist, is to note that, while processing a product query, what we > really want to do is ignore any VALUES RTEs from the original query, > since they will have already been processed. There should then

Re: predefined role(s) for VACUUM and ANALYZE

2022-11-28 Thread Nathan Bossart
On Mon, Nov 28, 2022 at 12:13:13PM -0500, Andrew Dunstan wrote: > pushed. Thanks! -- Nathan Bossart Amazon Web Services: https://aws.amazon.com

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Alvaro Herrera
On 2022-Nov-28, Simon Riggs wrote: > A narrative description of the issue follows: > session1 - requests multiple nested subtransactions like this: > BEGIN; ... > SAVEPOINT subxid1; ... > SAVEPOINT subxid2; ... > However, if subxid2 subcommits, then the lock wait moves from subxid2 > to the topxi

Re: fixing CREATEROLE

2022-11-28 Thread walther
Robert Haas: I don't know if changing the syntax from A to B is really getting us anywhere. I generally agree that the ALTER DEFAULT PRIVILEGES syntax looks nicer than the CREATE/ALTER ROLE syntax, but I'm not sure that's a sufficient reason to move the control over this behavior to ALTER DEFAULT

Re: Add tracking of backend memory allocated to pg_stat_activity

2022-11-28 Thread Andres Freund
On 2022-11-26 22:10:06 -0500, Reid Thompson wrote: >- zero allocated bytes after fork to avoid double counting postmaster > allocations I still don't understand this - postmaster shouldn't be counted at all. It doesn't have a PgBackendStatus. There simply shouldn't be any tracked allocations

Re: TAP output format in pg_regress

2022-11-28 Thread Nikolay Shaplov
В письме от понедельник, 28 ноября 2022 г. 21:28:48 MSK пользователь Andres Freund написал: > On 2022-11-28 14:13:16 +0100, Daniel Gustafsson wrote: > > > On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > > > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > > > Gustafsso

Re: Collation version tracking for macOS

2022-11-28 Thread Robert Haas
On Wed, Nov 23, 2022 at 12:09 AM Thomas Munro wrote: > OK. Time for a new list of the various models we've discussed so far: > > 1. search-by-collversion: We introduce no new "library version" > concept to COLLATION and DATABASE object and little or no new syntax. > > 2. lib-version-in-provide

Re: TAP output format in pg_regress

2022-11-28 Thread Daniel Gustafsson
> On 28 Nov 2022, at 20:02, Nikolay Shaplov wrote: > From my reviewer's point of view patch is ready for commit. > > Thank you for your patience :-) Thanks for review. The attached tweaks a few comments and attempts to address the compiler warning error in the CFBot CI. Not sure I entirely ag

Re: fixing CREATEROLE

2022-11-28 Thread David G. Johnston
On Mon, Nov 28, 2022 at 11:57 AM wrote: > Robert Haas: > > I don't know if changing the syntax from A to B is really getting us > > anywhere. I generally agree that the ALTER DEFAULT PRIVILEGES syntax > > looks nicer than the CREATE/ALTER ROLE syntax, but I'm not sure that's > > a sufficient reas

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Simon Riggs
On Mon, 28 Nov 2022 at 18:53, Alvaro Herrera wrote: > > On 2022-Nov-28, Simon Riggs wrote: > > > A narrative description of the issue follows: > > session1 - requests multiple nested subtransactions like this: > > BEGIN; ... > > SAVEPOINT subxid1; ... > > SAVEPOINT subxid2; ... > > > However, if s

Re: fixing CREATEROLE

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 1:56 PM wrote: > And now this reason is gone - there is no reason NOT to implement it as > DEFAULT PRIVILEGES. I think there is, and it's this, which you wrote further down: > In my proposal, the "object" is not the GRANT of that role. It's the > role itself. So the defau

Re: fixing CREATEROLE

2022-11-28 Thread walther
David G. Johnston: A quick tally of the thread so far: No Defaults needed: David J., Mark?, Tom? Defaults needed - attached to role directly: Robert Defaults needed - defined within Default Privileges: Walther? s/Walther/Wolfgang The capability itself seems orthogonal to the rest of the patch

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Simon Riggs
On Mon, 28 Nov 2022 at 17:38, Robert Haas wrote: > > On Mon, Nov 28, 2022 at 10:28 AM Simon Riggs > wrote: > > So we have these options > > > > 1. Removing the XactLockTableDelete() call in CommitSubTransaction(). > > That releases lock waiters earlier than expected, which requires > > pushups in

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 2:45 PM Simon Riggs wrote: > An easy point to confuse: > "subtransaction to end": The subtransaction is "still running" to > other backends even AFTER it has been subcommitted, but its state now > transfers to the parent. > > So the subtransaction doesn't cease running unti

Re: fixing CREATEROLE

2022-11-28 Thread David G. Johnston
On Mon, Nov 28, 2022 at 12:42 PM wrote: > David G. Johnston: > > A quick tally of the thread so far: > > > > No Defaults needed: David J., Mark?, Tom? > > Defaults needed - attached to role directly: Robert > > Defaults needed - defined within Default Privileges: Walther? > > s/Walther/Wolfgang >

Re: fixing CREATEROLE

2022-11-28 Thread walther
Robert Haas: In my proposal, the "object" is not the GRANT of that role. It's the role itself. So the default privileges express what should happen when the role is created. The default privileges would NOT affect a regular GRANT role TO role_spec command. They only run that command when a role i

Re: Failed Assert in pgstat_assoc_relation

2022-11-28 Thread Tom Lane
Andres Freund writes: > On 2022-11-28 13:37:16 -0500, Tom Lane wrote: >> As far as HEAD is concerned, maybe it's time to nuke the whole >> convert-table-to-view kluge entirely? Only pg_dump older than >> 9.4 will emit such code, so we're really about out of reasons >> to keep on maintaining it.

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Tom Lane
Robert Haas writes: > That's not what "running" means to me. Running means it's started and > hasn't yet committed or rolled back. A subxact definitely can't be considered committed until its topmost parent commits. However, it could be known to be rolled back before its parent. IIUC, the curre

Re: fixing CREATEROLE

2022-11-28 Thread Mark Dilger
> On Nov 28, 2022, at 11:34 AM, David G. Johnston > wrote: > > No Defaults needed: David J., Mark?, Tom? As Robert has the patch organized, I think defaults are needed, but I see that as a strike against the patch. > Defaults needed - attached to role directly: Robert > Defaults needed - d

Re: fixing CREATEROLE

2022-11-28 Thread walther
Mark Dilger: Robert's patch tries to deal with the (possibly unwanted) role membership by setting up defaults to mitigate the effects, but that is more confusing to me than just de-conflating role membership from role administration, and giving role creators administration over roles they crea

Re: fixing CREATEROLE

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 3:02 PM Mark Dilger wrote: > Robert's patch tries to deal with the (possibly unwanted) role membership by > setting up defaults to mitigate the effects, but that is more confusing to me > than just de-conflating role membership from role administration, and giving > role

Re: fixing CREATEROLE

2022-11-28 Thread Mark Dilger
> On Nov 28, 2022, at 12:08 PM, walt...@technowledgy.de wrote: > > Isn't this just GRANT .. WITH SET FALSE, INHERIT FALSE, ADMIN TRUE? That > should allow role administration, without actually granting membership in > that role, yet, right? Can you clarify what you mean here? Are you invent

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Andres Freund
Hi, On 2022-11-28 18:34:20 +0300, Nikita Malakhov wrote: > While working on Pluggable TOAST we've detected a defective behavior > on tables with large amounts of TOASTed data - queries freeze and DB stalls. > Further investigation led us to the loop with GetNewOidWithIndex function > call - when a

Re: fixing CREATEROLE

2022-11-28 Thread Mark Dilger
> On Nov 28, 2022, at 12:33 PM, Mark Dilger > wrote: > >> Isn't this just GRANT .. WITH SET FALSE, INHERIT FALSE, ADMIN TRUE? That >> should allow role administration, without actually granting membership in >> that role, yet, right? > > Can you clarify what you mean here? Are you inventi

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Nikita Malakhov
Hi! We've already encountered this issue on large production databases, and 4 billion rows is not so much for modern bases, so this issue already arises from time to time and would arise more and more often. I agree that global oid counter is the main issue, and better solution would be local coun

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-28 Thread Bruce Momjian
On Sun, Nov 27, 2022 at 11:26:50AM -0800, Andrey Borodin wrote: > Some funny stuff. If a user tries to cancel a non-replicated transaction > Azure Postgres will answer: "user requested cancel while waiting for > synchronous replication ack. The COMMIT record has already flushed to > WAL locally and

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-28 Thread Bruce Momjian
On Mon, Nov 28, 2022 at 12:03:06PM +0530, Bharath Rupireddy wrote: > Thanks for verifying the behaviour. And many thanks for an off-list chat. > > FWIW, I'm planning to prepare a patch as per the below idea which is > something similar to the initial proposal in this thread. Meanwhile, > thoughts

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Andres Freund
Hi, On 2022-11-28 23:54:53 +0300, Nikita Malakhov wrote: > We've already encountered this issue on large production databases, and > 4 billion rows is not so much for modern bases, so this issue already arises > from time to time and would arise more and more often. Was the issue that you exceede

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Tom Lane
Andres Freund writes: > I think the first step to improve the situation is to not use a global oid > counter for toasted values. One way to do that would be to use the sequence > code to do oid assignment, but we likely can find a more efficient > representation. I don't particularly buy that, be

Re: Failed Assert in pgstat_assoc_relation

2022-11-28 Thread Andres Freund
Hi, On 2022-11-28 10:50:13 -0800, Andres Freund wrote: > On 2022-11-28 13:37:16 -0500, Tom Lane wrote: > > Uh-huh. I've not bothered to trace this in detail, but presumably > > what is happening is that the first CREATE RULE converts the table > > to a view, and then the ROLLBACK undoes that so f

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Peter Geoghegan
On Mon, Nov 28, 2022 at 8:53 AM Robert Haas wrote: > It is true that if the table is progressively bloating, it is likely > to be more bloated by the time you are 8 billion XIDs behind than it > was when you were 800 million XIDs behind. I don't see that as a very > good reason not to adopt this p

Re: Bug in wait time when waiting on nested subtransaction

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 3:01 PM Tom Lane wrote: > One thing we need to be pretty careful of here is to not break the > promise of atomic commit. At topmost commit, all subxacts must > appear committed simultaneously. It's not quite clear to me whether > we need a similar guarantee in the rollbac

Re: fixing CREATEROLE

2022-11-28 Thread David G. Johnston
On Mon, Nov 28, 2022 at 1:28 PM Robert Haas wrote: > On Mon, Nov 28, 2022 at 3:02 PM Mark Dilger > wrote: > > You can argue that a grant with INHERIT FALSE, SET FALSE, ADMIN TRUE > still grants membership, and I think formally that's true, but I also > think it's just picking something to bicke

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Andres Freund
Hi, On 2022-11-28 16:04:12 -0500, Tom Lane wrote: > Andres Freund writes: > > - to > > combat the space usage we likely should switch to representing the ids as > > variable width integers or such, otherwise the space increase would likely > > be > > prohibitive. > > And I don't buy that either

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Nikita Malakhov
Hi, Andres Freund writes: >Was the issue that you exceeded 4 billion toasted datums, or that assignment >took a long time? How many toast datums did you actually have? Was this due to >very wide rows leading to even small datums getting toasted? Yep, we had 4 billion toasted datums. I remind tha

Re: Patch: Global Unique Index

2022-11-28 Thread Bruce Momjian
On Fri, Nov 25, 2022 at 05:03:06PM -0800, David Zhang wrote: > Hi Bruce, > > Thank you for helping review the patches in such detail. > > On 2022-11-25 9:48 a.m., Bruce Momjian wrote: > > Looking at the patch, I am unclear how the the patch prevents concurrent > duplicate value insertion

Re: Slow standby snapshot

2022-11-28 Thread Tom Lane
Michail Nikolaev writes: >> * when idle - since we have time to do it when that happens, which >> happens often since most workloads are bursty > I have added getting of ProcArrayLock for this case. That seems like a fairly bad idea: it will add extra contention on ProcArrayLock, and I see no re

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 4:09 PM Peter Geoghegan wrote: > Granted, the specifics of the current XidStopLimit mechanism are > unlikely to directly carry over to 64-bit XIDs. XidStopLimit is > structured in a way that doesn't actually consider freeze debt in > units like unfrozen pages. Like Chris, I

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-28 Thread Andrey Borodin
On Mon, Nov 28, 2022 at 12:59 PM Bruce Momjian wrote: > > You can prepare a patch, but it unlikely to get much interest until you > get agreement on what the behavior should be. We discussed the approach on 2020's Unconference [0]. And there kind of was an agreement. Then I made a presentation on

Re: Failed Assert in pgstat_assoc_relation

2022-11-28 Thread Tom Lane
Andres Freund writes: > Something like the attached. Still needs a bit of polish, e.g. adding the test > case from above. > I'm a bit uncomfortable adding a function call below >* Perform swapping of the relcache entry contents. Within this >* process the old entr

Re: CF 2022-11: entries "Ready for Committer" with recent activity

2022-11-28 Thread Bruce Momjian
On Sun, Nov 27, 2022 at 01:29:18PM +0900, Ian Lawrence Barwick wrote: > Transaction Management docs (2022-11-23) > > > - https://commitfest.postgresql.org/40/3899/ > - > https://www.postgresql.org/message-id/flat/canbhv-e_iy9fmrerxrch8tztyenpfo72hf_xd2hldp

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Tom Lane
Andres Freund writes: > On 2022-11-28 16:04:12 -0500, Tom Lane wrote: >> And I don't buy that either. An extra 4 bytes with a 2K payload is not >> "prohibitive", it's more like "down in the noise". > The space usage for the the the toast relation + index itself is indeed > irrelevant. Where it's

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Andres Freund
Hi, On 2022-11-29 00:24:49 +0300, Nikita Malakhov wrote: > 2) Upgrading to 64-bit id would require re-toasting old TOAST tables. Or > there is some way to distinguish old tables from new ones? The catalog / relcache entry should suffice to differentiate between the two. Greetings, Andres Freund

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Bruce Momjian
On Mon, Nov 28, 2022 at 04:30:22PM -0500, Robert Haas wrote: > What is the purpose of using 64-bit XIDs, if not to avoid having to > stop the world when we run short of XIDs? > > I'd say that if this patch, or any patch with broadly similar goals, > fails to remove xidStopLimit, it might as well n

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Peter Geoghegan
On Mon, Nov 28, 2022 at 1:30 PM Robert Haas wrote: > What is the purpose of using 64-bit XIDs, if not to avoid having to > stop the world when we run short of XIDs? I agree that the xidStopLimit mechanism was designed with the specific goal of preventing "true" physical XID wraparound that result

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-11-28 Thread Bruce Momjian
On Mon, Nov 28, 2022 at 01:31:39PM -0800, Andrey Borodin wrote: > On Mon, Nov 28, 2022 at 12:59 PM Bruce Momjian wrote: > > > > You can prepare a patch, but it unlikely to get much interest until you > > get agreement on what the behavior should be. > > We discussed the approach on 2020's Unconfe

Re: fixing CREATEROLE

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 4:19 PM David G. Johnston wrote: > That's fine, but are you saying this patch is incapable (or simply > undesirable) of having the parts about handling defaults separated out from > the parts that define how the system works with a given set of permissions; > and the one

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Tom Lane
Andres Freund writes: > On 2022-11-29 00:24:49 +0300, Nikita Malakhov wrote: >> 2) Upgrading to 64-bit id would require re-toasting old TOAST tables. Or >> there is some way to distinguish old tables from new ones? > The catalog / relcache entry should suffice to differentiate between the two. Y

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Peter Geoghegan
On Mon, Nov 28, 2022 at 1:52 PM Bruce Momjian wrote: > I think the problem is that we still have bloat with 64-bit XIDs, > specifically pg_xact and pg_multixact files. Yes, that bloat is less > serious, but it is still an issue worth reporting in the server logs, > though not serious enough to st

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Andres Freund
Hi, On 2022-11-28 16:57:53 -0500, Tom Lane wrote: > As I said before, I think there's a decent argument that some people > will want the option to stay with 4-byte TOAST OIDs indefinitely, > at least for smaller tables. I think we'll need to do something about the width of varatt_external to make

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Nikita Malakhov
Hi, I'll check that tomorrow. If it is so then there won't be a problem keeping old tables without re-toasting. On Tue, Nov 29, 2022 at 1:10 AM Andres Freund wrote: > Hi, > > On 2022-11-28 16:57:53 -0500, Tom Lane wrote: > > As I said before, I think there's a decent argument that some people >

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Tom Lane
Andres Freund writes: > On 2022-11-28 16:57:53 -0500, Tom Lane wrote: >> As I said before, I think there's a decent argument that some people >> will want the option to stay with 4-byte TOAST OIDs indefinitely, >> at least for smaller tables. > And as you said earlier, the increased overhead insi

Re: [PATCH] Infinite loop while acquiring new TOAST Oid

2022-11-28 Thread Nikita Malakhov
I've missed that - >4 billion external datums >typically use a lot of space. Not quite so. It's 8 Tb for the minimal size of toasted data (about 2 Kb). In my practice tables with more than 5 billions of rows are not of something out of the ordinary (highly loaded databases with large amounts of d

Re: Preventing indirection for IndexPageGetOpaque for known-size page special areas

2022-11-28 Thread Tom Lane
Bruce Momjian writes: > Uh, XTS doesn't require a nonce, so why are talking about nonces in this > thread? Because some other proposals do, or could, require a per-page nonce. After looking through this thread, I side with Robert: we should reject the remainder of this patch. It gives up page l

Re: Add 64-bit XIDs into PostgreSQL 15

2022-11-28 Thread Peter Geoghegan
On Mon, Nov 28, 2022 at 1:52 PM Peter Geoghegan wrote: > I'm not claiming to know how to build this "better xidStopLimit > mechanism", by the way. I'm not seriously proposing it. Mostly I'm > just saying that the question "where do you draw the line if not at 2 > billion XIDs?" is a very pertinent

Re: Reducing power consumption on idle servers

2022-11-28 Thread Thomas Munro
I found some more comments and some documentation to word-smith very lightly, and pushed. The comments were stray references to the trigger file. It's a little confusing because the remaining mechanism also uses a file, but it uses a signal first so seems better to refer to promotion requests rat

Re: Report roles in pg_upgrade pg_ prefix check

2022-11-28 Thread Michael Paquier
On Mon, Nov 28, 2022 at 09:58:46AM +0100, Daniel Gustafsson wrote: > We are a bit inconsistent in how much details we include in the report > textfiles, so could do that without breaking any consistency in reporting. > Looking at other checks, the below format would match what we already do > fair

Re: fixing CREATEROLE

2022-11-28 Thread David G. Johnston
On Mon, Nov 28, 2022 at 2:55 PM Robert Haas wrote: > On Mon, Nov 28, 2022 at 4:19 PM David G. Johnston > wrote: > > That's fine, but are you saying this patch is incapable (or simply > undesirable) of having the parts about handling defaults separated out from > the parts that define how the sys

Re: O(n) tasks cause lengthy startups and checkpoints

2022-11-28 Thread Nathan Bossart
Okay, here is a new patch set. 0004 adds logic to prevent custodian tasks from delaying shutdown. I haven't added any logging for long-running tasks yet. Tasks might ordinarily take a while, so such logs wouldn't necessarily indicate something is wrong. Perhaps we could add a GUC for the amount

Re: fixing CREATEROLE

2022-11-28 Thread Robert Haas
On Mon, Nov 28, 2022 at 4:55 PM Robert Haas wrote: > But so far nobody has actually reviewed anything, ... Actually this isn't true. Mark did review. Thanks, Mark. -- Robert Haas EDB: http://www.enterprisedb.com

  1   2   >