Re: Hash index build performance tweak from sorting

2022-07-29 Thread Simon Riggs
On Thu, 28 Jul 2022 at 19:50, Tom Lane wrote: > > Simon Riggs writes: > > Thanks for the nudge. New version attached. > > I also see a speed improvement from this, so pushed (after minor comment > editing). Thanks > I notice though that if I feed it random data, > &

Re: Maximize page freezing

2022-07-29 Thread Simon Riggs
page together appears in earlier patching that were posted. These were > things that didn't make it into Postgres 15. Yes, my patch from 2020 was never reviewed, which is why I was resubmitting here. > I should be able to post something in a couple of weeks. How do you see t

Re: Maximize page freezing

2022-07-29 Thread Simon Riggs
On Thu, 28 Jul 2022 at 14:55, Matthias van de Meent wrote: > > On Thu, 28 Jul 2022 at 15:36, Simon Riggs > wrote: > > > > Starting new thread with updated patch to avoid confusion, as > > mentioned by David Steele on the original thread: > > Original

Re: Slow standby snapshot

2022-07-29 Thread Simon Riggs
, aiming to keep the array from spreading out and impacting snapshot performance for SELECTs, yet not doing it so often that the startup process has a higher burden of work. -- Simon Riggshttp://www.EnterpriseDB.com/ subx_compress_knownassignedxids_more_often.v2.patch Description: Binary data

Re: Hash index build performance tweak from sorting

2022-08-01 Thread Simon Riggs
On Fri, 29 Jul 2022 at 13:49, Simon Riggs wrote: > > On Thu, 28 Jul 2022 at 19:50, Tom Lane wrote: > > > > Simon Riggs writes: > > > Thanks for the nudge. New version attached. > > > > I also see a speed improvement from this > > --- > > D

Re: Dump/Restore of non-default PKs

2022-08-01 Thread Simon Riggs
hash > > should be able to avoid any future clashes. That seems to conflict with USING INDEX TABLESPACE. I've tried a few things but have not found anything yet. Any other ideas on syntax? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Slow standby snapshot

2022-08-02 Thread Simon Riggs
re may be some benefit in varying those numbers to see the effects. It did also occur to me that we might need a separate process to perform the compressions, which we might be able to give to WALWriter. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Slow standby snapshot

2022-08-02 Thread Simon Riggs
e that significantly better by starting each binary search from the next element, rather than the start of the array. Something like the attached might help, but we can probably make that cache conscious to improve things even more. -- Simon Riggshttp://www.Enterprise

Re: Smoothing the subtrans performance catastrophe

2022-08-04 Thread Simon Riggs
On Wed, 3 Aug 2022 at 20:18, Andres Freund wrote: > On 2022-08-01 17:42:49 +0100, Simon Riggs wrote: > > The reason for the slowdown is clear: when we overflow we check every > > xid against subtrans, producing a large stream of lookups. Some > > previous hackers have tried

SUBTRANS: Minimizing calls to SubTransSetParent()

2022-08-08 Thread Simon Riggs
On Thu, 4 Aug 2022 at 13:11, Simon Riggs wrote: > > On Wed, 3 Aug 2022 at 20:18, Andres Freund wrote: > > > I think we should consider redesigning subtrans more substantially - even > > with > > the changes you propose here, there's still plenty ways to hit

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-08-09 Thread Simon Riggs
appy to provide more detailed review thoughts, so please keep asking questions. -- Simon Riggshttp://www.EnterpriseDB.com/

Blocking the use of TRIGGER privilege

2022-08-09 Thread Simon Riggs
ting usage of the trigger privilege would not be touched, only new usage. (No, this does not mean I want to ban triggers, only the trigger privilege). Thoughts? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-08-10 Thread Simon Riggs
CSnapdhot() That is one way to resolve the issue, but not the only one. I can also change AssignTransactionId() to recursively register parent xids for all of a subxid's parents. I will add in a test case and resolve the dependency in my next patch. Thanks again. -- Simon Riggs

Re: MERGE and parsing with prepared statements

2022-08-12 Thread Simon Riggs
. By changing the examples so they are the same, the sentence at line 573 now makes no sense. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: WIP: System Versioned Temporal Table

2021-09-12 Thread Simon Riggs
On Fri, 10 Sept 2021 at 19:30, Jaime Casanova wrote: > > On Tue, Aug 10, 2021 at 01:20:14PM +0100, Simon Riggs wrote: > > On Wed, 14 Jul 2021 at 12:48, vignesh C wrote: > > > > > The patch does not apply on Head anymore, could you rebase and post a > > &g

Re: Hook for extensible parsing.

2021-09-15 Thread Simon Riggs
execute which language types, allow us to switch between languages and have default languages for specific users or databases. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: WIP: System Versioned Temporal Table

2021-09-19 Thread Simon Riggs
orey suggested earlier that we just put the syntax in there, this was the direction I was thinking. After waiting a day since I wrote the above, I think we should go with (2) as Corey suggests, at least for now, and we can always add (3) later. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Hook for extensible parsing.

2021-09-22 Thread Simon Riggs
entation -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Using indexUnchanged with nbtree

2021-10-01 Thread Simon Riggs
On Fri, 1 Oct 2021 at 15:20, Jaime Casanova wrote: > This has been stalled since July, and based on Peter's comment i feel we > should mark this as RwF. Which i'm doing now. > > Please feel free to resubmit for Next Commitfest. Agreed, thank you Jaime. -- Simon Rig

Re: Reduce lock level for ALTER TABLE ... ADD CHECK .. NOT VALID

2021-10-03 Thread Simon Riggs
this and see how complex it is. At very least I will add a longer comment patch to mention this for the future. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Next Steps with Hash Indexes

2021-10-05 Thread Simon Riggs
optimizer code, as Tom points out. It also needs fine tuning to make the all-column approach beneficial for the additional cases without losing against what the "first column" approach gives. I did consider both approaches and after this discussion I am still in favour of committing the very simple "first column" approach to multi-col hash indexes now. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Next Steps with Hash Indexes

2021-10-05 Thread Simon Riggs
ultiple buckets, so we would expect the contention to be less than we would get on a monotonically increasing btree. So I don't now see any problem from holding the buffer lwlock on the bucket while we do multi-buffer operations. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Next Steps with Hash Indexes

2021-10-05 Thread Simon Riggs
On Tue, 5 Oct 2021 at 12:24, Dilip Kumar wrote: > > On Tue, Oct 5, 2021 at 4:08 PM Simon Riggs > wrote: > > > > On Mon, 27 Sept 2021 at 06:52, Amit Kapila wrote: > > > > > > On Thu, Sep 23, 2021 at 11:11 AM Dilip Kumar > > > wrote: > > &g

Re: Detecting File Damage & Inconsistencies

2021-07-13 Thread Simon Riggs
On Tue, Jul 6, 2021 at 4:21 AM Amit Kapila wrote: > > On Fri, Jul 2, 2021 at 8:29 PM Simon Riggs > wrote: > > > > On Fri, Jul 2, 2021 at 5:34 AM Craig Ringer > > wrote: > > > > > > > > If you don't think the sorts of use cases I presen

Re: PG 14 release notes, first draft

2021-07-13 Thread Simon Riggs
On Fri, Jul 2, 2021 at 12:50 AM Bruce Momjian wrote: > > On Thu, Jul 1, 2021 at 03:13:30PM +0100, Simon Riggs wrote: > > On Wed, Jun 30, 2021 at 11:20 PM Bruce Momjian wrote: > > > > * "row expiration" is a term not currently used in PG docs, so we > >

Re: Reduce lock level for ALTER TABLE ... ADD CHECK .. NOT VALID

2021-07-14 Thread Simon Riggs
On Sat, Jul 10, 2021 at 2:50 PM John Naylor wrote: > On Thu, Apr 22, 2021 at 8:01 AM Simon Riggs > wrote: > > > > 897795240cfaaed724af2f53ed2c50c9862f951f forgot to reduce the lock > > level for CHECK constraints when allowing them to be NOT VALID. > > > >

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-07-15 Thread Simon Riggs
On Wed, 14 Jul 2021 at 17:22, vignesh C wrote: > > On Thu, Apr 8, 2021 at 11:40 PM Simon Riggs wrote: > > > > On Thu, 8 Apr 2021 at 18:15, Alvaro Herrera wrote: > > > > > > On 2021-Apr-08, Simon Riggs wrote: > > > > > > > On Thu, 8 Apr

Next Steps with Hash Indexes

2021-07-15 Thread Simon Riggs
us from discussion of the main requirements. I have other performance tuning ideas, but they can wait. Anyway, that's what I think at present. Thoughts? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Next Steps with Hash Indexes

2021-07-20 Thread Simon Riggs
On Tue, Jul 20, 2021 at 1:00 PM Amit Kapila wrote: > > On Thu, Jul 15, 2021 at 10:11 PM Simon Riggs > wrote: > > > > 2. Unique Hash Indexes have been summarized here: > > https://www.postgresql.org/message-id/CAA4eK1KATC1TA5bR5eobYQVO3RWsnH6djNpk3P376em4V8MuUA%40ma

Re: Next Steps with Hash Indexes

2021-07-20 Thread Simon Riggs
s more than 2 long (bucket plus overflow), so it seems like mostly wasted effort at this point. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Detecting File Damage & Inconsistencies

2021-07-22 Thread Simon Riggs
ut for wal_sessioninfo data, if it exists * pg_waldump --user=USERNAME as a filter on username to demonstrate the use of this -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Next Steps with Hash Indexes

2021-07-23 Thread Simon Riggs
o tests yet for splitting or deferred uniqueness checks. The latter is because there are parse analysis changes needed to undo the assumption that only btrees support uniqueness, but nothing there looks like an issue. Thanks for your input, have a good weekend. -- Simon Riggs

Re: Skip temporary table schema name from explain-verbose output.

2021-07-26 Thread Simon Riggs
t myTempNamespace. Basically try to make that alias > > a bit less leaky. > > +1, let's replace it by "pg_temp" -- did the same in that attached 0001 patch. Sounds like a good change. Surely we need a test to exercise this works? Otherwise ready... -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Skip temporary table schema name from explain-verbose output.

2021-07-26 Thread Simon Riggs
On Mon, 26 Jul 2021 at 17:49, Tom Lane wrote: > > I wrote: > > Simon Riggs writes: > >> Surely we need a test to exercise this works? Otherwise ready... > > > There are lots of places in the core regression tests where we'd have > > used a temp table, ex

Re: Lowering the ever-growing heap->pd_lower

2021-08-02 Thread Simon Riggs
remove that from the patch and consider that as a separate issue, or close this. -- Simon Riggshttp://www.EnterpriseDB.com/

Commitfest overflow

2021-08-03 Thread Simon Riggs
n there be some additional coordination to actively clear down this problem? I won't attempt to come up with ideas to do this, but others may wish to suggest ways that avoid Committer burn-out? I will be happy to volunteer to be part of an organized approach that spreads out the work. Tha

Re: Commitfest overflow

2021-08-03 Thread Simon Riggs
On Tue, 3 Aug 2021 at 17:13, Tom Lane wrote: > > Bruce Momjian writes: > > On Tue, Aug 3, 2021 at 04:53:40PM +0100, Simon Riggs wrote: > >> There are 273 patches in the queue for the Sept Commitfest already, so > >> it seems clear the queue is not being

Re: Lowering the ever-growing heap->pd_lower

2021-08-03 Thread Simon Riggs
will cause multiple additional FPIs in WAL. I don't call that a future optimization, I call that essential additional work. +1 on committing the first part of the patch, -1 on the second. I suggest you split the patch and investigate the second part further. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reduce lock level for ALTER TABLE ... ADD CHECK .. NOT VALID

2021-08-03 Thread Simon Riggs
On Thu, 15 Jul 2021 at 07:47, Simon Riggs wrote: > > On Sat, Jul 10, 2021 at 2:50 PM John Naylor > wrote: > > On Thu, Apr 22, 2021 at 8:01 AM Simon Riggs > > wrote: > > > > > > 897795240cfaaed724af2f53ed2c50c9862f951f forgot to reduce the lock > >

Re: Lowering the ever-growing heap->pd_lower

2021-08-04 Thread Simon Riggs
On Wed, 4 Aug 2021 at 01:43, Peter Geoghegan wrote: > > On Mon, Aug 2, 2021 at 11:57 PM Simon Riggs > wrote: > > 1. Allow same thing as PageTruncateLinePointerArray() during HOT cleanup > > That is going to have a clear benefit for HOT workloads, which by > > their na

Accidentally dropped constraints: bug?

2021-08-05 Thread Simon Riggs
bc drop column c restrict; \d abc Table "public.abc" Column | Type | Collation | Nullable | Default +-+---+--+- a | integer | | not null | b | integer | | not null | Noting that all constra

Re: Lowering the ever-growing heap->pd_lower

2021-08-05 Thread Simon Riggs
XLOG_HEAP2_PRUNE record. If that is not safe... then we have a PG14 bug. If we do want to see this in WAL, both xl_heap_vacuum and xl_heap_prune lend themselves to just adding one more OffsetNumber onto the existing array, to represent the highest offset after truncation. So we don't need to change the WAL format. -- Simon Riggshttp://www.EnterpriseDB.com/

Default to TIMESTAMP WITH TIME ZONE?

2021-08-12 Thread Simon Riggs
zed ALTER TABLE command to make switching from WITHOUT to WITH TIME ZONE easy, so it is clearly an important thing to solve. So add a parameter called default_timestamp_with_timezone = off (default) | on Thoughts? -- Simon Riggshttp://www.EnterpriseDB.com/ default_timestamp_w

Re: Default to TIMESTAMP WITH TIME ZONE?

2021-08-12 Thread Simon Riggs
On Thu, 12 Aug 2021 at 20:07, Tom Lane wrote: > > Bruce Momjian writes: > > On Thu, Aug 12, 2021 at 07:24:58PM +0100, Simon Riggs wrote: > >> I heard the moan about "Why doesn't TIMESTAMP mean TIMESTAMP WITH TIME > >> ZONE" again today,

Re: Default to TIMESTAMP WITH TIME ZONE?

2021-08-13 Thread Simon Riggs
don't know and my patch didn't help them either. The only hope is to eventually change the default, so probably the best thing is to apply pressure via the SQL Std process. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Default to TIMESTAMP WITH TIME ZONE?

2021-08-14 Thread Simon Riggs
On Sat, 14 Aug 2021 at 09:03, Peter Eisentraut wrote: > > On 13.08.21 19:07, Tom Lane wrote: > > "David G. Johnston" writes: > >> On Fri, Aug 13, 2021 at 9:28 AM Simon Riggs > >> wrote: > >>> The only hope is to eventually change the d

Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
her with a new libpq call to utilise this. In summary: the patch is easy, the point is we need agreement to allow and encourage interoperation between clients and middleware. Thoughts? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
> over to replica x.x.x.x, please follow" That sounds like a server->middleware message and existing mechanisms might work for that already. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
icely with Logical Decoding is another can of > worms needing to be solved, but it is a start to becoming "Cloud > Native" :) Good ideas! -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
? I see this as a generalized interface that can be used for a great many things. * Failover managers * Load Balancers * Routing agents * Things I haven't thought of yet, but others may have etc.. We are currently limited by the messages we can send efficiently. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Middleware Messages for FE/BE

2021-08-19 Thread Simon Riggs
The question is how does a client communicate with middleware? The message that would be sent would relate to the features of the middleware, while being agnostic as to the client driver. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Middleware Messages for FE/BE

2021-08-20 Thread Simon Riggs
y defined to ignore unknown field types, so is suitable and extensible. We could add a new field type of 'm' to represent a message sent from middleware to the client. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Hash index build performance tweak from sorting

2022-08-30 Thread Simon Riggs
On Fri, 5 Aug 2022 at 20:46, David Zhang wrote: > > On 2022-08-01 8:37 a.m., Simon Riggs wrote: > > Using the above test case, I'm getting a further 4-7% improvement on > > already committed code with the attached patch, which follows your > > proposal. > >

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-08-30 Thread Simon Riggs
s well as smoothing the overflow case. Some discussion needed on this; there are various options. This combines the work originally posted here with another patch posted on the thread "Smoothing the subtrans performance catastrophe". I will do some performance testing also, but mo

Re: Smoothing the subtrans performance catastrophe

2022-09-06 Thread Simon Riggs
On Thu, 4 Aug 2022 at 13:11, Simon Riggs wrote: > I will post my patch, when complete, in a different thread. To avoid confusion, I will withdraw this patch from the CF, in favour of my other patch on a similar topic, Minimizing calls to SubTransSetParent() https://commitfest.postgresql.org

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-06 Thread Simon Riggs
On Tue, 6 Sept 2022 at 12:37, Dilip Kumar wrote: > > On Tue, Aug 30, 2022 at 10:16 PM Simon Riggs > wrote: > > > PFA two patches, replacing earlier work > > 001_new_isolation_tests_for_subxids.v3.patch > > 002_minimize_calls

New docs chapter on Transaction Management and related changes

2022-09-06 Thread Simon Riggs
New chapter on transaction management, plus a few related changes. Markup and links are not polished yet, so please comment initially on the topics, descriptions and wording. -- Simon Riggshttp://www.EnterpriseDB.com/ xact_docs.v2.patch Description: Binary data

Re: New docs chapter on Transaction Management and related changes

2022-09-06 Thread Simon Riggs
On Tue, 6 Sept 2022 at 17:19, Erik Rijkers wrote: > > Op 06-09-2022 om 17:16 schreef Simon Riggs: > > New chapter on transaction management, plus a few related changes. > > > > Markup and links are not polished yet, so please comment initially on > > the to

Re: New docs chapter on Transaction Management and related changes

2022-09-07 Thread Simon Riggs
On Tue, 6 Sept 2022 at 21:33, Justin Pryzby wrote: > > On Tue, Sep 06, 2022 at 04:16:02PM +0100, Simon Riggs wrote: > > New chapter on transaction management, plus a few related changes. > > > > Markup and links are not polished yet, so please comment initially on > >

Re: New docs chapter on Transaction Management and related changes

2022-09-09 Thread Simon Riggs
On Thu, 8 Sept 2022 at 08:42, Alvaro Herrera wrote: > > On 2022-Sep-06, Simon Riggs wrote: > > > On Tue, 6 Sept 2022 at 17:19, Erik Rijkers wrote: > > > > > > Op 06-09-2022 om 17:16 schreef Simon Riggs: > > > > New chapter on trans

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-13 Thread Simon Riggs
On Tue, 6 Sept 2022 at 13:14, Simon Riggs wrote: > I will update the patch, thanks for your scrutiny. I attach a diff showing what has changed between v8 and v9, and will reattach a full set of new patches in the next post, so patchtester doesn't squeal. -- Simon Riggs

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-13 Thread Simon Riggs
On Tue, 13 Sept 2022 at 11:56, Simon Riggs wrote: > > On Tue, 6 Sept 2022 at 13:14, Simon Riggs > wrote: > > > I will update the patch, thanks for your scrutiny. > > I attach a diff showing what has changed between v8 and v9, and will > reattach a full set of new p

Re: New docs chapter on Transaction Management and related changes

2022-09-13 Thread Simon Riggs
On Sun, 11 Sept 2022 at 21:35, Robert Treat wrote: > > On Wed, Sep 7, 2022 at 8:05 AM Simon Riggs > wrote: > > > > On Tue, 6 Sept 2022 at 21:33, Justin Pryzby wrote: > > > > > > On Tue, Sep 06, 2022 at 04:16:02PM +0100, Simon Riggs wrote: > > >

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-15 Thread Simon Riggs
On Wed, 14 Sept 2022 at 15:21, Alvaro Herrera wrote: > > On 2022-Aug-30, Simon Riggs wrote: > > > 001_new_isolation_tests_for_subxids.v3.patch > > Adds new test cases to master without adding any new code, specifically > > addressing the two areas of code that are n

Error for WITH options on partitioned tables

2022-09-16 Thread Simon Riggs
partitions instead -- Simon Riggshttp://www.EnterpriseDB.com/ 001_test_options_with_partitioned_table.v1.patch Description: Binary data 002_better_error_for_options_on_partitioned.v1.patch Description: Binary data

Re: SUBTRANS: Minimizing calls to SubTransSetParent()

2022-09-16 Thread Simon Riggs
On Thu, 15 Sept 2022 at 12:36, Japin Li wrote: > > > On Thu, 15 Sep 2022 at 18:04, Simon Riggs > wrote: > > On Wed, 14 Sept 2022 at 15:21, Alvaro Herrera > > wrote: > >> > >> On 2022-Aug-30, Simon Riggs wrote: > >> > >> > 001_new_

Pruning never visible changes

2022-09-16 Thread Simon Riggs
urs.) Please let me know what you think. -- Simon Riggshttp://www.EnterpriseDB.com/ never_visible.v1.patch Description: Binary data

Re: Pruning never visible changes

2022-09-16 Thread Simon Riggs
On Fri, 16 Sept 2022 at 15:26, Tom Lane wrote: > > Simon Riggs writes: > > A user asked me whether we prune never visible changes, such as > > BEGIN; > > INSERT... > > UPDATE.. (same row) > > COMMIT; > > Didn't we just have this discussion in a

Re: Slow standby snapshot

2022-09-16 Thread Simon Riggs
mail.gmail.com#379c1be7b8134ada5a574078d51b64c6 I've cleaned up the comments and used a #define for the constant. IMHO this is ready for commit. I will add it to the next CF. -- Simon Riggshttp://www.EnterpriseDB.com/ v8c-new-heuristic-to-compress-KnownAssignedXids.patch Description: Binary data

Re: Pruning never visible changes

2022-09-16 Thread Simon Riggs
On Fri, 16 Sept 2022 at 21:07, Laurenz Albe wrote: > > On Fri, 2022-09-16 at 10:26 -0400, Tom Lane wrote: > > Simon Riggs writes: > > > A user asked me whether we prune never visible changes, such as > > > BEGIN; > > > INSERT... > > > UPDATE.. (sam

Re: Pruning never visible changes

2022-09-16 Thread Simon Riggs
On Fri, 16 Sept 2022 at 18:37, Tom Lane wrote: > > Simon Riggs writes: > > On Fri, 16 Sept 2022 at 15:26, Tom Lane wrote: > >> You cannot > >> do that, at least not without checking that the originating > >> transaction has no snapshots that could see

Re: Hash index build performance tweak from sorting

2022-09-21 Thread Simon Riggs
On Wed, 21 Sept 2022 at 02:32, David Rowley wrote: > > On Tue, 2 Aug 2022 at 03:37, Simon Riggs wrote: > > Using the above test case, I'm getting a further 4-7% improvement on > > already committed code with the attached patch, which follows your > > proposal. > &

Re: Pruning never visible changes

2022-09-22 Thread Simon Riggs
On Mon, 19 Sept 2022 at 00:16, Greg Stark wrote: > > On Fri, 16 Sept 2022 at 10:27, Tom Lane wrote: > > > > Simon Riggs writes: > > > A user asked me whether we prune never visible changes, such as > > > BEGIN; > > > INSERT... > > > UPDATE.

Re: Pruning never visible changes

2022-09-22 Thread Simon Riggs
On Thu, 22 Sept 2022 at 15:16, Alvaro Herrera wrote: > > On 2022-Sep-22, Simon Riggs wrote: > > > On Mon, 19 Sept 2022 at 00:16, Greg Stark wrote: > > > > VACUUM was willing to remove a committed-dead tuple immediately if it > > > was > > >

Re: [Patch] ALTER SYSTEM READ ONLY

2020-06-18 Thread Simon Riggs
still better than what > we have now. > > If there are open transactions that have acquired an XID, the sessions are > killed > before the barrier is absorbed. > > inbuilt graceful failover for PostgreSQL > That doesn't appear to be very graceful. Perhaps objections

Re: A patch for get origin from commit_ts.

2020-06-30 Thread Simon Riggs
ning both > timestamp and origin for a given transaction ID. > No need to change existing APIs. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> Mission Critical Databases

Re: A patch for get origin from commit_ts.

2020-07-02 Thread Simon Riggs
On Thu, 2 Jul 2020 at 02:58, wrote: > On Tue, Jun 30, 2020 at 01:58:17PM +0100, Simon Riggs wrote: > > On Tue, 30 Jun 2020 at 02:17, Madan Kumar > wrote: > >> It may be better to have one single function returning both > >> timestamp and origin for a given tran

Re: Cleanup - Removed unused function parameter in reorder buffer & parallel vacuum

2020-07-03 Thread Simon Riggs
tains the changes for the same. > Thoughts? > Unused? To confirm that, everybody that has a logical decoding plugin needs to check their code so we are certain this is sensible. Seems like a change with low utility. -- Simon Riggshttp://www.2ndQuadrant.com/ <http:/

Re: ALTER TABLE validate foreign key dependency problem

2020-07-12 Thread Simon Riggs
validation of the foreign key > regardless of if there's a pending table rewrite, but the patch as it > is now only delays if there's a pending rewrite. > Consistency seems the better choice, so I agree we should validate later in all cases. Does changing that have any other eff

Re: Our naming of wait events is a disaster.

2020-05-12 Thread Simon Riggs
ist of possible changes. > If we add in extensions and lwlocks, will they show up as well? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> Mission Critical Databases

Re: Our naming of wait events is a disaster.

2020-05-12 Thread Simon Riggs
On Tue, 12 May 2020 at 21:00, Tom Lane wrote: > Simon Riggs writes: > > On Tue, 12 May 2020 at 19:11, Tom Lane wrote: > >> Anyway, I was just throwing this idea out to see if there would be > >> howls of "you can't rename anything" anguish. Since th

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-05-20 Thread Simon Riggs
eldom get those with many-way joins. So +1 for adding this technique. My question is whether it should be added as an optional facility of a parameterised sub plan, rather than an always-needed full-strength node. That way the choice of whether to use it can happen at execution time once we notice that

Re: [PATCH] Allow UNLISTEN during recovery

2019-01-26 Thread Simon Riggs
this is now in good shape, let me > > know if you think anything else needs to be done. > > Lotta work for a one-line code change, eh? Pushed now. > Good decision, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL

Re: Allowing extensions to supply operator-/function-specific info

2019-01-27 Thread Simon Riggs
number of output rows, in cases where that is variable and dependent upon the input params? * Allow a normal term to match a functional index, e.g. WHERE x = 'abcdefgh' => WHERE substr(x, 1 , 5) = 'abcde' AND x = 'abcdefgh' * Allow us to realise that ORDER BY f(

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2019-01-28 Thread Simon Riggs
might need to adjust in the future > during a detach will be changes that require AccessExclusiveLock > either, but that sounds awfully risky to me. We have very little DDL > that runs with less than AccessExclusiveLock, and I've already found > lots of subtle problems that have to be patched up just for the > particular case of allowing attach/detach to take a lesser lock on the > parent table, and I bet that there are a whole bunch more similar > problems when you start talking about weakening the lock on the child > table, and I'm not convinced that there are any reasonable solutions > to some of those problems, let alone that we can come up with good > solutions to all of them in the very near future. > I've not read every argument on this thread, but many of the later points made here are spurious, by which I mean they sound like they could apply but in fact do not. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Allowing extensions to supply operator-/function-specific info

2019-01-28 Thread Simon Riggs
se I'm not clear > on what your concern is. > To be able to extract indexable clauses where none existed before. Hash functions assume that x = N => hash(x) = hash(N) AND x = N so I want to be able to assume x = K => f(x) = f(K) AND x = K for specific f() to allow indexable operation

Re: Allowing extensions to supply operator-/function-specific info

2019-01-28 Thread Simon Riggs
On Tue, 29 Jan 2019 at 09:55, Tom Lane wrote: > Simon Riggs writes: > > On Sun, 27 Jan 2019 at 19:17, Tom Lane wrote: > >> ... I don't > >> know whether that would satisfy your concern, because I'm not clear > >> on what your concern is. > >

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-02-12 Thread Simon Riggs
On Wed, 13 Feb 2019 at 00:46, Alvaro Herrera wrote: > Here's a sample > concurrent index build on a 100M tuple table. Cool > There are no concurrent > transactions, so phases that wait for lockers don't appear. Would prefer to see them, so we know they are

Re: Compressed TOAST Slicing

2019-02-16 Thread Simon Riggs
good. Could we get an similarly optimized implementation of -> operator for JSONB as well? Are there any other potential uses? Best to fix em all up at once and then move on to other things. Thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> Po

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-02-19 Thread Simon Riggs
tables. For that case, it seems like reset hashtable is too much. Can we use our knowledge of the structure of locks, i.e. partition locks are all children of the partitioned table, to do a better job? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Compressed TOAST Slicing

2019-02-20 Thread Simon Riggs
On Tue, 19 Feb 2019 at 23:09, Paul Ramsey wrote: > On Sat, Feb 16, 2019 at 7:25 AM Simon Riggs wrote: > > > Could we get an similarly optimized implementation of -> operator for > JSONB as well? > > Are there any other potential uses? Best to fix em all up at once and

Re: propagating replica identity to partitions

2019-02-20 Thread Simon Riggs
ing it for legacy inheritance, but if the majority is to change it > for both, let's do that. > -1 for changing legacy inheritance. Two separate features. Inheritance has been around for many years and its feature set is stable. No need to touch it. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Compressed TOAST Slicing

2019-02-20 Thread Simon Riggs
Review comments != holding hostages. If we add one set of code now and need to add another different one later, we will have 2 sets of code that do similar things. I'm surprised to hear you think that is a good thing. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: propagating replica identity to partitions

2019-02-20 Thread Simon Riggs
e largest footgun in Postgres, given that command's current behavior and the size of partitioned tables. If it moved partitions concurrently I'd feel differently. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

pgbench MAX_ARGS

2019-02-26 Thread Simon Riggs
the error message when you hit the limit. The highest argument id is MAX_ARGS - 1, but the max number of arguments is MAX_ARGS. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training &

Re: pgbench MAX_ARGS

2019-02-26 Thread Simon Riggs
omewhere? There is no limit discussed in the > PREPARE documentation, I tested up to 20. I'd sugggest to add something at > the end of the paragraph about variable substitution in the "Custom > Script" section, eg "A maximum of XX variable references can appe

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-02-26 Thread Simon Riggs
On Thu, 21 Feb 2019 at 15:38, Kuntal Ghosh wrote: > Thank you for the patch. It seems to me that while performing COPY > FREEZE, if we've copied tuples in a previously emptied page There won't be any previously emptied pages because of the pre-conditions required for FREEZE.

Re: No-rewrite timestamp<->timestamptz conversions

2019-02-26 Thread Simon Riggs
for such a facility, so instead I think > ATColumnChangeRequiresRewrite() should have a hard-wired call for > F_TIMESTAMPTZ_TIMESTAMP and F_TIMESTAMP_TIMESTAMPTZ. Patch attached. If > we > find more applications of this concept, it shouldn't be hard to migrate > this > logic into SupportRequestSimplify. Does anyone think that's better to do > from > the start? > Looks good, would need docs. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: pgbench MAX_ARGS

2019-02-26 Thread Simon Riggs
On Tue, 26 Feb 2019 at 17:38, Andres Freund wrote: > Hi, > > On 2019-02-26 12:57:14 +0000, Simon Riggs wrote: > > On Tue, 26 Feb 2019 at 12:19, Fabien COELHO wrote: > > I've put it as 256 args now. > > > > The overhead of that is about 2kB, so not really

Re: pgsql: Avoid duplicate XIDs at recovery when building initial snapshot

2018-12-01 Thread Simon Riggs
On Fri, 30 Nov 2018 at 23:08, Michael Paquier wrote: > On Fri, Nov 30, 2018 at 02:55:47PM +0000, Simon Riggs wrote: > > 1df21ddb looks OK to me and was simple enough to backpatch safely. > > Thanks for the feedback! > > > Seems excessive to say that the WAL record is c

<    1   2   3   4   5   6   7   8   >