Re: Reducing power consumption on idle servers

2022-02-21 Thread Simon Riggs
On Sat, 19 Feb 2022 at 17:03, Andres Freund wrote: > > Hi, > > On 2022-02-19 14:10:39 +, Simon Riggs wrote: > > Some years ago we did a pass through the various worker processes to > > add hibernation as a mechanism to reduce power consumption on an idle > > ser

Re: Reducing power consumption on idle servers

2022-02-21 Thread Simon Riggs
it is being removed. v3 attached. -- Simon Riggshttp://www.EnterpriseDB.com/ hibernate_to_reduce_power_consumption.v3.patch Description: Binary data

Buffer Manager and Contention

2022-02-24 Thread Simon Riggs
eason of doing the Delete after the Insert? Put that another way, it looks like BufTable functions are used in a way that mismatches against the way dynahash is designed. Thoughts? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Logical insert/update/delete WAL records for custom table AMs

2022-02-24 Thread Simon Riggs
On Mon, 17 Jan 2022 at 09:05, Jeff Davis wrote: > > On Wed, 2022-01-05 at 20:19 +, Simon Riggs wrote: > > I spoke with Jeff in detail about this patch in NYC Dec 21, and I now > > think it is worth pursuing. It seems much more likely that this would > > be acceptab

Re: Buffer Manager and Contention

2022-02-24 Thread Simon Riggs
hmarking in the thread > showed a good result. I'd appreciate your input on that thread. Certainly, I will stop this thread and contribute to Yura's thread. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: BufferAlloc: don't take two simultaneous locks

2022-02-24 Thread Simon Riggs
artitioned dynahash from 32 to maybe 8, as originally speculated by Robert in 2016: https://www.postgresql.org/message-id/CA%2BTgmoZkg-04rcNRURt%3DjAG0Cs5oPyB-qKxH4wqX09e-oXy-nw%40mail.gmail.com since the freelists will be much less contended with the above approach It would be useful to see performance with a higher number of connections, >400. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Simon Riggs
t may be I'm missing > something. Sounds reasonable. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: suboverflowed subtransactions concurrency performance optimize

2022-03-07 Thread Simon Riggs
On Mon, 7 Mar 2022 at 09:49, Julien Rouhaud wrote: > > Hi, > > On Mon, Jan 17, 2022 at 01:44:02PM +, Simon Riggs wrote: > > > > Re-attached, so that the CFapp isn't confused between the multiple > > patches on this thread. > > Thanks a lot for wor

Re: Reducing power consumption on idle servers

2022-03-09 Thread Simon Riggs
y (the original API). Once that is gone the mechanism would be to request promotion via pg_ctl promote (the new API), which then creates the trigger file and sends a signal. So in both APIs the trigger file is still used. In this patch, if you create the trigger file directly/explicitly/y

Re: support for MERGE

2022-03-10 Thread Simon Riggs
ithout such a constraint there is no conflict. Concurrent inserts are checked by merge-insert-update.spec, which correctly raises an ERROR in this case, as documented. Various cases were not tested in the patch - additional patch attached, but nothing surprising there. ExecInsert() does not return from such an ERROR, so the code fragment appears correct to me. -- Simon Riggshttp://www.EnterpriseDB.com/ new_insert_tests_for_merge.patch Description: Binary data

Re: Parameter for planner estimate of recursive queries

2022-03-10 Thread Simon Riggs
d I can't see > us realistically doing a lot better with any reasonable amount of > work. Shall I set this as Ready For Committer? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reducing power consumption on idle servers

2022-03-10 Thread Simon Riggs
0s would decrease this. I don't see much difference between power consumption with timeouts of 60s and 300s. In the latest patch, I chose 300s. Does anyone have an opinion on the value here? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
On Tue, 22 Mar 2022 at 00:04, Andres Freund wrote: > On 2022-03-10 17:42:14 +0000, Simon Riggs wrote: > > Shall I set this as Ready For Committer? > > Currently this CF entry fails on cfbot: > https://cirrus-ci.com/task/4531771134967808?logs=test_world#L1158 > > [16:27

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
ure what a reasonable > non-zero lower bound would be. Agreed, makes sense. > * The proposed docs claim that a smaller setting works by biasing > the planner towards fast-start plans, but I don't think I believe > that explanation. I'd venture that we want text more

Re: Reducing power consumption on idle servers

2022-03-23 Thread Simon Riggs
On Tue, 22 Mar 2022 at 00:54, Andres Freund wrote: > > On 2022-02-21 21:04:19 +, Simon Riggs wrote: > > On Mon, 21 Feb 2022 at 16:49, Chapman Flack wrote: > > > > > Shouldn't the comment be "with work_done=false" ? > > > > Good c

Re: Parameter for planner estimate of recursive queries

2022-03-23 Thread Simon Riggs
On Wed, 23 Mar 2022 at 18:20, Simon Riggs wrote: > [New patch version pending] -- Simon Riggshttp://www.EnterpriseDB.com/ recursive_worktable_estimate.v3.patch Description: Binary data

Re: Huge memory consumption on partitioned table with FKs

2020-12-02 Thread Simon Riggs
and repeated planning of sub-plans for similar partitions. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: SQL/JSON: functions

2020-12-15 Thread Simon Riggs
n skip to PG15. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reloptions for table access methods

2020-12-15 Thread Simon Riggs
ee with the premise that all existing heap options should be common across all new or extension tableAMs. There are dozens of such options and I don't believe that they would all have meaning in all future storage plugins. I think options should just work exactly the same for Indexes and Ta

Discarding DISCARD ALL

2020-12-23 Thread Simon Riggs
s if we are dropping them all anyway. Patch attached, passes make check with new tests added. Comments welcome. -- Simon Riggshttp://www.EnterpriseDB.com/ transaction_cleanup.v4.patch Description: Binary data

Re: Discarding DISCARD ALL

2020-12-23 Thread Simon Riggs
nced to do whatever else we agree is desirable. Do we need something like DISCARD ALL EXCEPT PREPARED STATEMENTS; ?? If there are different requirements for each pooler, what are they? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Disable WAL logging to speed up data loading

2020-12-28 Thread Simon Riggs
hen back up again. It must be a one-way transition from "unsafe" to a higher level, so if people want to use this for temp reporting servers or initial loading, great, but they can't use it as a quick speed-up for databases containing needs-to-be-safe data. Possibly the state change mig

Re: create table like: ACCESS METHOD

2020-12-30 Thread Simon Riggs
bility in the release notes. > > This patch should have more tests. Something could be added in > > create_am.sql where there is a fake heap2 as table AM. > > Yes, I had already done that locally. There are no tests for the new functionality, please could you add some? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Allow CURRENT_ROLE in GRANTED BY

2020-12-30 Thread Simon Riggs
rhaps at a later date. > > Here is the highly anticipated and quite underwhelming second part of > this patch set. Looks great, but no test to confirm it works. I would suggest adding a test and committing directly since I don't see any cause for further discussion. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Disable WAL logging to speed up data loading

2020-12-30 Thread Simon Riggs
On Wed, 30 Dec 2020 at 13:04, Michael Paquier wrote: > > Yes, we did think of this feature already and rejected it. > > I don't recall seeing such a discussion. Do you have some references? > Just a matter of curiosity :) Off-list discussions. -- Simon Riggs

Re: Multi Inserts in CREATE TABLE AS - revived patch

2020-12-30 Thread Simon Riggs
haviour of > table_multi_insert() API for unlogged tables? I found the additional performance of Paul Guo's work to be compelling and the idea workable for very large loads. Surely LockRelationForExtension() is all the inter-process coordination we need to make this work for parallel loads? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Reloptions for table access methods

2020-12-30 Thread Simon Riggs
On Tue, 15 Dec 2020 at 23:59, Jeff Davis wrote: > > On Tue, 2020-12-15 at 17:37 +, Simon Riggs wrote: > > > > I definitely don't agree with the premise that all existing heap > > options should be common across all new or extension tableAMs. There > > are

Re: Zedstore - compressed in-core columnar storage

2020-12-31 Thread Simon Riggs
nly. Your data and/or query accuracy may be at risk if you rely on this." -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Safety/validity of resetting permissions by updating system tables

2021-01-03 Thread Simon Riggs
ABLES IN SCHEMA test" at the top of your script? You say there is a problem, but don't describe the precise problem. Can you give a fully worked example so we can understand how to resolve? The meaning of GRANT and REVOKE is now defined by SQL Standard, so not something we can easily change. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: WIP: System Versioned Temporal Table

2021-01-07 Thread Simon Riggs
could be earlier than start time, so that is just wrong. Ideally we would use commit timestamp and fill the values in later. So the only thing that makes sense for me is to use the dynamic time of insertion while we hold the buffer lock, otherwise we will get anomalies. The wo

Re: Reloptions for table access methods

2021-01-07 Thread Simon Riggs
On Sat, Jan 2, 2021 at 6:44 PM Jeff Davis wrote: > > On Wed, 2020-12-30 at 21:23 +, Simon Riggs wrote: > > But you cannot seriously argue that a one line patch that changes > > user > > visible behavior can be acceptable in Postgres core without tests or > >

Re: WIP: System Versioned Temporal Table

2021-01-07 Thread Simon Riggs
On Thu, Jan 7, 2021 at 5:59 PM Simon Riggs wrote: > > On Mon, Jan 4, 2021 at 2:24 PM Masahiko Sawada wrote: > > Please also note the v7 patch cannot be applied to the current HEAD. I'm > > switching the patch as Waiting on Author. > > Surafel, please say whether y

Re: WIP: System Versioned Temporal Table

2021-01-09 Thread Simon Riggs
itional tests so we can see more clearly that the tests fail on the present patch. I will post more on this next week. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: NOT VALID for Unique Indexes

2021-02-26 Thread Simon Riggs
On Mon, Jan 18, 2021 at 12:34 AM David Fetter wrote: > > On Thu, Jan 14, 2021 at 04:22:17PM +, Simon Riggs wrote: > > As you may be aware the NOT VALID qualifier currently only applies to > > CHECK and FK constraints, but not yet to unique indexes. I have had > > cus

Re: NOT VALID for Unique Indexes

2021-02-26 Thread Simon Riggs
On Mon, Jan 18, 2021 at 11:19 PM japin wrote: > > > On Fri, 15 Jan 2021 at 00:22, Simon Riggs > wrote: > > As you may be aware the NOT VALID qualifier currently only applies to > > CHECK and FK constraints, but not yet to unique indexes. I have had > > cu

Re: Deleting older versions in unique indexes to avoid page splits

2020-10-27 Thread Simon Riggs
t this is a distinct > argument to my favorite argument, which is that we cannot afford to > *not* go a bit slower in certain extreme cases). > > I welcome debate about this. Agreed, we can trade initial speed for long term consistency. I guess there are some heuristics there on that tradeoff. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Background writer and checkpointer in crash recovery

2020-11-11 Thread Simon Riggs
ffers, with the right workload. We don't have any stats to show whether this patch is worthwhile or not, so I suggest adding the attached instrumentation patch as well so we can see on production systems whether checkpoint_timeout is too high by comparison with pg_stat_bgwriter. The patch is writ

Detecting File Damage & Inconsistencies

2020-11-11 Thread Simon Riggs
that were not in the old index This approach piggybacks on existing operations. AccessShareLock is held on both indexes before the old one is dropped. Other ideas are welcome. Thoughts? -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Detecting File Damage & Inconsistencies

2020-11-11 Thread Simon Riggs
On Thu, 12 Nov 2020 at 06:42, tsunakawa.ta...@fujitsu.com wrote: > > From: Simon Riggs > > I would like to propose a few points that will help us detect file > > damage, inconsistencies in files and track actions of users. > > Hello, Simon san. Long time no see. I&#x

Re: Detecting File Damage & Inconsistencies

2020-11-13 Thread Simon Riggs
On Fri, 13 Nov 2020 at 00:50, tsunakawa.ta...@fujitsu.com wrote: > > From: Simon Riggs > > If a rogue user/process is suspected, this would allow you to identify > > more easily the changes made by specific sessions/users. > > Isn't that kind of auditing a job of p

Re: remove spurious CREATE INDEX CONCURRENTLY wait

2020-11-16 Thread Simon Riggs
s name are there two copies?" Agreed to all of the above, but I think the issue is miniscule because ProcArrayLock is acquired in LW_EXCLUSIVE mode at transaction commit. So it doesn't seem like there is much need to optimize this particular aspect of the patch. -- Simon Riggshttp://www.EnterpriseDB.com/

VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-16 Thread Simon Riggs
way from this feature by saying "is intended to be used only when the contents of the visibility map are suspect". Reworded docs patch attached. -- Simon Riggshttp://www.EnterpriseDB.com/ aggressive_rewording.v1.patch Description: Binary data

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-17 Thread Simon Riggs
On Mon, 16 Nov 2020 at 22:53, Masahiko Sawada wrote: > On Tue, Nov 17, 2020 at 5:52 AM Simon Riggs wrote: > I don't think the doc is wrong. If DISABLE_PAGE_SKIPPING is specified, > we not only set aggressive = true but also skip checking visibility > map. For instance, see line

Re: Detecting File Damage & Inconsistencies

2020-11-17 Thread Simon Riggs
On Fri, 13 Nov 2020 at 11:24, Simon Riggs wrote: > > On Fri, 13 Nov 2020 at 00:50, tsunakawa.ta...@fujitsu.com > wrote: > > > > From: Simon Riggs > > > If a rogue user/process is suspected, this would allow you to identify > > > more easily the

Re: Detecting File Damage & Inconsistencies

2020-11-18 Thread Simon Riggs
On Wed, 18 Nov 2020 at 06:42, Craig Ringer wrote: > > On Fri, Nov 13, 2020 at 7:24 PM Simon Riggs wrote: >> >> >> What I'm proposing is an option to add 16 bytes onto each COMMIT >> record > > > Would it make sense to write this at the time we

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-18 Thread Simon Riggs
zation, if we do find a row that needs freezing > > on a data block, we should simply freeze *all* row versions on the > > page, not just the ones below the selected cutoff. This is justified > > since writing the block is the biggest cost and it doesn't make much >

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-19 Thread Simon Riggs
On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: > > On Wed, Nov 18, 2020 at 12:54 PM Simon Riggs wrote: > > Patches attached. > > 1. vacuum_anti_wraparound.v2.patch > > 2. vacuumdb_anti_wrap.v1.patch - depends upon (1) > > I don't like the use of ANTI_WRAPA

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-19 Thread Simon Riggs
On Wed, 18 Nov 2020 at 02:04, Alvaro Herrera wrote: > > On 2020-Nov-17, Simon Riggs wrote: > > > As an additional optimization, if we do find a row that needs freezing > > on a data block, we should simply freeze *all* row versions on the > > page, not just the one

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Simon Riggs
On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > > > On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: > > > > > > On Wed, Nov 18, 2020 at 12:54 PM Simon Riggs > > &g

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Simon Riggs
since that would slow it down when we want to speed it up. I've updated the patch to match your proposal, so we can compare. It seems a shorter patch. (This patch is an optimization that is totally independent to the other proposals on this thread). -- Simon Riggshttp://www.EnterpriseDB.com/ one_freeze_then_max_freeze.v6.patch Description: Binary data

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Simon Riggs
On Fri, 20 Nov 2020 at 10:15, Simon Riggs wrote: > > On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > > > On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > > > > > On Wed, 18 Nov 2020 at 17:59, Robert Haas wrote: > > > > >

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-20 Thread Simon Riggs
they may contain more tuples than before and will increase the probability that the page is marked all_frozen. So yes, as you say, the net effect will be to reduce the number of write I/Os in subsequent vacuums required to move forward relfrozenxid. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-29 Thread Simon Riggs
On Fri, 20 Nov 2020 at 11:47, Simon Riggs wrote: > > On Fri, 20 Nov 2020 at 10:15, Simon Riggs wrote: > > > > On Fri, 20 Nov 2020 at 01:40, Masahiko Sawada wrote: > > > > > > On Thu, Nov 19, 2020 at 8:02 PM Simon Riggs wrote: > > > > > >

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2020-11-29 Thread Simon Riggs
I'm changing the patch to only work with Xids not both xids and multixacts. If this gets committed we can think more about multixacts and whether to optimize freezing them in the same situation or not. -- Simon Riggshttp://www.EnterpriseDB.com/ one_freeze_then_max_freeze.v7.patch Description: Binary data

Re: support for MERGE

2021-01-13 Thread Simon Riggs
would exist. > 9) parsenodes.h > > Should we rename mergeTarget_relation to mergeTargetRelation? The > current name seems like a mix between two naming schemes. +1 We've had code from 4-5 people in the patch now, so I will re-review myself to see if I can shed light on anything. -- Simon Riggshttp://www.EnterpriseDB.com/

NOT VALID for Unique Indexes

2021-01-14 Thread Simon Riggs
bility, syntax and requirements before I do that. I can also add similar syntax for UNIQUE and PK constraints. Thoughts please? -- Simon Riggshttp://www.EnterpriseDB.com/ alter_index_set_unique_not_valid.v4.patch Description: Binary data

Re: WIP: System Versioned Temporal Table

2021-01-14 Thread Simon Riggs
WEEN x AND y SHOULD include the Start and End timestamp columns since this form of query can include multiple row versions for the same row, so it makes sense to see the validity times -- Simon Riggshttp://www.EnterpriseDB.com/

Re: WIP: System Versioned Temporal Table

2021-01-14 Thread Simon Riggs
On Thu, Jan 14, 2021 at 5:42 PM Surafel Temesgen wrote: > > Hi Simon, > Thank you for all the work you does No problem. > On Mon, Jan 11, 2021 at 5:02 PM Simon Riggs > wrote: >> >> >> >> * Anomalies around use of CURRENT_TIMESTAMP are not discussed or r

Re: WIP: System Versioned Temporal Table

2021-01-15 Thread Simon Riggs
On Fri, Jan 15, 2021 at 4:46 PM Surafel Temesgen wrote: > > > > On Fri, Jan 15, 2021 at 12:27 AM Simon Riggs > wrote: >> >> >> Yes, I think it can. The current situation is that the Start or End is >> set to the Transaction Start Timestamp. >> So if

Re: WIP: System Versioned Temporal Table

2021-01-15 Thread Simon Riggs
On Fri, Jan 15, 2021 at 4:56 PM Surafel Temesgen wrote: > > > > On Fri, Jan 15, 2021 at 12:22 AM Simon Riggs > wrote: >> >> SELECT * FROM foo FOR SYSTEM_TIME AS OF ... >> should NOT include the Start and End timestamp columns >> because this acts li

Re: Discarding DISCARD ALL

2021-01-20 Thread Simon Riggs
SETs to be SET LOCAL. The point is that if we declare ahead of time that the transaction will be reset then we can act differently and more easily for various circumstances, for SETs, for Temp tables and others. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: WIP: System Versioned Temporal Table

2021-01-26 Thread Simon Riggs
On Tue, Jan 26, 2021 at 11:33 AM Vik Fearing wrote: > > On 1/11/21 3:02 PM, Simon Riggs wrote: > > * UPDATE foo SET start_timestamp = DEFAULT should fail but currently doesn't > > I'm still in the weeds of reviewing this patch, but why should this > fail? It sho

Re: WIP: System Versioned Temporal Table

2021-01-26 Thread Simon Riggs
On Tue, Jan 26, 2021 at 12:51 PM Vik Fearing wrote: > > On 1/26/21 1:16 PM, Simon Riggs wrote: > > On Tue, Jan 26, 2021 at 11:33 AM Vik Fearing > > wrote: > >> > >> On 1/11/21 3:02 PM, Simon Riggs wrote: > >>> * UPDATE foo SET start_timestamp =

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-01-28 Thread Simon Riggs
On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada wrote: > This entry has been "Waiting on Author" status and the patch has not > been updated since Nov 30. Are you still planning to work on this? Yes, new patch version tomorrow. Thanks for the nudge and the review.

Re: suboverflowed subtransactions concurrency performance optimize

2021-12-03 Thread Simon Riggs
when nsubxids < PGPROC_MAX_CACHED_SUBXIDS so we need not consult pg_subtrans in that case, see step 4 of TransactionIdIsInProgress() -- Simon Riggshttp://www.EnterpriseDB.com/

Re: suboverflowed subtransactions concurrency performance optimize

2021-12-08 Thread Simon Riggs
a little more > than 64 subtransactions and all the system is stuck. Or will it affect only > backend using more than 64 subtransactions? That is the objective: to isolate the effect to only those that overflow. It seems possible. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: suboverflowed subtransactions concurrency performance optimize

2021-12-08 Thread Simon Riggs
On Fri, 3 Dec 2021 at 06:27, Dilip Kumar wrote: > > On Tue, Nov 30, 2021 at 5:49 PM Simon Riggs > wrote: > > > transam.c uses a single item cache to prevent thrashing from repeated > > lookups, which reduces problems with shared access to SLRUs. > > multitrans

Documenting when to retry on serialization failure

2021-12-09 Thread Simon Riggs
the user to declare. Could we allow a user-defined auto_retry parameter? We don't mention that a transaction might just repeatedly fail either. Anyway, know y'all would have some opinions on this. Happy to document whatever we agree. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Documenting when to retry on serialization failure

2021-12-29 Thread Simon Riggs
On Wed, 29 Dec 2021 at 03:30, Thomas Munro wrote: > > On Fri, Dec 10, 2021 at 1:43 AM Simon Riggs > wrote: > > "Applications using this level must be prepared to retry transactions > > due to serialization failures." > > ... > > "When an applicat

Re: Parameter for planner estimate of recursive queries

2021-12-31 Thread Simon Riggs
On Wed, 27 Oct 2021 at 15:58, Simon Riggs wrote: > The poor performance is traced to the planner cost estimates for > recursive queries. Specifically, the cost of the recursive arm of the > query is evaluated based upon both of these hardcoded assumptions: > > 1. The recursion w

Re: Documenting when to retry on serialization failure

2022-01-04 Thread Simon Riggs
all cases. Some problem cases would help us decide either way. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: SKIP LOCKED assert triggered

2022-01-04 Thread Simon Riggs
ALID)); > > AFAICS, this assertion condition is constant-true, > because TM_WouldBlock is a nonzero constant. Perhaps you meant > >Assert(result == TM_WouldBlock || !(tuple->t_data->t_infomask & > HEAP_XMAX_INVALID)); Yes, I think that's what I meant. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: support for MERGE

2022-01-04 Thread Simon Riggs
On Wed, 22 Dec 2021 at 11:35, Simon Riggs wrote: > > On Mon, 15 Nov 2021 at 22:45, Alvaro Herrera wrote: > > > > On 2021-Nov-15, Alvaro Herrera wrote: > > > > > Thanks everyone for the feedback. I attach a version with the fixes > > > that were sub

Re: Pluggable toaster

2022-01-05 Thread Simon Riggs
so we can have more pointers in a table * We want to avoid putting the data length into the toast pointer, so we can allow the toasted data to be expanded without rewriting everything (to avoid O(N^2) cost) -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-05 Thread Simon Riggs
he user visible behavior. Please explain the various new states that pages can be in and what the effects are, My understanding is this would be backwards compatible, so we can upgrade to it. Please confirm. Thanks -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Logical insert/update/delete WAL records for custom table AMs

2022-01-05 Thread Simon Riggs
to be written in *some* common format, and the current heap format currently works, so de facto it is the format we should use. Right now, TAMs have to reformat back into this same format, so it is the way the APIs work. Put it another way: I don't see any other format that makes sense to use, now, but that could change in the future. So I'm signing up as a reviewer and we'll see if this is good to go. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Collecting statistics about contents of JSONB columns

2022-01-05 Thread Simon Riggs
> "a.b.c" > "a.b.c.d" > > and of course many of the paths will often have JSONB documents as > values, not simple scalar values. I wonder if we should limit the depth > somehow, and maybe build stats only for scalar values. The user interface for designing filters sounds hard, so I'd hope we can ignore that for now. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Simon Riggs
fects. Tested using the attached test, so seems to work correctly. On review of docs, no additions or changes required. Perhaps add something to README? If so, minor doc patch attached. Otherwise, this sub-patch is READY FOR COMMITTER. -- Simon Riggshttp://www.Ent

Re: Logical insert/update/delete WAL records for custom table AMs

2022-01-06 Thread Simon Riggs
and I will review with you as you go. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-06 Thread Simon Riggs
all required software is delivered in one shot, with fast upgrade 2. As each table is VACUUMed, we confirm/clean/groom data blocks so each table is individually confirmed as being ready. The pace that this happens at is under user control. 3. When all tables have been prepared, then restart to allow xid64 format usage -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-07 Thread Simon Riggs
w and atypical restriction. I don't see that restriction. Anyone upgrading from before PG15 would apply the transform. Just because we introduce a transform in PG15 doesn't mean we can't apply that transform in later releases as well, to allow say PG14 -> PG16. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Add 64-bit XIDs into PostgreSQL 15

2022-01-12 Thread Simon Riggs
we discuss the main approach. 2. Write up the approach in a detailed README, so people can understand the proposal and assess if there are problems. A few short notes and a link back to old conversations isn't enough to allow wide review and give confidence on such a major patch

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-03-17 Thread Simon Riggs
On Fri, 12 Mar 2021 at 22:16, Tomas Vondra wrote: > > On 1/28/21 2:33 PM, Simon Riggs wrote: > > On Thu, 28 Jan 2021 at 12:53, Masahiko Sawada wrote: > > > >> This entry has been "Waiting on Author" status and the patch has not > >> been updated s

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Simon Riggs
On Thu, 8 Apr 2021 at 16:23, David Steele wrote: > > On 3/17/21 4:50 PM, Simon Riggs wrote: > > On Fri, 12 Mar 2021 at 22:16, Tomas Vondra > > wrote: > >> > >> On 1/28/21 2:33 PM, Simon Riggs wrote: > >>> On Thu, 28 Jan 2021 at 12:53, Masahiko S

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Simon Riggs
CF entry > to next CF and clarify which patch is current? Entry: Maximize page freezing has this patch, perfectly fine, awaiting review from Alvaro since 29 Nov. one_freeze_then_max_freeze.v7.patch Other patch is awaiting changes. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Simon Riggs
On Thu, 8 Apr 2021 at 17:44, David Steele wrote: > > On 4/8/21 12:29 PM, Simon Riggs wrote: > > On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > > > >>>> It has been five months since this patch was updated, so marking > >>>> Returned with Feedba

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Simon Riggs
On Thu, 8 Apr 2021 at 18:15, Alvaro Herrera wrote: > > On 2021-Apr-08, Simon Riggs wrote: > > > On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > > > > It's not clear to me which patch is which, so perhaps move one CF entry > > > to next CF and clar

Re: PROC_IN_ANALYZE stillborn 13 years ago

2020-08-06 Thread Simon Riggs
dn't come from a single snapshot, but then who cares? There is no requirement for consistency - the sample would be arguably *more* stable because it comes from multiple points in time, not just one. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> Mission Critical Databases

Re: PROC_IN_ANALYZE stillborn 13 years ago

2020-08-06 Thread Simon Riggs
default statistics_target, so that default behavior would be unaffected. Larger settings would be chunked according to the default, so stats_target=10k(max) would take a 1/100 = 100 snapshots. That approach allows people to vary that using an existing parameter if needed. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> Mission Critical Databases

Re: massive FPI_FOR_HINT load after promote

2020-08-14 Thread Simon Riggs
hat sets all tuples at once, or possibly emit an FPI and then follow that with a second message that sets all other hints on the page. -- Simon Riggshttp://www.2ndQuadrant.com/ Mission Critical Databases

Re: Next Steps with Hash Indexes

2021-10-13 Thread Simon Riggs
would only be used by explicit user choice. If there are some minor > > sub-optimal aspects of using hash indexes, then btree was already the > > default and a great choice for many cases. > > > > But we can't select arbitrary column order, because the first column is > used to select the bucket. Which means it's also about what columns are > used by the queries. If the query is not using the first column, it > can't use the index. Neither approach works in that case, so it is moot. i.e. you cannot use a first-column hash, nor an all-column hash. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: Next Steps with Hash Indexes

2021-10-14 Thread Simon Riggs
On Wed, 13 Oct 2021 at 20:16, Peter Geoghegan wrote: > > On Wed, Oct 13, 2021 at 3:44 AM Simon Riggs > wrote: > > > IMO it'd be nice to show some numbers to support the claims that storing > > > the extra hashes and/or 8B hashes is not worth it ... > > >

Re: Next Steps with Hash Indexes

2021-10-17 Thread Simon Riggs
On Thu, 14 Oct 2021 at 16:09, Peter Geoghegan wrote: > > On Thu, Oct 14, 2021 at 12:48 AM Simon Riggs > wrote: > > The hash index tuples are 20-bytes each. If that were rounded up to > > 8-byte alignment, then that would be 24 bytes. > > > > Using pageinspect

Parameter for planner estimate of recursive queries

2021-10-27 Thread Simon Riggs
d up the query. One is to set enable_seqscan = off, which helps about 20%, but may have other consequences. Two is to set work_mem = 512MB, so that the poor plan (hash) works as fast as possible, but that is far from optimal either. Getting the right plan is x20 faster than either of those alter

Re: PITR: enhance getRecordTimestamp()

2021-11-03 Thread Simon Riggs
On Wed, 3 Nov 2021 at 13:28, Daniel Gustafsson wrote: > > > On 30 Jun 2021, at 11:59, Simon Riggs wrote: > > > For PITR, getRecordTimestamp() did not include all record types that > > contain times. > > Add handling for checkpoints, end of recovery and p

Re: lastOverflowedXid does not handle transaction ID wraparound

2021-11-03 Thread Simon Riggs
nvalidTransactionId; > LWLockRelease(ProcArrayLock); > } So I agree with this fix. It is however, an undocumented modularity violation. I think that is acceptable because of the ProcArrayLock traffic, but needs to have a comment to explain this at the call to ExpireOldKnownAssignedTransactionIds() i.e. " and potentially reset lastOverflowedXid", as well as a comment on the ExpireOldKnownAssignedTransactionIds() function. -- Simon Riggshttp://www.EnterpriseDB.com/

Re: lastOverflowedXid does not handle transaction ID wraparound

2021-11-04 Thread Simon Riggs
On Wed, 3 Nov 2021 at 22:07, Alexander Korotkov wrote: > > Hi! > > On Wed, Nov 3, 2021 at 8:51 PM Simon Riggs > wrote: > > It is however, an undocumented modularity violation. I think that is > > acceptable because of the ProcArrayLock traffic, but needs to have a >

SKIP LOCKED assert triggered

2021-11-12 Thread Simon Riggs
yway), but this code is different from the pgbench test. -- Simon Riggshttp://www.EnterpriseDB.com/ skip_locked_assert.v1.patch Description: Binary data skip_locked_then_update_test.v1.patch Description: Binary data

Re: WIP: System Versioned Temporal Table

2021-11-15 Thread Simon Riggs
On Mon, 15 Nov 2021 at 09:47, Daniel Gustafsson wrote: > > > On 19 Sep 2021, at 20:32, Simon Riggs wrote: > > > My preferred approach would be to do this "for free" in the table > > access method, but we're a long way from this in terms of actual > >

Re: increase size of pg_commit_ts buffers

2021-11-30 Thread Simon Riggs
On Fri, 12 Nov 2021 at 17:39, Tomas Vondra wrote: > So +1 to just get this committed, as it is. This is a real issue affecting Postgres users. Please commit this soon. -- Simon Riggshttp://www.EnterpriseDB.com/

  1   2   3   4   5   6   7   8   >