Re: suboverflowed subtransactions concurrency performance optimize

2021-11-30 Thread Simon Riggs
AX_CACHED_SUBXIDS. This would make subtrans much smaller and avoid one-entry-per-page which is a major source of cacheing. This would means some light changes in GetSnapshotData(). Let me know if that seems interesting also? -- Simon Riggshttp://www.EnterpriseDB.com/ subtrans_single_item_cache.v1.patch Description: Binary data

Re: SKIP LOCKED assert triggered

2021-12-01 Thread Simon Riggs
On Wed, 1 Dec 2021 at 14:33, Bossart, Nathan wrote: > > On 11/12/21, 8:56 AM, "Simon Riggs" wrote: > > The combination of these two statements in a transaction hits an > > Assert in heapam.c at line 4770 on REL_14_STABLE > > I've been unable to reproduce

Re: [HACKERS] Surjective functional indexes

2018-03-22 Thread Simon Riggs
een around the actual user interface to request it. Index option handling has changed (and this needs rebase!), but other than that I think we want this and am planning to commit something early next week. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-23 Thread Simon Riggs
On 2 February 2018 at 02:17, Michael Paquier wrote: > On Fri, Feb 02, 2018 at 12:21:49AM +0000, Simon Riggs wrote: >> Yes, it would be about 99% of the time. > > When it comes to recovery, I don't think that 99% is a guarantee > sufficient. (Wondering about the maths behin

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-23 Thread Simon Riggs
On 23 March 2018 at 08:35, Pavan Deolasee wrote: > > > On Fri, Mar 9, 2018 at 8:49 PM, Peter Eisentraut > wrote: >> >> On 2/1/18 19:21, Simon Riggs wrote: >> > If we really can't persuade you of that, it doesn't sink the patch. We >> > can have

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-23 Thread Simon Riggs
On 23 March 2018 at 09:22, Michael Paquier wrote: > On Fri, Mar 23, 2018 at 09:04:55AM +0000, Simon Riggs wrote: >> So it shows clear benefit for both bulk actions and OLTP, with no >> regressions. >> >> No objection exists to the approach used in the patch, so I&#x

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-23 Thread Simon Riggs
tests, and so far it >>> holds up. Peter, if you have the code and you consider it important that this subfeature is in PostgreSQL, why not post the code so we can commit it? Why would we repeat what has already been done? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] logical decoding of two-phase transactions

2018-03-23 Thread Simon Riggs
patch 4 so that this is a whole, committable patch * added comments to make abort and commit structs look same Attached patch is proposed for a separate, early commit as part of this -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Trai

Re: [HACKERS] Surjective functional indexes

2018-03-23 Thread Simon Riggs
tions it. Above that in RelationData we have other structures that are List of OIDs, so Alvaro's proposal make sense. That would simplify the code in ProjectionIsNotChanged() by just looping over the list of projection indexes rather than the list of indexes So please could you make the

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-24 Thread Simon Riggs
I suggest we focus on the engineering. I've not discussed this patch with Pavan offline. On 23 March 2018 at 23:32, Michael Paquier wrote: > On Fri, Mar 23, 2018 at 11:06:48AM +0000, Simon Riggs wrote: >> Your assumption that I would commit a new patch that was 29 mins old >&g

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-24 Thread Simon Riggs
On 24 March 2018 at 11:58, Robert Haas wrote: > On Sat, Mar 24, 2018 at 7:42 AM, Simon Riggs wrote: >> I suggest we focus on the engineering. I've not discussed this patch >> with Pavan offline. > > Well then proposing to commit moments after it's been posted is &g

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-24 Thread Simon Riggs
against b.foo and the other half against a.foo. I don't > know whether that will crash or bomb out with a strange error or just > make some unexpected modification to one of those tables, but the > behavior, even if not insecure, will certainly be wrong. MERGE uses multiple RTEs in

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-24 Thread Simon Riggs
On 24 March 2018 at 12:19, Robert Haas wrote: > On Sat, Mar 24, 2018 at 8:11 AM, Simon Riggs wrote: >> On 24 March 2018 at 11:58, Robert Haas wrote: >>> On Sat, Mar 24, 2018 at 7:42 AM, Simon Riggs wrote: >>>> I suggest we focus on the engineering. I've not d

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-26 Thread Simon Riggs
rror if they aren't. Since we now have MVCC catalog scans, all the name lookups are performed using the same snapshot so in the above scenario the newly created object would be invisible to the second name lookup. So I don't see anyway for the ERROR to occur and hence no need for a cross check, for UPDATE or MERGE. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-26 Thread Simon Riggs
v25 is the "commit candidate" and we can add other patches to it. Given recent bugfix/changes I don't plan to commit this tomorrow anymore. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-26 Thread Simon Riggs
o system column is allowed" 11. In comment "Since the plan re-evaluated by EvalPlanQual uses the second RTE", suggest using "join RTE" to make it more explicit which RTE we are discussing 12. Missed out merge.sgml from v25 patch. 13. For triggers we say &qu

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-26 Thread Simon Riggs
On 26 March 2018 at 16:09, Robert Haas wrote: > On Mon, Mar 26, 2018 at 5:53 AM, Simon Riggs wrote: >> Since we now have MVCC catalog scans, all the name lookups are >> performed using the same snapshot so in the above scenario the newly >> created object would be invisib

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-26 Thread Simon Riggs
On 26 March 2018 at 17:52, Robert Haas wrote: > On Mon, Mar 26, 2018 at 12:16 PM, Simon Riggs wrote: >> On 26 March 2018 at 16:09, Robert Haas wrote: >>> On Mon, Mar 26, 2018 at 5:53 AM, Simon Riggs wrote: >>>> Since we now have MVCC catalog scans, all the name lo

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-27 Thread Simon Riggs
On 26 March 2018 at 23:10, Peter Geoghegan wrote: > On Mon, Mar 26, 2018 at 12:17 PM, Simon Riggs wrote: >>> As far as I >>> know, the proposed MERGE patch has that issue an existing DML commands >>> don't; but someone else may have better information. >&

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-27 Thread Simon Riggs
On 26 March 2018 at 17:06, Simon Riggs wrote: > On 26 March 2018 at 15:39, Pavan Deolasee wrote: > > That's all I can see so far. * change comment “once to” to “once” in src/include/nodes/execnodes.h * change comment “and to run” to “and once to run” * change “result relati

Re: [HACKERS] logical decoding of two-phase transactions

2018-03-27 Thread Simon Riggs
On 23 March 2018 at 15:26, Simon Riggs wrote: > Reviewing 0003-Add-support-for-logging-GID-in-commit-abort-WAL-reco > > Looks fine, reworked patch attached > * added changes to xact.h from patch 4 so that this is a whole, > committable patch > * added comments to make abort

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-27 Thread Simon Riggs
de since the > mergrSourceTargetList only includes user columns and so set_plan_refs() > complains about a system column. > > I am not sure what's the best way to handle this. May be we can add system > columns to the mergrSourceTargetList. I haven't yet found a neat way to do > that. I was saying the comment needs changing, not the code. Cool, thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-27 Thread Simon Riggs
coverage of these ERROR messages Named security policy violation SELECT not allowed in MERGE INSERT... Multiple VALUES clauses not... MERGE is not supported for this... MERGE is not supported for relations with inheritance MERGE is not supported for relations with rules -- Simon Riggs

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-27 Thread Simon Riggs
On 27 March 2018 at 11:46, Simon Riggs wrote: > On 27 March 2018 at 10:31, Pavan Deolasee wrote: > >> Fixed in v26. > > More comments on v26 In terms of further performance optimization, if there is just one WHEN AND condition and no unconditional WHEN clauses then we ca

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-27 Thread Simon Riggs
gt;> 2. Does the new logic in pg_rewind to search backward for a checkpoint >> work reliably, and will it be slow? > > If you have to search backwards, this breaks it. Full stop. You don't have to search backwards. We only need to locate the last checkpoint record. -- Simon Ri

Re: [HACKERS] Surjective functional indexes

2018-03-27 Thread Simon Riggs
On 23 March 2018 at 15:54, Simon Riggs wrote: > So please could you make the change? Committed, but I still think that change would be good. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] logical decoding of two-phase transactions

2018-03-28 Thread Simon Riggs
On 28 March 2018 at 16:28, Nikhil Sontakke wrote: > Simon, 0003-Add-GID-and-replica-origin-to-two-phase-commit-abort.patch > is the exact patch that you had posted for an earlier commit. 0003 Pushed -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 S

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-28 Thread Simon Riggs
On 29 March 2018 at 07:37, Pavan Deolasee wrote: > > > On Tue, Mar 27, 2018 at 5:00 PM, Simon Riggs wrote: >> >> >> In terms of further performance optimization, if there is just one >> WHEN AND condition and no unconditional WHEN clauses then we can add >>

Re: Commit fest 2017-11

2018-03-29 Thread Simon Riggs
future of course, unless somebody wants to go > back and backfill existing patches with such information (which they might > be). The focus of this is on the Committers, which seems wrong. I suggest someone does another analysis that shows how many patch reviews have been conducted by patch authors, so we can highlight people who are causing the problem yet not helping solve the problem. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A design for amcheck heapam verification

2018-03-29 Thread Simon Riggs
CREATE INDEX afterwards anyway, which makes it more likely that people would just run CREATE INDEX and not bother with the check. So my initial questions are about when we would run this and making sure that is documented. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-29 Thread Simon Riggs
ill no doubt discover fairly soon after commit, such as additional runs by SQLsmith and more eyeballs. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-29 Thread Simon Riggs
e. I know the approach is new and surprising but I thought about it a lot before proposing it and I couldn't see any holes; still can't. Please give this some thought so we can get comfortable with this idea and increase performance as a result. Thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-29 Thread Simon Riggs
On 29 March 2018 at 18:13, Tomas Vondra wrote: > On 03/29/2018 06:42 PM, Tom Lane wrote: >> Simon Riggs writes: >>> I know the approach is new and surprising but I thought about it a lot >>> before proposing it and I couldn't see any holes; still can't. Pl

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-29 Thread Simon Riggs
/boundaries) >> >> That seems very much like xl_prev. > > I don't think so, because this ignores, for example, timeline > switches, or multiple clusters accidentally sharing an archive > directory. I'm not hearing any actual technical problems. > Given where we are

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-29 Thread Simon Riggs
in fact >> strong enough that it's good enough for government work. He might be > > Is he? I think the claims in this thread were pretty much that xl_curr > and xl_prev provide the same level of protection. Yes, the blockchain analogy breaks down because we don't include p

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-30 Thread Simon Riggs
gain the observation that they are no longer used for replication and if they are used as well as replication, have a bad effect on performance. I think it would be easily possible to add some more detail to the WAL stream if needed. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-30 Thread Simon Riggs
On 29 March 2018 at 10:50, Simon Riggs wrote: > On 28 March 2018 at 12:00, Pavan Deolasee wrote: > >> v27 attached, though review changes are in >> the add-on 0005 patch. > > This all looks good now, thanks for making all of those changes. > > I propose [v27 patch1

Re: hot_standby_feedback vs excludeVacuum and snapshots

2018-04-01 Thread Simon Riggs
VACUUM FULL was the thing that needed to be excluded in the past because it needed an xid to move rows. Greg's concern is a good one and his noticing that we hadn't specifically excluded VACUUMs is valid, so we should exclude them. Well spotted, Greg. So although this doesn't have t

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-01 Thread Simon Riggs
e logically >> decoded. >> + * We could optimize this away when !RelationIsLogicallyLogged(rel) >> + * but that doesn't save much space or time. > > What you're saying isn't that you're not logging anything, but that > you're allocating the

Re: [HACKERS] logical decoding of two-phase transactions

2018-04-02 Thread Simon Riggs
r make an unknown task happen automatically? We can't. We have a reasonable approach here. Locking shared resources before using them is not a radical new approach, its just standard development. If we find a better way in the future, we can use that, but requiring a better solution wh

Re: [HACKERS] logical decoding of two-phase transactions

2018-04-02 Thread Simon Riggs
; which would indicate this fact - logical decoding is using it's own > snapshots it could inject the information about being inside the 2PC > decoding. Perhaps, but how do we know we've covered all the right places? We don't know what every plugin will require, do we? The pl

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-02 Thread Simon Riggs
>>> >>> >> >> Why is this a good idea? > > I think it seemed like a good idea at the time, so to speak, but several > people have argued against it, so we should probably change this in the > final version. Who has argued aga

Re: [PATCH] Logical decoding of TRUNCATE

2018-04-04 Thread Simon Riggs
s reasonable to treat it > specially. Real DDL is being worked on in the 2PC decoding thread. > What we are discussing here isn't going to be applicable there and vice > versa, I think. In fact, one of the reasons for this effort is that in > BDR TRUNCATE is currently handled like

Re: Add support for printing/reading MergeAction nodes

2018-04-04 Thread Simon Riggs
mt? Could you explain what command you think should be supported? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Add support for printing/reading MergeAction nodes

2018-04-04 Thread Simon Riggs
d Delete and these could be output in various debug modes. I'm not clear what meaning we might attach to them if we looked since that differs from normal INSERTs, UPDATEs, DELETEs, but lets see. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Add support for printing/reading MergeAction nodes

2018-04-04 Thread Simon Riggs
On 4 April 2018 at 18:51, Tom Lane wrote: > Simon Riggs writes: >> On 4 April 2018 at 17:19, Tom Lane wrote: >>> If the MERGE patch has broken this, I'm going to push back on that >>> and push back on it hard, because it probably means there are a >>>

Re: pgsql: New files for MERGE

2018-04-04 Thread Simon Riggs
. There has been no discussion of what exactly would be better, only that what we have is somehow wrong, a point which both Pavan and I dispute, not least because the executor has already been rewritten once at Peter's request. I was under no pressure at all to commit this. In my opinion this

Re: pgsql: New files for MERGE

2018-04-04 Thread Simon Riggs
gnored those requests. Pavan tells me he has replied to you and is working on specific changes. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-05 Thread Simon Riggs
e. (Now that the patch is committed, I don't know if there > would be a rethink about changing file names. May be not, just raising that > concern) My review notes suggest a file called execMerge.c. I didn't spot the filename change. I think it's important to do that because there is no executor node called Merge. That is especially confusing because there *is* an executor node called MergeAppend and we want some cognitive distance between those things. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: pgsql: New files for MERGE

2018-04-05 Thread Simon Riggs
On 4 April 2018 at 21:28, Simon Riggs wrote: > On 4 April 2018 at 21:14, Andres Freund wrote: > >>> The normal way is to make review comments that allow change. Your >>> request for change of the parser data structures is fine and can be >>> done, possibly by Sat

Re: Add support for printing/reading MergeAction nodes

2018-04-05 Thread Simon Riggs
ags. No problem there too. OK, so $OP fixed. > This now also enforces single VALUES clause in the grammar itself instead of > doing that check at parse-analyse time. So that's a net improvement too. OK, that's good. I've updated the docs to show this restriction correctly.

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-05 Thread Simon Riggs
On 5 April 2018 at 12:38, Jesper Pedersen wrote: > Hi Simon and Paven, > > On 04/04/2018 08:46 AM, Jesper Pedersen wrote: >> >> On 03/30/2018 07:10 AM, Simon Riggs wrote: >>> >>> No problems found, but moving proposed commit to 2 April pm >>>

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-05 Thread Simon Riggs
On 5 April 2018 at 12:56, Jesper Pedersen wrote: > Hi, > > On 04/05/2018 07:48 AM, Simon Riggs wrote: >>> >>> Updated version due to latest refactoring. >> >> >> Thanks for your input. Removing that seems to prevent compilation. >> >> Did s

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-05 Thread Simon Riggs
On 5 April 2018 at 13:19, Jesper Pedersen wrote: > Hi, > > On 04/05/2018 08:04 AM, Simon Riggs wrote: >> >> On 5 April 2018 at 12:56, Jesper Pedersen >> wrote: >>> >>> Updated for non-assert build. >> >> >> Thanks, pushed. Sorry to h

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-05 Thread Simon Riggs
.co.jp > > and his suggestion to use special macro looks better for me: > - charrelkind; > + charrelkind PG_USED_FOR_ASSERTS_ONLY; Thanks both, I already fixed that. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Excessive PostmasterIsAlive calls slow down WAL redo

2018-04-05 Thread Simon Riggs
is would only > affect the case where we're actively replaying records. +1 -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-05 Thread Simon Riggs
List **namespace); > ^ How's this as a fix? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services fnamespace.v1.patch Description: Binary data

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-05 Thread Simon Riggs
On 5 April 2018 at 17:07, Alvaro Herrera wrote: > Simon Riggs wrote: >> On 5 April 2018 at 16:09, Alvaro Herrera wrote: >> > Quick item: parse_clause.h fails cpluspluscheck because it has a C++ >> > keyword as a function argument name: >> > >> > ./s

Re: pgsql: New files for MERGE

2018-04-06 Thread Simon Riggs
On 5 April 2018 at 21:02, Bruce Momjian wrote: > On Thu, Apr 5, 2018 at 11:15:20AM +0100, Simon Riggs wrote: >> On 4 April 2018 at 21:28, Simon Riggs wrote: >> > On 4 April 2018 at 21:14, Andres Freund wrote: >> > >> >>> The normal way is to ma

Re: [HACKERS] MERGE SQL Statement for PG11

2018-04-06 Thread Simon Riggs
think that people could spend their time better - and issue that concerns me also. But that certainly doesn't apply to parts of the code like this where we have full test coverage. It may not even apply to recovery now we have the ability to check in real-time the results of recovery and

Re: Add support for printing/reading MergeAction nodes

2018-04-06 Thread Simon Riggs
On 4 April 2018 at 18:08, Tom Lane wrote: > Simon Riggs writes: >> On 4 April 2018 at 17:19, Tom Lane wrote: >>> BTW, poking around in the grammar, I notice that MergeStmt did not >>> get added to RuleActionStmt. That seems like a rather serious >>> omiss

Re: pgsql: New files for MERGE

2018-04-07 Thread Simon Riggs
On 6 April 2018 at 17:22, Bruce Momjian wrote: > On Fri, Apr 6, 2018 at 09:21:54AM +0100, Simon Riggs wrote: >> On 5 April 2018 at 21:02, Bruce Momjian wrote: >> > Simon, you have three committers in this thread suggesting this patch be >> > reverted. Are you just

Re: pgsql: New files for MERGE

2018-04-11 Thread Simon Riggs
On 7 April 2018 at 18:45, Tom Lane wrote: > Simon Riggs writes: >> On 6 April 2018 at 17:22, Bruce Momjian wrote: >>> My point was that people didn't ask you to work harder on fixing the >>> patch, but in reverting it. You can work harder on fixing things i

Re: Bugs in TOAST handling, OID assignment and redo recovery

2018-04-11 Thread Simon Riggs
On 11 April 2018 at 19:57, Tom Lane wrote: > Pavan Deolasee writes: >> Ok. I propose attached patches, more polished this time. > > I'll take these, unless some other committer is hot to do so? Please go ahead. -- Simon Riggshttp://www.2ndQuadrant.com/ Po

Re: reloption to prevent VACUUM from truncating empty pages at the end of relation

2018-04-18 Thread Simon Riggs
lock itself can be cancelled, so the user would also be interested in explicitly requesting a retry with a separate command/function. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Query is over 2x slower with jit=on

2018-04-18 Thread Simon Riggs
ns before this gets too far into the wild. SSL is somewhat understandable because its not a Postgres-private term. geqo is regrettable and we really don't want any more too short/abbreviated parameter names. Think of our EOU if every GUC was a TLA. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Fix of fake unlogged LSN initialization

2019-10-24 Thread Simon Riggs
> So it seems to me that you have caught a bug here, and that we had > better back-patch to v12 so as recovery and pg_resetwal don't mess up > with AMs using lower values than that. > I wonder why is that value 1000, rather than an aligned value or a whole WAL page? -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Solutions for the Enterprise

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2019-03-21 Thread Simon Riggs
by code at 52ac6cd2d0cd70e01291e0ac4ee6d068b69bc478 or later then we will have problems, since deleteXid will not be set correctly. It seems this should not have been backpatched. Please give your assessment. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2019-03-21 Thread Simon Riggs
On Thu, 21 Mar 2019 at 15:18, Alexander Korotkov wrote: > On Thu, Mar 21, 2019 at 9:26 PM Simon Riggs wrote: > > > > It's been pointed out to me that 52ac6cd2d0cd70e01291e0ac4ee6d068b69bc478 > > introduced a WAL incompatibility that has not been flagged. > > &

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Simon Riggs
s exactly right. But ISTM that LIMIT itself is the issue there and it need more smarts to correctly calculate costs. I don't see LIMIT costing being broken as a reason to restrict this optimization. I would ask that we allow improvements to the important use case of ORDER BY/LIMIT, then spend time on making LIMIT work correctly. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Ordered Partitioned Table Scans

2019-03-22 Thread Simon Riggs
On Fri, 22 Mar 2019 at 11:39, Tom Lane wrote: > Simon Riggs writes: > > I agree that the issue of mixing sorts at various points will make > nonsense > > of the startup cost/total cost results. > > Right. > > > I don't see LIMIT costing being broken as a re

Error message inconsistency

2019-03-22 Thread Simon Riggs
is easy to fix once/if we agree to change. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services rationalize_constraint_error_messages.v1.patch Description: Binary data

Re: Error message inconsistency

2019-03-24 Thread Simon Riggs
On Sun, 24 Mar 2019 at 13:02, Amit Kapila wrote: > On Sat, Mar 23, 2019 at 4:33 AM Fabrízio de Royes Mello > wrote: > > > > On Fri, Mar 22, 2019 at 2:25 PM Simon Riggs > wrote: > > > > > > As noted by a PostgreSQL user to me, error messages for NOT NULL &

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-28 Thread Simon Riggs
preference, then I say we should have numbers. Numbering is natural for people. If we say "It's currently doing phase XYZ", they will say "Is that the 3rd phase?", we'll say "No, actually the 5th", and then they will say "Why didn't you just num

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-28 Thread Simon Riggs
On Thu, 28 Mar 2019 at 15:39, Alvaro Herrera wrote: > On 2019-Mar-28, Simon Riggs wrote: > > > On Thu, 28 Mar 2019 at 14:56, Alvaro Herrera > > wrote: > > > > > I have not reinstated phase numbers; I have Rahila's positive vote for > > > them. Do

Re: propagating replica identity to partitions

2019-03-28 Thread Simon Riggs
o the region administrator, down to the city administrators. > That use case is possible using different privileges. Having different owners makes it *very* difficult to administer. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Dev

Re: propagating replica identity to partitions

2019-03-28 Thread Simon Riggs
could recurse by default. IMHO this should be renamed to ALTER TABLE ... MOVE TO TABLESPACE, so its actual effect is clearer. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Simon Riggs
ne less heap fetch we need to perform when we delete the page - it's possible we optimize that away completely by doing this. Since this point of the code is clearly going to be a performance issue it seems like something we should do now. -- Simon Riggshttp://www.2ndQuadrant

Re: propagating replica identity to partitions

2019-03-29 Thread Simon Riggs
On Fri, 29 Mar 2019 at 09:51, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-03-28 18:16, Simon Riggs wrote: > > SET TABLESPACE should not recurse because it copies the data, while > > holding long locks. If that was ever fixed so it happened concurrent

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Simon Riggs
On Fri, 29 Mar 2019 at 15:29, Andres Freund wrote: > On 2019-03-29 09:37:11 +0000, Simon Riggs wrote: > > > While trying to understand this, I see there is an even better way to > > optimize this. Since we are removing dead index tuples, we could alter > the > > k

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-29 Thread Simon Riggs
On Fri, 29 Mar 2019 at 16:12, Andres Freund wrote: > On 2019-03-29 15:58:14 +0000, Simon Riggs wrote: > > On Fri, 29 Mar 2019 at 15:29, Andres Freund wrote: > > > That's far from a trivial feature imo. It seems quite possible that > we'd > > > end

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-31 Thread Simon Riggs
On Fri, 29 Mar 2019 at 16:32, Andres Freund wrote: > On 2019-03-29 16:20:54 +0000, Simon Riggs wrote: > > On Fri, 29 Mar 2019 at 16:12, Andres Freund wrote: > > > > > > > On 2019-03-29 15:58:14 +, Simon Riggs wrote: > > > > On Fri, 29

Re: pgbench - allow to create partitioned tables

2019-07-23 Thread Simon Riggs
o parts = 0.071 ms > # 1 hash = 0.071 ms (did someone optimize this case?!) > # 2 hash ~ 0.126 ms (+ 0.055 ms) > # 50 hash ~ 0.155 ms > # 100 hash ~ 0.178 ms > # 150 hash ~ 0.232 ms > # 200 hash ~ 0.279 ms > # overhead ~ (0.050 + [0.0005-0.0008] *

Re: Duplicated LSN in ReorderBuffer

2019-07-28 Thread Simon Riggs
p the one > in XLOG_HEAP2_MULTI_INSERT. > Fix proposed by Petr, with comments as explained by Andres. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Solutions for the Enterprise allow_XLOG_HEAP2_NEW_CID_while_building_snapshot.v1.patch Description: Binary data

Re: Problem with default partition pruning

2019-08-07 Thread Simon Riggs
lease could we do perf checks, with tests up to 1000s of partitions? And if there is a regression, I would vote to revoke this patch or address the request in a less general way. Hopefully I have misunderstood and/or there is no regression. -- Simon Riggshttp://www.2ndQuadrant.com/ <http://www.2ndquadrant.com/> PostgreSQL Solutions for the Enterprise

Re: Problem with default partition pruning

2019-08-08 Thread Simon Riggs
On Wed, 7 Aug 2019 at 21:27, Alvaro Herrera wrote: > On 2019-Aug-07, Simon Riggs wrote: > > > I saw your recent commit and it scares me in various places, noted below. > > > > "Commit: Apply constraint exclusion more generally in partitioning" > > >

Re: Problem with default partition pruning

2019-08-12 Thread Simon Riggs
On Mon, 12 Aug 2019 at 18:45, Alvaro Herrera wrote: > I think that should appease > Simon's performance concern for the most common case of default > partition not existing. > Much appreciated, thank you. -- Simon Riggshttp://www.2ndQuadrant.com/ <http:/

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-04 Thread Simon Riggs
On 29 December 2017 at 19:55, Andres Freund wrote: > Hi, > > On 2017-12-29 14:15:22 +0100, Marco Nenciarini wrote: >> This patch implements support for TRUNCATE statements >> in logical replication. The work has mainly done by Simon Riggs then >> finished by

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-04 Thread Simon Riggs
On 4 January 2018 at 17:29, Robert Haas wrote: > On Sat, Dec 30, 2017 at 6:01 AM, Simon Riggs wrote: >> Patch uses mechanism as agreed previously with Peter G et al. on this thread. > > I'm not sure that an agreement was reached, or what the substance of > that agreemen

Re: [HACKERS] Timeline ID in backup_label file

2018-01-05 Thread Simon Riggs
ood so I will mark it ready for committer. Sounds good. No tests? No docs/extended explanatory comments? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Fix permissions check on pg_stat_get_wal_senders

2018-01-05 Thread Simon Riggs
y for > committer. Simon, as the original committer of 25fff407, could you look > at what is proposed here? Yup, I got this. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Creating backup history files for backups taken from standbys

2018-01-05 Thread Simon Riggs
re >> still written to disk, so my take on the matter is to keep the code >> simple. > > I'm OK with that. I'm not. If we want to do this why not only do it in the modes that have meaning? i.e. put an if() test in for archive_mode == always Which also makes it a smaller and clearer patch -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: FOR EACH ROW triggers on partitioned tables

2018-01-06 Thread Simon Riggs
n make pg_dump do the right things. That's all mostly legwork, I > think. > > Also, does ALTER TABLE ... ENABLE/DISABLE TRIGGER do the right things on > partitioned tables? Not sure I care about that, since it just breaks FKs and other things, but we can add it later. -- S

Re: unique indexes on partitioned tables

2018-01-06 Thread Simon Riggs
. I agree we want 0004 also, but it would be simpler to just push 0002 and 0003 now and come back later for 0004. That would also avoid any confusion over patch credits. > This serves as basis to build foreign keys on top; I'll post that > separately. -- Simon Riggshttp://w

Re: Rangejoin rebased

2018-01-06 Thread Simon Riggs
e want this, it might be better for the EXPLAIN to say "Time Range Join" when the ranges being joined are Time Ranges, and for other cases to just say "Range Join". The use of the word Merge doesn't help much there. -- Simon Riggshttp://www.2ndQuadrant

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-06 Thread Simon Riggs
as needed So the idea is we enable Postgres to allow major new functionality, as was done for PostGIS so successfully. We can adopt syntax into the main parser later once SQLStandard accepts this, or some munged version of it. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL D

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-07 Thread Simon Riggs
ADME explains the ALIGN operation step-by-step with a > TEMPORAL LEFT OUTER JOIN example. That is, we start from a query > input, show how we rewrite it during parser stage, and show how the > final execution generates result tuples. Sorry, this was too complex for me. Can we get a muc

Re: [HACKERS] Replication status in logical replication

2018-01-07 Thread Simon Riggs
this removed (v4alt2) I've removed the edit that fusses over English grammar: both ways are correct. > I think this patch can be > back-patched to 9.4 as Simon mentioned. This patch appears to cause this DEBUG1 message "standby \"%s\" has now caught up with primary&q

Re: Logical decoding fast-forward and slot advance

2018-01-07 Thread Simon Riggs
ere to make sure we understand and avoid breakage. In pg_replication_slot_advance() the moveto variable is set to PG_GETARG_LSN(1) and then unconditionally overwritten before it is used for anything. Why? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Statement-level rollback

2018-01-07 Thread Simon Riggs
On 6 November 2017 at 12:36, MauMau wrote: > when I submit the next revision of my patch. When will the next version be posted? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

<    1   2   3   4   5   6   7   8   >