On Fri, 22 Feb 2019 at 05:33, Robert Haas wrote:
>
> On Wed, Feb 20, 2019 at 4:56 PM David Rowley
> wrote:
> > I've made a pass over this again and updated the header comments in
> > functions that now obtain a lock to mention that fact.
>
> Thanks. I have committed this version. I know Tomas V
On 2/21/19 5:33 PM, Robert Haas wrote:
> On Wed, Feb 20, 2019 at 4:56 PM David Rowley
> wrote:
>> I've made a pass over this again and updated the header comments in
>> functions that now obtain a lock to mention that fact.
>
> Thanks. I have committed this version. I know Tomas Vondra was
> pl
On Wed, Feb 20, 2019 at 4:56 PM David Rowley
wrote:
> I've made a pass over this again and updated the header comments in
> functions that now obtain a lock to mention that fact.
Thanks. I have committed this version. I know Tomas Vondra was
planning to do that, but it's been close to a month s
Robert Haas writes:
> On Wed, Feb 20, 2019 at 3:57 PM Tom Lane wrote:
>> Looking at the patch itself, I agree that a bit more attention to comments
>> is needed, and I wonder whether David has found all the places where
>> it's now necessary to s/NoLock/RowExclusiveLock/. I don't have any
>> oth
On Thu, 21 Feb 2019 at 10:32, Robert Haas wrote:
> I spent some time thinking about that exact issue this morning and
> studying the code to try to figure that out. I wasn't able to find
> any other places that seemed to need updating, but it could be that I
> missed something that David also mis
On Wed, Feb 20, 2019 at 3:57 PM Tom Lane wrote:
> I agree that any deadlock would have to involve somebody doing something
> quite odd --- not just one partition-oriented operation, but something
> taking multiple strong locks without regard to the partition structure.
> So I don't see a problem w
Robert Haas writes:
> On Wed, Feb 20, 2019 at 11:03 AM Tom Lane wrote:
>> What I was wondering about was the possibility of the set of
>> tables-that-need-to-be-locked-at-all changing. Maybe that won't
>> create an issue either, but I'm not quite sure.
> That's pretty much what I was thinking,
On Wed, Feb 20, 2019 at 11:03 AM Tom Lane wrote:
> Right, that's the same thing I was trying to say.
OK, thanks.
> > ... So my question is - what do
> > you mean by the parenthetical note about the partitioning info not
> > changing? Regardless of whether it does or does not, I think the same
Robert Haas writes:
> On Mon, Feb 18, 2019 at 6:15 PM Tom Lane wrote:
>> I'm inclined to think that if we already have lock on the parent
>> partitioned table (thereby, IIUC, guaranteeing that its partitioning
>> info can't change) that the order in which we acquire the same lock
>> level on its
On Tue, Feb 19, 2019 at 4:07 PM David Rowley
wrote:
> I'd say that here we should only discuss what this patch is doing, ...
On that note, I spent some more time looking at what the patch is doing today.
/*
* We locked all the partitions in ExecSetupPartitionTupleRouting
* inclu
On Tue, Feb 19, 2019 at 4:07 PM David Rowley
wrote:
> I'd say that here we should only discuss what this patch is doing, not
> anything else that's in flight that you're concerned will conflict
> with the ATTACH/DETACH PARTITION CONCURRENTLY patch.
>
> During INSERT and UPDATE, not all partitions
On Wed, 20 Feb 2019 at 06:36, Robert Haas wrote:
>
> On Mon, Feb 18, 2019 at 6:15 PM Tom Lane wrote:
> > I'm inclined to think that if we already have lock on the parent
> > partitioned table (thereby, IIUC, guaranteeing that its partitioning
> > info can't change) that the order in which we acqu
On Mon, Feb 18, 2019 at 6:15 PM Tom Lane wrote:
> I'm inclined to think that if we already have lock on the parent
> partitioned table (thereby, IIUC, guaranteeing that its partitioning
> info can't change) that the order in which we acquire the same lock
> level on its partition(s) isn't very imp
On Fri, Feb 15, 2019 at 9:22 PM Andres Freund wrote:
> On 2019-01-31 13:46:33 -0500, Robert Haas wrote:
> > I have reviewed this patch and I am in favor of it. I think it likely
> > needs minor rebasing because of the heap_open -> table_open renaming.
> > I also agree that it's worth taking some
On Tue, 19 Feb 2019 at 12:15, Tom Lane wrote:
>
> David Rowley writes:
> > Looks like you're looking at the patch from the "Delay locking
> > partitions during query execution" thread [1]. Different thing
> > altogether, although the names are confusingly similar.
>
> My apologies --- I searched
On Tue, 19 Feb 2019 at 12:15, Tom Lane wrote:
> Now that I've looked at *this* thread's patch, I agree that having
> the executor acquire locks based on find_all_inheritors() is a
> pretty horrid idea. However, it still seems like this is being
> done in a vacuum without attention to locks alread
David Rowley writes:
> On Tue, 19 Feb 2019 at 11:47, Tom Lane wrote:
>> Are we talking about the same patch? The one I'm looking at seems
>> to be mostly planner and plancache changes:
> Looks like you're looking at the patch from the "Delay locking
> partitions during query execution" thread [
On Tue, 19 Feb 2019 at 11:47, Tom Lane wrote:
> Are we talking about the same patch? The one I'm looking at seems
> to be mostly planner and plancache changes:
>
> src/backend/catalog/dependency.c | 1 +
> src/backend/commands/createas.c | 1 +
> src/backend/executor/execUtils
Hi,
On 2019-02-18 17:47:07 -0500, Tom Lane wrote:
> David Rowley writes:
> > Not sure what you mean here. This is an executor change. What does
> > the planner care about what the executor does?
>
> Are we talking about the same patch? The one I'm looking at seems
> to be mostly planner and pl
David Rowley writes:
> On Tue, 19 Feb 2019 at 11:11, Tom Lane wrote:
>> What I'd like to understand about this patch is how it relates
>> to Amit L.'s work on making the planner faster for partitioned
>> UPDATE/DELETE cases (https://commitfest.postgresql.org/22/1778/).
>> I think that that might
On Tue, 19 Feb 2019 at 11:11, Tom Lane wrote:
>
> What I'd like to understand about this patch is how it relates
> to Amit L.'s work on making the planner faster for partitioned
> UPDATE/DELETE cases (https://commitfest.postgresql.org/22/1778/).
It's close to being unrelated and completely unrela
What I'd like to understand about this patch is how it relates
to Amit L.'s work on making the planner faster for partitioned
UPDATE/DELETE cases (https://commitfest.postgresql.org/22/1778/).
I think that that might render this moot? And if it doesn't,
how much does it really matter? You can't re
Hi,
On 2019-01-31 13:46:33 -0500, Robert Haas wrote:
> I have reviewed this patch and I am in favor of it. I think it likely
> needs minor rebasing because of the heap_open -> table_open renaming.
> I also agree that it's worth taking some deadlock risk for the rather
> massive performance gain,
On 1/25/19, Tomas Vondra wrote:
> Yes, I don't see why would the patch change that and I've been looking
> for such cases. I think David was looking at that this week too, and I
> assume he'll send an update if he finds anything. If not, I plan to get
> it committed soon-ish (possibly next week af
On Fri, Feb 1, 2019 at 9:16 AM David Rowley
wrote:
> On Sat, 2 Feb 2019 at 03:07, Robert Haas wrote:
> > I'm now wondering whether the same issues discussed in
> > https://www.postgresql.org/message-id/CA%2BTgmoZN-80143F8OhN8Cn5-uDae5miLYVwMapAuc%2B7%2BZ7pyNg%40mail.gmail.com
> > also need discus
On Sat, 2 Feb 2019 at 03:07, Robert Haas wrote:
> I'm now wondering whether the same issues discussed in
> https://www.postgresql.org/message-id/CA%2BTgmoZN-80143F8OhN8Cn5-uDae5miLYVwMapAuc%2B7%2BZ7pyNg%40mail.gmail.com
> also need discussion with respect to this patch. But I haven't
> thought ab
On Thu, Jan 31, 2019 at 4:48 PM David Rowley
wrote:
> On Fri, 1 Feb 2019 at 07:46, Robert Haas wrote:
> > I have reviewed this patch and I am in favor of it. I think it likely
> > needs minor rebasing because of the heap_open -> table_open renaming.
>
> Many thanks for looking at it. The v2 pa
On Fri, 1 Feb 2019 at 07:46, Robert Haas wrote:
> I have reviewed this patch and I am in favor of it. I think it likely
> needs minor rebasing because of the heap_open -> table_open renaming.
Many thanks for looking at it. The v2 patch was based on top of the
heap_open -> table_open change.
On Thu, Jan 24, 2019 at 4:43 PM Tomas Vondra
wrote:
> Yes, I don't see why would the patch change that and I've been looking
> for such cases. I think David was looking at that this week too, and I
> assume he'll send an update if he finds anything. If not, I plan to get
> it committed soon-ish (p
On 1/24/19 10:34 PM, John Naylor wrote:
> On Thu, Jan 24, 2019 at 4:17 PM Tomas Vondra
> wrote:
>> I can still see about the same performance as before (on both clusters).
>
> Thanks for checking! I think the thing to do now is have a committer
> look at it. It's a small patch with obvious per
On Thu, Jan 24, 2019 at 4:17 PM Tomas Vondra
wrote:
> I can still see about the same performance as before (on both clusters).
Thanks for checking! I think the thing to do now is have a committer
look at it. It's a small patch with obvious performance effects --
there's just the question of wheth
On 1/24/19 9:50 PM, John Naylor wrote:
> On Wed, Jan 23, 2019 at 7:56 PM David Rowley
> wrote:
>> On Thu, 24 Jan 2019 at 13:38, John Naylor
>> wrote:
>>> Hmm, now instead of an 85x speedup over master in the 10k partition
>>> case, I only get 20x. Anyone else see this?
>>
>> What's it like wit
On Wed, Jan 23, 2019 at 7:56 PM David Rowley
wrote:
> On Thu, 24 Jan 2019 at 13:38, John Naylor wrote:
> > Hmm, now instead of an 85x speedup over master in the 10k partition
> > case, I only get 20x. Anyone else see this?
>
> What's it like with fsync off?
No change. Just in case, I tried git b
On Thu, 24 Jan 2019 at 13:38, John Naylor wrote:
>
> On Tue, Jan 22, 2019 at 4:31 PM David Rowley
> wrote:
> > Thanks. I've attached a rebased version.
>
> Hmm, now instead of an 85x speedup over master in the 10k partition
> case, I only get 20x. Anyone else see this?
What's it like with fsync
On Tue, Jan 22, 2019 at 4:31 PM David Rowley
wrote:
> Thanks. I've attached a rebased version.
Hmm, now instead of an 85x speedup over master in the 10k partition
case, I only get 20x. Anyone else see this?
--
John Naylorhttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7
On Wed, 23 Jan 2019 at 04:35, John Naylor wrote:
> The cfbot reports this patch no longer applies.
Thanks. I've attached a rebased version.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
v2-0001-Delay-locking-of-partit
The cfbot reports this patch no longer applies.
--
John Naylorhttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
gards,
sho kato
> -Original Message-
> From: David Rowley [mailto:david.row...@2ndquadrant.com]
> Sent: Friday, January 18, 2019 7:41 PM
> To: Kato, Sho/加藤 翔
> Cc: PostgreSQL Hackers ; David
> Rowley
> Subject: Re: Delay locking partitions during INSERT and UPDATE
&
On Sat, 19 Jan 2019 at 12:05, John Naylor wrote:
>
> On 11/22/18, David Rowley wrote:
> > If required, such operations could LOCK TABLE the top partitioned
> > table to block the DML operation. There's already a risk of similar
> > deadlocks from such operations done on multiple separate tables w
On 1/20/19 5:45 AM, John Naylor wrote:
> On Sat, Jan 19, 2019 at 10:59 AM Tomas Vondra
> wrote:
>>
>> On 1/19/19 12:05 AM, John Naylor wrote:
>>> I used a similar test, but with unlogged tables, and "-c 2", and got:
>>>
>>> normal table: 32000tps
>>> 10k partitions / master: 82tps
>>> 10k partitio
On Sat, Jan 19, 2019 at 10:59 AM Tomas Vondra
wrote:
>
> On 1/19/19 12:05 AM, John Naylor wrote:
> > I used a similar test, but with unlogged tables, and "-c 2", and got:
> >
> > normal table: 32000tps
> > 10k partitions / master: 82tps
> > 10k partitions / patch: 7000tps
> >
> > So far I haven't
On 1/19/19 12:05 AM, John Naylor wrote:
> On 11/22/18, David Rowley wrote:
>> If required, such operations could LOCK TABLE the top partitioned
>> table to block the DML operation. There's already a risk of similar
>> deadlocks from such operations done on multiple separate tables when
>> the orde
On 11/22/18, David Rowley wrote:
> If required, such operations could LOCK TABLE the top partitioned
> table to block the DML operation. There's already a risk of similar
> deadlocks from such operations done on multiple separate tables when
> the order they're done is not the same as the order th
On Fri, 18 Jan 2019 at 19:08, sho kato wrote:
> I confirmed that this patch improve performance by 10 times or more.
Thanks for testing this out.
> Also, I did make installcheck world, but test partition_prune failed.
> However, this test case failed even before applying a patch, so this patch i
The following review has been posted through the commitfest application:
make installcheck-world: tested, failed
Implements feature: tested, failed
Spec compliant: tested, failed
Documentation:tested, failed
Hi,
Increasing the number of clients, I benchmarked with a t
Hi,
On 11/23/18 1:14 AM, David Rowley wrote:
> As a follow-on from [1] and also discussed in [2], I'd like to propose
> that we don't obtain locks on all partitions during INSERT into a
> partitioned table and UPDATE of a partitioned key and instead, only
> lock the partition when we first route a
46 matches
Mail list logo
