Erik Rijkers wrote:
Everything together: the raid is what Areca call 'raid10(1E)'.
(to be honest I don't remember what that 1E exactly means -
extra flexibility in the number of disks, I think).
Standard RAID10 only supports an even number of disks. The 1E variants
also allow putting an od
On Tue, 2010-05-04 at 21:34 +0200, Stefan Kaltenbrunner wrote:
> FWIW - I'm seeing a behaviour here under pgbench -S workloads that looks
> kinda related.
>
> using -j 16 -c 16 -T 120 I get either 10tps and around 66
> contextswitches per second or on some runs I end up with 15tps a
On Tue, May 4, 2010 20:26, Greg Smith wrote:
> Erik Rijkers wrote:
>> OS: Centos 5.4
>> 2 quadcores: Intel(R) Xeon(R) CPU X5482 @ 3.20GHz
>> Areca 1280ML
>> primary and standby db both on a 12 disk array (sata 7200rpm, Seagat
>> Barracuda ES.2)
>>
>
> To fill in from data you already menti
Erik Rijkers wrote:
Hi Simon,
In another thread you mentioned you were lacking information from me:
On Tue, May 4, 2010 17:10, Simon Riggs wrote:
There is no evidence that Erik's strange performance has anything to do
with HS; it hasn't been seen elsewhere and he didn't respond to
questions ab
Erik Rijkers wrote:
OS: Centos 5.4
2 quadcores: Intel(R) Xeon(R) CPU X5482 @ 3.20GHz
Areca 1280ML
primary and standby db both on a 12 disk array (sata 7200rpm, Seagat
Barracuda ES.2)
To fill in from data you already mentioned upthread:
32 GB RAM
CentOS release 5.4 (Final), x86_64 Li
On Tue, May 4, 2010 18:19, Simon Riggs wrote:
> On Tue, 2010-05-04 at 18:10 +0200, Erik Rijkers wrote:
>> It would be interesting if anyone repeated these simple tests and
>> produced evidence that these non-HS.
>>
>> (Unfortunately, I have at the moment not much time for more testing)
>
> Would yo
On Tue, 2010-05-04 at 18:10 +0200, Erik Rijkers wrote:
> It would be interesting if anyone repeated these simple tests and
> produced
> evidence that these non-HS.
>
> (Unfortunately, I have at the moment not much time for more testing)
Would you be able to make those systems available for furthe
Hi Simon,
In another thread you mentioned you were lacking information from me:
On Tue, May 4, 2010 17:10, Simon Riggs wrote:
>
> There is no evidence that Erik's strange performance has anything to do
> with HS; it hasn't been seen elsewhere and he didn't respond to
> questions about the test se
On Tue, 2010-04-27 at 20:13 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > On Tue, 2010-04-27 at 18:08 -0400, Tom Lane wrote:
> >> Huh? How is a filter as coarse as an oldest-running-XID filter going
> >> to prevent that? And aren't we initializing from trustworthy data in
> >> ProcArrayApplyR
Simon Riggs writes:
> On Tue, 2010-04-27 at 18:08 -0400, Tom Lane wrote:
>> Huh? How is a filter as coarse as an oldest-running-XID filter going
>> to prevent that? And aren't we initializing from trustworthy data in
>> ProcArrayApplyRecoveryInfo, anyway?
>>
>> I still say it's useless.
> Quit
On Tue, 2010-04-27 at 18:08 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > On Tue, 2010-04-27 at 17:24 -0400, Tom Lane wrote:
> >> I think we should just lose that test, as well as the variable.
>
> > Yes, though it looks like it is still necessary in creating a valid
> > initial state because
Simon Riggs writes:
> On Tue, 2010-04-27 at 17:24 -0400, Tom Lane wrote:
>> I think we should just lose that test, as well as the variable.
> Yes, though it looks like it is still necessary in creating a valid
> initial state because otherwise we may have xids in KnownAssigned array
> that are al
On Tue, 2010-04-27 at 17:24 -0400, Tom Lane wrote:
> Isn't the snapshotOldestActiveXid filter in
> RecordKnownAssignedTransactionIds completely wrong/useless/bogus?
>
> AFAICS, snapshotOldestActiveXid is only set once at the start of
> recovery. This means it will soon be too old to provide any
Isn't the snapshotOldestActiveXid filter in
RecordKnownAssignedTransactionIds completely wrong/useless/bogus?
AFAICS, snapshotOldestActiveXid is only set once at the start of
recovery. This means it will soon be too old to provide any useful
filtering. But what's far worse is that the XID space
On Tue, 2010-04-27 at 16:18 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > On Tue, 2010-04-27 at 13:52 -0400, Tom Lane wrote:
> >> WTF? Either the comment is wrong or this should not be an elog
> >> condition.
>
> > That section of code has been rewritten many times. I think it is now
> > inac
Simon Riggs writes:
> On Tue, 2010-04-27 at 13:52 -0400, Tom Lane wrote:
>> WTF? Either the comment is wrong or this should not be an elog
>> condition.
> That section of code has been rewritten many times. I think it is now
> inaccurate and should be removed. I left it there because the
> unfor
On Tue, 2010-04-27 at 14:53 -0400, Tom Lane wrote:
> Hmm ... there's another point here, which is that the array size
> creates
> a hard maximum on the number of entries, whereas the hash table was a
> bit more forgiving. What is the proof that the array won't overflow?
> The fact that the equival
Hmm ... there's another point here, which is that the array size creates
a hard maximum on the number of entries, whereas the hash table was a
bit more forgiving. What is the proof that the array won't overflow?
The fact that the equivalent data structure on the master can't hold
more than this ma
On Tue, 2010-04-27 at 13:52 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > v3 attached
>
> This patch changes KnownAssignedXidsRemove() so that failure to find
> the target XID is elog(ERROR) (ie, a PANIC, since this is in the
> startup process).
Not in all cases. The code is correct, as far
Simon Riggs writes:
> v3 attached
This patch changes KnownAssignedXidsRemove() so that failure to find
the target XID is elog(ERROR) (ie, a PANIC, since this is in the
startup process). However, this comment is still there:
/*
* We can fail to find an xid if the xid came from a
Simon Riggs writes:
> v3 attached
Thanks, will work on this tomorrow.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, 2010-04-25 at 13:51 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > On Sun, 2010-04-25 at 13:33 -0400, Tom Lane wrote:
> >> If you like I'll have a go at rewriting the comments for this patch,
> >> because I am currently thinking that the problem is not so much with
> >> the code as with
On Mon, April 26, 2010 09:43, Simon Riggs wrote:
> On Sun, 2010-04-25 at 23:52 +0200, Erik Rijkers wrote:
>
>> I'll try to repeat this pattern on other hardware; although
>> if my tests were run with faulty hardware I wouldn't know how/why
>> that would give the above effect (such a 'regular aberra
On Mon, April 26, 2010 08:52, Fujii Masao wrote:
> On Mon, Apr 26, 2010 at 3:25 AM, Erik Rijkers wrote:
>> FWIW, here are some more results from pgbench comparing
>> primary and standby (both with Simon's patch).
>
> Was there a difference in CPU utilization between the primary
> and standby?
>
On Sun, 2010-04-25 at 23:52 +0200, Erik Rijkers wrote:
> I'll try to repeat this pattern on other hardware; although
> if my tests were run with faulty hardware I wouldn't know how/why
> that would give the above effect (such a 'regular aberration').
> testing is more difficult than I thought...
On Sun, 2010-04-25 at 19:18 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > [ v2 patch ]
>
> I've been studying this some more while making notes for improved
> comments, and I've about come to the conclusion that having readers
> move the tail pointer (at the end of KnownAssignedXidsGetAndSetXm
On Mon, Apr 26, 2010 at 3:25 AM, Erik Rijkers wrote:
> FWIW, here are some more results from pgbench comparing
> primary and standby (both with Simon's patch).
Was there a difference in CPU utilization between the primary
and standby?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE COR
Simon Riggs writes:
> [ v2 patch ]
I've been studying this some more while making notes for improved
comments, and I've about come to the conclusion that having readers
move the tail pointer (at the end of KnownAssignedXidsGetAndSetXmin)
is overly tricky and probably not a performance improvement
On Sun, April 25, 2010 20:55, Tom Lane wrote:
>
> That seems weird. Why do most of the runs show primary and standby
> as having comparable speed, but a few show the standby as much slower?
> The parameters for those runs don't seem obviously different from cases
> where it's fast. I think there
On Sun, 2010-04-25 at 20:25 +0200, Erik Rijkers wrote:
> Sorry if it's too much data, but to me at least it was illuminating;
> I now understand the effects of the different parameters better.
That's great, many thanks.
A few observations
* Standby performance is actually slightly above normal
"Erik Rijkers" writes:
> FWIW, here are some more results from pgbench comparing
> primary and standby (both with Simon's patch).
That seems weird. Why do most of the runs show primary and standby
as having comparable speed, but a few show the standby as much slower?
The parameters for those run
On Sat, April 24, 2010 01:17, Erik Rijkers wrote:
> On Sat, April 24, 2010 00:39, Simon Riggs wrote:
>> On Fri, 2010-04-23 at 11:32 -0400, Robert Haas wrote:
>>> >
>>> > 99% of transactions happen in similar times between primary and standby,
>>> > everything dragged down by rare but severe spikes.
Simon Riggs writes:
> On Sun, 2010-04-25 at 13:33 -0400, Tom Lane wrote:
>> If you like I'll have a go at rewriting the comments for this patch,
>> because I am currently thinking that the problem is not so much with
>> the code as with the poor explanation of what it's doing. Sometimes
>> the au
On Sun, 2010-04-25 at 13:33 -0400, Tom Lane wrote:
> If you like I'll have a go at rewriting the comments for this patch,
> because I am currently thinking that the problem is not so much with
> the code as with the poor explanation of what it's doing. Sometimes
> the author is too close to the c
Simon Riggs writes:
> On Sun, 2010-04-25 at 12:51 -0400, Tom Lane wrote:
>> If the comments were correct, I wouldn't be complaining. They're
>> misleading or outright wrong on many points. In particular, I don't
>> think you actually understand the weak-memory-ordering issue, because
>> the comm
On Sun, 2010-04-25 at 12:51 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > On Sun, 2010-04-25 at 11:50 -0400, Tom Lane wrote:
> >> This needs a redesign before it can be considered committable. I don't
> >> really care whether it makes things faster; it's too complicated and too
> >> poorly doc
On Sun, 2010-04-25 at 12:43 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > [ v2 patch ]
>
> BTW, while I'm looking at this, why bother with the separate
> KnownAssignedXidsValid[] array? Wouldn't it be cheaper
> (certainly so in storage, probably so in access/update times)
> to have just the K
Simon Riggs writes:
> On Sun, 2010-04-25 at 11:50 -0400, Tom Lane wrote:
>> This needs a redesign before it can be considered committable. I don't
>> really care whether it makes things faster; it's too complicated and too
>> poorly documented to be maintainable.
> There are more than 60 lines o
On Sun, 2010-04-25 at 11:50 -0400, Tom Lane wrote:
> Robert Haas writes:
> > On Sat, Apr 24, 2010 at 5:17 AM, Simon Riggs wrote:
> > Both Heikki and I objected to that patch.
> >>
> >> Please explain your objection, based upon the patch and my explanations.
>
> > Well, we objected to the lockin
Simon Riggs writes:
> [ v2 patch ]
BTW, while I'm looking at this, why bother with the separate
KnownAssignedXidsValid[] array? Wouldn't it be cheaper
(certainly so in storage, probably so in access/update times)
to have just the KnownAssignedXids[] array and store
InvalidTransactionId in unused
Robert Haas writes:
> On Sat, Apr 24, 2010 at 5:17 AM, Simon Riggs wrote:
> Both Heikki and I objected to that patch.
>>
>> Please explain your objection, based upon the patch and my explanations.
> Well, we objected to the locking.
Not only is the locking overly complex, but it's outright wro
On Sun, Apr 25, 2010 at 8:50 AM, Simon Riggs wrote:
> On Sun, 2010-04-25 at 06:53 -0400, Robert Haas wrote:
>> On Sat, Apr 24, 2010 at 5:17 AM, Simon Riggs wrote:
>> >> Both Heikki and I objected to that patch.
>> >
>> > Please explain your objection, based upon the patch and my explanations.
>>
On Sun, 2010-04-25 at 06:53 -0400, Robert Haas wrote:
> On Sat, Apr 24, 2010 at 5:17 AM, Simon Riggs wrote:
> >> Both Heikki and I objected to that patch.
> >
> > Please explain your objection, based upon the patch and my explanations.
>
> Well, we objected to the locking. Having reread the patc
On Sat, Apr 24, 2010 at 5:17 AM, Simon Riggs wrote:
>> Both Heikki and I objected to that patch.
>
> Please explain your objection, based upon the patch and my explanations.
Well, we objected to the locking. Having reread the patch a few times
though, I think I'm starting to wrap my head around
On Fri, 2010-04-23 at 19:07 -0400, Robert Haas wrote:
> On Fri, Apr 23, 2010 at 6:39 PM, Simon Riggs wrote:
> > On Fri, 2010-04-23 at 11:32 -0400, Robert Haas wrote:
> >> >
> >> > 99% of transactions happen in similar times between primary and standby,
> >> > everything dragged down by rare but se
On Sat, April 24, 2010 00:39, Simon Riggs wrote:
> On Fri, 2010-04-23 at 11:32 -0400, Robert Haas wrote:
>> >
>> > 99% of transactions happen in similar times between primary and standby,
>> > everything dragged down by rare but severe spikes.
>> >
>> > We're looking for something that would delay
On Fri, Apr 23, 2010 at 6:39 PM, Simon Riggs wrote:
> On Fri, 2010-04-23 at 11:32 -0400, Robert Haas wrote:
>> >
>> > 99% of transactions happen in similar times between primary and standby,
>> > everything dragged down by rare but severe spikes.
>> >
>> > We're looking for something that would de
On Fri, 2010-04-23 at 11:32 -0400, Robert Haas wrote:
> >
> > 99% of transactions happen in similar times between primary and standby,
> > everything dragged down by rare but severe spikes.
> >
> > We're looking for something that would delay something that normally
> > takes <0.1ms into something
On 4/23/10, Tom Lane wrote:
> Marko Kreen writes:
> > Um, you have been burned by exactly this on x86 also:
> > http://archives.postgresql.org/pgsql-hackers/2009-03/msg01265.php
>
>
> Yeah, we never did figure out exactly how come you were observing that
> failure on Intel-ish hardware. I
Marko Kreen writes:
> Um, you have been burned by exactly this on x86 also:
> http://archives.postgresql.org/pgsql-hackers/2009-03/msg01265.php
Yeah, we never did figure out exactly how come you were observing that
failure on Intel-ish hardware. I was under the impression that Intel
machines d
On 4/18/10, Simon Riggs wrote:
> On Sat, 2010-04-17 at 16:48 -0400, Tom Lane wrote:
> > There are some places where we suppose that a *single* write into shared
> > memory can safely be done without a lock, if we're not too concerned
> > about how soon other transactions will see the effects.
On Fri, Apr 23, 2010 at 11:14 AM, Simon Riggs wrote:
> On Thu, 2010-04-22 at 23:45 +0100, Simon Riggs wrote:
>> On Thu, 2010-04-22 at 20:39 +0200, Erik Rijkers wrote:
>> > On Sun, April 18, 2010 13:01, Simon Riggs wrote:
>>
>> > any comment is welcome...
>>
>> Please can you re-run with -l and pos
On Thu, 2010-04-22 at 23:45 +0100, Simon Riggs wrote:
> On Thu, 2010-04-22 at 20:39 +0200, Erik Rijkers wrote:
> > On Sun, April 18, 2010 13:01, Simon Riggs wrote:
>
> > any comment is welcome...
>
> Please can you re-run with -l and post me the file of times
Erik has sent me details of a test r
Erik Rijkers wrote:
This is the same behaviour (i.e. extreme slow standby) that I saw earlier (and
which caused the
original post, btw). In that earlier instance, the extreme slowness
disappeared later, after many
hours maybe even days (without bouncing either primary or standby).
I have no
On Thu, 2010-04-22 at 20:39 +0200, Erik Rijkers wrote:
> On Sun, April 18, 2010 13:01, Simon Riggs wrote:
> any comment is welcome...
Please can you re-run with -l and post me the file of times
Please also rebuild using --enable-profile so we can see what's
happening.
Can you also try the enclo
On Thu, April 22, 2010 23:54, Mark Kirkwood wrote:
> Greg Smith wrote:
>> Erik Rijkers wrote:
>>> This is the same behaviour (i.e. extreme slow standby) that I saw
>>> earlier (and which caused the
>>> original post, btw). In that earlier instance, the extreme slowness
>>> disappeared later, after
Greg Smith wrote:
Erik Rijkers wrote:
This is the same behaviour (i.e. extreme slow standby) that I saw
earlier (and which caused the
original post, btw). In that earlier instance, the extreme slowness
disappeared later, after many
hours maybe even days (without bouncing either primary or sta
Erik Rijkers wrote:
This is the same behaviour (i.e. extreme slow standby) that I saw earlier (and
which caused the
original post, btw). In that earlier instance, the extreme slowness
disappeared later, after many
hours maybe even days (without bouncing either primary or standby).
Any pos
On Sun, April 18, 2010 13:01, Simon Riggs wrote:
>>
>> OK, I'll put a spinlock around access to the head of the array.
>
> v2 patch attached
>
knownassigned_sortedarray.v2.diff applied to cvs HEAD (2010.04.21 22:36)
I have done a few smaller tests (scale 500, clients 1, 20):
init:
pgbench -h /
On Thu, 2010-04-22 at 08:57 +0300, Heikki Linnakangas wrote:
> >
> > I think the assert is a good idea. If there's no real problem here,
> > the assert won't trip. It's just a safety precaution.
>
> Right. And assertions also act as documentation, they are a precise and
> compact way to documen
Robert Haas wrote:
> On Wed, Apr 21, 2010 at 9:37 AM, Simon Riggs wrote:
>> On Wed, 2010-04-21 at 15:27 +0300, Heikki Linnakangas wrote:
>>
>>> Given the discussion about the cyclic nature of XIDs, it would be good
>>> to add an assertion that when a new XID is added to the array, it is
>>>
>>> a)
On Apr 21, 2010, at 16:49 , Simon Riggs wrote:
> On Wed, 2010-04-21 at 16:22 +0200, marcin mank wrote:
>
>> Is that not a good idea that (at least for dev-builds, like with
>> enable-cassert) the xid counter start at like 2^31 - 1000 ? It could
>> help catch some bugs.
>
> It is a good idea, I'm
On Wed, Apr 21, 2010 at 10:12 AM, Simon Riggs wrote:
> On Wed, 2010-04-21 at 09:51 -0400, Robert Haas wrote:
>> >
>> > Adding an assertion isn't going to do much because it's unlikely anybody
>> > is going to be running for 2^31 transactions with asserts enabled.
>> >
>
>> I think the assert is a
On Wed, 2010-04-21 at 16:22 +0200, marcin mank wrote:
> Is that not a good idea that (at least for dev-builds, like with
> enable-cassert) the xid counter start at like 2^31 - 1000 ? It could
> help catch some bugs.
It is a good idea, I'm sure that would help catch bugs.
It wouldn't help here be
On Wed, Apr 21, 2010 at 4:12 PM, Simon Riggs wrote:
> On Wed, 2010-04-21 at 09:51 -0400, Robert Haas wrote:
>> >
>> > Adding an assertion isn't going to do much because it's unlikely anybody
>> > is going to be running for 2^31 transactions with asserts enabled.
>> >
>
>> I think the assert is a g
On Wed, 2010-04-21 at 09:51 -0400, Robert Haas wrote:
> >
> > Adding an assertion isn't going to do much because it's unlikely anybody
> > is going to be running for 2^31 transactions with asserts enabled.
> >
> I think the assert is a good idea. If there's no real problem here,
> the assert won'
On Wed, Apr 21, 2010 at 8:20 AM, Heikki Linnakangas
wrote:
> The locking seems overly complex to me.
I tend to agree.
! /*
!* Callers must hold either ProcArrayLock in Exclusive mode or
!* ProcArrayLock in Shared mode *and* known_assigned_xids_lck
!* to update these
On Wed, Apr 21, 2010 at 9:37 AM, Simon Riggs wrote:
> On Wed, 2010-04-21 at 15:27 +0300, Heikki Linnakangas wrote:
>
>> Given the discussion about the cyclic nature of XIDs, it would be good
>> to add an assertion that when a new XID is added to the array, it is
>>
>> a) larger than the biggest va
On Wed, 2010-04-21 at 15:27 +0300, Heikki Linnakangas wrote:
> Given the discussion about the cyclic nature of XIDs, it would be good
> to add an assertion that when a new XID is added to the array, it is
>
> a) larger than the biggest value already in the array
> (TransactionIdFollows(new, head)
On Wed, 2010-04-21 at 15:20 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > On Sun, 2010-04-18 at 08:24 +0100, Simon Riggs wrote:
> >> On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote:
> >>> Simon Riggs writes:
> What I'm not clear on is why you've used a spinlock everywhere when o
Simon Riggs wrote:
> On Sun, 2010-04-18 at 08:24 +0100, Simon Riggs wrote:
>> On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote:
>>> Simon Riggs writes:
What I'm not clear on is why you've used a spinlock everywhere when only
weak-memory thang CPUs are a problem. Why not have a weak-memo
Simon Riggs wrote:
> On Sun, 2010-04-18 at 08:24 +0100, Simon Riggs wrote:
>> On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote:
>>> Simon Riggs writes:
What I'm not clear on is why you've used a spinlock everywhere when only
weak-memory thang CPUs are a problem. Why not have a weak-memo
On Wed, 2010-04-21 at 15:09 +1200, Mark Kirkwood wrote:
> I did some testing of this patch (v2). Unfortunately I don't have access
> to hardware capable of doing tests at the same scale as Erik used.
> However I was still able to show a consistent difference (I think)
> between standby performa
On Tue, Apr 20, 2010 at 11:09 PM, Mark Kirkwood
wrote:
> Robert Haas wrote:
>>
>> So, does anyone have a few cycles to test this out? We are down to
>> handful of remaining open items, so getting this tested and committed
>> sooner = beta sooner.
>>
>>
>>
>
> I did some testing of this patch (v2)
Robert Haas wrote:
So, does anyone have a few cycles to test this out? We are down to
handful of remaining open items, so getting this tested and committed
sooner = beta sooner.
I did some testing of this patch (v2). Unfortunately I don't have access
to hardware capable of doing tests a
On Sun, Apr 18, 2010 at 4:22 PM, Simon Riggs wrote:
> On Sun, 2010-04-18 at 13:16 -0700, David Fetter wrote:
>> On Sun, Apr 18, 2010 at 12:01:05PM +0100, Simon Riggs wrote:
>> > On Sun, 2010-04-18 at 08:24 +0100, Simon Riggs wrote:
>> > > On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote:
>> > > >
On Sun, Apr 18, 2010 at 09:22:21PM +0100, Simon Riggs wrote:
> On Sun, 2010-04-18 at 13:16 -0700, David Fetter wrote:
> > On Sun, Apr 18, 2010 at 12:01:05PM +0100, Simon Riggs wrote:
> > > On Sun, 2010-04-18 at 08:24 +0100, Simon Riggs wrote:
> > > > On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrot
On Sun, 2010-04-18 at 13:16 -0700, David Fetter wrote:
> On Sun, Apr 18, 2010 at 12:01:05PM +0100, Simon Riggs wrote:
> > On Sun, 2010-04-18 at 08:24 +0100, Simon Riggs wrote:
> > > On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote:
> > > > Simon Riggs writes:
> > > > > What I'm not clear on is wh
On Sun, Apr 18, 2010 at 12:01:05PM +0100, Simon Riggs wrote:
> On Sun, 2010-04-18 at 08:24 +0100, Simon Riggs wrote:
> > On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote:
> > > Simon Riggs writes:
> > > > What I'm not clear on is why you've used a spinlock everywhere
> > > > when only weak-memory
On Sun, 2010-04-18 at 08:24 +0100, Simon Riggs wrote:
> On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote:
> > Simon Riggs writes:
> > > What I'm not clear on is why you've used a spinlock everywhere when only
> > > weak-memory thang CPUs are a problem. Why not have a weak-memory-protect
> > > mac
On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > What I'm not clear on is why you've used a spinlock everywhere when only
> > weak-memory thang CPUs are a problem. Why not have a weak-memory-protect
> > macro that does does nada when the hardware already protects us? (i
Simon Riggs writes:
> What I'm not clear on is why you've used a spinlock everywhere when only
> weak-memory thang CPUs are a problem. Why not have a weak-memory-protect
> macro that does does nada when the hardware already protects us? (i.e. a
> spinlock only for the hardware that needs it).
Wel
On Sat, 2010-04-17 at 16:48 -0400, Tom Lane wrote:
> > We search the array between tail and head. If the head moves by integer
> > overwrite just as already happens for xid assignment, then we would use
> > the new head for the search. The code is careful to fetch only once.
>
> ... but this will
Simon Riggs writes:
> On Sat, 2010-04-17 at 15:45 -0400, Tom Lane wrote:
>> How do you know that just adding items at the right will produce a
>> sorted array?
> Xids don't arrive in sequence, but "known assigned xids" are added in
> sequence because we infer the existence of the intermediate xid
On Sat, 2010-04-17 at 15:45 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > On Sat, 2010-04-17 at 11:13 -0400, Tom Lane wrote:
> >> It'd be cheaper anyway to sort and search the
> >> array using plain <, so why are you so eager to use
> >> TransactionIdFollows?
>
> > The array grows to the right
Simon Riggs writes:
> On Sat, 2010-04-17 at 11:13 -0400, Tom Lane wrote:
>> It'd be cheaper anyway to sort and search the
>> array using plain <, so why are you so eager to use
>> TransactionIdFollows?
> The array grows to the right and is laid out one xid per element,
> resulting in a sequence o
On Sat, 2010-04-17 at 11:13 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > AFAICS the example you give isn't correct.
>
> > We would lay out the values like this
>
> > W-3 W-2 W-1 W 3 4 5
>
> > where W is the wrap value
>
> Stop right there, you're already failing to think clearly. There is
On Sat, Apr 17, 2010 at 11:13 AM, Tom Lane wrote:
> Simon Riggs writes:
>> AFAICS the example you give isn't correct.
>
>> We would lay out the values like this
>
>> W-3 W-2 W-1 W 3 4 5
>
>> where W is the wrap value
>
> Stop right there, you're already failing to think clearly. There is no
> un
Simon Riggs writes:
> AFAICS the example you give isn't correct.
> We would lay out the values like this
> W-3 W-2 W-1 W 3 4 5
> where W is the wrap value
Stop right there, you're already failing to think clearly. There is no
unique "wrap value", all values act the same in circular XID space.
On Fri, 2010-04-16 at 11:10 -0400, Tom Lane wrote:
> Simon Riggs writes:
> > On Fri, 2010-04-16 at 10:39 -0400, Tom Lane wrote:
> >> I think you're outsmarting yourself there. A binary search will in fact
> >> *not work* with circular xid comparison (this is exactly why there's no
> >> btree opcl
On Fri, 2010-04-16 at 13:00 +0100, Simon Riggs wrote:
> Almost done
Here's the finished patch.
Internal changes only, no functional or user visible changes, so no
docs, just detailed explanatory comments.
Expectation is
* performance no longer depends upon max_connections
* recovery will be abou
Simon Riggs writes:
> On Fri, 2010-04-16 at 10:39 -0400, Tom Lane wrote:
>> I think you're outsmarting yourself there. A binary search will in fact
>> *not work* with circular xid comparison (this is exactly why there's no
>> btree opclass for XID).
> I don't understand the exact, please explain
On Fri, 2010-04-16 at 10:39 -0400, Tom Lane wrote:
> Heikki Linnakangas writes:
> > I didn't handle xid wraparound correctly in the binary search, need to use
> > TransactionIdFollows instead of plan >.
>
> I think you're outsmarting yourself there. A binary search will in fact
> *not work* with
Heikki Linnakangas writes:
> I didn't handle xid wraparound correctly in the binary search, need to use
> TransactionIdFollows instead of plan >.
I think you're outsmarting yourself there. A binary search will in fact
*not work* with circular xid comparison (this is exactly why there's no
btree
On Fri, 2010-04-16 at 14:47 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > On Fri, 2010-04-16 at 11:29 +0300, Heikki Linnakangas wrote:
> >> How does the attached patch look like? It's probably similar to what you
> >> had in mind.
> >
> > It looks like a second version of what I'm work
Simon Riggs wrote:
> On Fri, 2010-04-16 at 11:29 +0300, Heikki Linnakangas wrote:
>> How does the attached patch look like? It's probably similar to what you
>> had in mind.
>
> It looks like a second version of what I'm working on and about to
> publish. I'll take that as a compliment!
>
> My pa
On Fri, 2010-04-16 at 11:29 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > On Tue, 2010-04-13 at 21:09 +0300, Heikki Linnakangas wrote:
> >> A quick fix would be to check if there's any entries in the hash table
> >> before scanning it. That would eliminate the overhead when there's no
>
Simon Riggs wrote:
> On Tue, 2010-04-13 at 21:09 +0300, Heikki Linnakangas wrote:
>> A quick fix would be to check if there's any entries in the hash table
>> before scanning it. That would eliminate the overhead when there's no
>> in-progress transactions in the master. But as soon as there's even
On Tue, 2010-04-13 at 21:09 +0300, Heikki Linnakangas wrote:
> Heikki Linnakangas wrote:
> > I could reproduce this on my laptop, standby is about 20% slower. I ran
> > oprofile, and what stands out as the difference between the master and
> > standby is that on standby about 20% of the CPU time is
Heikki Linnakangas writes:
> Changing the KnownAssignedXids data structure from
> hash table into something that's quicker to scan. Preferably something
> with O(N), where N is the number of entries in the data structure, not
> the maximum number of entries it can hold as it is with the hash tabl
1 - 100 of 113 matches
Mail list logo
