Re: [HACKERS] CLOG contention, part 2

2012-02-28 Thread Simon Riggs
On Tue, Feb 28, 2012 at 6:11 PM, Robert Haas wrote: > On Mon, Feb 27, 2012 at 4:03 AM, Simon Riggs wrote: >> So please use a scale factor that the hardware can cope with. > > OK.  I tested this out on Nate Boley's 32-core AMD machine, using > scale factor 100 and scale factor 300. I initialized i

Re: [HACKERS] CLOG contention, part 2

2012-02-28 Thread Robert Haas
On Mon, Feb 27, 2012 at 4:03 AM, Simon Riggs wrote: > So please use a scale factor that the hardware can cope with. OK. I tested this out on Nate Boley's 32-core AMD machine, using scale factor 100 and scale factor 300. I initialized it with Simon's patch, which should have the effect of renderi

Re: [HACKERS] CLOG contention, part 2

2012-02-27 Thread Simon Riggs
On Sun, Feb 26, 2012 at 10:53 PM, Robert Haas wrote: > On Sat, Feb 25, 2012 at 2:16 PM, Simon Riggs wrote: >> On Wed, Feb 8, 2012 at 11:26 PM, Robert Haas wrote: >>> Given that, I obviously cannot test this at this point, >> >> Patch with minor corrections attached here for further review. > > A

Re: [HACKERS] CLOG contention, part 2

2012-02-26 Thread Robert Haas
On Sat, Feb 25, 2012 at 2:16 PM, Simon Riggs wrote: > On Wed, Feb 8, 2012 at 11:26 PM, Robert Haas wrote: >> Given that, I obviously cannot test this at this point, > > Patch with minor corrections attached here for further review. All right, I will set up some benchmarks with this version, and

Re: [HACKERS] CLOG contention, part 2

2012-02-25 Thread Simon Riggs
On Wed, Feb 8, 2012 at 11:26 PM, Robert Haas wrote: > Given that, I obviously cannot test this at this point, Patch with minor corrections attached here for further review. > but let me go > ahead and theorize about how well it's likely to work.  What Tom > suggested before (and after some refl

Re: [HACKERS] CLOG contention, part 2

2012-02-10 Thread Simon Riggs
On Fri, Feb 10, 2012 at 7:01 PM, Ants Aasma wrote: > > On Feb 9, 2012 1:27 AM, "Robert Haas" > >> However, there is a potential fly in the ointment: in other cases in >> which we've reduced contention at the LWLock layer, we've ended up >> with very nasty contention at the spinlock layer that can

Re: [HACKERS] CLOG contention, part 2

2012-02-10 Thread Ants Aasma
On Feb 9, 2012 1:27 AM, "Robert Haas" > However, there is a potential fly in the ointment: in other cases in > which we've reduced contention at the LWLock layer, we've ended up > with very nasty contention at the spinlock layer that can sometimes > eat more CPU time than the LWLock contention did

Re: [HACKERS] CLOG contention, part 2

2012-02-08 Thread Robert Haas
On Sun, Jan 29, 2012 at 6:04 PM, Simon Riggs wrote: > On Sun, Jan 29, 2012 at 9:41 PM, Jeff Janes wrote: > >> If I cast to a int, then I see advancement: > > I'll initialise it as 0, rather than -1 and then we don't have a > problem in any circumstance. > > >>> I've specifically designed the pgbe

Re: [HACKERS] CLOG contention, part 2

2012-01-31 Thread Jeff Janes
On Mon, Jan 30, 2012 at 12:24 PM, Robert Haas wrote: > On Fri, Jan 27, 2012 at 8:21 PM, Jeff Janes wrote: >> On Fri, Jan 27, 2012 at 3:16 PM, Merlin Moncure wrote: >>> On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes wrote: Also, I think the general approach is wrong.  The only reason to have >

Re: [HACKERS] CLOG contention, part 2

2012-01-30 Thread Robert Haas
On Fri, Jan 27, 2012 at 8:21 PM, Jeff Janes wrote: > On Fri, Jan 27, 2012 at 3:16 PM, Merlin Moncure wrote: >> On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes wrote: >>> Also, I think the general approach is wrong.  The only reason to have >>> these pages in shared memory is that we can control acce

Re: [HACKERS] CLOG contention, part 2

2012-01-29 Thread Simon Riggs
On Sun, Jan 29, 2012 at 9:41 PM, Jeff Janes wrote: > If I cast to a int, then I see advancement: I'll initialise it as 0, rather than -1 and then we don't have a problem in any circumstance. >> I've specifically designed the pgbench changes required to simulate >> conditions of clog contention

Re: [HACKERS] CLOG contention, part 2

2012-01-29 Thread Jeff Janes
On Sun, Jan 29, 2012 at 1:41 PM, Jeff Janes wrote: > On Sun, Jan 29, 2012 at 12:18 PM, Simon Riggs wrote: >> On Fri, Jan 27, 2012 at 10:05 PM, Jeff Janes wrote: >>> On Sat, Jan 21, 2012 at 7:31 AM, Simon Riggs wrote: Yes, it was. Sorry about that. New version attached, retesting while

Re: [HACKERS] CLOG contention, part 2

2012-01-29 Thread Jeff Janes
On Sun, Jan 29, 2012 at 12:18 PM, Simon Riggs wrote: > On Fri, Jan 27, 2012 at 10:05 PM, Jeff Janes wrote: >> On Sat, Jan 21, 2012 at 7:31 AM, Simon Riggs wrote: >>> >>> Yes, it was. Sorry about that. New version attached, retesting while >>> you read this. >> >> In my hands I could never get th

Re: [HACKERS] CLOG contention, part 2

2012-01-29 Thread Simon Riggs
On Fri, Jan 27, 2012 at 10:05 PM, Jeff Janes wrote: > On Sat, Jan 21, 2012 at 7:31 AM, Simon Riggs wrote: >> >> Yes, it was. Sorry about that. New version attached, retesting while >> you read this. > > In my hands I could never get this patch to do anything.  The new > cache was never used. > >

Re: [HACKERS] CLOG contention, part 2

2012-01-29 Thread Simon Riggs
On Sat, Jan 28, 2012 at 1:52 PM, Simon Riggs wrote: >> Also, I think the general approach is wrong.  The only reason to have >> these pages in shared memory is that we can control access to them to >> prevent write/write and read/write corruption.  Since these pages are >> never written, they don

Re: [HACKERS] CLOG contention, part 2

2012-01-28 Thread Simon Riggs
On Fri, Jan 27, 2012 at 10:05 PM, Jeff Janes wrote: > On Sat, Jan 21, 2012 at 7:31 AM, Simon Riggs wrote: >> >> Yes, it was. Sorry about that. New version attached, retesting while >> you read this. > > In my hands I could never get this patch to do anything.  The new > cache was never used. > >

Re: [HACKERS] CLOG contention, part 2

2012-01-27 Thread Jeff Janes
On Fri, Jan 27, 2012 at 3:16 PM, Merlin Moncure wrote: > On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes wrote: >> Also, I think the general approach is wrong.  The only reason to have >> these pages in shared memory is that we can control access to them to >> prevent write/write and read/write corru

Re: [HACKERS] CLOG contention, part 2

2012-01-27 Thread Merlin Moncure
On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes wrote: > Also, I think the general approach is wrong.  The only reason to have > these pages in shared memory is that we can control access to them to > prevent write/write and read/write corruption.  Since these pages are > never written, they don't nee

Re: [HACKERS] CLOG contention, part 2

2012-01-27 Thread Jeff Janes
On Sat, Jan 21, 2012 at 7:31 AM, Simon Riggs wrote: > > Yes, it was. Sorry about that. New version attached, retesting while > you read this. In my hands I could never get this patch to do anything. The new cache was never used. I think that that was because RecentXminPageno never budged from -

Re: [HACKERS] CLOG contention, part 2

2012-01-22 Thread Jeff Janes
On Fri, Jan 20, 2012 at 6:44 AM, Simon Riggs wrote: > > OT: It would save lots of time if we had 2 things for the CF app: > .. > 2. Something that automatically tests patches. If you submit a patch > we run up a blank VM and run patch applies on all patches. As soon as > we get a fail, an email go

Re: [HACKERS] CLOG contention, part 2

2012-01-21 Thread Simon Riggs
On Sat, Jan 21, 2012 at 1:57 PM, Robert Haas wrote: > On Fri, Jan 20, 2012 at 10:44 AM, Robert Haas wrote: D'oh.  You're right.  Looks like I accidentally tried to apply this to the 9.1 sources.  Sigh... >>> >>> No worries. It's Friday. > > Server passed 'make check' with this patch, bu

Re: [HACKERS] CLOG contention, part 2

2012-01-21 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:44 AM, Robert Haas wrote: >>> D'oh.  You're right.  Looks like I accidentally tried to apply this to >>> the 9.1 sources.  Sigh... >> >> No worries. It's Friday. Server passed 'make check' with this patch, but when I tried to fire it up for some test runs, it fell over

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: > On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: >> I've taken that idea and used it to build a second Clog cache, known >> as ClogHistory which allows access to the read-only tail of pages in >> the clog. Once a page has been written to for

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:38 AM, Simon Riggs wrote: > On Fri, Jan 20, 2012 at 3:32 PM, Robert Haas wrote: >> On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs wrote: >>> On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: > I've taken th

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 3:32 PM, Robert Haas wrote: > On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs wrote: >> On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: >>> On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: I've taken that idea and used it to build a second Clog cache, known

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 10:16 AM, Simon Riggs wrote: > On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: >> On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: >>> I've taken that idea and used it to build a second Clog cache, known >>> as ClogHistory which allows access to the read-only tail o

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Fri, Jan 20, 2012 at 9:44 AM, Simon Riggs wrote: > On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: >> On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: >>> I've taken that idea and used it to build a second Clog cache, known >>> as ClogHistory which allows access to the read-only tail of

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Simon Riggs
On Fri, Jan 20, 2012 at 1:37 PM, Robert Haas wrote: > On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: >> I've taken that idea and used it to build a second Clog cache, known >> as ClogHistory which allows access to the read-only tail of pages in >> the clog. Once a page has been written to for

Re: [HACKERS] CLOG contention, part 2

2012-01-20 Thread Robert Haas
On Sun, Jan 8, 2012 at 9:25 AM, Simon Riggs wrote: > I've taken that idea and used it to build a second Clog cache, known > as ClogHistory which allows access to the read-only tail of pages in > the clog. Once a page has been written to for the last time, it will > be accessed via the ClogHistory

Re: [HACKERS] CLOG contention, part 2

2012-01-12 Thread Simon Riggs
On Sun, Jan 8, 2012 at 2:25 PM, Simon Riggs wrote: > I've taken that idea and used it to build a second Clog cache, known > as ClogHistory which allows access to the read-only tail of pages in > the clog. Once a page has been written to for the last time, it will > be accessed via the ClogHistory

[HACKERS] CLOG contention, part 2

2012-01-08 Thread Simon Riggs
Recent results from Robert show clog contention is still an issue. In various discussions Tom noted that pages prior to RecentXmin are readonly and we might find a way to make use of that fact in providing different mechanisms or resources. I've taken that idea and used it to build a second Clog