Re: [HACKERS] Process local hint bit cache

2011-06-29 Thread Merlin Moncure
On Wed, Jun 29, 2011 at 10:09 AM, Merlin Moncure wrote: > That is not correct.   Any cache 'miss' on a page continues to fall > through to SetHintBitsCache() which dirties the page as it always has. er, SetHintBits(), not SetHintBitsCache() merlin -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] Process local hint bit cache

2011-06-29 Thread Merlin Moncure
On Wed, Jun 29, 2011 at 9:09 AM, Simon Riggs wrote: > On Sat, Apr 2, 2011 at 8:40 PM, Tom Lane wrote: >> Merlin Moncure writes: > >>> aside: >>> Moving TransactionIdInProgress below TransactionIdDidCommit can help >>> in once sense: TransactionIdDidCommit grabs the XidStatus but discards >>> the

Re: [HACKERS] Process local hint bit cache

2011-06-29 Thread Simon Riggs
On Sat, Apr 2, 2011 at 8:40 PM, Tom Lane wrote: > Merlin Moncure writes: >> aside: >> Moving TransactionIdInProgress below TransactionIdDidCommit can help >> in once sense: TransactionIdDidCommit grabs the XidStatus but discards >> the knowledge if the transaction is known aborted. > > Doesn't t

Re: [HACKERS] Process local hint bit cache

2011-06-29 Thread Robert Haas
On Wed, Jun 29, 2011 at 9:39 AM, Merlin Moncure wrote: > On Tue, Jun 28, 2011 at 10:03 PM, Robert Haas wrote: >> On Thu, Apr 7, 2011 at 2:49 PM, Merlin Moncure wrote: >>> On Thu, Apr 7, 2011 at 1:28 PM, Merlin Moncure wrote:                        int ByteOffset = xid / BITS_PER_BYTE; >>>

Re: [HACKERS] Process local hint bit cache

2011-06-29 Thread Merlin Moncure
On Tue, Jun 28, 2011 at 10:03 PM, Robert Haas wrote: > On Thu, Apr 7, 2011 at 2:49 PM, Merlin Moncure wrote: >> On Thu, Apr 7, 2011 at 1:28 PM, Merlin Moncure wrote: >>>                        int ByteOffset = xid / BITS_PER_BYTE; >> >> whoops, I just notice this was wrong -- the byte offset nee

Re: [HACKERS] Process local hint bit cache

2011-06-28 Thread Robert Haas
On Thu, Apr 7, 2011 at 2:49 PM, Merlin Moncure wrote: > On Thu, Apr 7, 2011 at 1:28 PM, Merlin Moncure wrote: >>                        int ByteOffset = xid / BITS_PER_BYTE; > > whoops, I just notice this was wrong -- the byte offset needs to be > taking bucket into account.  I need to clean this

Re: [HACKERS] Process local hint bit cache

2011-04-07 Thread Merlin Moncure
On Thu, Apr 7, 2011 at 1:28 PM, Merlin Moncure wrote: >                        int ByteOffset = xid / BITS_PER_BYTE; whoops, I just notice this was wrong -- the byte offset needs to be taking bucket into account. I need to clean this up some more obviously, but the issues at play remain the same

Re: [HACKERS] Process local hint bit cache

2011-04-07 Thread Merlin Moncure
On Mon, Apr 4, 2011 at 9:25 AM, Merlin Moncure wrote: > On Sun, Apr 3, 2011 at 6:40 PM, Merlin Moncure wrote: >> On Sat, Apr 2, 2011 at 8:37 PM, Tom Lane wrote: >>> Merlin Moncure writes: On Thu, Mar 31, 2011 at 5:38 PM, Merlin Moncure wrote: > working on exanding the cache to # xid >

Re: [HACKERS] Process local hint bit cache

2011-04-04 Thread Merlin Moncure
On Sun, Apr 3, 2011 at 6:40 PM, Merlin Moncure wrote: > On Sat, Apr 2, 2011 at 8:37 PM, Tom Lane wrote: >> Merlin Moncure writes: >>> On Thu, Mar 31, 2011 at 5:38 PM, Merlin Moncure wrote: working on exanding the cache to # xid > 1. >> >>> patch attached.  this is essentially my original i

Re: [HACKERS] Process local hint bit cache

2011-04-03 Thread Merlin Moncure
On Sat, Apr 2, 2011 at 8:37 PM, Tom Lane wrote: > Merlin Moncure writes: >> On Thu, Mar 31, 2011 at 5:38 PM, Merlin Moncure wrote: >>> working on exanding the cache to # xid > 1. > >> patch attached.  this is essentially my original idea except it's >> injected directly in to tqual.c as a kind o

Re: [HACKERS] Process local hint bit cache

2011-04-02 Thread Tom Lane
Merlin Moncure writes: > On Thu, Mar 31, 2011 at 5:38 PM, Merlin Moncure wrote: >> working on exanding the cache to # xid > 1. > patch attached. this is essentially my original idea except it's > injected directly in to tqual.c as a kind of a expansion of the 'last > seen' concept. Because the

Re: [HACKERS] Process local hint bit cache

2011-04-02 Thread Tom Lane
Merlin Moncure writes: > On Wed, Mar 30, 2011 at 2:35 PM, Merlin Moncure wrote: >> btw I haven't forgotten your idea to move TransactionIdInProgress >> Down. I think this is a good idea, and will experiment with it pre and >> post cache. The reason it's done in that order is to avoid race condit

Re: [HACKERS] Process local hint bit cache

2011-04-01 Thread Merlin Moncure
On Thu, Mar 31, 2011 at 5:38 PM, Merlin Moncure wrote: > On Thu, Mar 31, 2011 at 5:33 PM, Merlin Moncure wrote: >> On Wed, Mar 30, 2011 at 2:35 PM, Merlin Moncure wrote: >>> On Wed, Mar 30, 2011 at 11:23 AM, Heikki Linnakangas >>> wrote: On 30.03.2011 18:02, Robert Haas wrote: > >

Re: [HACKERS] Process local hint bit cache

2011-03-31 Thread Merlin Moncure
On Thu, Mar 31, 2011 at 5:33 PM, Merlin Moncure wrote: > On Wed, Mar 30, 2011 at 2:35 PM, Merlin Moncure wrote: >> On Wed, Mar 30, 2011 at 11:23 AM, Heikki Linnakangas >> wrote: >>> On 30.03.2011 18:02, Robert Haas wrote: On Wed, Mar 30, 2011 at 10:40 AM, Greg Stark  wrote: > >

Re: [HACKERS] Process local hint bit cache

2011-03-31 Thread Merlin Moncure
On Wed, Mar 30, 2011 at 2:35 PM, Merlin Moncure wrote: > On Wed, Mar 30, 2011 at 11:23 AM, Heikki Linnakangas > wrote: >> On 30.03.2011 18:02, Robert Haas wrote: >>> >>> On Wed, Mar 30, 2011 at 10:40 AM, Greg Stark  wrote: But one way or another the hint bits have to get set sometime. T

Re: [HACKERS] Process local hint bit cache

2011-03-31 Thread Robert Haas
On Wed, Mar 30, 2011 at 6:30 PM, Merlin Moncure wrote: > On Wed, Mar 30, 2011 at 2:35 PM, Merlin Moncure wrote: >> >> btw I haven't forgotten your idea to move TransactionIdInProgress >> Down. I think this is a good idea, and will experiment with it pre and >> post cache. > > aside: > Moving Tran

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Merlin Moncure
On Wed, Mar 30, 2011 at 2:35 PM, Merlin Moncure wrote: > > btw I haven't forgotten your idea to move TransactionIdInProgress > Down. I think this is a good idea, and will experiment with it pre and > post cache. aside: Moving TransactionIdInProgress below TransactionIdDidCommit can help in once s

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Merlin Moncure
On Wed, Mar 30, 2011 at 11:23 AM, Heikki Linnakangas wrote: > On 30.03.2011 18:02, Robert Haas wrote: >> >> On Wed, Mar 30, 2011 at 10:40 AM, Greg Stark  wrote: >>> >>> But one way or another the hint bits have to get set sometime. The >>> sooner that happens the less clog i/o has to happen in the

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Merlin Moncure
On Wed, Mar 30, 2011 at 11:23 AM, Heikki Linnakangas wrote: > Even if clog access was free, hint bits still give a significant speedup > thanks to skipping all the other overhead like TransactionIdIsInProgress() > and TransactionIdIsCurrentTransactionId(). Speeding up clog access is > important; w

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Heikki Linnakangas
On 30.03.2011 18:02, Robert Haas wrote: On Wed, Mar 30, 2011 at 10:40 AM, Greg Stark wrote: But one way or another the hint bits have to get set sometime. The sooner that happens the less clog i/o has to happen in the meantime. I talked about this with Merlin a bit yesterday. I think that hi

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Merlin Moncure
On Wed, Mar 30, 2011 at 10:43 AM, Alvaro Herrera wrote: > Excerpts from Merlin Moncure's message of mié mar 30 12:14:20 -0300 2011: > >> It is very different -- the slru layer is in shared memory and >> requires locks to access.   The entire point is trying to avoid >> accessing this structure in

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Alvaro Herrera
Excerpts from Merlin Moncure's message of mié mar 30 12:14:20 -0300 2011: > It is very different -- the slru layer is in shared memory and > requires locks to access. The entire point is trying to avoid > accessing this structure in tight code paths. I'm actually very > skeptical the slru layer

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Merlin Moncure
On Wed, Mar 30, 2011 at 9:40 AM, Greg Stark wrote: > On Tue, Mar 29, 2011 at 10:34 PM, Merlin Moncure wrote: >> So I went back to the drawing board, reviewed the archives, and came >> up with a new proposal.  I'd like to see a process local clog page >> cache of around 1-4 pages (8-32kb typically

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Robert Haas
On Wed, Mar 30, 2011 at 10:40 AM, Greg Stark wrote: > But one way or another the hint bits have to get set sometime. The > sooner that happens the less clog i/o has to happen in the meantime. I talked about this with Merlin a bit yesterday. I think that his thought is that most transactions will

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Greg Stark
On Tue, Mar 29, 2011 at 10:34 PM, Merlin Moncure wrote: > So I went back to the drawing board, reviewed the archives, and came > up with a new proposal.  I'd like to see a process local clog page > cache of around 1-4 pages (8-32kb typically) that would replace the > current TransactionLogFetch la

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Heikki Linnakangas
On 30.03.2011 17:05, Merlin Moncure wrote: *) Maybe the shared buffer cache currently being maintained over the clog can be scrapped. I'm going to leave it alone for now, but I'm quite skeptical it provides much benefit even without local process cache. clog page have a very nice property that y

Re: [HACKERS] Process local hint bit cache

2011-03-30 Thread Merlin Moncure
On Tue, Mar 29, 2011 at 4:34 PM, Merlin Moncure wrote: > In a previous thread > (http://postgresql.1045698.n5.nabble.com/Set-hint-bits-upon-eviction-from-BufMgr-td4264323.html) > I was playing with the idea of granting the bgwriter the ability to > due last chance hint bit setting before evicting

[HACKERS] Process local hint bit cache

2011-03-29 Thread Merlin Moncure
In a previous thread (http://postgresql.1045698.n5.nabble.com/Set-hint-bits-upon-eviction-from-BufMgr-td4264323.html) I was playing with the idea of granting the bgwriter the ability to due last chance hint bit setting before evicting the page out. I still think might be a good idea, and it might b