Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Jaime Casanova (ja...@2ndquadrant.com) wrote: > On Tue, May 28, 2013 at 10:53 AM, Andres Freund > wrote: > > But I agree. This needs to work without much manual intervention. I > > think we just need to make autovacuum truncate only if it finds more > > free space than whatever amount we might

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 10:53 AM, Andres Freund wrote: > > But I agree. This needs to work without much manual intervention. I > think we just need to make autovacuum truncate only if it finds more > free space than whatever amount we might have added at that relation > size plus some slop. > And

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2013-05-28 10:07:06 -0400, Stephen Frost wrote: > > I'm really not, at all, excited about adding in GUCs for this. > > But I thought you were in favor of doing complex stuff like mapping > segments filled somewhere else into place :P That would

Re: [HACKERS] Extent Locks

2013-05-28 Thread Andres Freund
On 2013-05-28 10:07:06 -0400, Stephen Frost wrote: > * Jaime Casanova (ja...@2ndquadrant.com) wrote: > > btw, we can also use a next_extend_blocks GUC/reloption as a limit for > > autovacuum so it will allow that empty pages at the end of the table > > I'm really not, at all, excited about adding

Re: [HACKERS] Extent Locks

2013-05-28 Thread Merlin Moncure
On Tue, May 28, 2013 at 9:07 AM, Stephen Frost wrote: > * Jaime Casanova (ja...@2ndquadrant.com) wrote: >> btw, we can also use a next_extend_blocks GUC/reloption as a limit for >> autovacuum so it will allow that empty pages at the end of the table > > I'm really not, at all, excited about adding

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Jaime Casanova (ja...@2ndquadrant.com) wrote: > btw, we can also use a next_extend_blocks GUC/reloption as a limit for > autovacuum so it will allow that empty pages at the end of the table I'm really not, at all, excited about adding in GUCs for this. We just need to realize when the only avai

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 8:38 AM, Jaime Casanova wrote: > > We can also think in GUC/reloption for next_extend_blocks so formula > is needed, or of course the automated calculation that has been > proposed > s/so formula is needed/so *no* formula is needed btw, we can also use a next_extend_block

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 7:36 AM, Stephen Frost wrote: > > On the other hand, I do feel like people are worried about > over-extending a relation and wasting disk space- but with the way that > vacuum can clean up pages at the end, that would only be a temporary > situation anyway. > Hi, Maybe i'

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > On 05/17/2013 11:38 AM, Robert Haas wrote: > > maybe with a bit of modest pre-extension. > When it comes to pre-extension, is it realistic to get a count of > backends waiting on the lock and extend the relation by (say) 2x the > number of waiting ba

Re: [HACKERS] Extent Locks

2013-05-27 Thread Craig Ringer
On 05/18/2013 03:15 AM, Josh Berkus wrote: > The drawback to this is whatever size we choose is liable to be wrong > for some users. Users who currently have a lot of 16K tables would see > their databases grow alarmingly. This only becomes a problem for tables that're tiny, right? If your table

Re: [HACKERS] Extent Locks

2013-05-27 Thread Craig Ringer
On 05/17/2013 11:38 AM, Robert Haas wrote: > maybe with a bit of modest pre-extension. When it comes to pre-extension, is it realistic to get a count of backends waiting on the lock and extend the relation by (say) 2x the number of waiting backends? Getting a list of lock waiters is always a racey

Re: [HACKERS] Extent Locks

2013-05-17 Thread Josh Berkus
Robert, > But I still feel like that thought > experiment indicates that there must be a solution here just by > rejiggering the locking, and maybe with a bit of modest pre-extension. > The mediocre results of my last couple tries must indicate that I > wasn't entirely successful in getting the b

Re: [HACKERS] Extent Locks

2013-05-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, May 16, 2013 at 11:55 PM, Stephen Frost wrote: > > You don't change that. However, when a seq scan asks the storage layer > > for blocks that it knows don't actually exist, it can simply skip over > > them or return "empty" records or somethi

Re: [HACKERS] Extent Locks

2013-05-17 Thread Robert Haas
On Thu, May 16, 2013 at 11:55 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> I think it's pretty unrealistic to suppose that this can be made to >> work. The most obvious problem is that a sequential scan is coded to >> assume that every block between 0 and the last bl

Re: [HACKERS] Extent Locks

2013-05-17 Thread Robert Haas
On Thu, May 16, 2013 at 9:36 PM, Stephen Frost wrote: >> What we do have is >> a relation extension lock, but the size of the segment on disk has >> nothing to do with that: there's only one for the whole relation, and >> you hold it when adding a block to the relation. > > Yes, which is farrr too

Re: [HACKERS] Extent Locks

2013-05-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > I think it's pretty unrealistic to suppose that this can be made to > work. The most obvious problem is that a sequential scan is coded to > assume that every block between 0 and the last block in the relation > is worth reading, You don't change th

Re: [HACKERS] Extent Locks

2013-05-16 Thread Stephen Frost
Robert, For not understanding me, we seem to be in violent agreement. ;) * Robert Haas (robertmh...@gmail.com) wrote: > I think you might be confused, or else I'm confused, because I don't > believe we have any such thing as an extent lock. The relation extension lock is what I was referring t

Re: [HACKERS] Extent Locks

2013-05-16 Thread Robert Haas
On Wed, May 15, 2013 at 8:54 PM, Stephen Frost wrote: > Starting a new thread to avoid hijacking Heikki's original, but.. > > * Heikki Linnakangas (hlinnakan...@vmware.com) wrote: >> Truncating a heap at the end of vacuum, to release unused space back to >> the OS, currently requires taking an A

[HACKERS] Extent Locks

2013-05-15 Thread Stephen Frost
All, Starting a new thread to avoid hijacking Heikki's original, but.. * Heikki Linnakangas (hlinnakan...@vmware.com) wrote: > Truncating a heap at the end of vacuum, to release unused space back to > the OS, currently requires taking an AccessExclusiveLock. Although > it's only held for a shor