Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-23 Thread Simon Riggs
On 15 May 2013 16:35, Heikki Linnakangas wrote: > Shared memory space is limited, but we only need the watermarks for any > in-progress truncations. Let's keep them in shared memory, in a small > fixed-size array. That limits the number of concurrent truncations that can > be in-progress, but tha

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-22 Thread Robert Haas
On Mon, May 20, 2013 at 10:19 AM, Heikki Linnakangas wrote: >> It shouldn't be difficult to restrict the set of backends that have to >> be signaled to those that have the relation open. You could have a >> special kind of catchup signal that means "catch yourself up, but >> don't chain" > > What

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-20 Thread Heikki Linnakangas
On 17.05.2013 12:35, Andres Freund wrote: On 2013-05-17 10:45:26 +0300, Heikki Linnakangas wrote: On 16.05.2013 04:15, Andres Freund wrote: Couldn't we "just" take the extension lock and then walk backwards from the rechecked end of relation ConditionalLockBufferForCleanup() the buffers? For ev

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-20 Thread Heikki Linnakangas
On 20.05.2013 16:59, Robert Haas wrote: On Fri, May 17, 2013 at 3:38 AM, Heikki Linnakangas wrote: If we could use the catchup interrupts to speed that up though, that would be much better. I think vacuum could simply send a catchup interrupt, and wait until everyone has caught up. That would

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-20 Thread Robert Haas
On Fri, May 17, 2013 at 3:38 AM, Heikki Linnakangas wrote: > If we could use the catchup interrupts to speed that up though, that would > be much better. I think vacuum could simply send a catchup interrupt, and > wait until everyone has caught up. That would significantly increase the > traffic o

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-17 Thread Andres Freund
On 2013-05-17 10:45:26 +0300, Heikki Linnakangas wrote: > On 16.05.2013 04:15, Andres Freund wrote: > >On 2013-05-15 18:35:35 +0300, Heikki Linnakangas wrote: > >>Truncating a heap at the end of vacuum, to release unused space back to > >>the OS, currently requires taking an AccessExclusiveLock. Al

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-17 Thread Heikki Linnakangas
On 16.05.2013 04:15, Andres Freund wrote: On 2013-05-15 18:35:35 +0300, Heikki Linnakangas wrote: Truncating a heap at the end of vacuum, to release unused space back to the OS, currently requires taking an AccessExclusiveLock. Although it's only held for a short duration, it can be enough to ca

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-17 Thread Heikki Linnakangas
On 16.05.2013 00:18, Robert Haas wrote: On Wed, May 15, 2013 at 11:35 AM, Heikki Linnakangas wrote: Shared memory space is limited, but we only need the watermarks for any in-progress truncations. Let's keep them in shared memory, in a small fixed-size array. That limits the number of concurre

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-16 Thread Robert Haas
On Thu, May 16, 2013 at 1:15 PM, Tom Lane wrote: > Robert Haas writes: >>> 2. If you don't find an entry for your target rel in the cache, aren't >>> you still going to have to do an lseek? > >> Don't think of it as a cache. The caching happens inside each >> backend's relcache; the shared memor

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-16 Thread Tom Lane
Robert Haas writes: >> 2. If you don't find an entry for your target rel in the cache, aren't >> you still going to have to do an lseek? > Don't think of it as a cache. The caching happens inside each > backend's relcache; the shared memory structure is just a tool to > force those caches to be

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-16 Thread Robert Haas
On Wed, May 15, 2013 at 8:24 PM, Tom Lane wrote: > Robert Haas writes: >> I've been thinking for a while that we need some other system for >> managing other kinds of invalidations. For example, suppose we want >> to cache relation sizes in blocks. So we allocate 4kB of shared >> memory, interp

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Andres Freund
On 2013-05-15 18:35:35 +0300, Heikki Linnakangas wrote: > Truncating a heap at the end of vacuum, to release unused space back to > the OS, currently requires taking an AccessExclusiveLock. Although it's only > held for a short duration, it can be enough to cause a hiccup in query > processing whil

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Tom Lane
Robert Haas writes: > I've been thinking for a while that we need some other system for > managing other kinds of invalidations. For example, suppose we want > to cache relation sizes in blocks. So we allocate 4kB of shared > memory, interpreted as an array of 512 8-byte entries. Whenever you >

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Robert Haas
On Wed, May 15, 2013 at 7:10 PM, Tom Lane wrote: >> Another problem is that sinval resets are bad for performance, and >> anything we do that pushes more messages through sinval will increase >> the frequency of resets. > > I've been thinking that we should increase the size of the sinval ring; >

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Tom Lane
Robert Haas writes: > On Wed, May 15, 2013 at 11:35 AM, Heikki Linnakangas > wrote: >> To not slow down common backend >> operations, the values (or lack thereof) are cached in relcache. To sync the >> relcache when the values change, there will be a new shared cache >> invalidation event to forc

Re: [HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Robert Haas
On Wed, May 15, 2013 at 11:35 AM, Heikki Linnakangas wrote: > Shared memory space is limited, but we only need the watermarks for any > in-progress truncations. Let's keep them in shared memory, in a small > fixed-size array. That limits the number of concurrent truncations that can > be in-progre

[HACKERS] Heap truncation without AccessExclusiveLock (9.4)

2013-05-15 Thread Heikki Linnakangas
Truncating a heap at the end of vacuum, to release unused space back to the OS, currently requires taking an AccessExclusiveLock. Although it's only held for a short duration, it can be enough to cause a hiccup in query processing while it's held. Also, if there is a continuous stream of querie