Re: [HACKERS] WIP: Barriers

2017-01-06 Thread Thomas Munro
On Wed, Nov 23, 2016 at 2:28 PM, Robert Haas wrote: > The code here looks OK. A few thoughts: > > - I'm a little unsure whether it's a good idea to remove the existing > barrier.h and immediately add a new barrier.h that does something > totally different. It's tempting to try to find another na

Re: [HACKERS] WIP: Barriers

2016-11-22 Thread Robert Haas
On Tue, Nov 22, 2016 at 4:42 PM, Thomas Munro wrote: > On Tue, Nov 1, 2016 at 5:03 PM, Thomas Munro >> Here's a new version which is rebased and adds support for passing >> wait_event through to pg_stat_activity. > > Here's a version updated for the new conditional variables interface > which has

Re: [HACKERS] WIP: Barriers

2016-11-22 Thread Thomas Munro
On Tue, Nov 1, 2016 at 5:03 PM, Thomas Munro wrote: > On Thu, Aug 18, 2016 at 1:55 PM, Thomas Munro > wrote: >> On Tue, Aug 16, 2016 at 1:55 AM, Robert Haas wrote: >>> If we're going to remove barrier.h, I think that should be a separate >>> commit from creating a new barrier.h. >> >> OK. Here'

Re: [HACKERS] WIP: Barriers

2016-10-31 Thread Thomas Munro
On Thu, Aug 18, 2016 at 1:55 PM, Thomas Munro wrote: > On Tue, Aug 16, 2016 at 1:55 AM, Robert Haas wrote: >> If we're going to remove barrier.h, I think that should be a separate >> commit from creating a new barrier.h. > > OK. Here's a patch to remove the old header, and the v2 barrier patch >

Re: [HACKERS] WIP: Barriers

2016-08-17 Thread Thomas Munro
On Tue, Aug 16, 2016 at 1:55 AM, Robert Haas wrote: > On Sat, Aug 13, 2016 at 7:18 PM, Thomas Munro > wrote: >> My use case for this is coordinating the phases of parallel hash >> joins, but I strongly suspect there are other cases. Parallel sort >> springs to mind, which is why I wanted to post

Re: [HACKERS] WIP: Barriers

2016-08-16 Thread Peter Geoghegan
On Sat, Aug 13, 2016 at 4:18 PM, Thomas Munro wrote: > First, you initialise a Barrier object somewhere in shared memory, > most likely in the DSM segment used by parallel query, by calling > BarrierInit(&barrier, nworkers). Then workers can call > BarrierWait(&barrier) when they want to block un

Re: [HACKERS] WIP: Barriers

2016-08-16 Thread Peter Geoghegan
On Mon, Aug 15, 2016 at 6:55 AM, Robert Haas wrote: > A sort of dumb way of handling all this is to assume that once a > worker joins the hash join, it won't go off and do anything else until > the hash join is done. Under that assumption, you just need some sort > of BarrierAttach() operation; w

Re: [HACKERS] WIP: Barriers

2016-08-16 Thread Peter Geoghegan
On Mon, Aug 15, 2016 at 6:55 AM, Robert Haas wrote: > The simple version of this is that when a worker gets done with its > own probe phase for batch X, it can immediately start building the > hash table for phase X+1, stopping if it fills up the unused portion > of work_mem before the old hash ta

Re: [HACKERS] WIP: Barriers

2016-08-15 Thread Konstantin Knizhnik
On 15.08.2016 15:42, Thomas Munro wrote: This implementation is using a spinlock for the arrival counter, and signals (via Robert's condition variables and latches) for waking up peer processes when the counter reaches the target. I realise that using signals for this sort of thing is a bit un

Re: [HACKERS] WIP: Barriers

2016-08-15 Thread Robert Haas
On Sat, Aug 13, 2016 at 7:18 PM, Thomas Munro wrote: > I would like to propose "barriers" for Postgres processes. A barrier > is a very simple mechanism for coordinating parallel computation, as > found in many threading libraries. > > First, you initialise a Barrier object somewhere in shared me

Re: [HACKERS] WIP: Barriers

2016-08-15 Thread Thomas Munro
On Sun, Aug 14, 2016 at 6:54 PM, konstantin knizhnik wrote: > Barriers are really very simple and convenient mechanism for process > synchronization. > But it is actually a special case of semaphores: having semaphore primitive > it is trivial to implement a barrier. > We have semaphores in Post

Re: [HACKERS] WIP: Barriers

2016-08-13 Thread konstantin knizhnik
Hi Thomas, Barriers are really very simple and convenient mechanism for process synchronization. But it is actually a special case of semaphores: having semaphore primitive it is trivial to implement a barrier. We have semaphores in Postgres, but ... them can not be used by extensions: there is