Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-09 Thread Peter Geoghegan
On Thu, Mar 9, 2017 at 6:25 PM, Robert Haas wrote: > On Thu, Mar 9, 2017 at 7:29 PM, Thomas Munro > wrote: >> Suppressing ENOENT because it's not clear which backend actually owns >> a file is a bit different from using it to detect that there are no >> more segments... > > +1, emphatically. I d

Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-09 Thread Robert Haas
On Thu, Mar 9, 2017 at 7:29 PM, Thomas Munro wrote: > Suppressing ENOENT because it's not clear which backend actually owns > a file is a bit different from using it to detect that there are no > more segments... +1, emphatically. > Obviously I screwed some things up in the code I > posted, but

Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-09 Thread Robert Haas
On Thu, Mar 9, 2017 at 2:19 PM, Peter Geoghegan wrote: > Quite a lot of thought seems to have gone into making the > fd.c/resowner mechanism leak-proof in the face of errors. So, quite > apart from what that approaches misses out on, I really don't want to > change resource management too much. As

Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-09 Thread Peter Geoghegan
On Thu, Mar 9, 2017 at 4:29 PM, Thomas Munro wrote: > On Fri, Mar 10, 2017 at 8:19 AM, Peter Geoghegan wrote: >> by having state for each segment, it ends up actually *relying on* >> ENOENT-on-unlink() as a condition that terminates execution: > > Yeah, this seems to fall out of the requirement t

Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-09 Thread Thomas Munro
On Fri, Mar 10, 2017 at 8:19 AM, Peter Geoghegan wrote: > by having state for each segment, it ends up actually *relying on* > ENOENT-on-unlink() as a condition that terminates execution: Yeah, this seems to fall out of the requirement to manage a growable number of partition files in a fixed spa

Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-09 Thread Peter Geoghegan
On Thu, Mar 9, 2017 at 11:19 AM, Peter Geoghegan wrote: > That patch seems to be solving the problem by completely taking over > management of temp files from fd.c. That is, these temp files are not > marked as temp files in the way ordinary temp BufFiles are, with > explicit buy-in from resowner.

Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-09 Thread Peter Geoghegan
On Wed, Mar 8, 2017 at 5:23 PM, Robert Haas wrote: > I think something like 0007-hj-shared-buf-file-v6.patch from > https://www.postgresql.org/message-id/CAEepm=3g=dMG+84083fkFzLvgMJ7HdhbGB=aezabnukbzm3...@mail.gmail.com > is probably a good approach to this problem. In essence, it dodges > the p

Re: [HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-08 Thread Robert Haas
On Mon, Mar 6, 2017 at 7:33 PM, Peter Geoghegan wrote: > Recap > = > > A design goal of parallel tuplesort is that the parallel case be as > close to possible as the serial case is already. Very little new code > is needed in tuplesort.c and logtape.c, most of which is about using a > new lowe

[HACKERS] on_dsm_detach() callback and parallel tuplesort BufFile resource management

2017-03-06 Thread Peter Geoghegan
Recap = A design goal of parallel tuplesort is that the parallel case be as close to possible as the serial case is already. Very little new code is needed in tuplesort.c and logtape.c, most of which is about using a new lower-level facility which is very well encapsulated. And, even buffile.c