Re: [HACKERS] bg worker: overview

2010-07-26 Thread Dimitri Fontaine
Markus Wanner writes: > To simplify, you might want to start a bgworker on database 'postgres', > which then acts as a sub-coordinator (and doesn't really need to use its > database connection). Yeah, that sounds like the simplest way forward, so that it's easy for this "user daemon" to communica

Re: [HACKERS] bg worker: overview

2010-07-25 Thread Markus Wanner
Hey Dimitri, On 07/24/2010 07:26 PM, Dimitri Fontaine wrote: Trying to figure out how it would fit the PGQ and pgagent needs. But maybe user defined daemons should be sub-coordinators (I used to think about them as "supervisors") able to talk to the coordinator to get a backend connected to some

Re: [HACKERS] bg worker: overview

2010-07-24 Thread Dimitri Fontaine
Markus Wanner writes: > For one, yes, I want to avoid having to start ones too often. I did look > into letting these background workers switch the database connection, but > that turned out not to be worth the effort. > > Would you prefer a background worker that's not connected to a database, or

Re: [HACKERS] bg worker: overview

2010-07-24 Thread Markus Wanner
Hi, On 07/23/2010 09:45 PM, Dimitri Fontaine wrote: Yeah, I guess user daemons would have to be workers, not plugins you want to load into the coordinator. Okay. On the other side, the background workers have a connection to exactly one database. They are supposed to do work on that database

Re: [HACKERS] bg worker: overview

2010-07-23 Thread Dimitri Fontaine
Markus Wanner writes: > Daemon code? That sounds like it could be an addition to the > coordinator, which I'm somewhat hesitant to extend, as it's a pretty > critical process (especially for Postgres-R). [...] > However, note that the coordinator is designed to be just a message > passing or routi

Re: [HACKERS] bg worker: overview

2010-07-17 Thread Simon Riggs
On Sat, 2010-07-17 at 13:47 +0200, Markus Wanner wrote: > Are the descriptive mails I sent for each patch going into the right > direction and just need to be extended, in your opinion? Or are you > really missing something in there? Not detailed enough, for me, by a long way. Your notes read l

Re: [HACKERS] bg worker: overview

2010-07-17 Thread Markus Wanner
Sorry, hit send too early. On 07/17/2010 01:47 PM, Markus Wanner wrote: I think that I commented the source code pretty extensively, however, that's a subjective feeling. Take this phrase. I'm under the impression, that I commented the source code pretty well. Scratch that, please. :-)

Re: [HACKERS] bg worker: overview

2010-07-17 Thread Markus Wanner
Hello Simon, On 07/17/2010 12:30 PM, Simon Riggs wrote: The code itself merely reflects your design, so what I would really like to see is a full explanation of this. Are the descriptive mails I sent for each patch going into the right direction and just need to be extended, in your opinion?

Re: [HACKERS] bg worker: overview

2010-07-17 Thread Simon Riggs
On Tue, 2010-07-13 at 16:30 +0200, Markus Wanner wrote: > I've combined these two components into a single, general purpose > background worker infrastructure component I think many people want such a feature, so the requirement is good. The code itself merely reflects your design, so what I wo

Re: [HACKERS] bg worker: overview

2010-07-15 Thread Alvaro Herrera
Excerpts from Jaime Casanova's message of jue jul 15 15:51:10 -0400 2010: > On Thu, Jul 15, 2010 at 1:28 PM, Markus Wanner wrote: > > > > However, note that the coordinator is designed to be just a message > > passing or routing process, which should not do any kind of time > > consuming processin

Re: [HACKERS] bg worker: overview

2010-07-15 Thread Markus Wanner
Hi, On 07/15/2010 09:51 PM, Jaime Casanova wrote: > so, merging this with the autovacuum will drop our hopes of having a > time based autovacuum? not that i'm working on that nor i was thinking > on working on that... just asking to know what the implications are, > and what the future improves co

Re: [HACKERS] bg worker: overview

2010-07-15 Thread Jaime Casanova
On Thu, Jul 15, 2010 at 1:28 PM, Markus Wanner wrote: > > However, note that the coordinator is designed to be just a message > passing or routing process, which should not do any kind of time > consuming processing. It must *coordinate* things (well, jobs) and react > promptly. Nothing else. > s

Re: [HACKERS] bg worker: overview

2010-07-15 Thread Markus Wanner
Hi, On 07/15/2010 03:45 PM, Dimitri Fontaine wrote: > We've been talking about this topic on -performance: Thank for pointing out this discussion, I'm not following -performance too closely. > So, do you think we could use your work as a base for allowing custom > daemon code? Daemon code? That

Re: [HACKERS] bg worker: overview

2010-07-15 Thread Dimitri Fontaine
Hi, We've been talking about this topic on -performance: Markus Wanner writes: > I've combined these two components into a single, general purpose background > worker infrastructure component, which is now capable to serve autovacuum as > well as Postgres-R. And it might be of use for other purp

Re: [HACKERS] bg worker: overview

2010-07-14 Thread Markus Wanner
Hi, On 07/13/2010 08:45 PM, Kevin Grittner wrote: > You could submit them as Work In Progress patches Okay, I added them. I guess they get more attention that way. Regards Markus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: ht

Re: [HACKERS] bg worker: overview

2010-07-13 Thread Kevin Grittner
Markus Wanner wrote: > (I don't dare to add these patches to the commit fest, as this > refactoring doesn't have any immediate benefit for Postgres > itself, at the moment.) You could submit them as Work In Progress patches -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@

[HACKERS] bg worker: overview

2010-07-13 Thread Markus Wanner
Hi, I've been working on modularizing Postgres-R to ease review and maybe allow code reuse. As threatened at the Cluster Meeting in Tokyo and again at CHAR(10), I'm now presenting more results of that effort: the background workers infrastructure module. Postgres-R so far used custom backend