On Fri, May 24, 2013 at 12:35 PM, Robert Haas <robertmh...@gmail.com> wrote: > But I don't know what to do about the problem of needing to know how > many backends there are. I agree with Andres that it's not very > friendly to enforce a restriction that all the same worker processes > must be present on the standby.
OK, I have an idea for fixing this. This would be 9.4 material; I'm happy enough to leave this broken for 9.3, as the chances of anyone hitting it in practice seem remote, and if they do, the workaround is not hard. What I'd like to do is change the way that we allocate slots for bgworkers. Instead of computing the number of slots based on how many bgworkers are registered at postmaster startup time, I'd like to allocate a fixed number of slots, controlled by a GUC called max_worker_processes, which will obviously be PGC_POSTMASTER. I would propose to give this GUC a small but non-zero default value; I was thinking of 8. Under this proposal, it would no longer possible to register an unlimited number of background workers without any changes to postgresql.conf; if you want to register more of them than the default value of this parameter, you'll need to increase the parameter. That's a downside, but it's one that I think will affect very few users; how many people are going to be running installations with more than 8 background workers? (And, those that are probably pretty sophisticated users who won't suffer greatly from having to change two GUCs rather than just one.) So what's the upside? One advantage, obviously, is that we fix this bug. max_worker_processes would get propagated the standby, and checked, just as max_connections does today. The standby doesn't have to actually load the same extensions; it just has to allow for at least as many slots, which is cleaner. But that's pretty minor. I see a much larger advantage: we wouldn't be limited to worker procesess that are started up at fixed times in the postmaster lifecycle, as we currently are. Right now, background workers can only be started at three points: BgWorkerStart_PostmasterStart, BgWorkerStart_ConsistentState, and BgWorkerStart_RecoveryFinished. This change would allow us to move towards worker processes that can be started "on the fly", whenever needed, just as long as there are available slots from the total allocation carved out by max_worker_processes. I think you can see where I'm going here: aside from giving extension code more options in terms of how it uses worker processes, this would give us a facility that would be a useful building block toward parallel sort/copy/query/whatever. Even without parallelism in core, it's not too difficult to imagine a contrib module that does some kind of ad-hoc parallelism by firing up worker backends, which would be awkward with the 9.3 infrastructure. But the real big win is that the core facilities, when and as they arrive, can draw from that same pool of workers. Thoughts? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
