Simon,
> It's hard to work out how to reply to this because its just so off > base. I don't agree with the restrictions you think you see at all, > saying it politely rather than giving a one word answer. You have inside knowledge of Hannu's design. I am merely going from his description *on this list*, because that's all I have to go in. He requested comments, so here I am, commenting. I'm *hoping* that it's merely the description which is poor and not the conception of the feature. *As Hannu described the feature* it sounds useless and obscure, and miles away from powering any kind of general queueing mechanism. Or anything we discussed at the clustering meetings. And, again, if you didn't want comments, you shouldn't have posted an RFC. > All we're discussing is moving a successful piece of software into > core, which has been discussed for years at the international > technical meetings we've both been present at. I think an open > viewpoint on the feasibility of that would be reasonable, especially > when it comes from one of the original designers. When I ask you for technical clarification or bring up potential problems with a 2Q feature, you consistently treat it as a personal attack and are emotionally defensive instead of answering my technical questions. This, in turn, frustrates the heck out of me (and others) because we can't get the technical questions answered. I don't want you to justify yourself, I want a clear technical spec. I'm asking these questions because I'm excited about ReplicationII, and I want it to be the best feature it can possibly be. Or, as we tell many new contributors, "We wouldn't bring up potential problems and ask lots of questions if we weren't interested in the feature." Now, on to the technical questions: >> QUEUE emphasizes the aspect of logged only table that it accepts >> "records" in a certain order, persists these and then quarantees >> that they can be read out in exact the same order - all this being >> guaranteed by existing WAL mechanisms. >> >> It is not meant to be a full implementation of application level queuing >> system though but just the capture, persisting and distribution parts >> >> Using this as an "application level queue" needs a set of interface >> functions to extract the events and also to keep track of the processed >> events. As there is no general consensus what these shoul be (like if >> processing same event twice is allowed) this part is left for specific >> queue consumer implementations. While implementations vary, I think you'll find that the set of operations required for a full-featured application queue are remarkably similar across projects. Personally, I've worked with celery, Redis, AMQ, and RabbitMQ, as well as a custom solution on top of pgQ. The design, as you've described it, make several of these requirements unreasonably convoluted to implement. It sounds to me like the needs of internal queueing and application queueing may be hopelessly divergent. That was always possible, and maybe the answer is to forget about application queueing and focus on making this mechanism work for replication and for matviews, the two features we *know* we want it for. Which don't need the application queueing features I described AFAIK. > The two halves of the queue are the TAIL/entry point and the HEAD/exit > point. As you point out these could be on the different servers, > wherever the logical changes flow to, but could also be on the same > server. When the head and tail are on the same server, the MESSAGE > QUEUE syntax seems appropriate, but I agree that calling it that when > its just a head or just a tail seems slightly misleading. Yeah, that's why I was asking for clarification; the way Hannu described it, it sounded like it *couldn't* be read on the insert node, but only on a replica. > We do, I think, want a full queue implementation in core. We also want > to allow other queue implementations to interface with Postgres, so we > probably want to allow "first half" only as well. Meaning we want both > head and tail separately in core code. The question is whether we > require both head and tail in core before we allow commit, to which I > would say I think adding the tail first is OK, and adding the head > later when we know exactly the design. I'm just pointing out that some of the requirements of the design for the replication queue may conflict with a design for a full-featured application queue. I don't quite follow you on what you mean by "head" vs. "tail". Explain? > Having said that, the LOGGING ONLY syntax makes me shiver. Better name? I suck at names. Sorry. > I should also add that this is an switchable sync/asynchronous > transactional queue, whereas LISTEN/NOTIFY is a synchronous > transactional queue. Thanks for explaining. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
