On Wed, Oct 17, 2012 at 4:25 PM, Josh Berkus <j...@agliodbs.com> wrote: >> It is not meant to be a full implementation of application level queuing >> system though but just the capture, persisting and distribution parts >> >> Using this as an "application level queue" needs a set of interface >> functions to extract the events and also to keep track of the processed >> events. As there is no general consensus what these shoul be (like if >> processing same event twice is allowed) this part is left for specific >> queue consumer implementations. > > Well, but AFAICT, you've already prohibited features through your design > which are essential to application-level queues, and are implemented by, > for example, pgQ. > > 1. your design only allows the queue to be read on replicas, not on the > node where the item was inserted. > > 2. if you can't UPDATE or DELETE queue items -- or LOCK them -- how on > earth would a client know which items they have executed and which they > haven't? > > 3. Double-down on #2 in a multithreaded environment. > > For an application-level queue, the base functionality is: > > ADD ITEM > READ NEXT (#) ITEM(S) > LOCK ITEM > DELETE ITEM > > More sophisticated an useful queues also allow: > > READ NEXT UNLOCKED ITEM > LOCK NEXT UNLOCKED ITEM > UPDATE ITEM > READ NEXT (#) UNSEEN ITEM(S) > > The design you describe seems to prohibit pretty much all of the above > operations after READ NEXT. This makes it completely useless as a > application-level queue. > > And, for that matter, if your new queue only accepts INSERTs, why not > just improve LISTEN/NOTIFY so that it's readable on replicas? What does > this design buy you that that doesn't?
I've read the whole thread, but I still don't see that anyone's said it better than this, and I agree with these comments. (I don't find them ad hominem, either.) It's also worth noting that in order to be useful, this feature intrinsically requires the logical replication stuff to be committed first. It's not entirely clear that there's not enough time to get logical replication committed for 9.3, and the chances of getting any follow-on features getting committed as well seems remote. Besides the shortness of the time, I think all experience has shown that it's best not to rush into the design of follow-on features before we've got the basic feature well nailed down. This certainly can't be said of logical replication at this point. Andres seems to be making good progress and I'm grateful for his work on it, but I think there's a lot left to do before that one is in the bag (as I think Andres would agree). -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
