On Wed, May 9, 2012 at 3:17 PM, Tihomir Trifonov <t.trifo...@gmail.com>wrote:
> Hi Doug, > > not sure if you've hit the same problem, but I've spent some time on that > when I started using RabbitMQ. As I see from the example, you've provided: > > queue = Queue(name='notifications.info', > > exchange=Exchange(name='nova'... > > > > So you set explicitly a name for the queue. If you have two queues > declared with the same name, when they are bound to an Exchange, actually > each message is received only by the one queue at a time. Just leave the > name field empty(it will be auto-generated), and each queue will receive > its copy of the message. So the logic is that the first queue with that > name acknowledges the message, and the other one receives nothing. > > > Besides that, the topics are more powerful and handy to use than fanout. > Now that I see how it works, I agree. :-) > > > > On Wed, May 9, 2012 at 6:34 PM, Doug Hellmann <doug.hellm...@dreamhost.com > > wrote: > >> >> >> On Wed, May 9, 2012 at 6:42 AM, Kiall Mac Innes <ki...@managedit.ie>wrote: >> >>> Kinda! The queue has a name, but that name has no bearing on the set of >>> messages received. >> >> >>> If you create a queue called "MyCustomNotificationQueue", you can bind >>> that to the "notifications" exchange using the "notifications.info" >>> routing key. >>> >>> (I'm guessing some of the names here.. I know AMQP, and not the specific >>> naming nova uses!) >>> >> >> notifications.info is right. >> >> >>> >>> Nova just happens to use the same queue name and routing key. I believe >>> this is causing the confusion. >>> >> >> Exactly. I ended up creating a separate queue for each client that I have >> and setting them to auto-delete when the client disconnects. That way I can >> have as many clients connecting and listening as I want. The code is in >> https://github.com/dhellmann/metering-prototype if you want to take a >> look. >> >> >>> >> >>> >>> Does this make sense? >>> >>> Anyway - The RabbitMQ docs probably explain it better than I.. >>> http://www.rabbitmq.com/tutorials/tutorial-five-python.html >>> >>> Thanks, >>> Kiall >>> >>> >>> >>> On Wed, May 9, 2012 at 11:30 AM, Day, Phil <philip....@hp.com> wrote: >>> >>>> OK, get that so far – so both consumers need to declare and use the >>>> same exchange.**** >>>> >>>> ** ** >>>> >>>> But If I understand the next step right, to get multiple consumers of >>>> info notification messages they would all need to create separate “ >>>> notifications.info” queues into that exchange. And isn’t that >>>> exactly what Nova currently does to create a shared queue ?**** >>>> >>>> ** ** >>>> >>>> Phil**** >>>> >>>> ** ** >>>> >>>> *From:* Kiall Mac Innes [mailto:ki...@managedit.ie] >>>> *Sent:* 09 May 2012 10:51 >>>> *To:* Day, Phil >>>> *Cc:* openstack@lists.launchpad.net; Russell Bryant; Doug Hellmann >>>> >>>> *Subject:* Re: [Openstack] [nova] why does notification use a "topic" >>>> exchange instead of "fanout"?**** >>>> >>>> ** ** >>>> >>>> Your own queue listener should attempt to declare the exchange, using >>>> the same settings as Nova does. **** >>>> >>>> If the exchange exists, its a noop. Otherwise it's created for you.**** >>>> >>>> After that, if you start up Nova, it will do the same and reuse your >>>> exchange.**** >>>> >>>> Obviously this works both ways, and either nova or your code can >>>> declare the exchange.**** >>>> >>>> AMQP is designed to be a configuration-less protocol, where resources >>>> are configured by the first consumer attempting to use them.**** >>>> >>>> Thanks, >>>> Kiall**** >>>> >>>> Sent from my phone.**** >>>> >>>> On May 9, 2012 9:52 a.m., "Day, Phil" <philip....@hp.com> wrote:**** >>>> >>>> Hi Doug,**** >>>> >>>> **** >>>> >>>> > I think you missed my main point, which was that a topic exchange does >>>> > not impose a limitation that only one client can consume a given >>>> > notification. That's only true if each client is consuming from the >>>> > same queue bound to the exchange.**** >>>> >>>> **** >>>> >>>> So just to be clear, if I understand you correctly within the nova >>>> service/rpc abstraction layers the code is set up so that all services do >>>> bind to the same queue, and hence we get the round-robin delivery.**** >>>> >>>> But, if someone wanted to write a separate notification consumer so >>>> that they didn’t block anyone else from seeing the same messages then they >>>> (the consumer) should create a new queue on the existing topic exchange. >>>> **** >>>> >>>> **** >>>> >>>> Is that correct – and is there any worked example of doing this ?**** >>>> >>>> **** >>>> >>>> I thought within the nova code both the exchange and topic queues were >>>> set up by the consumer (so for example all compute_managers try to create >>>> the “compute” exchange and topic queue, but its only created by the first >>>> one and the others connect to the same queue). In that context I’m >>>> finding it hard to see how to change this model to have multiple “ >>>> notify.info” topic queues into the same exchange ?**** >>>> >>>> **** >>>> >>>> Cheers,**** >>>> >>>> Phil**** >>>> >>>> **** >>>> >>>> **** >>>> >>>> **** >>>> >>>> **** >>>> >>>> *From:* openstack-bounces+philip.day=hp....@lists.launchpad.net[mailto: >>>> openstack-bounces+philip.day=hp....@lists.launchpad.net] *On Behalf Of >>>> *Doug Hellmann >>>> *Sent:* 08 May 2012 23:34 >>>> *To:* Russell Bryant >>>> *Cc:* openstack@lists.launchpad.net >>>> *Subject:* Re: [Openstack] [nova] why does notification use a "topic" >>>> exchange instead of "fanout"?**** >>>> >>>> **** >>>> >>>> **** >>>> >>>> On Tue, May 8, 2012 at 6:04 PM, Russell Bryant <rbry...@redhat.com> >>>> wrote:**** >>>> >>>> On 05/08/2012 05:59 PM, Doug Hellmann wrote: >>>> > Here is a relevant section pulled out of the amqp 0-9-1 spec: >>>> > >>>> > 3.1.3.3 The Topic Exchange Type >>>> > >>>> > The topic exchange type works as follows: >>>> > >>>> > 1. A message queue binds to the exchange using a routing >>>> > pattern, P. >>>> > 2. A publisher sends the exchange a message with the >>>> routing >>>> > key R. >>>> > 3. The message is passed to the message queue if R matches >>>> P. >>>> > >>>> > The routing key used for a topic exchange MUST consist of zero >>>> or >>>> > more words delimited by dots. Each word may contain the >>>> letters A-Z >>>> > and a-z and digits 0-9. >>>> > >>>> > The routing pattern follows the same rules as the routing key >>>> with >>>> > the addition that * matches a single word, and # matches zero >>>> or >>>> > more words. Thus the routing pattern *.stock.# matches the >>>> routing >>>> > keys usd.stock and eur.stock.db but not stock.nasdaq. >>>> > >>>> > In nova, for a given topic such as 'scheduler', all of the >>>> consumers are >>>> > binding to the same queue on the topic exchange, resulting in >>>> > round-robin delivery to each of the consumers. If instead you >>>> make a >>>> > new queue, you can get your own copy of each message. >>>> > >>>> > There is an additional benefit of using a topic exchange here. >>>> The >>>> > topic used for notifications is 'notifications.<priority>'. That >>>> means >>>> > that when you create your queue, you can set it up to receive all >>>> > notifications, or only notifications of a certain priority. >>>> > >>>> > >>>> > Topic exchanges make a lot of sense for messages that should only be >>>> > consumed once, such as tasks. Notifications are different. Lots of >>>> > different clients might want to know that some event happened in the >>>> > system. The way things are in Nova today, they can't. The first client >>>> > who consumes a notification message will prevent all of the other >>>> > clients from seeing that message at all.**** >>>> >>>> I think you missed my main point, which was that a topic exchange does >>>> not impose a limitation that only one client can consume a given >>>> notification. That's only true if each client is consuming from the >>>> same queue bound to the exchange.**** >>>> >>>> **** >>>> >>>> Yes, that wasn't obvious from any of the kombu documentation I've seen >>>> so far. I'll keep looking.**** >>>> >>>> **** >>>> >>>> Thanks,**** >>>> >>>> Doug**** >>>> >>>> **** >>>> >>>> >>>> > I can change Nova's notification system to use a fanout exchange (in >>>> > impl_kombu.py changing the exchange type used by NotifyPublisher), but >>>> > before I submit a patch I want to make sure the current implementation >>>> > using a topic exchange wasn't selected deliberately for some reason.* >>>> *** >>>> >>>> I think using a fanout exchange would be a downgrade. As I mentioned >>>> before, a topic exchange allows you to create a queue to get all >>>> notifications or only notifications of a specific priority. If the >>>> exchange type is changed to fanout, it's everybody gets everything, and >>>> that's it. >>>> >>>> -- >>>> Russell Bryant**** >>>> >>>> **** >>>> >>>> >>>> _______________________________________________ >>>> Mailing list: https://launchpad.net/~openstack >>>> Post to : openstack@lists.launchpad.net >>>> Unsubscribe : https://launchpad.net/~openstack >>>> More help : https://help.launchpad.net/ListHelp**** >>>> >>> >>> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp >> >> > > > -- > Regards, > Tihomir Trifonov >
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
