We're not looking for a message queue, we're looking for a notifications system ala AWS SNS. CloudStack needs to be able to *push* events out to endpoints; not send them to an MQ for interested parties to pull.
-George On May 18, 2012, at 12:09 PM, Chiradeep Vittal wrote: > While embedding the publisher inside CloudStack could be one way to go, it is > probably more immediately useful if we push events to a popular queueing > system such as AMQP? > Having CS have its own unique way of publishing events does make it more work > for others to utilize these events. > Agree on having a design that allows for other systems of notifications. > > From: George Reese <george.re...@enstratus.com> > Date: Fri, 18 May 2012 05:24:42 -0700 > To: "cloudstack-dev@incubator.apache.org" > <cloudstack-dev@incubator.apache.org> > Cc: Nitin Mehta <nitin.me...@citrix.com>, Kishan Kavala > <kishan.kav...@citrix.com>, Dean <cl...@tizatron.com>, Chiradeep Vittal > <chiradeep.vit...@citrix.com> > Subject: Re: Event Publish and Subscribe ( was perhaps Re: Cloudstack > Questions ) > > Here's the way event pub sub should work… > > People interested in events will subscribe to them via API and CloudStack > will maintain a record that includes: > > * What type of event(s) the subscriber cares about > * The URI of an endpoint to notify on events > * An optional API key for signing event publications > > If you want it to be really rich, you allow for multiple notifications > endpoints, including SMS and email. > > When the existing event management system in CloudStack identifies a state > change, it published it to a message queue. > > A new CloudStack component is subscribed to the message queue and pulls the > state change off. It then sends the event to all interested parties. It does > not keep delivery state. It sends and forgets. The only thing resembling > state is the counting of failures, eventually killing an endpoint that fails > to often. > > -George > > On May 17, 2012, at 10:53 PM, Abhinandan Prateek wrote: > >> Cloudstack already have a well-defined set of events, currently being used >> by the Usage module. >> We can extend this to publish the events to registered listeners/subscribers. >> >> Nitin, >> We should evaluate the google guava package for this purpose. >> >> -abhi >> >>> -----Original Message----- >>> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] >>> Sent: Saturday, May 12, 2012 3:48 AM >>> To: cloudstack-dev@incubator.apache.org >>> Cc: Dean >>> Subject: Re: Event Publish and Subscribe ( was perhaps Re: Cloudstack >>> Questions ) >>> >>> +1. >>> This could also be used for example to update DNS records on a DNS server >>> whenever a VM is started / stopped in a network. >>> A subscriber to the event bus can do this, and potentially things like >>> updating a >>> CMDB or an LDAP server. >>> >>> The important caveat is that this will be asynchronous to the stuff >>> happening >>> inside the management server: it has limited utility for doing stuff that >>> is >>> required to succeed before an API operation can succeed. >>> For example, if a VM start requires a NAT rule to be programmed on the >>> firewall before the start can be considered successful. You could write a >>> event >>> bus subscriber that does this, but the management server would not know if >>> it was successful or not. >>> >>> There's at least a couple of entry points: >>> 1. com.cloud.utils.fsm.StateListener. Classes that implement this listener >>> get >>> notified on finite state event transitions. Examples include VM start / >>> stop 2. >>> com.cloud.network.element.NetworkElement. Classes that implement this >>> get notified of network state transitions. Examples include adding nics and >>> removing nics to/from a network. >>> >>> I think the first question regarding monitoring VMs on the isolated VLAN >>> had a >>> different origin though. >>> The intention there was to have a monitoring service (e.g., Nagios) reach >>> into >>> the VM and monitor stuff like CPU, IO, or even applications. >>> The issue there was around network reachability. >>> >>> -- >>> Chiradeep >>> >>> >>> >>> On 5/11/12 2:09 PM, "Adrian Cole" <adr...@jclouds.org> wrote: >>> >>>> +1 >>>> >>>> Makes sense to have pubsub. Inside the java codebase, we could >>>> consider a clean and idiomatic lib like guava which is easy to unit test. >>>> >>>> http://codingjunkie.net/guava-eventbus/ >>>> >>>> Then, expose out-of-JVM hooks for any of the popular services people use. >>>> >>>> -A >>>> On May 11, 2012 1:58 PM, "Dean" <cl...@tizatron.com> wrote: >>>> >>>>> Cross reference to: >>>>> >>>>> >>>>> >>>>> http://mail-archives.apache.org/mod_mbox/incubator-cloudstack- >>> dev/201204. >>>>> mbox/browser >>>>> >>>>> [ from: Marlon Davids ] >>>>> < munch > >>>>>> 2) How do we monitor VM's that are in Cloudstack when they are in >>>>>> an >>>>> isolated VLAN does >>>>>> anyone have a clever workaround? >>>>>> 3) Has anyone developed a script for parsing and alerting on >>>>>> warning >>>>> events in the >>>>>> management Log yet? >>>>> >>>>> I would like to propose cloudstack consider a pub/sub model for event >>>>> handling to complement API calls like listEvents. >>>>> >>>>> Polling can be problematic and sensitive to scaling. >>>>> >>>>> A simple example would be state change on a physical device. The >>>>> admin server can simply publish a message on a network socket >>>>> indicating that the device has changed it's state. >>>>> >>>>> If a subscriber was interested in that device, it could make an api >>>>> call to the admin server for state change information for that device >>>>> only. >>>>> The >>>>> admin server may choose to validate that physical device against the >>>>> current state table in the database. >>>>> >>>>> The API would reply that this node changed it's state from >>>>> Operational to Prep For Maintenance. (or whatever the transition >>>>> state would be) >>>>> >>>>> The message exchange could be wrapped around vm states, resource >>>>> additions/removals etc. >>>>> >>>>> Using a library like zeromq, a developer can write any number of >>>>> consumers in any language they wanted to subscribe to the Event Bus. >>>>> >>>>> Comments? >>>>> >>>>> -- >>>>> -Dean >> > > -- > George Reese - Chief Technology Officer, enStratus > e: george.re...@enstratus.com Skype: nspollution t: @GeorgeReese p: > +1.207.956.0217 > enStratus: Enterprise Cloud Management - @enStratus - http://www.enstratus.com > To schedule a meeting with me: http://tungle.me/GeorgeReese > -- George Reese - Chief Technology Officer, enStratus e: george.re...@enstratus.com Skype: nspollution t: @GeorgeReese p: +1.207.956.0217 enStratus: Enterprise Cloud Management - @enStratus - http://www.enstratus.com To schedule a meeting with me: http://tungle.me/GeorgeReese
