On Mon, Jan 28, 2013 at 10:17 AM, Murali Reddy <murali.re...@citrix.com> wrote: > I would like to request merge of events framework [4] branch changes in to > master to branch. > > I have proposed a while ago [1], about the need for a framework to > Publish-Subscribe CloudStack to events. Initially I was planning to use > simple in-memory based publish-subscribe mechanism using google event bus > library as described in the FS [3]. But considering the long term goal of > CloudStack to decouple orchestration subsystems to independently runnable > services as proposed architecture refactoring [2], and as per the > discussion followed, I have taken approach of publishing CloudStack events > to a messaging broker. Context diagram of how CloudStack and MOM (message > oriented middleware) fits in documented at [4]. > > Idea was to enable two use cases. > > - external components like CMDB, billing systems, etc can directly get the > events from the message broker, enabling complete decoupling. Also get > instant notification of the events (like new VM create etc) to trigger > further processing. > > - be able to create a notification service which can push notifications to > different end points (web call back, sms, email) line AWS SNS. > > As part of this effort I am introducing a event bus abstraction which is > realized by an interface [5] providing publish, subscribe, unsubscribe > semantics. One can have a plug in that implements this interface to > provide the actually event bus implementation. CloudStack core uses the > interface to publish/subscribe to the events. CloudStack will by default > will publish all Action Events, Usage Events, Alerts, Resource State > change events. Resource state change is new type of event introduced which > gets generated on resource (Vm, volume, snapshot, network etc) state > changes. > > I looked at the options to provide a implementation using the framework. > Integrating with a AMQP complaint message broker seemed better due to > availability of multiple implementations (Rabbit MQ, Active MQ, Qpid etc). > I used rabbitMQ [6] java AMQP client to interact with AMQP servers. > > AMQP has a publish-subscribe paradigm. AMQP has notion of 'Exchanges' and > 'Queues'. Messages are published on to the 'Exchanges' specifying a > routing key. Queue's are subscriber that bound to exchanges with a > specific binding key. AMQP implementation is responsible for delivering a > message in exchange to queue's that has the matching binding key. On > message arrival in a queue, AMQP server can send call back to consumers. > > So the plug-in I have implemented using rabbit MQ, publishes events by > CloudStack on to configured exchange on AMQP server. For each of the > subscriber plug-in is responsible for creating a queue with specified > event topic. > > Routing is designed to have the format. > > Event-source.Event-Category.Event-Type.Resource.ResourceUUID. For e.g. A > message is published with a routing key: > management-server:ActionEvent:SNAPSHOT-CREATE:Snapshot:0a7ea29e-691b-11e2-b > afa-2c3ba27d8c47. > > A subscriber interested in receiving all the events corresponding to a VM > with a UUID 9d827485-0f46-4db8-bd39-fede97cbac0c would result in a queue > with a binding key > > *.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c. > > This feature is contained, does not touch business logic of the > subsystems, and hooks in to convenient classes that are used to persist > Action, Usage and Alert events in to DB. Also at run time if no plug-in is > configured that provides implementation of EventBus abstraction, there > will not have any impact of CloudStack. > > I have added state machine to VirtualMachine, Network, Volume, Snapshot > resources, and on state transition generates a resource state change > notification. I can add port merge, if any other critical resource for > which state change event would be valuable. > > 'events-framework' branch is upto date with master and there are no > unit-test failures. > > [1] http://markmail.org/thread/tmqagkbj7urzcouf > [2] http://markmail.org/message/dz66se5biblj5nri > [3] > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notification+F > ramework+Proposal > [4] > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notification+w > ith+message+oriented+middleware+Proposal > [4] > https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=shortl > og;h=refs/heads/events-framework > [5] > https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=blob;f > =framework/events/src/org/apache/cloudstack/framework/events/EventBus.java; > h=c16ee6f96f4dfdca3a5c20e6476703c05c374df9;hb=refs/heads/events-framework > > [6] http://www.rabbitmq.com/java-client.html > [ > >
Great to see this being close to coming into master! A couple of questions: 1 - I don't seem to be able to find any new unit tests that cover the feature. Are there any that I'm missing? 2 - Doing a diff between master and the events-framework branches, I see a number of unrelated changes that would be applied. Perhaps events framework isn't completely up to date. I'd ask that you review the diff to see if there are any surprises in there. 3 - As I read the pom's (which may be incorrect), RabbitMQ's client is now a required dependency. Does everyone agree that we should include Rabbit as a dependency? 4 - Is the feature disabled by default? -chip
