I would like to request merge of events framework [4] branch changes in to master to branch.
I have proposed a while ago [1], about the need for a framework to Publish-Subscribe CloudStack to events. Initially I was planning to use simple in-memory based publish-subscribe mechanism using google event bus library as described in the FS [3]. But considering the long term goal of CloudStack to decouple orchestration subsystems to independently runnable services as proposed architecture refactoring [2], and as per the discussion followed, I have taken approach of publishing CloudStack events to a messaging broker. Context diagram of how CloudStack and MOM (message oriented middleware) fits in documented at [4]. Idea was to enable two use cases. - external components like CMDB, billing systems, etc can directly get the events from the message broker, enabling complete decoupling. Also get instant notification of the events (like new VM create etc) to trigger further processing. - be able to create a notification service which can push notifications to different end points (web call back, sms, email) line AWS SNS. As part of this effort I am introducing a event bus abstraction which is realized by an interface [5] providing publish, subscribe, unsubscribe semantics. One can have a plug in that implements this interface to provide the actually event bus implementation. CloudStack core uses the interface to publish/subscribe to the events. CloudStack will by default will publish all Action Events, Usage Events, Alerts, Resource State change events. Resource state change is new type of event introduced which gets generated on resource (Vm, volume, snapshot, network etc) state changes. I looked at the options to provide a implementation using the framework. Integrating with a AMQP complaint message broker seemed better due to availability of multiple implementations (Rabbit MQ, Active MQ, Qpid etc). I used rabbitMQ [6] java AMQP client to interact with AMQP servers. AMQP has a publish-subscribe paradigm. AMQP has notion of 'Exchanges' and 'Queues'. Messages are published on to the 'Exchanges' specifying a routing key. Queue's are subscriber that bound to exchanges with a specific binding key. AMQP implementation is responsible for delivering a message in exchange to queue's that has the matching binding key. On message arrival in a queue, AMQP server can send call back to consumers. So the plug-in I have implemented using rabbit MQ, publishes events by CloudStack on to configured exchange on AMQP server. For each of the subscriber plug-in is responsible for creating a queue with specified event topic. Routing is designed to have the format. Event-source.Event-Category.Event-Type.Resource.ResourceUUID. For e.g. A message is published with a routing key: management-server:ActionEvent:SNAPSHOT-CREATE:Snapshot:0a7ea29e-691b-11e2-b afa-2c3ba27d8c47. A subscriber interested in receiving all the events corresponding to a VM with a UUID 9d827485-0f46-4db8-bd39-fede97cbac0c would result in a queue with a binding key *.*.*.VirtualMachine.9d827485-0f46-4db8-bd39-fede97cbac0c. This feature is contained, does not touch business logic of the subsystems, and hooks in to convenient classes that are used to persist Action, Usage and Alert events in to DB. Also at run time if no plug-in is configured that provides implementation of EventBus abstraction, there will not have any impact of CloudStack. I have added state machine to VirtualMachine, Network, Volume, Snapshot resources, and on state transition generates a resource state change notification. I can add port merge, if any other critical resource for which state change event would be valuable. 'events-framework' branch is upto date with master and there are no unit-test failures. [1] http://markmail.org/thread/tmqagkbj7urzcouf [2] http://markmail.org/message/dz66se5biblj5nri [3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notification+F ramework+Proposal [4] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Event+Notification+w ith+message+oriented+middleware+Proposal [4] https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=shortl og;h=refs/heads/events-framework [5] https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=blob;f =framework/events/src/org/apache/cloudstack/framework/events/EventBus.java; h=c16ee6f96f4dfdca3a5c20e6476703c05c374df9;hb=refs/heads/events-framework [6] http://www.rabbitmq.com/java-client.html [
