Hello Isaku, Thanks for sharing this! Right now in Sahara project we think to use Marconi as a mean to communicate with VM. Seems like you are familiar with the discussions happened so far. If not, please see links at the bottom of UnifiedGuestAgent [1] wiki page. In short we see Marconi's supports for multi-tenancy as a huge advantage over other MQ solutions. Our agent is network-based, so tenant isolation is a real issue here. For clarity, here is the overview scheme of network based agent:
server <-> MQ (Marconi) <-> agent All communication goes over network. I've made a PoC of the Marconi driver for oslo.messaging, you can find it at [2] We also considered 'hypervisor-dependent' agents (as I called them in the initial thread) like the one you propose. They also provide tenant isolation. But the drawback is _much_ bigger development cost and more fragile and complex deployment. In case of network-based agent all the code is * Marconi driver for RPC library (oslo.messaging) * thin client for server to make calls * a guest agent with thin server-side If you write your agent on python, it will work on any OS with any host hypervisor. For hypervisor dependent-agent it becomes much more complex. You need one more additional component - a proxy-agent running on Compute host, which makes deployment harder. You also need to support various transports for various hypervisors: virtio-serial for KVM, XenStore for Xen, something for Hyper-V, etc. Moreover guest OS must have driver for these transports and you will probably need to write different implementation for different OSes. Also you mention that in some cases a second proxy-agent is needed and again in some cases only cast operations could be used. Using cast only is not an option for Sahara, as we do need feedback from the agent and sometimes getting the return value is the main reason to make an RPC call. I didn't see a discussion in Neutron on which approach to use (if it was, I missed it). I see simplicity of network-based agent as a huge advantage. Could you please clarify why you've picked design depending on hypervisor? Thanks, Dmitry [1] https://wiki.openstack.org/wiki/UnifiedGuestAgent [2] https://github.com/dmitrymex/oslo.messaging 2014-04-09 12:33 GMT+04:00 Isaku Yamahata <isaku.yamah...@gmail.com>: > Hello developers. > > > As discussed many times so far[1], there are many projects that needs > to propagate RPC messages into VMs running on OpenStack. Neutron in my case. > > My idea is to relay RPC messages from management network into tenant > network over file-like object. By file-like object, I mean virtio-serial, > unix domain socket, unix pipe and so on. > I've wrote some code based on oslo.messaging[2][3] and a documentation > on use cases.[4][5] > Only file-like transport and proxying messages would be in oslo.messaging > and agent side code wouldn't be a part of oslo.messaging. > > > use cases:([5] for more figures) > file-like object: virtio-serial, unix domain socket, unix pipe > > server <-> AMQP <-> agent in host <-virtio serial-> guest agent in VM > per VM > > server <-> AMQP <-> agent in host <-unix socket/pipe-> > agent in tenant network <-> guest agent in VM > > > So far there are security concerns to forward oslo.messaging from management > network into tenant network. One approach is to allow only cast-RPC from > server to guest agent in VM so that guest agent in VM only receives messages > and can't send anything to servers. With unix pipe, it's write-only > for server, read-only for guest agent. > > > Thoughts? comments? > > > Details of Neutron NFV use case[6]: > Neutron services so far typically runs agents in host, the host agent > in host receives RPCs from neutron server, then it executes necessary > operations. Sometimes the agent in host issues RPC to neutron server > periodically.(e.g. status report etc) > It's desirable to make such services virtualized as Network Function > Virtualizaton(NFV), i.e. make those features run in VMs. So it's quite > natural approach to propagate those RPC message into agents into VMs. > > > [1] https://wiki.openstack.org/wiki/UnifiedGuestAgent > [2] https://review.openstack.org/#/c/77862/ > [3] https://review.openstack.org/#/c/77863/ > [4] https://blueprints.launchpad.net/oslo.messaging/+spec/message-proxy-server > [5] https://wiki.openstack.org/wiki/Oslo/blueprints/message-proxy-server > [6] https://blueprints.launchpad.net/neutron/+spec/adv-services-in-vms > -- > Isaku Yamahata <isaku.yamah...@gmail.com> _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
