On 07/27/2015 11:54 AM, Yang Hongyang wrote: > > > On 07/27/2015 11:24 AM, Jason Wang wrote: >> >> >> On 07/24/2015 04:04 PM, Yang Hongyang wrote: >>> Hi Jason, >>> >>> On 07/24/2015 10:12 AM, Jason Wang wrote: >>>> >>>> >>>> On 07/24/2015 10:04 AM, Dong, Eddie wrote: >>>>> Hi Stefan: >>>>> Thanks for your comments! >>>>> >>>>>> On Mon, Jul 20, 2015 at 02:42:33PM +0800, Li Zhijian wrote: >>>>>>> We are planning to implement colo-proxy in qemu to cache and >>>>>>> compare >>>>>> packets. >>>>>> >>>>>> I thought there is a kernel module to do that? >>>>> Yes, that is the previous solution the COLO sub-community choose >>>>> to go, but we realized it might be not the best choices, and thus we >>>>> want to bring discussion back here :) More comments are welcome. >>>>> >>>> >>>> Hi: >>>> >>>> Could you pls describe more details on this decision? What's the >>>> reason >>>> that you realize it was not the best choice? >>> >>> Below is my opinion: >>> >>> We realized that there're disadvantages do it in kernel spaces: >>> 1. We need to recompile kernel: the colo-proxy kernel module is >>> implemented as a nf conntrack extension. Adding a extension need to >>> modify the extension struct in-kernel, so recompile kernel is >>> needed. >> >> There's no need to do all in kernel, you can use a separate process to >> do the comparing and trigger the state sync through monitor. > > I don't get it, colo-proxy kernel module using a kthread do the > comparing and > trigger the state sync. We implemented it as a nf conntrack extension > module, > so we need to extend the extension struct in-kernel, although it just > needs > few lines changes to kernel, but a recompile of kernel is needed. Are you > talking about not implement it as a nf conntrack extension?
Yes, I mean implement the comparing in userspace but not in qemu. > >> >>> 2. We need to recompile iptables/nftables to use together with the >>> colo-proxy >>> kernel module. >>> 3. Need to configure primary host to forward input packets to >>> secondary as >>> well as configure secondary to forward output packets to primary >>> host, the >>> network topology and configuration is too complex for a regular >>> user. >>> >> >> You can use current kernel primitives to mirror the traffic of both PVM >> and SVM to another process without any modification of kernel. And qemu >> can offload all network configuration to management in this case. And >> what's more import, this works for vhost. Filtering in qemu won't work >> for vhost. > > We are using tc to mirror/forward packets now. Implement in QEMU do > have some > limits, but there're also limits in kernel, if the packet do not pass > the host kernel TCP/IP stack, such as vhost-user. But the limits are much less than userspace, no? For vhost-user, maybe we could extend the backed to mirror the traffic also. > >> >> >>> You can refer to http://wiki.qemu.org/Features/COLO >>> to see the network topology and the steps to setup an env. >> >> The figure "COLO Framework" shows there's a proxy kernel module in >> primary node but in secondary node this is done through a process? This >> will complicate the environment a bit more. > > proxy kernel module also works for secondary node. > >> >>> >>> Setup a test env is too complex. The usability is so important to a >>> feature >>> like COLO which provide VM FT solution, if fewer people can/willing to >>> setup the env, the feature is useless. So we decide to develop user >>> space >>> colo-proxy. >> >> If the setup is too complex, need to consider to simplify or reuse codes >> and designs. Otherwise you probably introduce something new that needs >> fault tolerance. >> >>> >>> The advantage is obvious, >>> 1. we do not need to recompile kernel. >>> 2. No need to recompile iptables/nftables. >> >> As I descried above, looks like there's no need to modify kernel. >> >>> 3. we do not need to deal with the network configuration, we just >>> using a >>> socket connection between 2 QEMUs to forward packets. >> >> All network configurations should be offloaded to management. And you >> still need a dedicated topology according to the wiki. >> >>> 4. A complete VM FT solution in one go, we have already developed the >>> block >>> replication in QEMU, so with the network replication in QEMU, all >>> components we needed are within QEMU, this is very important, it >>> greatly >>> improves the usability of COLO feature! We hope it will gain more >>> testers, >>> users and developers. >> >> Is your block solution works for vhost? > > No, it can't works for vhost and dataplane, migration also won't work > for dataplane IIRC. > >> >>> 5. QEMU will gain a complete VM FT solution and the most advantage FT >>> solution >>> so far! >>> >>> Overall, usability is the most important factor that impact our choice. >>> >>> >> >> Usability will be improved if you can use exist primitives and decouple >> unnecessary codes from qemu. >> >> Thanks >> >>>> >>>> Thanks >>>> . >>>> >>> >> >> >> . >> >
