Thanks for the detailed reply. I will twiddle around with this and see if I can reproduce your results.
I would be interested in the matahari-related config files as I have no experience with that particular package, if you wouldn't mind shooting me copies. >Sure, > >As a warning, it is stupid complicated, and I am not certain exactly >why it works the way it does, multicast to a unicast broker still >confuses me. ;-) > >I have a guest cluster that is using the Centos HA packages (which >are basically a clone of what RedHat provides). In this setup I >needed to be able to do guest fencing, as I wanted my guest >cluster members to reboot rather than being fenced with the >fence_scsi option. > >In my config I have 8 hosts and 8 guests. Usually there is only 1 >guest on a host, but sometimes the guests migrate as we >manually balance things. The mapping from guest to host is not >always the same. > >The end result is that any guest can run fence_xvm -H guest-name >and the correct guest system will be rebooted regardless of what >host it is running on. The command fence_xvm -o list does not >work right for me, but my nodes should be able to fence if >necessary so it is mostly a solution.... If that is interesting read on. > >I drew inspiration from >http://clusterlabs.org/wiki/Guest_Fencing >http://www.daemonzone.net/e/3/ >http://oss.clusterlabs.org/pipermail/pacemaker/2011->November/012321.html > >This basically got luci and my /etc/cluster/cluster.conf to agree that >multicast and guest fencing is what my cluster should use. > >I installed fence-virt fence-virtd fence-virtd-multicast fence-virtd->libvirt libvirt-qmf matahari-broker on the host systems and fence->virt on the guests. > >On the hosts (after getting iptables and multicast packets to play >nicely) and getting libvirt-qmf running (for me this was just starting >the init.d script for it). I configured fence-virtd to use the libvirt-qpid >backend with a host and port configuration option that pointed at >the matahari-broker (which I think is a more recent qpidd) > >libvirt-qpid { > uri = "qemu:///system"; > host = "192.168.20.20"; > port = "49000"; > } > >This gets libvirt-qpid talking to the broker running on the host. I >tested a configuration where all hosts used the same broker and it >lets fencing occur. This was not optimal as if the host with the >broker failed.. bad things. > >I then got matahari-broker working on each host so each libvirt->qpid has something to talk to. I then looked at >http://matahari-dev.blogspot.com/2010/11/federation.html > >Which shows how to get the matahari-brokers to send messages >to each other in a federation configuration. I changed the init.d >script for matahari-broker to setup a route from the host executing >the script to each broker on the other hosts. > >So I think what happens (still not certain) is that the multicast gets >the information about what needs to be fenced to fence-virtd on >each host. Each hosts fence-virtd then passes the info to the hosts >matahari-broker, which sends the information to all the other >brokers in the federation. Each broker then tells its local libvirt-qmf >the information and if that libvirt-qmf actually has the guest to be >fenced it does so, otherwise it ignores the request because the >desired guest is not running. This also has the potential issue of >each libvirt-qmf will be told to fence a guest once for each broker >this includes the one that really has the guest. I believe (I have >looked at no source code so I am just speculating) fence_xvm -o >list does not work as expected because it runs with the first >response it receives so it becomes a race between all the hosts >as to which one will get there first. > >I hope that helps Adrian. I am happy to provide the iptables, >fence_virt.conf, and /etc/sysconfig/matahari and matahari-broker >files and my init.d script for matahari-broker that sets up all the >routes. _______________________________________________ Linux-HA mailing list [email protected] http://lists.linux-ha.org/mailman/listinfo/linux-ha See also: http://linux-ha.org/ReportingProblems
