Happy to help.
IPMI goes well beyond just powering on and off nodes, but from a fencing
perspective, that's all we're interested in. Outside that, you can check
power, voltage, fans and temperature readings.
Most remote management devices have a dedicated network interfaces, but
sometimes it piggy-backs on one of the onboard NICs (it will have it's
own MAC address, IP address, etc.).
So whether shared or dedicated, the trick is to give it an IP address on
the same network that the cluster will be on. The cluster uses the
network with the IP address resolved by the node's hostname. That is,
get the name from 'uname -n' and resolve it. The IP returned is the
interface the cluster will use. Here's a command that will get it for you;
ifconfig |grep -B 1 $(gethostip -d $(uname -n)) | grep HWaddr | awk '{
print $1 }'
Whatever that network is, give the IPMI interface an IP on the same
network. The cluster will then be able to reach the peer node's IPMI
interface and evict it from the cluster.
I've got a network map that might help you see this (note that I use
bonding and a dedicated cluster network separated from inter/intranet
facing traffic);
https://alteeve.ca/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Network
At the bottom you will see the links to the IPMI interfaces (and
switched PDUs, which I use as a backup fence device).
If you want to read a bit more detail on how fencing works, the same
tutorial covers how failures are detected in corosync. The main bit is here:
https://alteeve.ca/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Concept.3B_Fencing
Hope this helps. :)
digimer
On 10/25/2012 11:50 PM, Josh Bowling wrote:
> Wow, thanks for the links, they've really help to clear this up a bit for
> me.
> From what I understand, out-of-band IPMI is what allows admins/other
> cluster nodes to do things such as power cycling. In my case, that's all I
> need. I just what Node 2 to shoot Node 1 in the head if it misbehaves.
>
> I'll look around and see what's available for Ubuntu in terms of
> IPMI-related packages.
>
> The only thing I'm not so sure of is how Node 2 connects to Node 1 IPMI and
> vice-versa?
> I currently have 2 NICs in both machines (one for LAN and one crossover for
> DRBD).
> Can the STONITH+Pacemaker setup on Node 2 connect to the IPMI of Node 1
> through LAN or does it need to be serial? Or does this simply vary from
> vendor to vendor?
>
> Thanks again
>
> On Fri, Oct 26, 2012 at 11:58 AM, Digimer <[email protected]> wrote:
>
>> On 10/25/2012 08:55 PM, Josh Bowling wrote:
>>> I just got a couple IBM x3200 M3 servers and decided to turn them into an
>>> Ubuntu 12.04 64-bit based DRBD + Pacemaker/Corosync cluster. I have
>> pretty
>>> much everything working, but read time and time again that if I don't
>> have
>>> a STONITH device set up, I'm setting myself up for disaster. I've done my
>>> best to look around and try to figure out what the best device would be
>> for
>>> these servers, but really have no idea. In all honesty, STONITH devices
>> in
>>> general are still a complete mystery to me.
>>>
>>> According to the x3200 M3 manual, it does have IPMI built in for remote
>>> power management:
>>>
>>> Integrated IPMI 2.0 support alerts IBM Systems Director to anomalous
>>>> environmental factors, such as
>>>> voltage and thermal conditions. It also supports highly secure remote
>>>> power control using data
>>>> encryption.
>>>
>>>
>>> I've seen a few tutorials on how to get the device setup with Pacemaker,
>>> but like I said, have no idea as to what hardware I need.
>>> Any ideas for this STONITH newbie?
>>>
>>> Thanks in advance!
>>
>> If your servers have IPMI (usually part of IBM's RSA out-of-band
>> management), then you are most of the way there. I've got a basic IPMI
>> tutorial that might help (it's for RHEL / CentOS, but the tools should
>> be common to Debian / Ubuntu);
>>
>> https://alteeve.ca/w/IPMI
>>
>> As soon as you can check the power status (and power on/off) the nodes,
>> you will be good. Adding support to pacemaker is then done via the
>> 'fence_ipmilan' fence agent.
>>
>> hth
>>
>> --
>> Digimer
>> Papers and Projects: https://alteeve.ca/w/
>> What if the cure for cancer is trapped in the mind of a person without
>> access to education?
>>
>
--
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
