the changes take instant effect, i let "watch /usr/lib64/nagios/plugins/check_tcp -H my.vip.add.r -p 3306" on a 3rd instance running, and the second i issued on the new master node the aws command it changed from CRITICAL to OK
On Wed, Oct 2, 2013 at 4:13 PM, Peter Romfeld <peter.romfeld...@gmail.com>wrote: > In you VPC the change take effect after you got "true", you can check it > with describe-eip or in console :) > right now i just try to add the command with hardcoded variables to > IPaddr2 start, just to get it running for now > > i put my attempt to create a resource agent on github, maybe someone can > help me find why params are not working? > wget https://raw.github.com/peterromfeldhk/pacemaker/master/AWSFIP > > sorry im just started with scripting with nagios_nrpe last project :) > i have 2 problems, 1s2 the OCF_RESKEY_ params dont work as i thought, if i > hardcode the variables the commands are working at least in testrun, the > second big issue i have is the monitoring > > > On Wed, Oct 2, 2013 at 2:34 PM, David Lang <da...@lang.hm> wrote: > >> Unless something has changed in the AWS API in the last few months, when >> the aws command exits successfully, that doesn't mean the change has taken >> effect, just that the aws system has accepted the change and it will take >> effect 'soon' >> >> >> David Lang >> >> On Wed, 2 Oct 2013, Peter Romfeld wrote: >> >> yes i need to use the aws command, i am using a VPC, after issueing the >>> command i get a "true" statement and its done >>> >>> so i only want pacemaker to issue the one-shot command at failover. Here >>> is >>> what i have atm: (i know its still dirty, just lerning pacemaker) >>> >>> primitive drbd_mysql ocf:linbit:drbd \ >>> params drbd_resource="mydata" \ >>> op monitor interval="15s" >>> primitive fs_mysql ocf:heartbeat:Filesystem \ >>> params device="/dev/drbdx" directory="/mountpint" fstype="ext4" >>> options="relatime,barrier=1" \ >>> op start interval="0" timeout="60" \ >>> op stop interval="0" timeout="60" \ >>> op monitor interval="10s" timeout="60s" OCF_CHECK_LEVEL="20" \ >>> meta target-role="started" >>> primitive fvip ocf:heartbeat:AWSFIP \ >>> params fvip="192.168.2.10" region="ap-southeast-1" >>> primitive ip_mysql ocf:heartbeat:IPaddr2 \ >>> params ip="192.168.2.10" cidr_netmask="20" \ >>> op monitor interval="10" \ >>> meta target-role="started" >>> primitive mysqld lsb:mysql >>> group mysql fs_mysql ip_mysql mysqld >>> ms ms_drbd_mysql drbd_mysql \ >>> meta master-max="1" master-node-max="1" clone-max="2" >>> clone-node-max="1" notify="true" >>> colocation mysql_on_drbd inf: fvip mysql ms_drbd_mysql:Master >>> order mysql_after_drbd_and_fvip inf: ms_drbd_mysql:promote fvip:start >>> mysql:start >>> >>> my AWSFIP(adjusted Dummy :%s/dummy/awsfip/g|%s/Dummy/**AWSFIP/g): >>> >>> <parameter name="fvip" unique="1" required="1"> >>> <longdesc lang="en"> >>> The IPv4 address to be configured in dotted quad notation, for example >>> "192.168.1.1". >>> </longdesc> >>> <shortdesc lang="en">IPv4 address</shortdesc> >>> <content type="string" default="" /> >>> </parameter> >>> >>> <parameter name="region" unique="1" required="1"> >>> <longdesc lang="en"> >>> The name of the AWS region >>> </longdesc> >>> <shortdesc lang="en">AWS region</shortdesc> >>> <content type="string"/> >>> </parameter> >>> >>> awsfip_start() { >>> awsfip_monitor >>> Instance_ID=`/usr/bin/curl --silent >>> http://169.254.169.254/latest/**meta-data/instance-id`<http://169.254.169.254/latest/meta-data/instance-id> >>> ENI_ID=`aws ec2 describe-instances --instance-id $Instance_ID --region >>> $OCF_RESKEY_region | grep NetworkInterfaceId | cut -d '"' -f 4` >>> if [ $? = $OCF_SUCCESS ]; then >>> return $OCF_SUCCESS >>> fi >>> aws ec2 assign-private-ip-addresses --network-interface-id >>> $ENI_ID >>> --private-ip-addresses $OCF_RESKEY_fvip --allow-reassignment --region >>> $OCF_RESKEY_region >>> sleep 4 >>> aws ec2 assign-private-ip-addresses --network-interface-id >>> $ENI_ID >>> --private-ip-addresses $OCF_RESKEY_fvip --allow-reassignment --region >>> $OCF_RESKEY_region >>> /etc/init.d/networking restart >>> touch ${OCF_RESKEY_state} >>> } >>> >>> I couldn't get it to work yet, and i don't want to run a external script >>> for it. I can't be so hard to let pacemaker execute an additional >>> one-shot >>> command at failover (in the correct order..) >>> >>> Thanks for your help! >>> >>> >>> On Wednesday, October 02, 2013 07:33 AM, David Lang wrote: >>> >>> the aws command is making the call to inform aws, if you were to bring up >>> the address without making the aws command, would it work? If you are on >>> a >>> Virtual Private Cloud (VPC), it may, but I didn't think it would. >>> >>> If you can make it work without the aws command, then you can just use >>> the >>> standard pacemaker VIP configuration. I know that this doesn't work if >>> you >>> have an external IP that you are moving (you must use an aws call to tell >>> Amazon to move the IP), but it's possible that you don't have to for an >>> internal IP, but I would be surprised. >>> >>> David Lang >>> >>> >>> On Wed, 2 Oct 2013, Peter Romfeld wrote: >>> >>> Hey, >>> when i change the secondary IP per hand or with external script on a >>> Ubuntu >>> Instance I just need: >>> /etc/network/interfaces >>> auto eth0 >>> iface eth0 inet dhcp >>> address 192.168.32.12 >>> netmask 255.255.240.0 >>> gateway 192.168.32.1 >>> up ip addr add 192.168.32.11/20 dev eth0 >>> >>> and then run the script which basically just does: >>> #!/bin/sh >>> >>> VIP=172.32.32.11 >>> REGION=ap-southeast-1 >>> >>> Instance_ID=`/usr/bin/curl --silent >>> http://169.254.169.254/latest/**meta-data/instance-id`<http://169.254.169.254/latest/meta-data/instance-id> >>> ENI_ID=`aws ec2 describe-instances --instance-id $Instance_ID --region >>> $REGION | grep NetworkInterfaceId | cut -d '"' -f 4` >>> >>> aws ec2 assign-private-ip-addresses --network-interface-id $ENI_ID >>> --private-ip-addresses $VIP --allow-reassignment --region $REGION >>> >>> >>> I dont need to inform AWS or restart network, only the correct network >>> config and the one command, when i tested it with pinging from a 3rd >>> instance during IP change i didnt got any interupts. I dont know about >>> monitoring it >>> >>> >>> On Wed, Oct 2, 2013 at 1:38 AM, David Lang <da...@lang.hm> >>> <da...@lang.hm>wrote: >>> >>> On Tue, 1 Oct 2013, Dejan Muhamedagic wrote: >>> >>> On Tue, Oct 01, 2013 at 10:07:12AM -0700, David Lang wrote: >>> >>> >>> On Tue, 1 Oct 2013, Dejan Muhamedagic wrote: >>> >>> On Tue, Oct 01, 2013 at 07:22:20AM -0700, David Lang wrote: >>> >>> >>> On Tue, 1 Oct 2013, Dejan Muhamedagic wrote: >>> >>> Hi David, >>> >>> >>> On Mon, Sep 30, 2013 at 12:41:23PM -0700, David Lang wrote: >>> >>> On Mon, 30 Sep 2013, David Lang wrote: >>> >>> On Mon, 30 Sep 2013, Michael Schwartzkopff wrote: >>> >>> >>> Am Montag, 30. September 2013, 21:12:56 schrieb Peter Romfeld: >>> >>> >>> I am working in AWS i cant just use a VIP i need to use a floating >>> secondary IP which i reassign through script, i want to let >>> pacemaker >>> handle the reassignment... >>> >>> >>> Please explain the difference of a VIP and a "secondary IP" in >>> your opinion. >>> >>> >>> with AWS you need to inform amazon of the change, not just change >>> the IP on the local box, that requires much more work than a >>> simple local VIP >>> >>> >>> being more detailed, instead of just >>> ifconfig eth0:0 $vip >>> you have to do something like >>> >>> /opt/aws/bin/ec2-assign-****private-ip-addresses -n $ENI_ID >>> --secondary-private-ip-address $VIP --allow-reassignment --region $REGION >>> >>> >>> We may consider adding such an option to IPaddr2. Has anybody >>> ever tried that? >>> >>> pingresult=`ping -c 1 -W 1 $VIP | grep time= | wc -l` >>> >>> if [ "$pingresult" == "0" ]; then >>> echo `date` "-- Restarting network" >>> /sbin/service network restart > /dev/null 2>&1 >>> >>> >>> That may break the cluster communication, which may lead to split >>> brain, etc. Is that really the only way? >>> >>> >>> It's not the only way, but you do have the problem that the call to >>> aws management interface is asynchronous, you don't know when it's >>> going to complete, and until it does, the IP doesn't actually work. >>> >>> >>> Wouldn't it be then safer to wait until it starts working, i.e. >>> to monitor in a loop? >>> >>> >>> that's exactly what the snippet of code above is for, to detect when >>> the other box no longer has the address. >>> >>> >>> Hmm, perhaps I'm missing something, but I couldn't notice a loop >>> in that code. What I meant was something like this: >>> >>> while ! ping -c 1 -W 1 $VIP | grep -qs time=; do >>> : >>> done >>> >>> Then network restart wouldn't be necessary, right? Sorry, I don't >>> know much about aws. >>> >>> >>> I haven't used this exact script before, but I have seen the problem that >>> this script is designed to address. I am not saying that I agree with >>> this >>> script, but it's what Amazon is suggesting, so it's probably a reasonable >>> start. >>> >>> >>> this was a cut-n-paste from the URL provided earlier >>> http://aws.amazon.com/****articles/2127188135977316<http://aws.amazon.com/**articles/2127188135977316> >>> <http://aws.amazon.com/**articles/2127188135977316<http://aws.amazon.com/articles/2127188135977316> >>> ><htt**p://aws.amazon.com/articles/**2127188135977316<http://aws.amazon.com/articles/2127188135977316> >>> > >>> >>> >>> >>> #!/bin/sh >>> # This script will monitor another HA node and take over a Virtual IP >>> (VIP) >>> # if communication with the other node fails >>> >>> # High Availability IP variables >>> # Other node's IP to ping and VIP to swap if other node goes down >>> HA_Node_IP=10.0.0.11 >>> VIP=10.0.0.10 >>> >>> # Specify the EC2 region that this will be running in >>> REGION=us-west-2 >>> >>> # Run aws-apitools-common.sh to set up default environment variables and >>> to >>> # leverage AWS security credentials provided by EC2 roles >>> . /etc/profile.d/aws-apitools-****common.sh >>> >>> # Determine the instance and ENI IDs so we can reassign the VIP to the >>> # correct ENI. Requires EC2 describe-instances and >>> assign-private-ip-address >>> # permissions. The following example EC2 roles policy will authorize >>> these >>> # commands: >>> # { >>> # "Statement": [ >>> # { >>> # "Action": [ >>> # "ec2:AssignPrivateIpAddresses"****, >>> # "ec2:DescribeInstances" >>> # ], >>> # "Effect": "Allow", >>> # "Resource": "*" >>> # } >>> # ] >>> # } >>> >>> Instance_ID=`/usr/bin/curl --silent >>> http://169.254.169.254/latest/****<http://169.254.169.254/latest/**> >>> meta-data/instance-id`ENI_ID=`****/opt/aws/bin/ec2-describe-**** >>> instances >>> <http://169.254.169.254/**latest/meta-data/instance-** >>> idENI_ID=/opt/aws/bin/ec2-**describe-instances<http://169.254.169.254/latest/meta-data/instance-idENI_ID=/opt/aws/bin/ec2-describe-instances> >>> ><http://**169.254.169.254/latest/meta-**data/instance-idENI_ID=/opt/** >>> aws/bin/ec2-describe-instances<http://169.254.169.254/latest/meta-data/instance-idENI_ID=/opt/aws/bin/ec2-describe-instances> >>> **>$Instance_ID >>> >>> --region $REGION | grep eni -m 1 | awk '{print $2;}'` >>> >>> echo `date` "-- Starting HA monitor" >>> while [ . ]; do >>> pingresult=`ping -c 3 -W 1 $HA_Node_IP | grep time= | wc -l` >>> >>> if [ "$pingresult" == "0" ]; then >>> echo `date` "-- HA heartbeat failed, taking over VIP" >>> >>> /opt/aws/bin/ec2-assign-****private-ip-addresses -n $ENI_ID >>> --secondary-private-ip-address $VIP --allow-reassignment --region $REGION >>> pingresult=`ping -c 1 -W 1 $VIP | grep time= | wc -l` >>> if [ "$pingresult" == "0" ]; then >>> echo `date` "-- Restarting network" >>> /sbin/service network restart > /dev/null 2>&1 >>> fi >>> sleep 60 >>> fi >>> sleep 2 >>> done >>> >>> >>> >>> David Lang >>> >>> ______________________________****_________________ >>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>> http://oss.clusterlabs.org/****mailman/listinfo/pacemaker<http://oss.clusterlabs.org/**mailman/listinfo/pacemaker> >>> <http://oss.clusterlabs.org/**mailman/listinfo/pacemaker<http://oss.clusterlabs.org/mailman/listinfo/pacemaker> >>> ><ht**tp://oss.clusterlabs.org/**mailman/listinfo/pacemaker<http://oss.clusterlabs.org/mailman/listinfo/pacemaker> >>> > >>> >>> >>> Project Home: http://www.clusterlabs.org >>> Getting started: http://www.clusterlabs.org/**** >>> doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/**doc/Cluster_from_Scratch.pdf> >>> <http://www.clusterlabs.org/**doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf> >>> ><**http://www.clusterlabs.org/**doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf> >>> > >>> >>> Bugs: http://bugs.clusterlabs.org >>> >>> >>> >>> >>> ______________________________**_________________ >>> >>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>> http://oss.clusterlabs.org/**mailman/listinfo/pacemaker<http://oss.clusterlabs.org/mailman/listinfo/pacemaker> >>> >>> >>> >>> Project Home: http://www.clusterlabs.org >>> >>> Getting started: http://www.clusterlabs.org/** >>> doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf> >>> >>> Bugs: http://bugs.clusterlabs.org >>> >>> >>> >>> ______________________________**_________________ >>> Pacemaker mailing list: >>> Pacemaker@oss.clusterlabs.**orghttp://oss.clusterlabs.org/** >>> mailman/listinfo/pacemaker<http://oss.clusterlabs.org/mailman/listinfo/pacemaker> >>> >>> >>> Project Home: http://www.clusterlabs.org >>> Getting started: http://www.clusterlabs.org/** >>> doc/Cluster_from_Scratch.pdf<http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf> >>> Bugs: http://bugs.clusterlabs.org >>> >> >> _______________________________________________ >> >> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >> >> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >> >> >> >> Project Home: http://www.clusterlabs.org >> >> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >> >> Bugs: http://bugs.clusterlabs.org >> >> >> _______________________________________________ >> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >> >> Project Home: http://www.clusterlabs.org >> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >> Bugs: http://bugs.clusterlabs.org >> >> >
_______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
