On Thu, Jul 21, 2011 at 9:26 AM, Bob Schatz <bsch...@yahoo.com> wrote: > Okay, this configuration works on one node (I am waiting for a hardware > problem to be fixed before testing with second node): > node cnode-1-3-5 > node cnode-1-3-6 > primitive glance-drbd ocf:linbit:drbd \ > params drbd_resource="glance-repos-drbd" \ > op start interval="0" timeout="240" \ > op stop interval="0" timeout="100" \ > op monitor interval="59s" role="Master" timeout="30s" \ > op monitor interval="61s" role="Slave" timeout="30s" > primitive glance-fs ocf:heartbeat:Filesystem \ > params device="/dev/drbd1" directory="/glance-mount" fstype="ext4" \ > op start interval="0" timeout="60" \ > op monitor interval="60" timeout="60" OCF_CHECK_LEVEL="20" \ > op stop interval="0" timeout="120" > primitive glance-ip ocf:heartbeat:IPaddr2 \ > params ip="10.4.0.25" nic="br100:1" \ > op monitor interval="5s" > primitive glance-repos ocf:heartbeat:LVM \ > params volgrpname="glance-repos" exclusive="true" \ > op start interval="0" timeout="30" \ > op stop interval="0" timeout="30" > group glance-repos-fs-group glance-fs glance-ip \ > meta target-role="Started" > ms ms_drbd glance-drbd \ > meta master-node-max="1" clone-max="2" clone-node-max="1" > globally-unique="false" notify="true" target-role="Master" > colocation coloc-rule-w-master inf: ms_drbd:Master glance-repos-fs-group > colocation coloc-rule-w-master2 inf: glance-repos ms_drbd:Master > order glance-order-fs-after-drbd inf: glance-repos:start ms_drbd:start
This is the important one. > order glance-order-fs-after-drbd-stop inf: glance-repos-fs-group:stop > ms_drbd:demote > order glance-order-fs-after-drbd-stop2 inf: ms_drbd:demote ms_drbd:stop This one (above) is implied. > order glance-order-fs-after-drbd-stop3 inf: ms_drbd:stop glance-repos:stop > order glance-order-fs-after-drbd2 inf: ms_drbd:start ms_drbd:promote As is this one. > order glance-order-fs-after-drbd3 inf: ms_drbd:promote > glance-repos-fs-group:start > property $id="cib-bootstrap-options" \ > dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \ > cluster-infrastructure="openais" \ > expected-quorum-votes="1" \ > stonith-enabled="false" \ > no-quorum-policy="ignore" \ > last-lrm-refresh="1310768814" > I will let everyone know how testing goes. > > Thanks, > Bob > ----- Forwarded Message ----- > From: Bob Schatz <bsch...@yahoo.com> > To: "pacemaker@oss.clusterlabs.org" <pacemaker@oss.clusterlabs.org> > Sent: Wednesday, July 20, 2011 1:38 PM > Subject: [Pacemaker] Fw: Configuration for FS over DRBD over LVM > > One correction: > > I removed the "location" constraint and simply went with this: > colocation coloc-rule-w-master inf: glance-repos ms_drbd:Master > glance-repos-fs-group > order glance-order-fs-after-drbd inf: glance-repos:start > ms_drbd:promote glance-repos-fs-group:start > order glance-order-fs-after-drbd2 inf: glance-repos-fs-group:stop > ms_drbd:demote ms_drbd:stop glance-repos:stop > I called out the stop of DRBD before the stop of LVM. The syslog attached > previously is for this configuration. > > Thanks, > Bob > ________________________________ > From: Bob Schatz <bsch...@yahoo.com> > To: "pacemaker@oss.clusterlabs.org" <pacemaker@oss.clusterlabs.org> > Sent: Wednesday, July 20, 2011 11:32 AM > Subject: [Pacemaker] Fw: Configuration for FS over DRBD over LVM > > I tried another test based on this thread: > http://www.gossamer-threads.com/lists/linuxha/pacemaker/65928?search_string=lvm%20drbd;#65928 > I removed the "location" constraint and simply went with this: > colocation coloc-rule-w-master inf: glance-repos ms_drbd:Master > glance-repos-fs-group > order glance-order-fs-after-drbd inf: glance-repos:start > ms_drbd:promote glance-repos-fs-group:start > order glance-order-fs-after-drbd2 inf: glance-repos-fs-group:stop > ms_drbd:demote glance-repos:stop > The stop actions were called in this order: > stop file system > demote DRBD > stop LVM ***** > stop DRBD ***** > instead of: > stop file system > demote DRBD > stop DRBD ****** > stop LVM ****** > I see these messages in the log which I believe are debug messages based on > reading other threads: > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-0-start-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-0-start-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-0-stop-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-0-stop-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-1-promote-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-1-promote-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-1-demote-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-1-demote-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-2-start-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-2-start-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-2-stop-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd-2-stop-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-0-stop-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-0-stop-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-0-start-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-0-start-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-1-demote-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-1-demote-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-1-promote-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-1-promote-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-2-stop-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-2-stop-end > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-2-start-begin > pengine: [21021]: debug: text2task: Unsupported action: > glance-order-fs-after-drbd2-2-start-end > I have attached a syslog-pacemaker log of the "/etc/init.d/corosync start" > through "/etc/init.d/corosync stop" sequence. > > Thanks, > Bob > ----- Forwarded Message ----- > From: Bob Schatz <bsch...@yahoo.com> > To: "pacemaker@oss.clusterlabs.org" <pacemaker@oss.clusterlabs.org> > Sent: Tuesday, July 19, 2011 4:38 PM > Subject: [Pacemaker] Configuration for FS over DRBD over LVM > > Hi, > I am trying to configure an FS running on top of DRBD on top of LVM or: > FS > | > DRBD > | > LVM > I am using Pacemaker 1.0.8, Ubuntu 10.04 and DRBD 8.3.7. > Reviewing all the manuals (Pacemaker Explained 1.0, DRBD 8.4 User Guide, > etc) I came up with this Pacemaker configuration: > node cnode-1-3-5 > node cnode-1-3-6 > primitive glance-drbd ocf:linbit:drbd \ > params drbd_resource="glance-repos-drbd" \ > op start interval="0" timeout="240" \ > op stop interval="0" timeout="100" \ > op monitor interval="59s" role="Master" timeout="30s" \ > op monitor interval="61s" role="Slave" timeout="30s" > primitive glance-fs ocf:heartbeat:Filesystem \ > params device="/dev/drbd1" directory="/glance-mount" fstype="ext4" \ > op start interval="0" timeout="60" \ > op monitor interval="60" timeout="60" OCF_CHECK_LEVEL="20" \ > op stop interval="0" timeout="120" > primitive glance-repos ocf:heartbeat:LVM \ > params volgrpname="glance-repos" exclusive="true" \ > op start interval="0" timeout="30" \ > op stop interval="0" timeout="30" > group glance-repos-fs-group glance-fs > ms ms_drbd glance-drbd \ > meta master-node-max="1" clone-max="2" clone-node-max="1" > globally-unique="false" notify="true" target-role="Master" > location drbd_on_node1 ms_drbd \ > rule $id="drbd_on_node1-rule" $role="Master" 100: #uname eq > cnode-1-3-5 > colocation coloc-rule-w-master inf: glance-repos ms_drbd:Master > order glance-order-fs-after-drbd inf: glance-repos:start ms_drbd:promote > glance-repos-fs-group:start > property $id="cib-bootstrap-options" \ > dc-version="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd" \ > cluster-infrastructure="openais" \ > expected-quorum-votes="1" \ > stonith-enabled="false" \ > no-quorum-policy="ignore" \ > last-lrm-refresh="1310768814" > On one node, things come up cleanly. In fact, debug messages in the agent > show that the start() functions for the agent are called and exited in order > (LVM start, DRBD start and Filesystem start). > The problem occurs when I do "/etc/init.d/corosync stop" on a single node. > What happens is that the stop() functions are called in this order: > 1. LVM stop > 2. Filesystem stop > 3. DRBD stop > What I have tried: > 1. I tried setting the score of the order to "500" assuming that this would > mean the colocation rule would hit first. Still the same problem. > 2. I tried leaving off the ":start" and ":promote" options on the "order" > line. The stop order was still LVM, Filesystem, and DRBD > 3. I tried adding another colocation rule "colocation coloc-rule-w-master2 > inf: ms_drbd:Master glance-repos-fs-group" to tie glance-repos-fs-group to > the same node as DRBD. Stop still had the same issue. I assume that I > will still need this rule when I add a second node to the test. > Any suggestions would be appreciated. > A side note, the reason I have a group for the file system is that I would > like to add an application and IP address to the group once I get this > working. Also, the reason I have LVM under DRBD is that I want to be able > to grow the LVM volume as needed and then expand the DRBD volume. > > Thanks in advance, > Bob > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: > http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker > > > > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: > http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker > > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: > http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker > > > > > > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: > http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker > > > > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: > http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker > > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: > http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker > > _______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
