There was quite a lot of it (all were pe-input* files). I made one file out of them that has everything under /var/lib/pengine/ that was created the same hour than my previous logs: http://minus.com/dcHdfKuZD.gz
Since the test in the logs I have changed my corosync config a bit by removing the redundant ring settings because the second ring was always marked as faulty and some posts lately on the list indicate that rrp isn't quite ready for production yet. But that didn't solve my problem. However, I noticed that if I stop the virtual domain resources in crm before starting the ms_drbd_* resources, both DRBD devices are correctly promoted to masters on both nodes. And after that I can start the virtual domains in crm with no problems. So either this is some kind of timing issue, or I have something wrong in my pacemaker config? Also, if I have all resources running on s2 and I put s2 on standby I eventually get: Node s2: standby Online: [ s1 s3 ] Master/Slave Set: ms_drbd_www drbd_www:1 (ocf::linbit:drbd): Slave s2 (unmanaged) FAILED Masters: [ s1 ] Master/Slave Set: ms_drbd_www2 drbd_www2:1 (ocf::linbit:drbd): Slave s2 (unmanaged) FAILED Masters: [ s1 ] www-server (ocf::heartbeat:VirtualDomain): Started s2 (unmanaged) FAILED www2-server (ocf::heartbeat:VirtualDomain): Started s2 (unmanaged) FAILED Failed actions: drbd_www:1_demote_0 (node=s2, call=1152, rc=-2, status=Timed Out): unknown exec error drbd_www:1_stop_0 (node=s2, call=1157, rc=-2, status=Timed Out): unknown exec error drbd_www2:1_demote_0 (node=s2, call=1159, rc=-2, status=Timed Out): unknown exec error drbd_www2:1_stop_0 (node=s2, call=1162, rc=-2, status=Timed Out): unknown exec error www-server_stop_0 (node=s2, call=1147, rc=1, status=complete): unknown error www2-server_stop_0 (node=s2, call=1148, rc=1, status=complete): unknown error And DRBD is still running as primary on both nodes and virtual servers are both also still running on s2. The "unkown" errors in DC seems to be: Aug 15 21:38:13 s3-1 pengine: [20809]: WARN: unpack_rsc_op: Processing failed op drbd_www:1_stop_0 on s2: unknown exec error (-2) Aug 15 21:38:13 s3-1 pengine: [20809]: WARN: unpack_rsc_op: Processing failed op www-server_stop_0 on s2: unknown error (1) Just stopping the virtual domain resource also gives "unknown error", but the actual virtual server is really stopped. If I stop the virtual domain resources, and cleanup their errors. I can put s2 on standby and DRBD devices are stopped on s2. And if I then start the virtual domain resources, they are correctly started on s1. So looks again like some timing problem with DRBD + virtual domain RAs, or just some error in my config? > We'd need access to the files in /var/lib/pengine/ from the DC too. > > On Tue, Aug 2, 2011 at 7:08 PM, Matt Anderson <tux...@hotmail.com> wrote: > > > > Hi! > > > > Sorry for the repost, but the links in my previous message expired. > > Now these new ones shouldn't do that. I also added the DC's log at the end > > of this message. > > > > I've been trying to make a simple HA cluster with 3 servers (the 3rd server > > is there only to maintain quorum if one node fails). The idea is to run two > > virtual domains over dedicated DRBD devices in dual-primary mode (so that > > live migration would be possible). > > > > Things worked well for a while, but somewhere during my tests something > > went wrong and now the DRBD devices don't get promoted to primary mode by > > pacemaker. Pacemaker just keeps starting and stopping the devices in a loop. > > If I start DRBD from the init script, both devices are started and > > automaticly synced. At first I had this problem only with one device, but > > now it's the same with both devices under pacemaker. > > > > Pacemaker and DRBD write a lot of logs [1] [2] [3] (these are made when I > > try to start ms_drbd_www2, but I don't see a reason why pacemaker doesn't > > promote any masters. > > > > My guess is that this has something to do with my fencing rules in DRBD [4] > > or then just in my pacemaker config [5]. I used to have STONITH enabled, but > > since my STONITH devices share the power supply with the server, I've then > > removed those settings from my pacemaker config. > > > > I'm running Debian squeeze on amd64 with pacemaker (1.0.11-1~bpo60+1) and > > corosync (1.3.0-3~bpo60+1) from backports. > > > > Any ideas what's wrong and how to fix it? > > > > > > [1] http://paste.debian.net/124836/ (DRBD log from on node) > > > > [2] http://paste.debian.net/124838/ (pacemaker log from the same node as > > above) > > > > [3] http://paste.debian.net/124839/ (pacemaker log from DC) > > > > [4] http://paste.debian.net/124845/ (DRBD common config) > > > > [5] http://paste.debian.net/124846/ (pacemaker config) > > > > Pacemaker log from DC [3]: > > > > Jul 28 22:28:01 s3-1 cibadmin: [10292]: info: Invoked: cibadmin -Ql -o > > resources > > Jul 28 22:28:01 s3-1 cibadmin: [10295]: info: Invoked: cibadmin -p -R -o > > resources > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - <cib > > admin_epoch="0" epoch="439" num_updates="10" > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > <configuration > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > <resources > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > <master id="ms_drbd_www2" > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > <meta_attributes id="ms_drbd_www2-meta_attributes" > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > <nvpair value="Stopped" id="ms_drbd_www2-meta_attributes-target-role" > > /> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > </meta_attributes> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > </master> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > </resources> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > </configuration> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: - > > </cib> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + <cib > > admin_epoch="0" epoch="440" num_updates="1" > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > <configuration > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > <resources > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > <master id="ms_drbd_www2" > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > <meta_attributes id="ms_drbd_www2-meta_attributes" > > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > <nvpair value="Started" id="ms_drbd_www2-meta_attributes-target-role" > > /> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > </meta_attributes> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > </master> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > </resources> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > </configuration> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: log_data_element: cib:diff: + > > </cib> > > Jul 28 22:28:01 s3-1 cib: [21918]: info: cib_process_request: Operation > > complete: op cib_replace for section resources (origin=local/cibadmin/2, > > version=0.440.1): ok (rc=0) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: abort_transition_graph: > > need_abort:59 - Triggered transition abort (complete=1) : Non-status change > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: need_abort: Aborting on change to > > admin_epoch > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_state_transition: State > > transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL > > origin=abort_transition_graph ] > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_state_transition: All 3 > > cluster nodes are eligible to run resources. > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_pe_invoke: Query 1845: > > Requesting the current CIB: S_POLICY_ENGINE > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_pe_invoke_callback: Invoking > > the PE: query=1845, ref=pe_calc-dc-1311881281-3699, seq=190040, quorate=1 > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: unpack_config: Node scores: > > 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: determine_online_status: Node > > s3 is online > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: determine_online_status: Node > > s1 is online > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: determine_online_status: Node > > s2 is online > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: clone_print: Master/Slave > > Set: ms_drbd_www > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: short_print: Stopped: [ > > drbd_www:0 drbd_www:1 ] > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: clone_print: Master/Slave > > Set: ms_drbd_www2 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: short_print: Stopped: [ > > drbd_www2:0 drbd_www2:1 ] > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: native_print: > > www-server#011(ocf::heartbeat:VirtualDomain):#011Stopped > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: native_print: > > www2-server#011(ocf::heartbeat:VirtualDomain):#011Stopped > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: native_print: > > www2-mailto#011(ocf::heartbeat:MailTo):#011Stopped > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: native_print: > > www-mailto#011(ocf::heartbeat:MailTo):#011Stopped > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > drbd_www:0 cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > drbd_www:1 cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: master_color: ms_drbd_www: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: master_color: ms_drbd_www2: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: master_color: ms_drbd_www: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: rsc_merge_weights: www-server: > > Rolling back scores from www-mailto > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > www-server cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: master_color: ms_drbd_www2: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: rsc_merge_weights: > > www2-server: Rolling back scores from www2-mailto > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > www2-server cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > www2-mailto cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > www-mailto cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:0 on s2 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:1 on s1 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:0 on s2 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:1 on s1 > > Jul 28 22:28:01 s3-1 pengine: [21921]: ERROR: clone_rsc_order_rh_non_clone: > > Unknown action: www-server_demote_0 > > Jul 28 22:28:01 s3-1 pengine: [21921]: ERROR: clone_rsc_order_rh_non_clone: > > Unknown action: www2-server_demote_0 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > drbd_www:0#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > drbd_www:1#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Start > > drbd_www2:0#011(s2) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Start > > drbd_www2:1#011(s1) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www-server#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www2-server#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www2-mailto#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www-mailto#011(Stopped) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_state_transition: State > > transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS > > cause=C_IPC_MESSAGE origin=handle_response ] > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: unpack_graph: Unpacked transition > > 1548: 12 actions in 12 synapses > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_te_invoke: Processing graph > > 1548 (ref=pe_calc-dc-1311881281-3699) derived from > > /var/lib/pengine/pe-input-9218.bz2 > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 36 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 37 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 34 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 30: start drbd_www2:0_start_0 on s2 > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:0_start_0 (30) confirmed on s2 (rc=0) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 32: start drbd_www2:1_start_0 on s1 > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: abort_transition_graph: > > te_update_diff:150 - Triggered transition abort (complete=0, tag=nvpair, > > id=status-s1-master-drbd_www2:1, magic=NA, cib=0.440.3) : Transient > > attribute: update > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: update_abort_priority: Abort > > priority upgraded from 0 to 1000000 > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: update_abort_priority: Abort > > action done superceeded by restart > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:1_start_0 (32) confirmed on s1 (rc=0) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 35 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 38 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 80: notify drbd_www2:0_post_notify_start_0 on s2 > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:0_post_notify_start_0 (80) confirmed on s2 (rc=0) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 81: notify drbd_www2:1_post_notify_start_0 on s1 > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:1_post_notify_start_0 (81) confirmed on s1 (rc=0) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 39 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: run_graph: > > ==================================================== > > Jul 28 22:28:01 s3-1 crmd: [21922]: notice: run_graph: Transition 1548 > > (Complete=10, Pending=0, Fired=0, Skipped=2, Incomplete=0, > > Source=/var/lib/pengine/pe-input-9218.bz2): Stopped > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_graph_trigger: Transition 1548 > > is now complete > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_state_transition: State > > transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC > > cause=C_FSA_INTERNAL origin=notify_crmd ] > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_state_transition: All 3 > > cluster nodes are eligible to run resources. > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_pe_invoke: Query 1846: > > Requesting the current CIB: S_POLICY_ENGINE > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_pe_invoke_callback: Invoking > > the PE: query=1846, ref=pe_calc-dc-1311881281-3704, seq=190040, quorate=1 > > Jul 28 22:28:01 s3-1 cib: [10296]: info: write_cib_contents: Archived > > previous version as /var/lib/heartbeat/crm/cib-77.raw > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: process_pe_message: Transition > > 1548: PEngine Input stored in: /var/lib/pengine/pe-input-9218.bz2 > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: unpack_config: Node scores: > > 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: determine_online_status: Node > > s3 is online > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: determine_online_status: Node > > s1 is online > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: determine_online_status: Node > > s2 is online > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: clone_print: Master/Slave > > Set: ms_drbd_www > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: short_print: Stopped: [ > > drbd_www:0 drbd_www:1 ] > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: clone_print: Master/Slave > > Set: ms_drbd_www2 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: short_print: Slaves: [ > > s2 s1 ] > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: native_print: > > www-server#011(ocf::heartbeat:VirtualDomain):#011Stopped > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: native_print: > > www2-server#011(ocf::heartbeat:VirtualDomain):#011Stopped > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: native_print: > > www2-mailto#011(ocf::heartbeat:MailTo):#011Stopped > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: native_print: > > www-mailto#011(ocf::heartbeat:MailTo):#011Stopped > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > drbd_www:0 cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > drbd_www:1 cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: master_color: ms_drbd_www: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: master_color: ms_drbd_www2: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: master_color: ms_drbd_www: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: rsc_merge_weights: www-server: > > Rolling back scores from www-mailto > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > www-server cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: master_color: ms_drbd_www2: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: rsc_merge_weights: > > www2-server: Rolling back scores from www2-mailto > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > www2-server cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > www2-mailto cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: native_color: Resource > > www-mailto cannot run anywhere > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:0 on s1 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:1 on s2 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:0 on s1 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:1 on s2 > > Jul 28 22:28:01 s3-1 pengine: [21921]: ERROR: clone_rsc_order_rh_non_clone: > > Unknown action: www-server_demote_0 > > Jul 28 22:28:01 s3-1 pengine: [21921]: ERROR: clone_rsc_order_rh_non_clone: > > Unknown action: www2-server_demote_0 > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > drbd_www:0#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > drbd_www:1#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Move resource > > drbd_www2:0#011(Slave s2 -> s1) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Move resource > > drbd_www2:1#011(Slave s1 -> s2) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www-server#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www2-server#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www2-mailto#011(Stopped) > > Jul 28 22:28:01 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www-mailto#011(Stopped) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_state_transition: State > > transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS > > cause=C_IPC_MESSAGE origin=handle_response ] > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: unpack_graph: Unpacked transition > > 1549: 23 actions in 23 synapses > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: do_te_invoke: Processing graph > > 1549 (ref=pe_calc-dc-1311881281-3704) derived from > > /var/lib/pengine/pe-input-9219.bz2 > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 46 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 82: notify drbd_www2:0_pre_notify_stop_0 on s2 > > Jul 28 22:28:01 s3-1 pengine: [21921]: info: process_pe_message: Transition > > 1549: PEngine Input stored in: /var/lib/pengine/pe-input-9219.bz2 > > Jul 28 22:28:01 s3-1 cib: [10296]: info: write_cib_contents: Wrote version > > 0.440.0 of the CIB to disk (digest: 3fa86d20299acf9247c14b5760f9b9c3) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:0_pre_notify_stop_0 (82) confirmed on s2 (rc=0) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 83: notify drbd_www2:1_pre_notify_stop_0 on s1 > > Jul 28 22:28:01 s3-1 cib: [10296]: info: retrieveCib: Reading cluster > > configuration from: /var/lib/heartbeat/crm/cib.sLA4uT (digest: > > /var/lib/heartbeat/crm/cib.tXdeLK) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:1_pre_notify_stop_0 (83) confirmed on s1 (rc=0) > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 47 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 44 fired and confirmed > > Jul 28 22:28:01 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 31: stop drbd_www2:0_stop_0 on s2 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:0_stop_0 (31) confirmed on s2 (rc=0) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 35: stop drbd_www2:1_stop_0 on s1 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: abort_transition_graph: > > te_update_diff:164 - Triggered transition abort (complete=0, > > tag=transient_attributes, id=s1, magic=NA, cib=0.440.10) : Transient > > attribute: removal > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: update_abort_priority: Abort > > priority upgraded from 0 to 1000000 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: update_abort_priority: Abort > > action done superceeded by restart > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:1_stop_0 (35) confirmed on s1 (rc=0) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 45 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 48 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 49 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: run_graph: > > ==================================================== > > Jul 28 22:28:02 s3-1 crmd: [21922]: notice: run_graph: Transition 1549 > > (Complete=10, Pending=0, Fired=0, Skipped=8, Incomplete=5, > > Source=/var/lib/pengine/pe-input-9219.bz2): Stopped > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_graph_trigger: Transition 1549 > > is now complete > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_state_transition: State > > transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC > > cause=C_FSA_INTERNAL origin=notify_crmd ] > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_state_transition: All 3 > > cluster nodes are eligible to run resources. > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_pe_invoke: Query 1847: > > Requesting the current CIB: S_POLICY_ENGINE > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_pe_invoke_callback: Invoking > > the PE: query=1847, ref=pe_calc-dc-1311881282-3709, seq=190040, quorate=1 > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: unpack_config: Node scores: > > 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: determine_online_status: Node > > s3 is online > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: determine_online_status: Node > > s1 is online > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: determine_online_status: Node > > s2 is online > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: clone_print: Master/Slave > > Set: ms_drbd_www > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: short_print: Stopped: [ > > drbd_www:0 drbd_www:1 ] > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: clone_print: Master/Slave > > Set: ms_drbd_www2 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: short_print: Stopped: [ > > drbd_www2:0 drbd_www2:1 ] > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: native_print: > > www-server#011(ocf::heartbeat:VirtualDomain):#011Stopped > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: native_print: > > www2-server#011(ocf::heartbeat:VirtualDomain):#011Stopped > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: native_print: > > www2-mailto#011(ocf::heartbeat:MailTo):#011Stopped > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: native_print: > > www-mailto#011(ocf::heartbeat:MailTo):#011Stopped > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > drbd_www:0 cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > drbd_www:1 cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: master_color: ms_drbd_www: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: master_color: ms_drbd_www2: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: master_color: ms_drbd_www: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: rsc_merge_weights: www-server: > > Rolling back scores from www-mailto > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > www-server cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: master_color: ms_drbd_www2: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: rsc_merge_weights: > > www2-server: Rolling back scores from www2-mailto > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > www2-server cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > www2-mailto cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > www-mailto cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:0 on s2 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:1 on s1 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:0 on s2 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:1 on s1 > > Jul 28 22:28:02 s3-1 pengine: [21921]: ERROR: clone_rsc_order_rh_non_clone: > > Unknown action: www-server_demote_0 > > Jul 28 22:28:02 s3-1 pengine: [21921]: ERROR: clone_rsc_order_rh_non_clone: > > Unknown action: www2-server_demote_0 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > drbd_www:0#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > drbd_www:1#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Start > > drbd_www2:0#011(s2) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Start > > drbd_www2:1#011(s1) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www-server#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www2-server#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www2-mailto#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www-mailto#011(Stopped) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_state_transition: State > > transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS > > cause=C_IPC_MESSAGE origin=handle_response ] > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: unpack_graph: Unpacked transition > > 1550: 12 actions in 12 synapses > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_te_invoke: Processing graph > > 1550 (ref=pe_calc-dc-1311881282-3709) derived from > > /var/lib/pengine/pe-input-9220.bz2 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 36 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 37 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 34 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 30: start drbd_www2:0_start_0 on s2 > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: process_pe_message: Transition > > 1550: PEngine Input stored in: /var/lib/pengine/pe-input-9220.bz2 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:0_start_0 (30) confirmed on s2 (rc=0) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 32: start drbd_www2:1_start_0 on s1 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: abort_transition_graph: > > te_update_diff:150 - Triggered transition abort (complete=0, tag=nvpair, > > id=status-s1-master-drbd_www2:1, magic=NA, cib=0.440.13) : Transient > > attribute: update > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: update_abort_priority: Abort > > priority upgraded from 0 to 1000000 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: update_abort_priority: Abort > > action done superceeded by restart > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:1_start_0 (32) confirmed on s1 (rc=0) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 35 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 38 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 80: notify drbd_www2:0_post_notify_start_0 on s2 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:0_post_notify_start_0 (80) confirmed on s2 (rc=0) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 81: notify drbd_www2:1_post_notify_start_0 on s1 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:1_post_notify_start_0 (81) confirmed on s1 (rc=0) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 39 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: run_graph: > > ==================================================== > > Jul 28 22:28:02 s3-1 crmd: [21922]: notice: run_graph: Transition 1550 > > (Complete=10, Pending=0, Fired=0, Skipped=2, Incomplete=0, > > Source=/var/lib/pengine/pe-input-9220.bz2): Stopped > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_graph_trigger: Transition 1550 > > is now complete > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_state_transition: State > > transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC > > cause=C_FSA_INTERNAL origin=notify_crmd ] > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_state_transition: All 3 > > cluster nodes are eligible to run resources. > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_pe_invoke: Query 1848: > > Requesting the current CIB: S_POLICY_ENGINE > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_pe_invoke_callback: Invoking > > the PE: query=1848, ref=pe_calc-dc-1311881282-3714, seq=190040, quorate=1 > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: unpack_config: Node scores: > > 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: determine_online_status: Node > > s3 is online > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: determine_online_status: Node > > s1 is online > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: determine_online_status: Node > > s2 is online > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: clone_print: Master/Slave > > Set: ms_drbd_www > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: short_print: Stopped: [ > > drbd_www:0 drbd_www:1 ] > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: clone_print: Master/Slave > > Set: ms_drbd_www2 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: short_print: Slaves: [ > > s2 s1 ] > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: native_print: > > www-server#011(ocf::heartbeat:VirtualDomain):#011Stopped > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: native_print: > > www2-server#011(ocf::heartbeat:VirtualDomain):#011Stopped > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: native_print: > > www2-mailto#011(ocf::heartbeat:MailTo):#011Stopped > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: native_print: > > www-mailto#011(ocf::heartbeat:MailTo):#011Stopped > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > drbd_www:0 cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > drbd_www:1 cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: master_color: ms_drbd_www: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: master_color: ms_drbd_www2: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: master_color: ms_drbd_www: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: rsc_merge_weights: www-server: > > Rolling back scores from www-mailto > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > www-server cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: master_color: ms_drbd_www2: > > Promoted 0 instances of a possible 2 to master > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: rsc_merge_weights: > > www2-server: Rolling back scores from www2-mailto > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > www2-server cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > www2-mailto cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: native_color: Resource > > www-mailto cannot run anywhere > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:0 on s1 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:1 on s2 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:0 on s1 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: RecurringOp: Start > > recurring monitor (15s) for drbd_www2:1 on s2 > > Jul 28 22:28:02 s3-1 pengine: [21921]: ERROR: clone_rsc_order_rh_non_clone: > > Unknown action: www-server_demote_0 > > Jul 28 22:28:02 s3-1 pengine: [21921]: ERROR: clone_rsc_order_rh_non_clone: > > Unknown action: www2-server_demote_0 > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > drbd_www:0#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > drbd_www:1#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Move resource > > drbd_www2:0#011(Slave s2 -> s1) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Move resource > > drbd_www2:1#011(Slave s1 -> s2) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www-server#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www2-server#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www2-mailto#011(Stopped) > > Jul 28 22:28:02 s3-1 pengine: [21921]: notice: LogActions: Leave resource > > www-mailto#011(Stopped) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_state_transition: State > > transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS > > cause=C_IPC_MESSAGE origin=handle_response ] > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: unpack_graph: Unpacked transition > > 1551: 23 actions in 23 synapses > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: do_te_invoke: Processing graph > > 1551 (ref=pe_calc-dc-1311881282-3714) derived from > > /var/lib/pengine/pe-input-9221.bz2 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 46 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 82: notify drbd_www2:0_pre_notify_stop_0 on s2 > > Jul 28 22:28:02 s3-1 pengine: [21921]: info: process_pe_message: Transition > > 1551: PEngine Input stored in: /var/lib/pengine/pe-input-9221.bz2 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:0_pre_notify_stop_0 (82) confirmed on s2 (rc=0) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 83: notify drbd_www2:1_pre_notify_stop_0 on s1 > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:1_pre_notify_stop_0 (83) confirmed on s1 (rc=0) > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 47 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 44 fired and confirmed > > Jul 28 22:28:02 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 31: stop drbd_www2:0_stop_0 on s2 > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:0_stop_0 (31) confirmed on s2 (rc=0) > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: te_rsc_command: Initiating action > > 35: stop drbd_www2:1_stop_0 on s1 > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: abort_transition_graph: > > te_update_diff:164 - Triggered transition abort (complete=0, > > tag=transient_attributes, id=s1, magic=NA, cib=0.440.20) : Transient > > attribute: removal > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: update_abort_priority: Abort > > priority upgraded from 0 to 1000000 > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: update_abort_priority: Abort > > action done superceeded by restart > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: match_graph_event: Action > > drbd_www2:1_stop_0 (35) confirmed on s1 (rc=0) > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 45 fired and confirmed > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 48 fired and confirmed > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: te_pseudo_action: Pseudo action > > 49 fired and confirmed > > Jul 28 22:28:03 s3-1 crmd: [21922]: info: run_graph: > > ==================================================== > > > > > > _______________________________________________ > > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > > > Project Home: http://www.clusterlabs.org > > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > > Bugs: > > http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker > > _______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker
