On 13 Apr 2014, at 11:36 am, Caizhifeng <bluewin...@h3c.com> wrote: > Hi all, > I'm kinda new to Pacemaker and have some questions I hope some of you could > help me with, any idea(s) would be highly appreciated, thank you. > I'am building the HA cluster with corosync2.3.3 + cluster-glue + > pacemaker-1.1.11-rc5. The problem is, when I try to remove a node from HA > cluster with "crm_node -f -R nodeID", it seems to remove the wrong node. > > My setup is as follows(and it can be reproduced): > 1. The HA is ok with 2 resource and 2 nodes, and with stonith-enabled="false": > root@h1:/opt/bin# crm status > Last updated: Sat Apr 12 16:00:51 2014 > Last change: Sat Apr 12 16:00:43 2014 via cibadmin on h1 > Stack: corosync > Current DC: h1 (1084752017) - partition with quorum > Version: 1.1.10-33f9d09 > 2 Nodes configured > 2 Resources configured > > > Online: [ h0 h1 ] > > VMdos-1 (ocf::heartbeat:VirtualDomain): Started h1 > root@h1:/opt/bin# > > root@h1:/opt/bin# crm_node --list > 1084752017 h1 > 1084752016 h0 > root@h1:/opt/bin# > > 2. remove node h0 from HA > (1). Stop pacemakerd and corosync service on node h0 (2). Run the follow cmd > on node h1 to Remove node info from cib: > cibadmin --delete --obj_type status --crm_xml "<node_state > id=\"1084752016\"/>" > cibadmin --delete --obj_type nodes --crm_xml "<node id=\"1084752016\"/>" > > at this time; the result of "crm status" shows that node " h0 " is offline > and there's no information of node h0 in CIB. But the "crm_node --list " > still include node" h0" > > 3. crm_node -f -R 1084752016
Could you re-run this command but add -VVVVVVVV please?
I suspect the fault lies in this binary.
> After this cmd, node " h1" is removed, and result of "crm status " is as
> follow(with 0 node):
>
> root@h1:/opt/bin# crm status
> Last updated: Sat Apr 12 15:59:42 2014
> Last change: Sat Apr 12 15:59:37 2014 via crm_node on h1
> Stack: corosync
> Current DC: NONE
> 0 Nodes configured
> 2 Resources configured
> root@h1:/opt/bin
>
>
> The corosync.conf is as follow:
> .............
> quorum {
> provider: corosync_votequorum
> expected_votes: 2
> allow_downscale: 1
> two_node: 1
> }
> ...........
>
>
> The corosync.log is as follow, it seems node " h1" is removed due to fencing!
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ringbuffer.c:299
> Free'ing ringbuffer: /dev/shm/qb-cmap-request-25004-14152-27-header
> Apr 12 16:22:35 [25026] h1 cib: debug: activateCibXml:
> Triggering CIB write for cib_delete op
> Apr 12 16:22:35 [25031] h1 crmd: notice: crm_reap_dead_member:
> Removing h0/1084752016 from the membership list
> Apr 12 16:22:35 [25031] h1 crmd: notice: reap_crm_member:
> Purged 1 peers with id=1084752016 and/or uname=(null) from the membership
> cache
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: log_cib_diff: Config
> update: Local-only Change: 0.12.1
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> <cib admin_epoch="0" epoch="11" num_updates="1">
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> <configuration>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> <nodes>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <node id="1084752017" uname="h1"/>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> </nodes>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> </configuration>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> </cib>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: ++
> <cib epoch="12" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2"
> crm_feature_set="3.0.8" cib-last-written="Sat Apr 12 16:22:35 2014"
> update-origin="h1" update-client="crm_node" have-quorum="1"
> dc-uuid="1084752017"/>
> Apr 12 16:22:35 [25026] h1 cib: notice: log_cib_diff: cib:diff:
> Local-only Change: 0.12.1
> Apr 12 16:22:35 [25026] h1 cib: notice: cib:diff: --
> <node id="1084752017" uname="h1"/>
> Apr 12 16:22:35 [25026] h1 cib: notice: cib:diff: ++ <cib
> epoch="12" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2"
> crm_feature_set="3.0.8" cib-last-written="Sat Apr 12 16:22:35 2014"
> update-origin="h1" update-client="crm_node" have-quorum="1"
> dc-uuid="1084752017"/>
> Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
> Completed cib_delete operation for section nodes: OK (rc=0,
> origin=local/crm_node/2, version=0.12.1)
> Apr 12 16:22:35 [25031] h1 crmd: debug: te_update_diff:
> Processing diff (cib_delete): 0.11.1 -> 0.12.1 (S_IDLE)
> Apr 12 16:22:35 [25031] h1 crmd: info: abort_transition_graph:
> te_update_diff:126 - Triggered transition abort (complete=1, node=,
> tag=diff, id=(null), magic=NA, cib=0.12.1) : Non-status change
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <diff crm_feature_set="3.0.8"
> digest="3cccef06483ac4dfeadfb562f6f8293a">
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <diff-removed admin_epoch="0" epoch="11" num_updates="1">
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <cib admin_epoch="0" epoch="11" num_updates="1">
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <configuration>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <nodes>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <node id="1084752017" uname="h1"
> __crm_diff_marker__="removed:top"/>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause </nodes>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause </configuration>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause </cib>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause </diff-removed>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <diff-added>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <cib epoch="12" num_updates="1" admin_epoch="0"
> validate-with="pacemaker-1.2" crm_feature_set="3.0.8" cib-last-written="Sat
> Apr 12 16:22:35 2014" update-origin="h1" update-client="crm_node"
> have-quorum="1" dc-uuid="1084752017"/>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause </diff-added>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause </diff>
> Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
> I_PE_CALC: [ state=S_IDLE cause=C_FSA_INTERNAL origin=abort_transition_graph ]
> Apr 12 16:22:35 [25031] h1 crmd: notice: do_state_transition: State
> transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL
> origin=abort_transition_graph ]
> Apr 12 16:22:35 [25031] h1 crmd: debug: do_state_transition: All 1
> cluster nodes are eligible to run resources.
> Apr 12 16:22:35 [25031] h1 crmd: debug: do_pe_invoke: Query 38:
> Requesting the current CIB: S_POLICY_ENGINE
> Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
> Completed cib_query operation for section 'all': OK (rc=0,
> origin=local/crmd/38, version=0.12.1)
> Apr 12 16:22:35 [25031] h1 crmd: debug: do_pe_invoke_callback:
> Invoking the PE: query=38, ref=pe_calc-dc-1397290955-21, seq=1024, quorate=1
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: Diff:
> --- 0.12.1
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: Diff:
> +++ 0.12.2 3d673c27c3c92939b41c7207edee9f46
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> <cib num_updates="1">
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> <status>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <node_state id="1084752017" uname="h1" in_ccm="true" crmd="online"
> crm-debug-origin="post_cache_update" join="member" expected="member">
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <transient_attributes id="1084752017">
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <instance_attributes id="status-1084752017">
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <nvpair id="status-1084752017-shutdown" name="shutdown" value="0"/>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <nvpair id="status-1084752017-probe_complete" name="probe_complete"
> value="true"/>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> </instance_attributes>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> </transient_attributes>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <lrm id="1084752017">
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <lrm_resources>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <lrm_resource id="VMdos-1" type="VirtualDomain" class="ocf"
> provider="heartbeat">
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <lrm_rsc_op id="VMdos-1_last_failure_0"
> operation_key="VMdos-1_monitor_0" operation="monitor"
> crm-debug-origin="build_active_RAs" crm_feature_set="3.0.8"
> transition-key="6:4:7:affc1dba-30eb-458d-8f86-c37d0268e52c"
> transition-magic="0:0;6:4:7:affc1dba-30eb-458d-8f86-c37d0268e52c" call-id="5"
> rc-code="0" op-status="0" interval="0" last-run="1397290607"
> last-rc-change="1397290607" exec-time="92" queue-time="0" op-digest="
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <lrm_rsc_op id="VMdos-1_monitor_30000"
> operation_key="VMdos-1_monitor_30000" operation="monitor"
> crm-debug-origin="build_active_RAs" crm_feature_set="3.0.8"
> transition-key="7:5:0:affc1dba-30eb-458d-8f86-c37d0268e52c"
> transition-magic="0:0;7:5:0:affc1dba-30eb-458d-8f86-c37d0268e52c" call-id="6"
> rc-code="0" op-status="0" interval="30000" last-rc-change="1397290607"
> exec-time="67" queue-time="0" op-digest="0874c7ce5f61a12
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> </lrm_resource>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <lrm_resource id="VMdos-2" type="VirtualDomain" class="ocf"
> provider="heartbeat">
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> <lrm_rsc_op id="VMdos-2_last_0" operation_key="VMdos-2_monitor_0"
> operation="monitor" crm-debug-origin="build_active_RAs"
> crm_feature_set="3.0.8"
> transition-key="7:6:7:affc1dba-30eb-458d-8f86-c37d0268e52c"
> transition-magic="0:7;7:6:7:affc1dba-30eb-458d-8f86-c37d0268e52c"
> call-id="10" rc-code="7" op-status="0" interval="0" last-run="1397290608"
> last-rc-change="1397290608" exec-time="61" queue-time="0" op-digest="c7d22be
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> </lrm_resource>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> </lrm_resources>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> </lrm>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
> </node_state>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> </status>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
> </cib>
> Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: ++
> <cib epoch="12" num_updates="2" admin_epoch="0" validate-with="pacemaker-1.2"
> crm_feature_set="3.0.8" cib-last-written="Sat Apr 12 16:22:35 2014"
> update-origin="h1" update-client="crm_node" have-quorum="1"
> dc-uuid="1084752017"/>
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config:
> STONITH timeout: 60000
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config:
> STONITH of failed nodes is disabled
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Stop
> all active resources: false
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config:
> Cluster is symmetric - resources can run anywhere by default
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config:
> Default stickiness: 0
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: On
> loss of CCM Quorum: Stop ALL resources
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Node
> scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_domains:
> Unpacking domains
> Apr 12 16:22:35 [25030] h1 pengine: warning: unpack_status: Node
> h1 in status section no longer exists
> Apr 12 16:22:35 [25030] h1 pengine: info: unpack_status: Node
> 1084752017 is unknown
> Apr 12 16:22:35 [25030] h1 pengine: info: native_print: VMdos-1
> (ocf::heartbeat:VirtualDomain): Stopped
> Apr 12 16:22:35 [25030] h1 pengine: info: native_print: VMdos-2
> (ocf::heartbeat:VirtualDomain): Stopped
> Apr 12 16:22:35 [25030] h1 pengine: debug: native_assign_node: Could
> not allocate a node for VMdos-1
> Apr 12 16:22:35 [25030] h1 pengine: info: native_color: Resource
> VMdos-1 cannot run anywhere
> Apr 12 16:22:35 [25030] h1 pengine: debug: native_assign_node: Could
> not allocate a node for VMdos-2
> Apr 12 16:22:35 [25030] h1 pengine: info: native_color: Resource
> VMdos-2 cannot run anywhere
> Apr 12 16:22:35 [25030] h1 pengine: info: LogActions: Leave
> VMdos-1 (Stopped)
> Apr 12 16:22:35 [25030] h1 pengine: info: LogActions: Leave
> VMdos-2 (Stopped)
> Apr 12 16:22:35 [25030] h1 pengine: notice: process_pe_message:
> Calculated Transition 4: /var/lib/pacemaker/pengine/pe-input-4.bz2
> Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
> I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
> origin=handle_response ]
> Apr 12 16:22:35 [25031] h1 crmd: info: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Apr 12 16:22:35 [25031] h1 crmd: debug: unpack_graph: Unpacked
> transition 4: 0 actions in 0 synapses
> Apr 12 16:22:35 [25031] h1 crmd: info: do_te_invoke: Processing
> graph 4 (ref=pe_calc-dc-1397290955-21) derived from
> /var/lib/pacemaker/pengine/pe-input-4.bz2
> Apr 12 16:22:35 [25031] h1 crmd: debug: print_graph: Empty
> transition graph
> Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
> Completed cib_delete operation for section status: OK (rc=0,
> origin=local/crm_node/3, version=0.12.2)
> Apr 12 16:22:35 [25031] h1 crmd: debug: te_update_diff:
> Processing diff (cib_delete): 0.12.1 -> 0.12.2 (S_TRANSITION_ENGINE)
> Apr 12 16:22:35 [25031] h1 crmd: info: abort_transition_graph:
> te_update_diff:188 - Triggered transition abort (complete=0, node=h1,
> tag=transient_attributes, id=1084752017, magic=NA, cib=0.12.2) : Transient
> attribute: removal
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <transient_attributes id="1084752017">
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <instance_attributes id="status-1084752017">
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <nvpair id="status-1084752017-shutdown" name="shutdown"
> value="0"/>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause <nvpair id="status-1084752017-probe_complete"
> name="probe_complete" value="true"/>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause </instance_attributes>
> Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
> Cause </transient_attributes>
> Apr 12 16:22:35 [25031] h1 crmd: debug: update_abort_priority:
> Abort priority upgraded from 0 to 1000000
> Apr 12 16:22:35 [25031] h1 crmd: debug: update_abort_priority:
> Abort action done superceeded by restart
> Apr 12 16:22:35 [25031] h1 crmd: notice: run_graph: Transition 4
> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pacemaker/pengine/pe-input-4.bz2): Complete
> Apr 12 16:22:35 [25031] h1 crmd: debug: print_graph: Empty
> transition graph
> Apr 12 16:22:35 [25031] h1 crmd: debug: te_graph_trigger:
> Transition 4 is now complete
> Apr 12 16:22:35 [25031] h1 crmd: debug: notify_crmd: Processing
> transition completion in state S_TRANSITION_ENGINE
> Apr 12 16:22:35 [25031] h1 crmd: debug: notify_crmd: Transition 4
> status: restart - Transient attribute: removal
> Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
> I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
> origin=notify_crmd ]
> Apr 12 16:22:35 [25031] h1 crmd: info: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Apr 12 16:22:35 [25031] h1 crmd: debug: do_state_transition: All 1
> cluster nodes are eligible to run resources.
> Apr 12 16:22:35 [25031] h1 crmd: debug: do_pe_invoke: Query 39:
> Requesting the current CIB: S_POLICY_ENGINE
> Apr 12 16:22:35 [25024] h1 pacemakerd: info: crm_client_new:
> Connecting 0x25b4ea0 for uid=0 gid=0 pid=14152
> id=f3612e17-0806-4355-a3fc-2cf1feda1e6d
> Apr 12 16:22:35 [25024] h1 pacemakerd: debug: handle_new_connection:
> IPC credentials authenticated (25024-14152-10)
> Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_ipcs_shm_connect:
> connecting to client [14152]
> Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
> Completed cib_query operation for section 'all': OK (rc=0,
> origin=local/crmd/39, version=0.12.2)
> Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_rb_open_2: shm
> size:131085; real_size:135168; rb->word_size:33792
> Apr 12 16:22:35 [25026] h1 cib: debug:
> qb_ipcs_dispatch_connection_request: HUP conn (25026-14152-13)
> Apr 12 16:22:35 [25026] h1 cib: debug: qb_ipcs_disconnect:
> qb_ipcs_disconnect(25026-14152-13) state:2
> Apr 12 16:22:35 [25026] h1 cib: info: crm_client_destroy:
> Destroying 0 events
> Apr 12 16:22:35 [25026] h1 cib: debug: qb_rb_close: Free'ing
> ringbuffer: /dev/shm/qb-cib_rw-response-25026-14152-13-header
> Apr 12 16:22:35 [25026] h1 cib: debug: qb_rb_close: Free'ing
> ringbuffer: /dev/shm/qb-cib_rw-event-25026-14152-13-header
> Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_rb_open_2: shm
> size:131085; real_size:135168; rb->word_size:33792
> Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_rb_open_2: shm
> size:131085; real_size:135168; rb->word_size:33792
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ipc_setup.c:478 IPC
> credentials authenticated (25004-14152-27)
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ipc_shm.c:294 connecting
> to client [14152]
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ringbuffer.c:236 shm
> size:1048589; real_size:1052672; rb->word_size:263168
> Apr 12 16:22:35 [25031] h1 crmd: debug: do_pe_invoke_callback:
> Invoking the PE: query=39, ref=pe_calc-dc-1397290955-22, seq=1024, quorate=1
> Apr 12 16:22:35 [25031] h1 crmd: debug:
> qb_ipcs_dispatch_connection_request: HUP conn (25031-14152-14)
> Apr 12 16:22:35 [25031] h1 crmd: debug: qb_ipcs_disconnect:
> qb_ipcs_disconnect(25031-14152-14) state:2
> Apr 12 16:22:35 [25031] h1 crmd: info: crm_client_destroy:
> Destroying 0 events
> Apr 12 16:22:35 [25031] h1 crmd: debug: qb_rb_close: Free'ing
> ringbuffer: /dev/shm/qb-crmd-response-25031-14152-14-header
> ...
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config:
> STONITH timeout: 60000
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config:
> STONITH of failed nodes is disabled
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Stop
> all active resources: false
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config:
> Cluster is symmetric - resources can run anywhere by default
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config:
> Default stickiness: 0
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: On
> loss of CCM Quorum: Stop ALL resources
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Node
> scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
> Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_domains:
> Unpacking domains
> Apr 12 16:22:35 [25030] h1 pengine: info: native_print: VMdos-1
> (ocf::heartbeat:VirtualDomain): Stopped
> Apr 12 16:22:35 [25030] h1 pengine: info: native_print: VMdos-2
> (ocf::heartbeat:VirtualDomain): Stopped
> Apr 12 16:22:35 [25030] h1 pengine: debug: native_assign_node: Could
> not allocate a node for VMdos-1
> Apr 12 16:22:35 [25030] h1 pengine: info: native_color: Resource
> VMdos-1 cannot run anywhere
> Apr 12 16:22:35 [25030] h1 pengine: debug: native_assign_node: Could
> not allocate a node for VMdos-2
> Apr 12 16:22:35 [25030] h1 pengine: info: native_color: Resource
> VMdos-2 cannot run anywhere
> Apr 12 16:22:35 [25030] h1 pengine: info: LogActions: Leave
> VMdos-1 (Stopped)
> Apr 12 16:22:35 [25030] h1 pengine: info: LogActions: Leave
> VMdos-2 (Stopped)
> Apr 12 16:22:35 [25030] h1 pengine: notice: process_pe_message:
> Calculated Transition 5: /var/lib/pacemaker/pengine/pe-input-5.bz2
> Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
> I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
> origin=handle_response ]
> Apr 12 16:22:35 [25031] h1 crmd: info: do_state_transition: State
> transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
> cause=C_IPC_MESSAGE origin=handle_response ]
> Apr 12 16:22:35 [25031] h1 crmd: debug: unpack_graph: Unpacked
> transition 5: 0 actions in 0 synapses
> Apr 12 16:22:35 [25031] h1 crmd: info: do_te_invoke: Processing
> graph 5 (ref=pe_calc-dc-1397290955-22) derived from
> /var/lib/pacemaker/pengine/pe-input-5.bz2
> Apr 12 16:22:35 [25031] h1 crmd: debug: print_graph: Empty
> transition graph
> Apr 12 16:22:35 [25031] h1 crmd: notice: run_graph: Transition 5
> (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
> Source=/var/lib/pacemaker/pengine/pe-input-5.bz2): Complete
> Apr 12 16:22:35 [25031] h1 crmd: debug: print_graph: Empty
> transition graph
> Apr 12 16:22:35 [25031] h1 crmd: debug: te_graph_trigger:
> Transition 5 is now complete
> Apr 12 16:22:35 [25031] h1 crmd: debug: notify_crmd: Processing
> transition completion in state S_TRANSITION_ENGINE
> Apr 12 16:22:35 [25031] h1 crmd: debug: notify_crmd: Transition 5
> status: done - <null>
> Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
> I_TE_SUCCESS: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
> origin=notify_crmd ]
> Apr 12 16:22:35 [25031] h1 crmd: info: do_log: FSA: Input
> I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
> Apr 12 16:22:35 [25031] h1 crmd: notice: do_state_transition: State
> transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
> cause=C_FSA_INTERNAL origin=notify_crmd ]
> Apr 12 16:22:35 [25031] h1 crmd: debug: do_state_transition:
> Starting PEngine Recheck Timer
> Apr 12 16:22:35 [25031] h1 crmd: debug: crm_timer_start:
> Started PEngine Recheck Timer (I_PE_CALC:900000ms), src=62
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ringbuffer.c:236 shm
> size:1048589; real_size:1052672; rb->word_size:263168
> Apr 12 16:22:35 [25026] h1 cib: info: write_cib_contents:
> Archived previous version as /var/lib/pacemaker/cib/cib-9.raw
> Apr 12 16:22:35 [25026] h1 cib: debug: write_cib_contents:
> Writing CIB to disk
> Apr 12 16:22:35 [25026] h1 cib: debug: qb_rb_close: Free'ing
> ringbuffer: /dev/shm/qb-cib_rw-request-25026-14152-13-header
> Apr 12 16:22:35 [25026] h1 cib: info: write_cib_contents: Wrote
> version 0.12.0 of the CIB to disk (digest: bd7d26226d6aa75f28b9eb670a67e944)
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ringbuffer.c:236 shm
> size:1048589; real_size:1052672; rb->word_size:263168
> Apr 12 16:22:35 [25003] h1 corosync debug [MAIN ] ipc_glue.c:272
> connection created
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] cmap.c:306 lib_init_fn:
> conn=0x7f32d0f4f4b0
> Apr 12 16:22:35 [25024] h1 pacemakerd: notice: pcmk_ipc_dispatch:
> Instructing peers to remove references to node (null)/1084752016
> Apr 12 16:22:35 [25026] h1 cib: info: crm_client_new:
> Connecting 0xa23770 for uid=0 gid=0 pid=14152
> id=701dfa3c-d585-49d2-bd1e-f44636823e1b
> Apr 12 16:22:35 [25026] h1 cib: debug: handle_new_connection:
> IPC credentials authenticated (25026-14152-13)
> Apr 12 16:22:35 [25026] h1 cib: debug: qb_ipcs_shm_connect:
> connecting to client [14152]
> Apr 12 16:22:35 [25026] h1 cib: debug: write_cib_contents: Wrote
> digest bd7d26226d6aa75f28b9eb670a67e944 to disk
> Apr 12 16:22:35 [25026] h1 cib: info: retrieveCib: Reading
> cluster configuration from: /var/lib/pacemaker/cib/cib.y6wnAg (digest:
> /var/lib/pacemaker/cib/cib.ncXFvs)
> ...
> Apr 12 16:22:35 [25026] h1 cib: debug: write_cib_contents:
> Activating /var/lib/pacemaker/cib/cib.y6wnAg
> Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
> Completed cib_delete operation for section nodes: OK (rc=0,
> origin=local/crm_node/2, version=0.12.2)
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ipcs.c:757 HUP conn
> (25004-14152-27)
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ipcs.c:605
> qb_ipcs_disconnect(25004-14152-27) state:2
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] loop_poll_epoll.c:117
> epoll_ctl(del): Bad file descriptor (9)
> Apr 12 16:22:35 [25003] h1 corosync debug [MAIN ] ipc_glue.c:417
> cs_ipcs_connection_closed()
> Apr 12 16:22:35 [25003] h1 corosync debug [QB ] cmap.c:325 exit_fn for
> conn=0x7f32d0f4f4b0
> Apr 12 16:22:35 [25003] h1 corosync debug [MAIN ] ipc_glue.c:390
> cs_ipcs_connection_destroyed()
> Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
> Completed cib_delete operation for section status: OK (rc=0,
> origin=local/crm_node/3, version=0.12.2)
> ...
> Apr 12 16:22:35 [25024] h1 pacemakerd: debug:
> qb_ipcs_dispatch_connection_request: HUP conn (25024-14152-10)
> Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_ipcs_disconnect:
> qb_ipcs_disconnect(25024-14152-10) state:2
> Apr 12 16:22:35 [25024] h1 pacemakerd: info: crm_client_destroy:
> Destroying 0 events
> ...
> Apr 12 16:22:35 [25026] h1 cib: debug:
> qb_ipcs_dispatch_connection_request: HUP conn (25026-14152-13)
> Apr 12 16:22:35 [25026] h1 cib: debug: qb_ipcs_disconnect:
> qb_ipcs_disconnect(25026-14152-13) state:2
> Apr 12 16:22:35 [25026] h1 cib: info: crm_client_destroy:
> Destroying 0 events
> ...
> Apr 12 16:22:35 [25024] h1 pacemakerd: notice: crm_reap_dead_member:
> Removing h0/1084752016 from the membership list
> Apr 12 16:22:35 [25024] h1 pacemakerd: notice: reap_crm_member:
> Purged 1 peers with id=1084752016 and/or uname=(null) from the membership
> cache
>
>
> -------------------------------------------------------------------------------------------------------------------------------------
> 本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址中列出
> 的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
> 或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
> 邮件！
> This e-mail and its attachments contain confidential information from H3C,
> which is
> intended only for the person or entity whose address is listed above. Any use
> of the
> information contained herein in any way (including, but not limited to, total
> or partial
> disclosure, reproduction, or dissemination) by persons other than the intended
> recipient(s) is prohibited. If you receive this e-mail in error, please
> notify the sender
> by phone or email immediately and delete it!
> _______________________________________________
> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
