Jun 17 13:37:07 node1 lrmd: [11173]: WARN: pingd_child:1:monitor process (PID 1001) timed out (try 1).  Killing with signal SIGTERM (15).
Jun 17 13:37:09 node1 lrmd: [11173]: WARN: G_SIG_dispatch: Dispatch function for SIGCHLD was delayed 1210 ms (> 100 ms) before being called (GSource: 0xc50e9b8)
Jun 17 13:37:09 node1 lrmd: [11173]: info: G_SIG_dispatch: started at 496169216 should have started at 496169095
Jun 17 14:08:16 node1 lrmd: [11173]: WARN: pingd_child:1:monitor process (PID 5207) timed out (try 1).  Killing with signal SIGTERM (15).
Jun 17 14:08:16 node1 lrmd: [11173]: WARN: drbddisk_mysql:monitor process (PID 5205) timed out (try 1).  Killing with signal SIGTERM (15).
Jun 17 14:08:17 node1 lrmd: [11173]: WARN: There is something wrong: the first line isn't read in. Maybe the heartbeat does not ouput string correctly for status operation. Or the code (myself) is wrong.
Jun 17 14:08:17 node1 lrmd: [11173]: WARN: operation monitor[35] on heartbeat::drbddisk::drbddisk_mysql for client 11176, its parameters: target_role=[started] CRM_meta_interval=[30000] 1=[mysql_data] CRM_meta_id=[drbddisk_mysql_mon] CRM_meta_timeout=[10000] crm_feature_set=[2.1] CRM_meta_name=[monitor] : pid [5205] timed out
Jun 17 14:08:17 node1 lrmd: [11173]: WARN: operation monitor[49] on ocf::pingd::pingd_child:1 for client 11176, its parameters: multiplier=[100] CRM_meta_interval=[15000] CRM_meta_prereq=[nothing] dampen=[5s] CRM_meta_id=[pingd_child_mon] CRM_meta_timeout=[5000] crm_feature_set=[2.1] CRM_meta_clone_max=[2] CRM_meta_name=[monitor] CRM_meta_globally_unique=[false] CRM_meta_clone=[1] : pid [5207] timed out
Jun 17 14:08:17 node1 crmd: [11176]: ERROR: process_lrm_event: LRM operation drbddisk_mysql_monitor_30000 (35) Timed Out (timeout=10000ms)
Jun 17 14:08:17 node1 lrmd: [11173]: WARN: G_SIG_dispatch: Dispatch function for SIGCHLD took too long to execute: 930 ms (> 300 ms) (GSource: 0xc50e9b8)
Jun 17 14:08:17 node1 crmd: [11176]: ERROR: process_lrm_event: LRM operation pingd_child:1_monitor_15000 (49) Timed Out (timeout=5000ms)
Jun 17 14:08:17 node1 tengine: [17203]: info: process_graph_event: Detected action drbddisk_mysql_monitor_30000 from a different transition: 10 vs. 17
Jun 17 14:08:17 node1 tengine: [17203]: info: update_abort_priority: Abort priority upgraded to 1000000
Jun 17 14:08:17 node1 tengine: [17203]: WARN: update_failcount: Updating failcount for drbddisk_mysql on 88f6568d-e0ec-40a3-bed5-cd6b0762ef42 after failed monitor: rc=-2
Jun 17 14:08:17 node1 crmd: [11176]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Jun 17 14:08:17 node1 crmd: [11176]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Jun 17 14:08:17 node1 tengine: [17203]: info: process_graph_event: Action pingd_child:1_monitor_15000 arrived after a completed transition
Jun 17 14:08:17 node1 pengine: [17204]: info: determine_online_status: Node node2.mydomian.com is online
Jun 17 14:08:17 node1 tengine: [17203]: WARN: update_failcount: Updating failcount for pingd_child:1 on 88f6568d-e0ec-40a3-bed5-cd6b0762ef42 after failed monitor: rc=-2
Jun 17 14:08:18 node1 pengine: [17204]: info: determine_online_status: Node node1.mydomian.com is online
Jun 17 14:08:18 node1 tengine: [17203]: info: extract_event: Aborting on transient_attributes changes for 88f6568d-e0ec-40a3-bed5-cd6b0762ef42
Jun 17 14:08:18 node1 pengine: [17204]: info: unpack_find_resource: Internally renamed pingd_child:0 on node1.mydomian.com to pingd_child:1
Jun 17 14:08:18 node1 tengine: [17203]: info: extract_event: Aborting on transient_attributes changes for 88f6568d-e0ec-40a3-bed5-cd6b0762ef42
Jun 17 14:08:18 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op pingd_child:1_monitor_15000 on node1.mydomian.com: Timed Out
Jun 17 14:08:18 node1 tengine: [17203]: info: te_update_diff: Aborting on transient_attributes deletions
Jun 17 14:08:18 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op drbddisk_mysql_monitor_30000 on node1.mydomian.com: Timed Out
Jun 17 14:08:18 node1 pengine: [17204]: notice: group_print: Resource Group: rg_mysql
Jun 17 14:08:18 node1 pengine: [17204]: notice: native_print:     ip_mysql	(heartbeat::ocf:IPaddr):	Started node1.mydomian.com
Jun 17 14:08:18 node1 pengine: [17204]: notice: native_print:     drbddisk_mysql	(heartbeat:drbddisk):	Started node1.mydomian.com FAILED
Jun 17 14:08:18 node1 pengine: [17204]: notice: native_print:     fs_mysql	(heartbeat::ocf:Filesystem):	Started node1.mydomian.com
Jun 17 14:08:18 node1 pengine: [17204]: notice: native_print:     mysql	(lsb:mysql):	Started node1.mydomian.com
Jun 17 14:08:18 node1 pengine: [17204]: notice: native_print:     rg_mysql_mailto	(heartbeat::ocf:MailTo):	Started node1.mydomian.com
Jun 17 14:08:18 node1 pengine: [17204]: notice: clone_print: Clone Set: pingd_clone
Jun 17 14:08:18 node1 pengine: [17204]: notice: native_print:     pingd_child:0	(heartbeat::ocf:pingd):	Started node2.mydomian.com
Jun 17 14:08:18 node1 pengine: [17204]: notice: native_print:     pingd_child:1	(heartbeat::ocf:pingd):	Started node1.mydomian.com FAILED
Jun 17 14:08:18 node1 pengine: [17204]: notice: NoRoleChange: Leave resource ip_mysql	(node1.mydomian.com)
Jun 17 14:08:18 node1 pengine: [17204]: notice: NoRoleChange: Recover resource drbddisk_mysql	(node1.mydomian.com)
Jun 17 14:08:18 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop drbddisk_mysql
Jun 17 14:08:18 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start drbddisk_mysql
Jun 17 14:08:18 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   drbddisk_mysql_monitor_30000
Jun 17 14:08:18 node1 pengine: [17204]: notice: NoRoleChange: Leave resource fs_mysql	(node1.mydomian.com)
Jun 17 14:08:18 node1 pengine: [17204]: notice: NoRoleChange: Leave resource mysql	(node1.mydomian.com)
Jun 17 14:08:18 node1 pengine: [17204]: notice: NoRoleChange: Leave resource rg_mysql_mailto	(node1.mydomian.com)
Jun 17 14:08:18 node1 pengine: [17204]: notice: NoRoleChange: Leave resource pingd_child:0	(node2.mydomian.com)
Jun 17 14:08:18 node1 pengine: [17204]: notice: NoRoleChange: Recover resource pingd_child:1	(node1.mydomian.com)
Jun 17 14:08:18 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop pingd_child:1
Jun 17 14:08:18 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start pingd_child:1
Jun 17 14:08:18 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   pingd_child:1_monitor_15000
Jun 17 14:08:22 node1 pengine: [17204]: info: process_pe_message: Transition 18: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-161.bz2
Jun 17 14:08:22 node1 pengine: [17204]: info: determine_online_status: Node node2.mydomian.com is online
Jun 17 14:08:22 node1 pengine: [17204]: info: determine_online_status: Node node1.mydomian.com is online
Jun 17 14:08:22 node1 pengine: [17204]: info: unpack_find_resource: Internally renamed pingd_child:0 on node1.mydomian.com to pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op pingd_child:1_monitor_15000 on node1.mydomian.com: Timed Out
Jun 17 14:08:22 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op drbddisk_mysql_monitor_30000 on node1.mydomian.com: Timed Out
Jun 17 14:08:22 node1 pengine: [17204]: notice: group_print: Resource Group: rg_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     ip_mysql	(heartbeat::ocf:IPaddr):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     drbddisk_mysql	(heartbeat:drbddisk):	Started node1.mydomian.com FAILED
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     fs_mysql	(heartbeat::ocf:Filesystem):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     mysql	(lsb:mysql):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     rg_mysql_mailto	(heartbeat::ocf:MailTo):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: clone_print: Clone Set: pingd_clone
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     pingd_child:0	(heartbeat::ocf:pingd):	Started node2.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     pingd_child:1	(heartbeat::ocf:pingd):	Started node1.mydomian.com FAILED
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource ip_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Recover resource drbddisk_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop drbddisk_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start drbddisk_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   drbddisk_mysql_monitor_30000
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource fs_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource rg_mysql_mailto	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource pingd_child:0	(node2.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Recover resource pingd_child:1	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   pingd_child:1_monitor_15000
Jun 17 14:08:22 node1 pengine: [17204]: info: process_pe_message: Transition 19: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-162.bz2
Jun 17 14:08:22 node1 pengine: [17204]: info: determine_online_status: Node node2.mydomian.com is online
Jun 17 14:08:22 node1 pengine: [17204]: info: determine_online_status: Node node1.mydomian.com is online
Jun 17 14:08:22 node1 pengine: [17204]: info: unpack_find_resource: Internally renamed pingd_child:0 on node1.mydomian.com to pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op pingd_child:1_monitor_15000 on node1.mydomian.com: Timed Out
Jun 17 14:08:22 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op drbddisk_mysql_monitor_30000 on node1.mydomian.com: Timed Out
Jun 17 14:08:22 node1 pengine: [17204]: notice: group_print: Resource Group: rg_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     ip_mysql	(heartbeat::ocf:IPaddr):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     drbddisk_mysql	(heartbeat:drbddisk):	Started node1.mydomian.com FAILED
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     fs_mysql	(heartbeat::ocf:Filesystem):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     mysql	(lsb:mysql):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     rg_mysql_mailto	(heartbeat::ocf:MailTo):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: clone_print: Clone Set: pingd_clone
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     pingd_child:0	(heartbeat::ocf:pingd):	Started node2.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     pingd_child:1	(heartbeat::ocf:pingd):	Started node1.mydomian.com FAILED
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource ip_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Recover resource drbddisk_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop drbddisk_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start drbddisk_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   drbddisk_mysql_monitor_30000
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource fs_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource rg_mysql_mailto	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource pingd_child:0	(node2.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Recover resource pingd_child:1	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   pingd_child:1_monitor_15000
Jun 17 14:08:22 node1 pengine: [17204]: info: process_pe_message: Transition 20: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-163.bz2
Jun 17 14:08:22 node1 pengine: [17204]: info: determine_online_status: Node node2.mydomian.com is online
Jun 17 14:08:22 node1 pengine: [17204]: info: determine_online_status: Node node1.mydomian.com is online
Jun 17 14:08:22 node1 pengine: [17204]: info: unpack_find_resource: Internally renamed pingd_child:0 on node1.mydomian.com to pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op pingd_child:1_monitor_15000 on node1.mydomian.com: Timed Out
Jun 17 14:08:22 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op drbddisk_mysql_monitor_30000 on node1.mydomian.com: Timed Out
Jun 17 14:08:22 node1 pengine: [17204]: notice: group_print: Resource Group: rg_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     ip_mysql	(heartbeat::ocf:IPaddr):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     drbddisk_mysql	(heartbeat:drbddisk):	Started node1.mydomian.com FAILED
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     fs_mysql	(heartbeat::ocf:Filesystem):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     mysql	(lsb:mysql):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     rg_mysql_mailto	(heartbeat::ocf:MailTo):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: clone_print: Clone Set: pingd_clone
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     pingd_child:0	(heartbeat::ocf:pingd):	Started node2.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     pingd_child:1	(heartbeat::ocf:pingd):	Started node1.mydomian.com FAILED
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource ip_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Recover resource drbddisk_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop drbddisk_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start drbddisk_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   drbddisk_mysql_monitor_30000
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource fs_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource rg_mysql_mailto	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource pingd_child:0	(node2.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Recover resource pingd_child:1	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   pingd_child:1_monitor_15000
Jun 17 14:08:22 node1 pengine: [17204]: info: process_pe_message: Transition 21: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-164.bz2
Jun 17 14:08:22 node1 pengine: [17204]: info: determine_online_status: Node node2.mydomian.com is online
Jun 17 14:08:22 node1 pengine: [17204]: info: determine_online_status: Node node1.mydomian.com is online
Jun 17 14:08:22 node1 pengine: [17204]: info: unpack_find_resource: Internally renamed pingd_child:0 on node1.mydomian.com to pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op pingd_child:1_monitor_15000 on node1.mydomian.com: Timed Out
Jun 17 14:08:22 node1 pengine: [17204]: WARN: unpack_rsc_op: Processing failed op drbddisk_mysql_monitor_30000 on node1.mydomian.com: Timed Out
Jun 17 14:08:22 node1 pengine: [17204]: notice: group_print: Resource Group: rg_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     ip_mysql	(heartbeat::ocf:IPaddr):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     drbddisk_mysql	(heartbeat:drbddisk):	Started node1.mydomian.com FAILED
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     fs_mysql	(heartbeat::ocf:Filesystem):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     mysql	(lsb:mysql):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     rg_mysql_mailto	(heartbeat::ocf:MailTo):	Started node1.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: clone_print: Clone Set: pingd_clone
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     pingd_child:0	(heartbeat::ocf:pingd):	Started node2.mydomian.com
Jun 17 14:08:22 node1 pengine: [17204]: notice: native_print:     pingd_child:1	(heartbeat::ocf:pingd):	Started node1.mydomian.com FAILED
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource ip_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Recover resource drbddisk_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop drbddisk_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start drbddisk_mysql
Jun 17 14:08:22 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   drbddisk_mysql_monitor_30000
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource fs_mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource mysql	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource rg_mysql_mailto	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Leave resource pingd_child:0	(node2.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: NoRoleChange: Recover resource pingd_child:1	(node1.mydomian.com)
Jun 17 14:08:22 node1 pengine: [17204]: notice: StopRsc:   node1.mydomian.com	Stop pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: notice: StartRsc:  node1.mydomian.com	Start pingd_child:1
Jun 17 14:08:22 node1 pengine: [17204]: notice: RecurringOp: node1.mydomian.com	   pingd_child:1_monitor_15000
Jun 17 14:08:22 node1 crmd: [11176]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Jun 17 14:08:22 node1 pengine: [17204]: info: process_pe_message: Transition 22: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-165.bz2
Jun 17 14:08:22 node1 tengine: [17203]: info: unpack_graph: Unpacked transition 22: 24 actions in 24 synapses
Jun 17 14:08:22 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 23 fired and confirmed
Jun 17 14:08:22 node1 tengine: [17203]: info: send_rsc_command: Initiating action 19: rg_mysql_mailto_stop_0 on node1.mydomian.com
Jun 17 14:08:22 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 30 fired and confirmed
Jun 17 14:08:22 node1 tengine: [17203]: info: send_rsc_command: Initiating action 4: pingd_child:1_stop_0 on node1.mydomian.com
Jun 17 14:08:22 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=rg_mysql_mailto_stop_0 key=19:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:08:22 node1 lrmd: [11173]: info: rsc:rg_mysql_mailto: stop
Jun 17 14:08:22 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=pingd_child:1_stop_0 key=4:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:08:22 node1 lrmd: [11173]: info: rsc:pingd_child:1: stop
Jun 17 14:08:22 node1 crmd: [11176]: info: process_lrm_event: LRM operation pingd_child:1_monitor_15000 (call=49, rc=-2) Cancelled 
Jun 17 14:08:22 node1 crmd: [11176]: info: process_lrm_event: LRM operation pingd_child:1_stop_0 (call=51, rc=0) complete 
Jun 17 14:08:22 node1 tengine: [17203]: info: match_graph_event: Action pingd_child:1_stop_0 (4) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:08:22 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 31 fired and confirmed
Jun 17 14:08:22 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 28 fired and confirmed
Jun 17 14:08:22 node1 tengine: [17203]: info: send_rsc_command: Initiating action 27: pingd_child:1_start_0 on node1.mydomian.com
Jun 17 14:08:22 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=pingd_child:1_start_0 key=27:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:08:22 node1 lrmd: [11173]: info: rsc:pingd_child:1: start
Jun 17 14:08:22 node1 crmd: [11176]: info: process_lrm_event: LRM operation rg_mysql_mailto_stop_0 (call=50, rc=0) complete 
Jun 17 14:08:22 node1 tengine: [17203]: info: match_graph_event: Action rg_mysql_mailto_stop_0 (19) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:08:22 node1 tengine: [17203]: info: send_rsc_command: Initiating action 17: mysql_stop_0 on node1.mydomian.com
Jun 17 14:08:22 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=mysql_stop_0 key=17:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:08:22 node1 lrmd: [11173]: info: rsc:mysql: stop
Jun 17 14:08:22 node1 lrmd: [5289]: WARN: For LSB init script, no additional parameters are needed.
Jun 17 14:08:22 node1 pingd: [5285]: info: Invoked: /usr/lib64/heartbeat/pingd -D -p /var/run/heartbeat/rsctmp/pingd-pingd_child:1 -a pingd -d 5s -m 100 
Jun 17 14:08:22 node1 crmd: [11176]: info: process_lrm_event: LRM operation pingd_child:1_start_0 (call=52, rc=0) complete 
Jun 17 14:08:22 node1 tengine: [17203]: info: match_graph_event: Action pingd_child:1_start_0 (27) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:08:22 node1 tengine: [17203]: info: send_rsc_command: Initiating action 3: pingd_child:1_monitor_15000 on node1.mydomian.com
Jun 17 14:08:22 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 29 fired and confirmed
Jun 17 14:08:22 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=pingd_child:1_monitor_15000 key=3:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:08:22 node1 crmd: [11176]: info: process_lrm_event: LRM operation pingd_child:1_monitor_15000 (call=54, rc=0) complete 
Jun 17 14:08:22 node1 tengine: [17203]: info: match_graph_event: Action pingd_child:1_monitor_15000 (3) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:08:22 node1 lrmd: [11173]: info: RA output: (mysql:stop:stdout) Shutting down MySQL
Jun 17 14:08:22 node1 lrmd: [11173]: info: RA output: (mysql:stop:stdout) .
Jun 17 14:08:27 node1 last message repeated 5 times
Jun 17 14:08:28 node1 pingd: [5294]: info: do_node_walk: Requesting the list of configured nodes
Jun 17 14:08:28 node1 pingd: [5294]: info: send_update: 1 active ping nodes
Jun 17 14:08:28 node1 pingd: [5294]: info: main: Starting pingd
Jun 17 14:08:28 node1 lrmd: [11173]: info: RA output: (mysql:stop:stdout) .
Jun 17 14:08:59 node1 last message repeated 30 times
Jun 17 14:09:15 node1 last message repeated 16 times
Jun 17 14:09:16 node1 lrmd: [11173]: info: RA output: (mysql:stop:stdout)  SUCCESS!  
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation mysql_stop_0 (call=53, rc=0) complete 
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action mysql_stop_0 (17) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 15: fs_mysql_stop_0 on node1.mydomian.com
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=fs_mysql_stop_0 key=15:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 lrmd: [11173]: info: rsc:fs_mysql: stop
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation fs_mysql_monitor_30000 (call=37, rc=-2) Cancelled 
Jun 17 14:09:16 node1 Filesystem[5544]: INFO: Running stop for /dev/drbd0 on /mnt/mysql_data
Jun 17 14:09:16 node1 Filesystem[5544]: INFO: Trying to unmount /mnt/mysql_data
Jun 17 14:09:16 node1 Filesystem[5544]: INFO: unmounted /mnt/mysql_data successfully
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation fs_mysql_stop_0 (call=55, rc=0) complete 
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action fs_mysql_stop_0 (15) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 6: drbddisk_mysql_stop_0 on node1.mydomian.com
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=drbddisk_mysql_stop_0 key=6:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 lrmd: [11173]: info: rsc:drbddisk_mysql: stop
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation drbddisk_mysql_monitor_30000 (call=35, rc=-2) Cancelled 
Jun 17 14:09:16 node1 kernel: drbd0: role( Primary -> Secondary ) 
Jun 17 14:09:16 node1 kernel: drbd0: Writing meta data super block now.
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation drbddisk_mysql_stop_0 (call=56, rc=0) complete 
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action drbddisk_mysql_stop_0 (6) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 24 fired and confirmed
Jun 17 14:09:16 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 11 fired and confirmed
Jun 17 14:09:16 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 21 fired and confirmed
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 13: ip_mysql_start_0 on node1.mydomian.com
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=ip_mysql_start_0 key=13:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 lrmd: [11173]: info: rsc:ip_mysql: start
Jun 17 14:09:16 node1 IPaddr[5608]: INFO: Using calculated nic for 192.168.0.20: bond0
Jun 17 14:09:16 node1 IPaddr[5608]: INFO: Using calculated netmask for 192.168.0.20: 255.255.254.0
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation ip_mysql_start_0 (call=57, rc=0) complete 
Jun 17 14:09:16 node1 crmd: [11176]: ERROR: lrm_get_rsc_type_metadata(572): got a return code HA_FAIL from a reply message of rmetadata with function get_ret_from_msg.
Jun 17 14:09:16 node1 crmd: [11176]: WARN: get_rsc_metadata: No metadata found for IPaddr::ocf:heartbeat
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action ip_mysql_start_0 (13) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 2: ip_mysql_monitor_30000 on node1.mydomian.com
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 14: drbddisk_mysql_start_0 on node1.mydomian.com
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=ip_mysql_monitor_30000 key=2:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=drbddisk_mysql_start_0 key=14:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 lrmd: [11173]: info: rsc:drbddisk_mysql: start
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation ip_mysql_monitor_30000 (call=33, rc=-2) Cancelled 
Jun 17 14:09:16 node1 kernel: drbd0: role( Secondary -> Primary ) 
Jun 17 14:09:16 node1 kernel: drbd0: Writing meta data super block now.
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation drbddisk_mysql_start_0 (call=59, rc=0) complete 
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action drbddisk_mysql_start_0 (14) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 5: drbddisk_mysql_monitor_30000 on node1.mydomian.com
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 16: fs_mysql_start_0 on node1.mydomian.com
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=drbddisk_mysql_monitor_30000 key=5:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=fs_mysql_start_0 key=16:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 lrmd: [11173]: info: rsc:fs_mysql: start
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation drbddisk_mysql_monitor_30000 (call=60, rc=0) complete 
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation ip_mysql_monitor_30000 (call=58, rc=0) complete 
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action drbddisk_mysql_monitor_30000 (5) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action ip_mysql_monitor_30000 (2) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 Filesystem[5693]: INFO: Running start for /dev/drbd0 on /mnt/mysql_data
Jun 17 14:09:16 node1 Filesystem[5693]: INFO: Starting filesystem check on /dev/drbd0
Jun 17 14:09:16 node1 lrmd: [11173]: info: RA output: (fs_mysql:start:stdout) fsck 1.39 (29-May-2006) 
Jun 17 14:09:16 node1 lrmd: [11173]: info: RA output: (fs_mysql:start:stdout) /dev/drbd0: clean, 915/52428800 files, 53498348/104854391 blocks 
Jun 17 14:09:16 node1 kernel: kjournald starting.  Commit interval 5 seconds
Jun 17 14:09:16 node1 kernel: EXT3 FS on drbd0, internal journal
Jun 17 14:09:16 node1 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation fs_mysql_start_0 (call=61, rc=0) complete 
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action fs_mysql_start_0 (16) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 7: fs_mysql_monitor_30000 on node1.mydomian.com
Jun 17 14:09:16 node1 tengine: [17203]: info: send_rsc_command: Initiating action 18: mysql_start_0 on node1.mydomian.com
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=fs_mysql_monitor_30000 key=7:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=mysql_start_0 key=18:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:16 node1 lrmd: [11173]: info: rsc:mysql: start
Jun 17 14:09:16 node1 lrmd: [5755]: WARN: For LSB init script, no additional parameters are needed.
Jun 17 14:09:16 node1 crmd: [11176]: info: process_lrm_event: LRM operation fs_mysql_monitor_30000 (call=62, rc=0) complete 
Jun 17 14:09:16 node1 tengine: [17203]: info: match_graph_event: Action fs_mysql_monitor_30000 (7) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:16 node1 lrmd: [11173]: info: RA output: (mysql:start:stdout) Starting MySQL
Jun 17 14:09:16 node1 lrmd: [11173]: info: RA output: (mysql:start:stdout) .
Jun 17 14:09:19 node1 last message repeated 3 times
Jun 17 14:09:20 node1 lrmd: [11173]: info: RA output: (mysql:start:stdout)  SUCCESS!  
Jun 17 14:09:20 node1 crmd: [11176]: info: process_lrm_event: LRM operation mysql_start_0 (call=63, rc=0) complete 
Jun 17 14:09:20 node1 tengine: [17203]: info: match_graph_event: Action mysql_start_0 (18) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:20 node1 tengine: [17203]: info: send_rsc_command: Initiating action 20: rg_mysql_mailto_start_0 on node1.mydomian.com
Jun 17 14:09:20 node1 crmd: [11176]: info: do_lrm_rsc_op: Performing op=rg_mysql_mailto_start_0 key=20:22:d797e1f5-028c-48d2-beb6-600a4796cecf)
Jun 17 14:09:20 node1 lrmd: [11173]: info: rsc:rg_mysql_mailto: start
Jun 17 14:09:20 node1 crmd: [11176]: info: process_lrm_event: LRM operation rg_mysql_mailto_start_0 (call=64, rc=0) complete 
Jun 17 14:09:20 node1 tengine: [17203]: info: match_graph_event: Action rg_mysql_mailto_start_0 (20) confirmed on node1.mydomian.com (rc=0)
Jun 17 14:09:20 node1 tengine: [17203]: info: te_pseudo_action: Pseudo action 22 fired and confirmed
Jun 17 14:09:20 node1 tengine: [17203]: info: run_graph: Transition 22: (Complete=24, Pending=0, Fired=0, Skipped=0, Incomplete=0)
Jun 17 14:09:20 node1 tengine: [17203]: info: notify_crmd: Transition 22 status: te_complete - <null>
Jun 17 14:09:20 node1 crmd: [11176]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Jun 17 14:11:37 node1 cib: [11172]: info: cib_stats: Processed 35 operations (13142.00us average, 0% utilization) in the last 10min
Jun 17 15:13:26 node1 cibadmin: [14958]: info: Invoked: cibadmin -o resources -R -x /etc/ha.d/cib/hb_mysql.xml 
