New event: Mar 28 14:37:32 ovirt-node3.ovirt vdsm[4288]: WARN executor state: count=5 workers={<Worker name=periodic/4 waiting task#=4 at 0x7fce0e9d6048>, <Worker name=periodic/5 waiting task#=0 at 0x7fcda8700860>, <Worker name=periodic/3 waiting task#=188 at 0x7fcdc0010470>, <Worker name=periodic/1 waiting task#=189 at 0x7fcdc00102b0>, <Worker name=periodic/2 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7fcdc007a048> at 0x7fcdc0010898> timeout=7.5, duration=7.50 at 0x7fcdc0010208> discarded task#=189 at 0x7fcdc0010390>} Mar 28 14:37:32 ovirt-node3.ovirt sanlock[1662]: 2023-03-28 14:37:32 829 [7438]: s4 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/glusterSD/ovirt-node3.ovirt:_gv0/4745320f-bfc3-46c4-8849-b4fe8f1b2de6/dom_md/ids Mar 28 14:37:32 ovirt-node3.ovirt sanlock[1662]: 2023-03-28 14:37:32 829 [7438]: s4 renewal error -202 delta_length 10 last_success 798 Mar 28 14:37:33 ovirt-node3.ovirt sanlock[1662]: 2023-03-28 14:37:33 830 [7660]: s6 delta_renew read timeout 10 sec offset 0 /rhev/data-center/mnt/ovirt-nfsha.ovirt:_dati_drbd0/2527ed0f-e91a-4748-995c-e644362e8408/dom_md/ids Mar 28 14:37:33 ovirt-node3.ovirt sanlock[1662]: 2023-03-28 14:37:33 830 [7660]: s6 renewal error -202 delta_length 10 last_success 799 Mar 28 14:37:36 ovirt-node3.ovirt pacemaker-controld[3145]: notice: High CPU load detected: 32.590000 Mar 28 14:37:36 ovirt-node3.ovirt kernel: drbd drbd0/0 drbd1 ovirt-node2.ovirt: We did not send a P_BARRIER for 14436ms > ko-count (7) * timeout (10 * 0.1s); drbd kernel thread blocked? Mar 28 14:37:41 ovirt-node3.ovirt libvirtd[2735]: Domain id=1 name='SSIS-microos' uuid=e41f8148-79ab-4a88-879f-894d5750e5fb is tainted: custom-ga-command Mar 28 14:37:49 ovirt-node3.ovirt kernel: drbd drbd0/0 drbd1 ovirt-node2.ovirt: We did not send a P_BARRIER for 7313ms > ko-count (7) * timeout (10 * 0.1s); drbd kernel thread blocked? Mar 28 14:37:56 ovirt-node3.ovirt kernel: drbd drbd0/0 drbd1 ovirt-node2.ovirt: We did not send a P_BARRIER for 14481ms > ko-count (7) * timeout (10 * 0.1s); drbd kernel thread blocked? Mar 28 14:38:06 ovirt-node3.ovirt pacemaker-controld[3145]: notice: High CPU load detected: 33.500000 Mar 28 14:38:09 ovirt-node3.ovirt kernel: drbd drbd0/0 drbd1 ovirt-node2.ovirt: Remote failed to finish a request within 7010ms > ko-count (7) * timeout (10 * 0.1s)
2023-03-28 14:37:32,601Z INFO [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-73) [] There is no host with more than 10 running guests, no balancing is needed 2023-03-28 14:37:50,662Z INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] VM 'ccb06298-33a3-4b6f-bff3-d0bcd494b18d'(TpayX2GO) moved from 'Up' --> 'NotResponding' 2023-03-28 14:37:50,666Z WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-41) [] EVENT_ID: VM_NOT_RESPONDING(126), VM TpayX2GO is not responding. 2023-03-28 14:38:01,087Z WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6602) [] domain '4745320f-bfc3-46c4-8849-b4fe8f1b2de6:gv0' in problem 'PROBLEMATIC'. vds: 'ovirt-node2.ovirt' 2023-03-28 14:38:05,676Z INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-20) [] VM 'ccb06298-33a3-4b6f-bff3-d0bcd494b18d'(TpayX2GO) moved from 'NotResponding' --> 'Up' 2023-03-28 14:38:16,107Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6609) [] Domain '2527ed0f-e91a-4748-995c-e644362e8408:drbd0' recovered from problem. vds: 'ovirt-node2.ovirt' 2023-03-28 14:38:16,107Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6609) [] Domain '4745320f-bfc3-46c4-8849-b4fe8f1b2de6:gv0' recovered from problem. vds: 'ovirt-node2.ovirt' 2023-03-28 14:38:16,107Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6610) [] Domain '2527ed0f-e91a-4748-995c-e644362e8408:drbd0' recovered from problem. vds: 'ovirt-node4.ovirt' 2023-03-28 14:38:16,107Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6610) [] Domain '4745320f-bfc3-46c4-8849-b4fe8f1b2de6:gv0' recovered from problem. vds: 'ovirt-node4.ovirt' 2023-03-28 14:38:16,327Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6612) [] Domain '2527ed0f-e91a-4748-995c-e644362e8408:drbd0' recovered from problem. vds: 'ovirt-node3.ovirt' 2023-03-28 14:38:16,327Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6612) [] Domain '2527ed0f-e91a-4748-995c-e644362e8408:drbd0' has recovered from problem. No active host in the DC is reporting it as problematic, so clearing the domain recovery timer. 2023-03-28 14:38:16,327Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6612) [] Domain '4745320f-bfc3-46c4-8849-b4fe8f1b2de6:gv0' recovered from problem. vds: 'ovirt-node3.ovirt' 2023-03-28 14:38:16,327Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyImpl] (EE-ManagedThreadFactory-engine-Thread-6612) [] Domain '4745320f-bfc3-46c4-8849-b4fe8f1b2de6:gv0' has recovered from problem. No active host in the DC is reporting it as problematic, so clearing the domain recovery timer. 2023-03-28 14:38:18,602Z INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-44) [] VM 'e41f8148-79ab-4a88-879f-894d5750e5fb'(SSIS-microos) moved from 'Up' --> 'NotResponding' 2023-03-28 14:38:18,607Z WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-44) [] EVENT_ID: VM_NOT_RESPONDING(126), VM SSIS-microos is not responding. 2023-03-28 14:38:32,603Z INFO [org.ovirt.engine.core.bll.scheduling.policyunits.EvenGuestDistributionBalancePolicyUnit] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-32) [] There is no host with more than 10 running guests, no balancing is needed _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/V5JGP6TJXTFKWZIPQPD5ZKSUCBIJMBGR/