Solution after 2 moths of random experimenting
1) if you free up host from any vms or stop them, it gets back as UP,
without any action
2) if you restart cloudstack-management server, all hosts goes to Alert
state...
Total resolution: UPDATE cloud.vm_instance SET host_id = NULL where id >
0; Hosts comes UP and CS displays real count of vms running.
Janis
On 2024-07-01 21:44, Janis Viklis | Files.fm wrote:
Hi,
looking for help after 2 weeks: What could be the reason that
suddenly after restarting the 4.13.1 Management server, all 4 XEN
(xcp-ng 8.1) hosts of one Intel cluster disconnects and goes into
"Alert state" with an error:
Monitor ComputeCapacityListener says there is an error in the connect
process for 248 due to null
I can't find the reason for 2 weeks. The other AMD Xenserver 6.5
cluster is working just fine.
Everything seems ok: network is working, I restarted: toolstack, both
system vms (SSVM, consolev), one of the hosts, then removed and added
back.
Previously there were 3 management servers via Haproxy and Galera
Mariadb, I left only one. (tried upgrade to 3.14.1, didn't help). I
can manage hosts via Xencenter. There ar 5 storage pools and 3 secondary.
Thanks, hoping on some clues or directions, Janis.
Below is LOG output:
[root@cs1 ~]# cat
/var/log/cloudstack/management/management-server.log | grep ctx-f411cc14
2024-07-01 16:31:25,641 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Simulating start for
resource 5ed7f899-a267-4278-b06d-5e2bc8fea685 id 248
2024-07-01 16:31:25,641 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Creating agent for
host 248
2024-07-01 16:31:25,802 DEBUG [c.c.h.x.r.CitrixResourceBase]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Management network
is on pif=cdb44e56-6c0b-32ab-9113-ae9c8dc5a93f
2024-07-01 16:31:25,816 DEBUG [c.c.h.x.r.XsLocalNetwork]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found a network
called Management on host=10.10.10.131;
Network=f94c6cef-2a03-f964-b272 -a214b6abdaa8;
pif=cdb44e56-6c0b-32ab-9113-ae9c8dc5a93f
2024-07-01 16:31:25,816 INFO [c.c.h.x.r.CitrixResourceBase]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) XenServer Version is
8.1.0 for host 10.10.10.131
2024-07-01 16:31:25,817 INFO [c.c.h.x.r.CitrixResourceBase]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Private Network is
Management for host 10.10.10.131
2024-07-01 16:31:25,817 INFO [c.c.h.x.r.CitrixResourceBase]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Guest Network is
Management for host 10.10.10.131
2024-07-01 16:31:25,817 INFO [c.c.h.x.r.CitrixResourceBase]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Public Network is
Management for host 10.10.10.131
2024-07-01 16:31:25,831 DEBUG [c.c.h.x.r.CitrixResourceBase]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total Ram:
257724032384 dom0 Ram: 8589934592
2024-07-01 16:31:26,103 ERROR [c.c.u.s.SshHelper]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) SSH execution of
command xe sm-list | grep "resigning of duplicates" has an error
status code i n return. Result output:
2024-07-01 16:31:26,113 DEBUG [c.c.h.x.r.XenServer620Resource]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) can't get patches
information for hotFix: 996dd2e7-ad95-49cc-a0be-2c9adc4dfb0b
2024-07-01 16:31:26,122 DEBUG [c.c.h.x.r.XenServer620Resource]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) can't get patches
information for hotFix: 0850b186-4d47-11e3-a720-001b2151a503
2024-07-01 16:31:26,165 DEBUG [c.c.h.x.r.CitrixResourceBase]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found 0 local
storage of type [lvm] for host [5ed7f899-a267-4278-b06d-5e2bc8fea685]
2024-07-01 16:31:26,171 DEBUG [c.c.h.x.r.CitrixResourceBase]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found 0 local
storage of type [ext] for host [5ed7f899-a267-4278-b06d-5e2bc8fea685]
2024-07-01 16:31:26,184 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq -1--1: Startup
request from directly connected host: { Cmd , MgmtId: -1, via: -1,
Ver: v1, F lags: 11,
[{"cpuSockets":2,"cpus":48,"speed":2500,"memory":257724032384,"dom0MinMemory":8589934592,"poolSync":false,"supportsClonedVolumes":false,"caps":"xen-3.0-x86_64
, xen-3.0-x86_32p , hvm-3. 0-x86_32 , hvm-3.0-x86_32p ,
hvm-3.0-x86_64","pool":"3e71abd3-f524-1cbd-da4d-47d0b77d85a3","hypervisorType":"XenServer","hostDetails":{"product_version":"8.1.0","com.cloud.network.Networks.Router
PrivateIpStrategy":"DcGlobal","product_version_text_short":"8.1","private.network.device":"Management","product_brand":"XCP-ng"},"hostTags":["GPU"],"hypervisorVersion":"8.1.0","groupDetails":{"Gr
oup of ASPEED Technology, Inc. ASPEED Graphics Family
GPUs":{}},"type":"Routing","dataCenter":"9","pod":"12","cluster":"29","guid":"5ed7f899-a267-4278-b06d-5e2bc8fea685","name":"xs31.failiem.lv",
"version":"4.13.1.0","iqn":"iqn.2020-06.lv.failiem:7264f035","publicIpAddress":"10.10.10.131","publicNetmask":"255.255.255.0","publicMacAddress":"0c:c4:7a:bb:cb:22","privateIpAddress":"10.10.10.1
31","privateMacAddress":"0c:c4:7a:bb:cb:22","privateNetmask":"255.255.255.0","storageIpAddress":"10.10.10.131","storageNetmask":"255.255.255.0","storageMacAddress":"0c:c4:7a:bb:cb:22","wait":0}]
}
2024-07-01 16:31:26,198 DEBUG [c.c.r.ResourceManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Dispatching resource
state event CREATE_HOST_VO_FOR_DIRECT_CONNECT to CiscoVnmcElement
2024-07-01 16:31:26,198 DEBUG [c.c.r.ResourceManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Dispatching resource
state event CREATE_HOST_VO_FOR_DIRECT_CONNECT to
BareMetalDiscover er
2024-07-01 16:31:26,198 DEBUG [c.c.r.ResourceManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Dispatching resource
state event CREATE_HOST_VO_FOR_DIRECT_CONNECT to NetscalerElement
2024-07-01 16:31:26,198 DEBUG [c.c.r.ResourceManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Dispatching resource
state event CREATE_HOST_VO_FOR_DIRECT_CONNECT to
JuniperSRXExterna lFirewallElement
2024-07-01 16:31:26,198 DEBUG [c.c.r.ResourceManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Dispatching resource
state event CREATE_HOST_VO_FOR_DIRECT_CONNECT to
HypervServerDisco verer
2024-07-01 16:31:26,198 DEBUG [c.c.r.ResourceManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Dispatching resource
state event CREATE_HOST_VO_FOR_DIRECT_CONNECT to
BaremetalPxeManag erImpl
2024-07-01 16:31:26,198 DEBUG [c.c.r.ResourceManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Dispatching resource
state event CREATE_HOST_VO_FOR_DIRECT_CONNECT to
XcpServerDiscover er
2024-07-01 16:31:26,202 INFO [c.c.h.x.d.XcpServerDiscoverer]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Host:
xs31.failiem.lv connected with hypervisor type: XenServer. Checking
CIDR...
2024-07-01 16:31:26,263 DEBUG [c.c.r.ResourceState]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Resource state
update: [id = 248; name = xs31.failiem.lv; old state = Enabled; event
= Intern alCreated; new state = Enabled]
2024-07-01 16:31:26,263 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Transition:[Resource
state = Enabled, Agent event = AgentConnected, Host id = 248,
name = xs31.failiem.lv]
2024-07-01 16:31:26,270 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) create
ClusteredDirectAgentAttache for 248
2024-07-01 16:31:26,270 INFO [c.c.a.m.DirectAgentAttache]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) StartupAnswer
received 248 Interval = 60
2024-07-01 16:31:26,273 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: XcpServerDiscoverer
2024-07-01 16:31:26,279 WARN [c.c.h.x.d.XcpServerDiscoverer]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) defaulting to
xenserver650 resource for product brand: XCP-ng with product
version: 8.1.0
2024-07-01 16:31:26,279 DEBUG [c.c.h.x.d.XcpServerDiscoverer]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Setting up host 248
2024-07-01 16:31:26,289 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381185: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100111,
[{"com.cloud.agent.api.SetupCommand":{"env":{},"multipath":false,"needSetup":false,"wait":0}}]
}
2024-07-01 16:31:26,289 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381185: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.SetupCommand":{"env":{},"multipath":false,"needSetup":false,"wait":0}}]
}
2024-07-01 16:31:29,517 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381185: Received: { Ans: , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 110, { SetupAnswer } }
2024-07-01 16:31:29,524 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: HypervServerDiscoverer
2024-07-01 16:31:29,524 DEBUG [c.c.h.h.d.HypervServerDiscoverer]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Not Hyper-V
hypervisor, so moving on.
2024-07-01 16:31:29,524 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: VmwareManagerImpl
2024-07-01 16:31:29,524 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: ClusteredVirtualMachineManagerImpl
2024-07-01 16:31:29,524 DEBUG [c.c.v.VirtualMachineManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Received startup
command from hypervisor host. host id: 248
2024-07-01 16:31:29,524 INFO [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Reset VM power state
sync for host: 248
2024-07-01 16:31:29,539 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381186: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100011,
[{"com.cloud.agent.api.ClusterVMMetaDataSyncCommand":{"_interval":180,"_clusterId":29,"wait":0}}]
}
2024-07-01 16:31:29,539 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381186: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ClusterVMMetaDataSyncCommand":{"_interval":180,"_clusterId":29,"wait":0}}]
}
2024-07-01 16:31:29,539 DEBUG [c.c.v.VirtualMachineManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Cluster VM metadata
sync started with jobid 1798343626204381186
2024-07-01 16:31:29,539 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: NetworkOrchestrator
2024-07-01 16:31:29,546 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Host's
hypervisorType is: XenServer
2024-07-01 16:31:29,581 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending
CheckNetworkCommand to check the Network is setup correctly on Agent
2024-07-01 16:31:29,594 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381187: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100111,
[{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":202,"privateNetworkName":"Management","storageNetworkName":"Management"},{"physicalNetwor
kId":203,"publicNetworkName":"Public","guestNetworkName":"Public"}],"wait":0}}]
}
2024-07-01 16:31:29,594 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381187: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":202,"privateNetworkName":"Management","storageNetworkName":"Management"},{"physicalNet
workId":203,"publicNetworkName":"Public","guestNetworkName":"Public"}],"wait":0}}]
}
2024-07-01 16:31:29,605 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381187: Received: { Ans: , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 110, {
CheckNetworkAnswer } }
2024-07-01 16:31:29,605 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Details from
executing class com.cloud.agent.api.CheckNetworkCommand: Network Setup
chec k by names is done
2024-07-01 16:31:29,605 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Network setup is
correct on Agent
2024-07-01 16:31:29,605 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: StoragePoolMonitor
2024-07-01 16:31:29,617 DEBUG [c.c.s.l.StoragePoolMonitor]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Host 248 connected,
connecting host to shared pool id 152 and sending storage pool
inf ormation ...
2024-07-01 16:31:29,620 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Adding pool null to
host 248
2024-07-01 16:31:29,635 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381188: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":152,"uuid":"47c142c0-770c-32b0-ba43-20e0fa2d7f9e","host":"10.10.10.41","path":"/xs3-SSD","port":2049
,"type":"NetworkFilesystem"},"localPath":"/mnt//47c142c0-770c-32b0-ba43-20e0fa2d7f9e","wait":0}}]
}
2024-07-01 16:31:29,636 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381188: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":152,"uuid":"47c142c0-770c-32b0-ba43-20e0fa2d7f9e","host":"10.10.10.41","path":"/xs3-SSD","port":2
049,"type":"NetworkFilesystem"},"localPath":"/mnt//47c142c0-770c-32b0-ba43-20e0fa2d7f9e","wait":0}}]
}
2024-07-01 16:31:29,839 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381188: Received: { Ans: , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 10, {
ModifyStoragePoolAnswer } }
2024-07-01 16:31:29,863 INFO [o.a.c.s.d.p.DefaultHostListener]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Connection
established between storage pool org.apache.cloudstack.storage.datasto
re.PrimaryDataStoreImpl@7df02a59 and host 248
2024-07-01 16:31:29,872 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found storage pool
csstorage2-xs3-SSD of type NetworkFilesystem with overprovisioning
fa ctor 2
2024-07-01 16:31:29,872 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity calculated is 2 * 3829052473344
2024-07-01 16:31:29,872 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity of the pool csstorage2-xs3-SSD id: 152 is
7658104946688
2024-07-01 16:31:29,877 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Successfully set
Capacity - 7658104946688 for capacity type - 3 , DataCenterId - 9,
Host OrPoolId - 152, PodId 12
2024-07-01 16:31:29,877 DEBUG [c.c.s.l.StoragePoolMonitor]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Host 248 connected,
connecting host to shared pool id 158 and sending storage pool
inf ormation ...
2024-07-01 16:31:29,879 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Adding pool null to
host 248
2024-07-01 16:31:29,888 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381189: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":158,"uuid":"56d6552e-b8a8-3f8b-b218-49b12f5443db","host":"csstorage3.failiem.lv","path":"/primary-NV
M/xs3","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//56d6552e-b8a8-3f8b-b218-49b12f5443db","wait":0}}]
}
2024-07-01 16:31:29,888 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381189: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":158,"uuid":"56d6552e-b8a8-3f8b-b218-49b12f5443db","host":"csstorage3.failiem.lv","path":"/primary
-NVM/xs3","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//56d6552e-b8a8-3f8b-b218-49b12f5443db","wait":0}}]
}
2024-07-01 16:31:30,086 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381189: Received: { Ans: , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 10, {
ModifyStoragePoolAnswer } }
2024-07-01 16:31:30,094 INFO [o.a.c.s.d.p.DefaultHostListener]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Connection
established between storage pool org.apache.cloudstack.storage.datasto
re.PrimaryDataStoreImpl@42d3c9d9 and host 248
2024-07-01 16:31:30,104 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found storage pool
csstorage3-xs3-NVM of type NetworkFilesystem with overprovisioning
fa ctor 2
2024-07-01 16:31:30,104 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity calculated is 2 * 1362611929088
2024-07-01 16:31:30,104 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity of the pool csstorage3-xs3-NVM id: 158 is
2725223858176
2024-07-01 16:31:30,109 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Successfully set
Capacity - 2725223858176 for capacity type - 3 , DataCenterId - 9,
Host OrPoolId - 158, PodId 12
2024-07-01 16:31:30,109 DEBUG [c.c.s.l.StoragePoolMonitor]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Host 248 connected,
connecting host to shared pool id 160 and sending storage pool
inf ormation ...
2024-07-01 16:31:30,111 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Adding pool null to
host 248
2024-07-01 16:31:30,121 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381190: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":160,"uuid":"f1af685f-f357-390e-9224-84feecd41343","host":"csstorage4.failiem.lv","path":"/primary-SA
TA2-XS3","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//f1af685f-f357-390e-9224-84feecd41343","wait":0}}]
}
2024-07-01 16:31:30,121 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381190: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":160,"uuid":"f1af685f-f357-390e-9224-84feecd41343","host":"csstorage4.failiem.lv","path":"/primary
-SATA2-XS3","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//f1af685f-f357-390e-9224-84feecd41343","wait":0}}]
}
2024-07-01 16:31:30,322 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381190: Received: { Ans: , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 10, {
ModifyStoragePoolAnswer } }
2024-07-01 16:31:30,330 INFO [o.a.c.s.d.p.DefaultHostListener]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Connection
established between storage pool org.apache.cloudstack.storage.datasto
re.PrimaryDataStoreImpl@306585d2 and host 248
2024-07-01 16:31:30,339 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found storage pool
csstorage4-xs3-SATA of type NetworkFilesystem with overprovisioning
f actor 2
2024-07-01 16:31:30,339 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity calculated is 2 * 59766547152896
2024-07-01 16:31:30,339 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity of the pool csstorage4-xs3-SATA id: 160 is
1195330943057 92
2024-07-01 16:31:30,343 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Successfully set
Capacity - 119533094305792 for capacity type - 3 , DataCenterId - 9,
Ho stOrPoolId - 160, PodId 12
2024-07-01 16:31:30,343 DEBUG [c.c.s.l.StoragePoolMonitor]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Host 248 connected,
connecting host to shared pool id 166 and sending storage pool
inf ormation ...
2024-07-01 16:31:30,346 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Adding pool null to
host 248
2024-07-01 16:31:30,354 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381191: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":166,"uuid":"e55b8062-4a38-3842-99ea-b1c18909e0e6","host":"10.10.10.103","path":"/primary-SSD/xs3","p
ort":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//e55b8062-4a38-3842-99ea-b1c18909e0e6","wait":0}}]
}
2024-07-01 16:31:30,354 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381191: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":166,"uuid":"e55b8062-4a38-3842-99ea-b1c18909e0e6","host":"10.10.10.103","path":"/primary-SSD/xs3"
,"port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//e55b8062-4a38-3842-99ea-b1c18909e0e6","wait":0}}]
}
2024-07-01 16:31:30,550 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381191: Received: { Ans: , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 10, {
ModifyStoragePoolAnswer } }
2024-07-01 16:31:30,558 INFO [o.a.c.s.d.p.DefaultHostListener]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Connection
established between storage pool org.apache.cloudstack.storage.datasto
re.PrimaryDataStoreImpl@1c98de8a and host 248
2024-07-01 16:31:30,567 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found storage pool
csstorage3-xs3-SSD of type NetworkFilesystem with overprovisioning
fa ctor 2
2024-07-01 16:31:30,567 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity calculated is 2 * 5753096110080
2024-07-01 16:31:30,567 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity of the pool csstorage3-xs3-SSD id: 166 is
11506192220160
2024-07-01 16:31:30,571 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Successfully set
Capacity - 11506192220160 for capacity type - 3 , DataCenterId - 9,
Hos tOrPoolId - 166, PodId 12
2024-07-01 16:31:30,571 DEBUG [c.c.s.l.StoragePoolMonitor]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Host 248 connected,
connecting host to shared pool id 169 and sending storage pool
inf ormation ...
2024-07-01 16:31:30,573 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Adding pool null to
host 248
2024-07-01 16:31:30,581 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381192: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":169,"uuid":"25418de5-a6fb-3026-a983-c6a2d8f61322","host":"csstorage5.failiem.lv","path":"/primary-NV
Me","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//25418de5-a6fb-3026-a983-c6a2d8f61322","wait":0}}]
}
2024-07-01 16:31:30,581 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381192: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":true,"pool":{"id":169,"uuid":"25418de5-a6fb-3026-a983-c6a2d8f61322","host":"csstorage5.failiem.lv","path":"/primary
-NVMe","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//25418de5-a6fb-3026-a983-c6a2d8f61322","wait":0}}]
}
2024-07-01 16:31:30,790 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381192: Received: { Ans: , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 10, {
ModifyStoragePoolAnswer } }
2024-07-01 16:31:30,798 INFO [o.a.c.s.d.p.DefaultHostListener]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Connection
established between storage pool org.apache.cloudstack.storage.datasto
re.PrimaryDataStoreImpl@55325ecf and host 248
2024-07-01 16:31:30,807 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found storage pool
csstorage5-xs3-NVMe of type NetworkFilesystem with overprovisioning
f actor 2
2024-07-01 16:31:30,807 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity calculated is 2 * 4079092760576
2024-07-01 16:31:30,807 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Total over
provisioned capacity of the pool csstorage5-xs3-NVMe id: 169 is
8158185521152
2024-07-01 16:31:30,815 DEBUG [c.c.s.StorageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Successfully set
Capacity - 8158185521152 for capacity type - 3 , DataCenterId - 9,
Host OrPoolId - 169, PodId 12
2024-07-01 16:31:30,815 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: SecondaryStorageListener
2024-07-01 16:31:30,815 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: DeploymentPlanningManagerImpl
2024-07-01 16:31:30,819 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: SecurityGroupListener
2024-07-01 16:31:30,819 INFO [c.c.n.s.SecurityGroupListener]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Received a host
startup notification
2024-07-01 16:31:30,822 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381193: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100011,
[{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2414,"wait":0}}]
}
2024-07-01 16:31:30,822 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381193: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":2414,"wait":0}}]
}
2024-07-01 16:31:30,822 INFO [c.c.n.s.SecurityGroupListener]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Scheduled network
rules cleanup, interval=2414
2024-07-01 16:31:30,823 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: BehindOnPingListener
2024-07-01 16:31:30,823 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: SetHostParamsListener
2024-07-01 16:31:30,823 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: DownloadListener
2024-07-01 16:31:30,826 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: ConsoleProxyListener
2024-07-01 16:31:30,826 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: SshKeysDistriMonitor
2024-07-01 16:31:30,838 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381194: Sending { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver : v1, Flags: 100011,
[{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}] }
2024-07-01 16:31:30,838 DEBUG [c.c.a.t.Request]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Seq
248-1798343626204381194: Executing: { Cmd , MgmtId: 95534596974, via:
248(xs31.failiem.lv), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifySshKeysCommand":{"wait":0}}] }
2024-07-01 16:31:30,838 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: VpcVirtualNetworkApplianceManagerImpl
2024-07-01 16:31:30,854 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: StorageCapacityListener
2024-07-01 16:31:30,854 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Connect to
listener: ComputeCapacityListener
2024-07-01 16:31:30,906 DEBUG [c.c.c.CapacityManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found 32 VMs on host
248
2024-07-01 16:31:30,910 DEBUG [c.c.c.CapacityManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Found 0 VMs are
Migrating from host 248
2024-07-01 16:31:30,993 ERROR [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Monitor
ComputeCapacityListener says there is an error in the connect process
for 248 du e to null
2024-07-01 16:31:30,993 INFO [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Host 248 is
disconnecting with event AgentDisconnected
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) The next status of
agent 248is Alert, current status is Connecting
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Deregistering link
for 248 with state Alert
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Remove Agent : 248
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.DirectAgentAttache]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Processing
disconnect 248(xs31.failiem.lv)
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.hypervisor.xenserver.discoverer.XcpServerDisco
verer
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.hypervisor.hyperv.discoverer.HypervServerDisco
verer
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.hypervisor.vmware.manager.VmwareManagerImpl
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: org.apache.cloudstack.engine.orchestration.NetworkOrches
trator
2024-07-01 16:31:30,996 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.storage.listener.StoragePoolMonitor
2024-07-01 16:31:30,997 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.storage.secondary.SecondaryStorageListener
2024-07-01 16:31:30,997 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.deploy.DeploymentPlanningManagerImpl
2024-07-01 16:31:30,997 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.network.security.SecurityGroupListener
2024-07-01 16:31:30,997 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.agent.manager.AgentManagerImpl$BehindOnPingLis
tener
2024-07-01 16:31:30,997 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.agent.manager.AgentManagerImpl$SetHostParamsLi
stener
2024-07-01 16:31:30,997 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.storage.download.DownloadListener
2024-07-01 16:31:30,997 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.consoleproxy.ConsoleProxyListener
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.network.SshKeysDistriMonitor
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.network.router.VpcVirtualNetworkApplianceManag
erImpl
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.capacity.StorageCapacityListener
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.capacity.ComputeCapacityListener
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.storage.LocalStoragePoolListener
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.network.SshKeysDistriMonitor
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.network.router.VirtualNetworkApplianceManagerI mpl
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.network.NetworkUsageManagerImpl$DirectNetworkS
tatsListener
2024-07-01 16:31:31,000 DEBUG [c.c.n.NetworkUsageManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Disconnected called
on 248 with status Alert
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Sending Disconnect
to listener: com.cloud.storage.upload.UploadListener
2024-07-01 16:31:31,000 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Transition:[Resource
state = Enabled, Agent event = AgentDisconnected, Host id = 248,
na me = xs31.failiem.lv]
2024-07-01 16:31:31,007 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Notifying other
nodes of to disconnect
2024-07-01 16:31:31,013 WARN [c.c.r.ResourceManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Unable to connect
due to
2024-07-01 16:31:31,014 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Transition:[Resource
state = Enabled, Agent event = AgentDisconnected, Host id = 248,
na me = xs31.failiem.lv]
2024-07-01 16:31:31,022 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentTaskPool-380:ctx-f411cc14) (logid:284129f8) Completed creating
agent for host 248
----------------
LAST WARNINGS: cat
/var/log/cloudstack/management/management-server.log | grep WARN
----------------
2024-07-01 21:37:48,038 WARN [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-2cd13b14) (logid:c460a47e) AlertType:: 24 |
dataCenterId:: 9 | podId:: null | clusterId:: null | message:: System
Alert: Number of unallocated shared network IPs is low in availability
zone LTC-DC
2024-07-01 21:37:48,115 WARN [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-2cd13b14) (logid:c460a47e) AlertType:: 1 |
dataCenterId:: 9 | podId:: 12 | clusterId:: 29 | message:: System
Alert: Low Unallocated CPU in cluster xs3-intel-xcp-ng pod 29/1 of
availability zone LTC-DC
2024-07-01 21:38:48,133 WARN [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-254eadd9) (logid:477960f4) AlertType:: 24 |
dataCenterId:: 9 | podId:: null | clusterId:: null | message:: System
Alert: Number of unallocated shared network IPs is low in availability
zone LTC-DC
2024-07-01 21:38:48,240 WARN [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-254eadd9) (logid:477960f4) AlertType:: 1 |
dataCenterId:: 9 | podId:: 12 | clusterId:: 29 | message:: System
Alert: Low Unallocated CPU in cluster xs3-intel-xcp-ng pod 29/1 of
availability zone LTC-DC
2024-07-01 21:39:17,689 WARN [c.c.v.UserVmManagerImpl]
(StatsCollector-6:ctx-7cde3df8) (logid:bef4aeac) Unable to obtain VM
network statistics.
2024-07-01 21:39:17,876 WARN
[c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-d811632c) (logid:7fc274df) Invalid last
alert timestamp received while collecting alerts from router: r-1485-VM
2024-07-01 21:39:25,981 WARN [c.c.a.m.ClusteredAgentManagerImpl]
(ClusteredAgentManager Timer:ctx-06f6388c) (logid:a7949301) can not
load directly connected host 18(xs13.failiem.lv) due to
2024-07-01 21:39:26,663 WARN [c.c.h.x.d.XcpServerDiscoverer]
(AgentTaskPool-496:ctx-5f6aa810) (logid:bc575c27) defaulting to
xenserver650 resource for product brand: XCP-ng with product version:
8.1.0
2024-07-01 21:39:26,860 WARN [c.c.h.x.d.XcpServerDiscoverer]
(AgentTaskPool-497:ctx-2718223e) (logid:ee799242) defaulting to
xenserver650 resource for product brand: XCP-ng with product version:
8.1.0
2024-07-01 21:39:27,002 WARN [c.c.h.x.d.XcpServerDiscoverer]
(AgentTaskPool-498:ctx-39723084) (logid:a026fd97) defaulting to
xenserver650 resource for product brand: XCP-ng with product version:
8.1.0
2024-07-01 21:39:27,059 WARN [c.c.h.x.d.XcpServerDiscoverer]
(AgentTaskPool-499:ctx-e7eaf7e0) (logid:887afb4f) defaulting to
xenserver650 resource for product brand: XCP-ng with product version:
8.1.0
2024-07-01 21:39:31,324 WARN [c.c.r.ResourceManagerImpl]
(AgentTaskPool-496:ctx-5f6aa810) (logid:bc575c27) Unable to connect
due to
2024-07-01 21:39:32,678 WARN [c.c.r.ResourceManagerImpl]
(AgentTaskPool-497:ctx-2718223e) (logid:ee799242) Unable to connect
due to
2024-07-01 21:39:32,886 WARN [c.c.r.ResourceManagerImpl]
(AgentTaskPool-499:ctx-e7eaf7e0) (logid:887afb4f) Unable to connect
due to
2024-07-01 21:39:32,973 WARN [c.c.r.ResourceManagerImpl]
(AgentTaskPool-498:ctx-39723084) (logid:a026fd97) Unable to connect
due to
2024-07-01 21:39:48,791 WARN [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-fd2d7789) (logid:5d6d6dad) AlertType:: 24 |
dataCenterId:: 9 | podId:: null | clusterId:: null | message:: System
Alert: Number of unallocated shared network IPs is low in availability
zone LTC-DC
2024-07-01 21:39:48,883 WARN [c.c.a.AlertManagerImpl]
(CapacityChecker:ctx-fd2d7789) (logid:5d6d6dad) AlertType:: 1 |
dataCenterId:: 9 | podId:: 12 | clusterId:: 29 | message:: System
Alert: Low Unallocated CPU in cluster xs3-intel-xcp-ng pod 29/1 of
availability zone LTC-DC
