Seems there was some issue like that and hence HA kicked in, but finally seems like the CS stopped the vm and then tried starting it. Can you please put more logs to see what happened post sending the startcommand
On 06/04/13 7:12 PM, "CK" <[email protected]> wrote: >The XS is a brand new setup and I have checked the files mentioned and >they >are there. However, I have noticed the in the logs (in the original email) >that there may be a connectivity issue. What do you think? > > >On 2 April 2013 09:39, Pranav Saxena <[email protected]> wrote: > >> Can you check if your systemvm.iso is copied to your host and also try >>to >> copy vhd-util to your XS host at /opt/xensource/bin if your XS host is a >> brand new setup . Sometimes the systemvm.iso is not found on your host >> when you build the repo and the systemVM's continue to remain in the >> starting state. >> >> Thanks, >> Pranav >> >> -----Original Message----- >> From: CK [mailto:[email protected]] >> Sent: Tuesday, April 02, 2013 2:31 AM >> To: [email protected] >> Subject: ConsoleProxy not starting up >> >> Hi, >> I am having a problem with the ConsoleProxy VM starting up, in the CS UI >> the consoleproxy is showing as "Starting up" and remains in this state. >>I >> have restarted the cloud-management service as well as the console VM on >> the host, but still appears to be hung. >> >> I am unable to restart or destroy the console VM in the UI, as no such >> option. >> >> Any help would be appreciated. >> >> The CS management log shows the following: >> >> >> 2013-04-01 21:35:35,553 INFO [utils.component.ComponentLocator] >> (Thread-5:null) Asking ConsoleProxyAlert to shutdown. >> 2013-04-01 21:35:36,626 INFO [utils.component.ComponentLocator] >> (Thread-5:null) Asking Manager ConsoleProxyManager to shutdown. >> 2013-04-01 21:35:36,626 INFO >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (Thread-5:null) Stop console proxy manager >> 2013-04-01 21:35:37,626 INFO [utils.component.ComponentLocator] >> (Thread-5:null) Skipping Manager ConsoleProxyManager because it is not >>in >> a state to shutdown. >> 2013-04-01 21:35:47,256 DEBUG [utils.component.ComponentLocator] >> (main:null) Looking for class >> com.cloud.consoleproxy.ConsoleProxyBalanceAllocator >> 2013-04-01 21:35:47,257 INFO [utils.component.ComponentLocator] >> (main:null) Found component: >>com.cloud.consoleproxy.ConsoleProxyAllocator >> in com.cloud.consoleproxy.ConsoleProxyBalanceAllocator - Balance >> 2013-04-01 21:35:47,333 DEBUG [utils.component.ComponentLocator] >> (main:null) Looking for class com.cloud.alert.ConsoleProxyAlertAdapter >> 2013-04-01 21:35:47,334 INFO [utils.component.ComponentLocator] >> (main:null) Found component: com.cloud.alert.AlertAdapter in >> com.cloud.alert.ConsoleProxyAlertAdapter - ConsoleProxyAlert >> 2013-04-01 21:35:47,518 INFO [utils.component.ComponentLocator] >> (main:null) Found component: com.cloud.vm.dao.ConsoleProxyDao in >> com.cloud.vm.dao.ConsoleProxyDaoImpl - ConsoleProxyDao >> 2013-04-01 21:35:47,697 INFO [utils.component.ComponentLocator] >> (main:null) Found component: com.cloud.consoleproxy.ConsoleProxyManager >>in >> com.cloud.consoleproxy.ConsoleProxyManagerImpl - ConsoleProxyManager >> 2013-04-01 21:35:47,697 INFO [utils.component.ComponentLocator] >> (main:null) Found component: com.cloud.consoleproxy.ConsoleProxyService >>in >> com.cloud.consoleproxy.ConsoleProxyManagerImpl - ConsoleProxyManager >> 2013-04-01 21:35:50,355 INFO [utils.component.ComponentLocator] >> (main:null) Starting singleton DAO: ConsoleProxyDao >> 2013-04-01 21:35:52,284 INFO [utils.component.ComponentLocator] >> (main:null) Instantiating Adapter: ConsoleProxyAlert >> 2013-04-01 21:35:52,682 INFO [utils.component.ComponentLocator] >> (main:null) Instantiating Manager: ConsoleProxyManager >> 2013-04-01 21:35:52,937 DEBUG [utils.component.ComponentLocator] >> (main:null) Injecting singleton Manager: ConsoleProxyManager >> 2013-04-01 21:35:53,364 INFO [utils.component.ComponentLocator] >> (main:null) Configuring singleton Manager: ConsoleProxyManager >> 2013-04-01 21:35:53,364 INFO >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (main:null) Start configuring console proxy manager : >>ConsoleProxyManager >> 2013-04-01 21:35:53,364 INFO >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (main:null) Console proxy max session soft limit : 50 >> 2013-04-01 21:35:53,364 INFO >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (main:null) Console proxy standby capacity : 10 >> 2013-04-01 21:35:53,380 DEBUG [agent.manager.AgentManagerImpl] >>(main:null) >> Registering listener ConsoleProxyListener with id 12 >> 2013-04-01 21:35:53,383 INFO >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (main:null) Console Proxy Manager is configured. >> 2013-04-01 21:35:53,511 INFO [utils.component.ComponentLocator] >> (main:null) Injecting singleton Adapter: ConsoleProxyAlert >> 2013-04-01 21:35:53,511 INFO [utils.component.ComponentLocator] >> (main:null) Configuring singleton Adapter: ConsoleProxyAlert >> 2013-04-01 21:35:53,511 INFO [cloud.alert.ConsoleProxyAlertAdapter] >> (main:null) Start configuring console proxy alert manager : >> ConsoleProxyAlert >> 2013-04-01 21:35:53,838 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (main:null) Schedule vm for HA: VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:35:53,880 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (main:null) Schedule vm for HA: VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:35:53,886 INFO [utils.component.ComponentLocator] >> (main:null) Starting singleton Manager: ConsoleProxyManager >> 2013-04-01 21:35:53,886 INFO >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (main:null) Start console proxy manager >> 2013-04-01 21:35:53,886 INFO [utils.component.ComponentLocator] >> (main:null) Started Manager: ConsoleProxyManager >> 2013-04-01 21:35:54,004 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-1:work-38) HA on VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:35:54,014 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-0:work-37) HA on VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:35:54,030 INFO [utils.component.ComponentLocator] >> (main:null) Starting singleton Adapter: ConsoleProxyAlert >> 2013-04-01 21:35:54,065 DEBUG [cloud.ha.CheckOnAgentInvestigator] >> (HA-Worker-1:work-38) Unable to reach the agent for >> VM[ConsoleProxy|v-2-VM]: Resource [Host:1] is unreachable: Host 1: Host >> with specified id is not in the right state: Disconnected >> 2013-04-01 21:35:54,066 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-1:work-38) SimpleInvestigator found VM[ConsoleProxy|v-2-VM]to >> be alive? null >> 2013-04-01 21:35:54,066 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-1:work-38) VmwareInvestigator found VM[ConsoleProxy|v-2-VM]to >> be alive? null >> 2013-04-01 21:35:54,075 DEBUG [cloud.ha.CheckOnAgentInvestigator] >> (HA-Worker-0:work-37) Unable to reach the agent for >> VM[ConsoleProxy|v-2-VM]: Resource [Host:1] is unreachable: Host 1: Host >> with specified id is not in the right state: Disconnected >> 2013-04-01 21:35:54,079 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-0:work-37) SimpleInvestigator found VM[ConsoleProxy|v-2-VM]to >> be alive? null >> 2013-04-01 21:35:54,079 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-0:work-37) VmwareInvestigator found VM[ConsoleProxy|v-2-VM]to >> be alive? null >> 2013-04-01 21:35:54,084 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-0:work-37) XenServerInvestigator found >> VM[ConsoleProxy|v-2-VM]to be alive? null >> 2013-04-01 21:35:54,084 DEBUG [cloud.ha.UserVmDomRInvestigator] >> (HA-Worker-0:work-37) Not a User Vm, unable to determine state of >> VM[ConsoleProxy|v-2-VM] returning null >> 2013-04-01 21:35:54,084 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-0:work-37) PingInvestigator found VM[ConsoleProxy|v-2-VM]to >>be >> alive? null >> 2013-04-01 21:35:54,084 DEBUG >>[cloud.ha.ManagementIPSystemVMInvestigator] >> (HA-Worker-0:work-37) Testing if VM[ConsoleProxy|v-2-VM] is alive >> 2013-04-01 21:35:54,090 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-1:work-38) XenServerInvestigator found >> VM[ConsoleProxy|v-2-VM]to be alive? null >> 2013-04-01 21:35:54,090 DEBUG [cloud.ha.UserVmDomRInvestigator] >> (HA-Worker-1:work-38) Not a User Vm, unable to determine state of >> VM[ConsoleProxy|v-2-VM] returning null >> 2013-04-01 21:35:54,094 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-1:work-38) PingInvestigator found VM[ConsoleProxy|v-2-VM]to >>be >> alive? null >> 2013-04-01 21:35:54,094 DEBUG >>[cloud.ha.ManagementIPSystemVMInvestigator] >> (HA-Worker-1:work-38) Testing if VM[ConsoleProxy|v-2-VM] is alive >> 2013-04-01 21:35:54,123 DEBUG >>[cloud.ha.ManagementIPSystemVMInvestigator] >> (HA-Worker-1:work-38) unable to determine state of >>VM[ConsoleProxy|v-2-VM] >> returning null >> 2013-04-01 21:35:54,123 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-1:work-38) ManagementIPSysVMInvestigator found >> VM[ConsoleProxy|v-2-VM]to be alive? null >> 2013-04-01 21:35:54,124 DEBUG >>[cloud.ha.ManagementIPSystemVMInvestigator] >> (HA-Worker-0:work-37) unable to determine state of >>VM[ConsoleProxy|v-2-VM] >> returning null >> 2013-04-01 21:35:54,124 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-0:work-37) ManagementIPSysVMInvestigator found >> VM[ConsoleProxy|v-2-VM]to be alive? null >> 2013-04-01 21:35:54,127 DEBUG [cloud.ha.XenServerFencer] >> (HA-Worker-1:work-38) Unable to fence off VM[ConsoleProxy|v-2-VM] on >> Host[-1-Routing] >> 2013-04-01 21:35:54,127 DEBUG [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-1:work-38) We were unable to fence off the VM >> VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:35:54,150 DEBUG [cloud.ha.XenServerFencer] >> (HA-Worker-0:work-37) Unable to fence off VM[ConsoleProxy|v-2-VM] on >> Host[-1-Routing] >> 2013-04-01 21:35:54,163 DEBUG [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-0:work-37) We were unable to fence off the VM >> VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:35:57,208 DEBUG [cloud.resource.ResourceManagerImpl] >> (AgentConnectTaskPool-1:null) Dispatching resource state event >> CREATE_HOST_VO_FOR_CONNECTED to >> ConsoleProxyManagerImpl$$EnhancerByCGLIB$$bdec6fa >> 2013-04-01 21:36:06,270 DEBUG [agent.manager.AgentManagerImpl] >> (AgentConnectTaskPool-1:null) Sending Connect to listener: >> ConsoleProxyListener >> 2013-04-01 21:36:08,058 DEBUG [cloud.resource.ResourceManagerImpl] >> (AgentTaskPool-1:null) Dispatching resource state event >> CREATE_HOST_VO_FOR_DIRECT_CONNECT to >> ConsoleProxyManagerImpl$$EnhancerByCGLIB$$bdec6fa >> 2013-04-01 21:36:10,577 DEBUG [cloud.vm.VirtualMachineManagerImpl] >> (AgentTaskPool-1:null) Found an outstanding work item for this vm >> VM[ConsoleProxy|v-2-VM] in state:Starting, work >> id:d1de55ad-2517-466a-9b23-9d3caee2c9e9 >> 2013-04-01 21:36:10,625 DEBUG [cloud.ha.HighAvailabilityManagerImpl] >> (AgentTaskPool-1:null) VM does not require investigation so I'm marking >>it >> as Stopped: VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:36:31,738 DEBUG [cloud.vm.VirtualMachineManagerImpl] >> (AgentTaskPool-1:null) VM[ConsoleProxy|v-2-VM] is stopped on the host. >> Proceeding to release resource held. >> 2013-04-01 21:36:31,809 DEBUG [cloud.vm.VirtualMachineManagerImpl] >> (AgentTaskPool-1:null) Successfully released network resources for the >>vm >> VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:36:31,809 DEBUG [cloud.vm.VirtualMachineManagerImpl] >> (AgentTaskPool-1:null) Successfully released storage resources for the >>vm >> VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:36:31,855 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (AgentTaskPool-1:null) Schedule vm for HA: VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:36:31,881 INFO [cloud.ha.HighAvailabilityManagerImpl] >> (HA-Worker-1:work-39) HA on VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:36:32,613 DEBUG [agent.manager.AgentManagerImpl] >> (AgentTaskPool-1:null) Sending Connect to listener: ConsoleProxyListener >> 2013-04-01 21:36:53,413 DEBUG >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (consoleproxy-1:null) Zone 1 is ready to launch console proxy >> 2013-04-01 21:36:53,413 DEBUG >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (consoleproxy-1:null) Expand console proxy standby capacity for zone >>London >> 2013-04-01 21:36:53,418 INFO >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (consoleproxy-1:null) Found a stopped console proxy, bring it up to >> running pool. proxy vm id : 2 >> 2013-04-01 21:36:53,442 DEBUG [cloud.vm.VirtualMachineManagerImpl] >> (consoleproxy-1:null) Successfully transitioned to start state for >> VM[ConsoleProxy|v-2-VM] reservation id = >> e060dfdb-73b7-4b3c-aef3-86e86fb41ea2 >> 2013-04-01 21:36:53,505 DEBUG [cloud.vm.VirtualMachineManagerImpl] >> (consoleproxy-1:null) Deployment found - P0=VM[ConsoleProxy|v-2-VM], >> >>P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Po >>ol(Id))] >> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()] >> 2013-04-01 21:36:53,744 DEBUG [network.guru.PodBasedNetworkGuru] >> (consoleproxy-1:null) Allocated a nic >> NicProfile[7-2-e060dfdb-73b7-4b3c-aef3-86e86fb41ea2-192.168.2.94-null >>for >> VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:36:53,752 DEBUG [cloud.storage.StorageManagerImpl] >> (consoleproxy-1:null) Checking if we need to prepare 1 volumes for >> VM[ConsoleProxy|v-2-VM] >> 2013-04-01 21:36:53,812 DEBUG >>[cloud.consoleproxy.ConsoleProxyManagerImpl] >> (consoleproxy-1:null) Boot Args for VM[ConsoleProxy|v-2-VM]: >> template=domP type=consoleproxy host=192.168.2.201 port=8250 >>name=v-2-VM >> premium=true >> zone=1 pod=1 guid=Proxy.2 proxy_vm=2 disable_rp_filter=true >> eth2ip=192.168.2.125 eth2mask=255.255.255.0 gateway=192.168.2.1 >> eth0ip=169.254.1.78 eth0mask=255.255.0.0 eth1ip=192.168.2.94 >> eth1mask=255.255.255.0 mgmtcidr=192.168.2.0/24 localgw=192.168.2.1 >> internaldns1=192.168.2.1 dns1=8.8.8.8 dns2=8.8.4.4 >> 2013-04-01 21:36:53,854 DEBUG [agent.transport.Request] >> (consoleproxy-1:null) Seq 1-557383690: Sending { Cmd , MgmtId: >> 52232380435, via: 1, Ver: v1, Flags: 100111, >> >>[{"StartCommand":{"vm":{"id":2,"name":"v-2-VM","bootloader":"PyGrub","typ >>e":"ConsoleProxy","cpus":1,"speed":500,"minRam":1073741824,"maxRam":10737 >>41824,"arch":"x86_64","os":"Debian >> GNU/Linux 6(64-bit)","bootArgs":" template=domP type=consoleproxy >> host=192.168.2.201 port=8250 name=v-2-VM premium=true zone=1 pod=1 >> guid=Proxy.2 proxy_vm=2 disable_rp_filter=true eth2ip=192.168.2.125 >> eth2mask=255.255.255.0 gateway=192.168.2.1 eth0ip=169.254.1.78 >> eth0mask=255.255.0.0 eth1ip=192.168.2.94 eth1mask=255.255.255.0 >>mgmtcidr= >> 192.168.2.0/24 localgw=192.168.2.1 internaldns1=192.168.2.1 dns1=8.8.8.8 >> >>dns2=8.8.4.4","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false, >>"vncPassword":"38be171d61aa8901","params":{},"disks":[{"id":2,"name":"ROO >>T-2","mountPoint":"/export/primary","path":"58945947-d40e-4532-b26a-f97f6 >>96715f8","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesy >>stem","storagePoolUuid":"c06d70ac-1552-3d7d-afe7-07d5690d2e09","deviceId" >>:0}],"nics":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"ip":" >>192.168.2.125","netmask":"255.255.255.0","gateway":"192.168.2.1","mac":"0 >>6:3a:44:00:00:2e","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan >>","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://u >>ntagged","isSecurityGroupEnabled":true},{"deviceId":0,"networkRateMbps":- >>1,"defaultNic":false,"ip":"169.254.1.78","netmask":"255.255.0.0","gateway >>":"169.254.0.1","mac":"0e:00:a9:fe:01:4e","broadcastType":"LinkLocal","ty >>pe":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateM >>bps":-1,"defaultNic":false,"ip":"192.168.2.94","netmask":"255.255.255.0", >>"gateway":"192.168.2.1","mac":"06:51:7e:00:00:0f","broadcastType":"Native >>","type":"Management","isSecurityGroupEnabled":false}]},"wait":0}},{"chec >>k.CheckSshCommand":{"ip":"169.254.1.78","port":3922,"interval":6,"retries >>":100,"name":"v-2-VM","wait":0}}] >> } >> 2013-04-01 21:36:53,859 DEBUG [agent.transport.Request] >> (consoleproxy-1:null) Seq 1-557383690: Executing: { Cmd , MgmtId: >> 52232380435, via: 1, Ver: v1, Flags: 100111, >> >>[{"StartCommand":{"vm":{"id":2,"name":"v-2-VM","bootloader":"PyGrub","typ >>e":"ConsoleProxy","cpus":1,"speed":500,"minRam":1073741824,"maxRam":10737 >>41824,"arch":"x86_64","os":"Debian >> GNU/Linux 6(64-bit)","bootArgs":" template=domP type=consoleproxy >> host=192.168.2.201 port=8250 name=v-2-VM premium=true zone=1 pod=1 >> guid=Proxy.2 proxy_vm=2 disable_rp_filter=true eth2ip=192.168.2.125 >> eth2mask=255.255.255.0 gateway=192.168.2.1 eth0ip=169.254.1.78 >> eth0mask=255.255.0.0 eth1ip=192.168.2.94 eth1mask=255.255.255.0 >>mgmtcidr= >> 192.168.2.0/24 localgw=192.168.2.1 internaldns1=192.168.2.1 dns1=8.8.8.8 >> >>dns2=8.8.4.4","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false, >>"vncPassword":"38be171d61aa8901","params":{},"disks":[{"id":2,"name":"ROO >>T-2","mountPoint":"/export/primary","path":"58945947-d40e-4532-b26a-f97f6 >>96715f8","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesy >>stem","storagePoolUuid":"c06d70ac-1552-3d7d-afe7-07d5690d2e09","deviceId" >>:0}],"nics":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"ip":" >>192.168.2.125","netmask":"255.255.255.0","gateway":"192.168.2.1","mac":"0 >>6:3a:44:00:00:2e","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan >>","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://u >>ntagged","isSecurityGroupEnabled":true},{"deviceId":0,"networkRateMbps":- >>1,"defaultNic":false,"ip":"169.254.1.78","netmask":"255.255.0.0","gateway >>":"169.254.0.1","mac":"0e:00:a9:fe:01:4e","broadcastType":"LinkLocal","ty >>pe":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateM >>bps":-1,"defaultNic":false,"ip":"192.168.2.94","netmask":"255.255.255.0", >>"gateway":"192.168.2.1","mac":"06:51:7e:00:00:0f","broadcastType":"Native >>","type":"Management","isSecurityGroupEnabled":false}]},"wait":0}},{"chec >>k.CheckSshCommand":{"ip":"169.254.1.78","port":3922,"interval":6,"retries >>":100,"name":"v-2-VM","wait":0}}] >> } >>
