[
https://issues.apache.org/jira/browse/CLOUDSTACK-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sangeetha Hariharan updated CLOUDSTACK-1007:
--------------------------------------------
Description:
Steps to reproduce the problem:
Set up - Advanced zone with KVM host.
Create a Zone wide Shared network.
Deploy Vm using this network.
Destroy this Vm.
Wait for the Vm to get Expunged.
Delete the network.
Network deletion fails because we are not able to destroy the router.
Management server logs:
2013-01-17 18:05:31,818 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-6:job-34) Sending destroy to
com.cloud.network.element.VirtualRouterElement$$EnhancerByCGLIB$$d659f5e7@447d4275
2013-01-17 18:05:31,820 DEBUG
[network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-6:job-34)
Attempting to destroy router 10
2013-01-17 18:05:31,840 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-34) VM state transitted from :Running to Stopping with
event: StopRequestedvm's original host id: 1 new host id: 1 host id before
state transition: 1
2013-01-17 18:05:31,850 DEBUG [agent.transport.Request] (Job-Executor-6:job-34)
Seq 1-772669469: Sending { Cmd , MgmtId: 206915885081428, via: 1, Ver: v1,
Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"r-10-VM","wait":0}}] }
2013-01-17 18:05:33,284 DEBUG [cloud.server.StatsCollector]
(StatsCollector-2:null) HostStatsCollector is running...
2013-01-17 18:05:33,883 DEBUG [agent.transport.Request] (StatsCollector-2:null)
Seq 1-772669470: Received: { Ans: , MgmtId: 206915885081428, via: 1, Ver: v1,
Flags: 10, { GetHostStatsAnswer } }
2013-01-17 18:05:36,715 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null)
===START=== 10.217.252.59 -- GET
command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464170203
2013-01-17 18:05:36,746 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null)
===END=== 10.217.252.59 -- GET
command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464170203
2013-01-17 18:05:38,232 DEBUG [agent.transport.Request]
(AgentManager-Handler-15:null) Seq 1-772669469: Processing: { Ans: , MgmtId:
206915885081428, via: 1, Ver: v1, Flags: 110,
[{"Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStorageAdaptor(KVMStoragePoolManager.java:41)\n\tat
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:66)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupDisk(LibvirtComputingResource.java:3153)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2687)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:968)\n\tat
com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat
com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat
com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
2013-01-17 18:05:38,232 DEBUG [agent.manager.AgentAttache]
(AgentManager-Handler-15:null) Seq 1-772669469: No more commands found
2013-01-17 18:05:38,232 DEBUG [agent.transport.Request] (Job-Executor-6:job-34)
Seq 1-772669469: Received: { Ans: , MgmtId: 206915885081428, via: 1, Ver: v1,
Flags: 110, { Answer } }
2013-01-17 18:05:38,232 WARN [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-34) Unable to stop vm VM[DomainRouter|r-10-VM]
2013-01-17 18:05:38,299 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-34) VM state transitted from :Stopping to Running with
event: OperationFailedvm's original host id: 1 new host id: 1 host id before
state transition: 1
2013-01-17 18:05:38,300 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-34) Unable to stop the VM so we can't expunge it.
2013-01-17 18:05:38,300 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-34) Unable to destroy the vm because it is not in the
correct state: VM[DomainRouter|r-10-VM]
2013-01-17 18:05:38,300 INFO [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-34) Did not expunge VM[DomainRouter|r-10-VM]
2013-01-17 18:05:38,300 WARN [cloud.network.NetworkManagerImpl]
(Job-Executor-6:job-34) Unable to complete destroy of the network: failed to
destroy network element VirtualRouter
2013-01-17 18:05:38,317 WARN [cloud.api.ApiDispatcher] (Job-Executor-6:job-34)
class com.cloud.api.ServerApiException : Failed to delete network
2013-01-17 18:05:38,318 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-6:job-34) Complete async job-34, jobStatus: 2, resultCode: 530,
result: Error Code: 530 Error text: Failed to delete network
2013-01-17 18:05:38,352 DEBUG [cloud.async.SyncQueueManagerImpl]
(Job-Executor-6:job-34) Sync queue (10) is currently empty
2013-01-17 18:05:38,352 WARN [cloud.async.AsyncJobManagerImpl]
(Job-Executor-6:job-34) Unable to unregister active job 34 from JMX monitoring
2013-01-17 18:05:41,714 DEBUG [cloud.api.ApiServlet] (catalina-exec-8:null)
===START=== 10.217.252.59 -- GET
command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464175201
2013-01-17 18:05:41,728 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-8:null) Async job-34 completed
in agent.log:
013-01-17 23:09:38,641 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Request:Seq 1-772669469: { Cmd , MgmtId: 206915885081428, via: 1, Ver: v1,
Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"r-10-VM","wait":0}}] }
2013-01-17 23:09:38,641 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Processing command: com.cloud.agent.api.StopCommand
2013-01-17 23:09:38,703 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Executing:
/usr/lib64/cloud/common/scripts/vm/network/security_group.py
destroy_network_rules_for_vm --vmname r-10-VM --vif vnet12
2013-01-17 23:09:38,857 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Execution is successful.
2013-01-17 23:09:38,858 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) Try to stop the vm at first
2013-01-17 23:09:40,083 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null)
Processing command: com.cloud.agent.api.GetHostStatsCommand
2013-01-17 23:09:40,083 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Executing: /bin/bash -c idle=$(top -b -n 1|grep
Cpu\(s\):|cut -d% -f4|cut -d, -f2);echo $idle
2013-01-17 23:09:40,612 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Execution is successful.
2013-01-17 23:09:40,613 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Executing: /bin/bash -c freeMem=$(free|grep
cache:|awk '{print $4}');echo $freeMem
2013-01-17 23:09:40,627 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Execution is successful.
2013-01-17 23:09:40,627 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Executing: /bin/bash -c free|grep Mem:|awk
'{print $2}'
2013-01-17 23:09:40,644 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Execution is successful.
2013-01-17 23:09:40,645 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Executing: /bin/bash -c cat
/sys/class/net/cloudbr0/statistics/rx_bytes
2013-01-17 23:09:40,656 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Execution is successful.
2013-01-17 23:09:40,657 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Executing: /bin/bash -c cat
/sys/class/net/cloudbr0/statistics/tx_bytes
2013-01-17 23:09:40,668 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-3:null) Execution is successful.
2013-01-17 23:09:45,014 WARN [cloud.agent.Agent] (agentRequest-Handler-2:null)
Caught:
java.lang.NullPointerException
at
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStorageAdaptor(KVMStoragePoolManager.java:41)
at
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:66)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupDisk(LibvirtComputingResource.java:3153)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2687)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:968)
at com.cloud.agent.Agent.processRequest(Agent.java:525)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-01-17 23:09:45,017 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null)
Seq 1-772669469: { Ans: , MgmtId: 206915885081428, via: 1, Ver: v1, Flags:
110, [{"Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStorageAdaptor(KVMStoragePoolManager.java:41)\n\tat
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:66)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupDisk(LibvirtComputingResource.java:3153)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2687)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:968)\n\tat
com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat
com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat
com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
was:
Steps to reproduce the problem:
Set up - Advanced zone with KVM host.
Create a Zone wide Shared network.
Deploy Vm using this network.
Destroy this Vm.
Wait for the Vm to get Expunged.
Delete the network.
Network deletion fails because we are not able to destroy the router.
Management server logs:
2013-01-17 18:05:31,818 DEBUG [cloud.network.NetworkManagerImpl]
(Job-Executor-6:job-34) Sending destroy to
com.cloud.network.element.VirtualRouterElement$$EnhancerByCGLIB$$d659f5e7@447d4275
2013-01-17 18:05:31,820 DEBUG
[network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-6:job-34)
Attempting to destroy router 10
2013-01-17 18:05:31,840 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-34) VM state transitted from :Running to Stopping with
event: StopRequestedvm's original host id: 1 new host id: 1 host id before
state transition: 1
2013-01-17 18:05:31,850 DEBUG [agent.transport.Request] (Job-Executor-6:job-34)
Seq 1-772669469: Sending { Cmd , MgmtId: 206915885081428, via: 1, Ver: v1,
Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"r-10-VM","wait":0}}] }
2013-01-17 18:05:33,284 DEBUG [cloud.server.StatsCollector]
(StatsCollector-2:null) HostStatsCollector is running...
2013-01-17 18:05:33,883 DEBUG [agent.transport.Request] (StatsCollector-2:null)
Seq 1-772669470: Received: { Ans: , MgmtId: 206915885081428, via: 1, Ver: v1,
Flags: 10, { GetHostStatsAnswer } }
2013-01-17 18:05:36,715 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null)
===START=== 10.217.252.59 -- GET
command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464170203
2013-01-17 18:05:36,746 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null)
===END=== 10.217.252.59 -- GET
command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464170203
2013-01-17 18:05:38,232 DEBUG [agent.transport.Request]
(AgentManager-Handler-15:null) Seq 1-772669469: Processing: { Ans: , MgmtId:
206915885081428, via: 1, Ver: v1, Flags: 110,
[{"Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStorageAdaptor(KVMStoragePoolManager.java:41)\n\tat
com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:66)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupDisk(LibvirtComputingResource.java:3153)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2687)\n\tat
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:968)\n\tat
com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat
com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat
com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
2013-01-17 18:05:38,232 DEBUG [agent.manager.AgentAttache]
(AgentManager-Handler-15:null) Seq 1-772669469: No more commands found
2013-01-17 18:05:38,232 DEBUG [agent.transport.Request] (Job-Executor-6:job-34)
Seq 1-772669469: Received: { Ans: , MgmtId: 206915885081428, via: 1, Ver: v1,
Flags: 110, { Answer } }
2013-01-17 18:05:38,232 WARN [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-34) Unable to stop vm VM[DomainRouter|r-10-VM]
2013-01-17 18:05:38,299 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-6:job-34) VM state transitted from :Stopping to Running with
event: OperationFailedvm's original host id: 1 new host id: 1 host id before
state transition: 1
2013-01-17 18:05:38,300 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-34) Unable to stop the VM so we can't expunge it.
2013-01-17 18:05:38,300 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-34) Unable to destroy the vm because it is not in the
correct state: VM[DomainRouter|r-10-VM]
2013-01-17 18:05:38,300 INFO [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-6:job-34) Did not expunge VM[DomainRouter|r-10-VM]
2013-01-17 18:05:38,300 WARN [cloud.network.NetworkManagerImpl]
(Job-Executor-6:job-34) Unable to complete destroy of the network: failed to
destroy network element VirtualRouter
2013-01-17 18:05:38,317 WARN [cloud.api.ApiDispatcher] (Job-Executor-6:job-34)
class com.cloud.api.ServerApiException : Failed to delete network
2013-01-17 18:05:38,318 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-6:job-34) Complete async job-34, jobStatus: 2, resultCode: 530,
result: Error Code: 530 Error text: Failed to delete network
2013-01-17 18:05:38,352 DEBUG [cloud.async.SyncQueueManagerImpl]
(Job-Executor-6:job-34) Sync queue (10) is currently empty
2013-01-17 18:05:38,352 WARN [cloud.async.AsyncJobManagerImpl]
(Job-Executor-6:job-34) Unable to unregister active job 34 from JMX monitoring
2013-01-17 18:05:41,714 DEBUG [cloud.api.ApiServlet] (catalina-exec-8:null)
===START=== 10.217.252.59 -- GET
command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464175201
2013-01-17 18:05:41,728 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-8:null) Async job-34 completed
> Not able to delete Shared network.
> ----------------------------------
>
> Key: CLOUDSTACK-1007
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1007
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.1.0
> Environment: Build from "network-refactor"
> Reporter: Sangeetha Hariharan
> Fix For: 4.1.0
>
> Attachments: agent.log, agent.log.2013-01-17.gz, management-server.zip
>
>
> Steps to reproduce the problem:
> Set up - Advanced zone with KVM host.
> Create a Zone wide Shared network.
> Deploy Vm using this network.
> Destroy this Vm.
> Wait for the Vm to get Expunged.
> Delete the network.
> Network deletion fails because we are not able to destroy the router.
> Management server logs:
> 2013-01-17 18:05:31,818 DEBUG [cloud.network.NetworkManagerImpl]
> (Job-Executor-6:job-34) Sending destroy to
> com.cloud.network.element.VirtualRouterElement$$EnhancerByCGLIB$$d659f5e7@447d4275
> 2013-01-17 18:05:31,820 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-6:job-34)
> Attempting to destroy router 10
> 2013-01-17 18:05:31,840 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-34) VM state transitted from :Running to Stopping with
> event: StopRequestedvm's original host id: 1 new host id: 1 host id before
> state transition: 1
> 2013-01-17 18:05:31,850 DEBUG [agent.transport.Request]
> (Job-Executor-6:job-34) Seq 1-772669469: Sending { Cmd , MgmtId:
> 206915885081428, via: 1, Ver: v1, Flags: 100111,
> [{"StopCommand":{"isProxy":false,"vmName":"r-10-VM","wait":0}}] }
> 2013-01-17 18:05:33,284 DEBUG [cloud.server.StatsCollector]
> (StatsCollector-2:null) HostStatsCollector is running...
> 2013-01-17 18:05:33,883 DEBUG [agent.transport.Request]
> (StatsCollector-2:null) Seq 1-772669470: Received: { Ans: , MgmtId:
> 206915885081428, via: 1, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2013-01-17 18:05:36,715 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null)
> ===START=== 10.217.252.59 -- GET
> command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464170203
> 2013-01-17 18:05:36,746 DEBUG [cloud.api.ApiServlet] (catalina-exec-1:null)
> ===END=== 10.217.252.59 -- GET
> command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464170203
> 2013-01-17 18:05:38,232 DEBUG [agent.transport.Request]
> (AgentManager-Handler-15:null) Seq 1-772669469: Processing: { Ans: , MgmtId:
> 206915885081428, via: 1, Ver: v1, Flags: 110,
> [{"Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat
> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStorageAdaptor(KVMStoragePoolManager.java:41)\n\tat
>
> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:66)\n\tat
>
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupDisk(LibvirtComputingResource.java:3153)\n\tat
>
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2687)\n\tat
>
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:968)\n\tat
> com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat
> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat
> com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
> java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
> 2013-01-17 18:05:38,232 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-15:null) Seq 1-772669469: No more commands found
> 2013-01-17 18:05:38,232 DEBUG [agent.transport.Request]
> (Job-Executor-6:job-34) Seq 1-772669469: Received: { Ans: , MgmtId:
> 206915885081428, via: 1, Ver: v1, Flags: 110, { Answer } }
> 2013-01-17 18:05:38,232 WARN [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-6:job-34) Unable to stop vm VM[DomainRouter|r-10-VM]
> 2013-01-17 18:05:38,299 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-6:job-34) VM state transitted from :Stopping to Running with
> event: OperationFailedvm's original host id: 1 new host id: 1 host id before
> state transition: 1
> 2013-01-17 18:05:38,300 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-6:job-34) Unable to stop the VM so we can't expunge it.
> 2013-01-17 18:05:38,300 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-6:job-34) Unable to destroy the vm because it is not in the
> correct state: VM[DomainRouter|r-10-VM]
> 2013-01-17 18:05:38,300 INFO [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-6:job-34) Did not expunge VM[DomainRouter|r-10-VM]
> 2013-01-17 18:05:38,300 WARN [cloud.network.NetworkManagerImpl]
> (Job-Executor-6:job-34) Unable to complete destroy of the network: failed to
> destroy network element VirtualRouter
> 2013-01-17 18:05:38,317 WARN [cloud.api.ApiDispatcher]
> (Job-Executor-6:job-34) class com.cloud.api.ServerApiException : Failed to
> delete network
> 2013-01-17 18:05:38,318 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-6:job-34) Complete async job-34, jobStatus: 2, resultCode: 530,
> result: Error Code: 530 Error text: Failed to delete network
> 2013-01-17 18:05:38,352 DEBUG [cloud.async.SyncQueueManagerImpl]
> (Job-Executor-6:job-34) Sync queue (10) is currently empty
> 2013-01-17 18:05:38,352 WARN [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-6:job-34) Unable to unregister active job 34 from JMX monitoring
> 2013-01-17 18:05:41,714 DEBUG [cloud.api.ApiServlet] (catalina-exec-8:null)
> ===START=== 10.217.252.59 -- GET
> command=queryAsyncJobResult&jobId=82e583fa-7829-4396-a2f7-4358e2abba80&response=json&sessionkey=hd3IkmVY5KODN5MiyOXg%2FapVTb4%3D&_=1358464175201
> 2013-01-17 18:05:41,728 DEBUG [cloud.async.AsyncJobManagerImpl]
> (catalina-exec-8:null) Async job-34 completed
> in agent.log:
> 013-01-17 23:09:38,641 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Request:Seq 1-772669469: { Cmd , MgmtId:
> 206915885081428, via: 1, Ver: v1, Flags: 100111,
> [{"StopCommand":{"isProxy":false,"vmName":"r-10-VM","wait":0}}] }
> 2013-01-17 23:09:38,641 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Processing command:
> com.cloud.agent.api.StopCommand
> 2013-01-17 23:09:38,703 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-2:null) Executing:
> /usr/lib64/cloud/common/scripts/vm/network/security_group.py
> destroy_network_rules_for_vm --vmname r-10-VM --vif vnet12
> 2013-01-17 23:09:38,857 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-2:null) Execution is successful.
> 2013-01-17 23:09:38,858 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-2:null) Try to stop the vm at first
> 2013-01-17 23:09:40,083 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-3:null) Processing command:
> com.cloud.agent.api.GetHostStatsCommand
> 2013-01-17 23:09:40,083 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Executing: /bin/bash -c idle=$(top -b -n 1|grep
> Cpu\(s\):|cut -d% -f4|cut -d, -f2);echo $idle
> 2013-01-17 23:09:40,612 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Execution is successful.
> 2013-01-17 23:09:40,613 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Executing: /bin/bash -c freeMem=$(free|grep
> cache:|awk '{print $4}');echo $freeMem
> 2013-01-17 23:09:40,627 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Execution is successful.
> 2013-01-17 23:09:40,627 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Executing: /bin/bash -c free|grep Mem:|awk
> '{print $2}'
> 2013-01-17 23:09:40,644 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Execution is successful.
> 2013-01-17 23:09:40,645 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Executing: /bin/bash -c cat
> /sys/class/net/cloudbr0/statistics/rx_bytes
> 2013-01-17 23:09:40,656 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Execution is successful.
> 2013-01-17 23:09:40,657 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Executing: /bin/bash -c cat
> /sys/class/net/cloudbr0/statistics/tx_bytes
> 2013-01-17 23:09:40,668 DEBUG [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-3:null) Execution is successful.
> 2013-01-17 23:09:45,014 WARN [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Caught:
> java.lang.NullPointerException
> at
> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStorageAdaptor(KVMStoragePoolManager.java:41)
> at
> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:66)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupDisk(LibvirtComputingResource.java:3153)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2687)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:968)
> at com.cloud.agent.Agent.processRequest(Agent.java:525)
> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
> at com.cloud.utils.nio.Task.run(Task.java:83)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-01-17 23:09:45,017 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Seq 1-772669469: { Ans: , MgmtId:
> 206915885081428, via: 1, Ver: v1, Flags: 110,
> [{"Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat
> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStorageAdaptor(KVMStoragePoolManager.java:41)\n\tat
>
> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:66)\n\tat
>
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupDisk(LibvirtComputingResource.java:3153)\n\tat
>
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2687)\n\tat
>
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:968)\n\tat
> com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat
> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat
> com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\n\tat
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\n\tat
> java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
