Hi,
I am getting the similar log after deleting the Network it says "Unable to
destroy network because of virtual router" & VR is in expunging state.
2014-06-23 13:02:21,589 WARN [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Unable to apply ip association,
virtual router is not in the right state Expunging
2014-06-23 13:02:21,594 DEBUG [c.c.n.IpAddressManagerImpl]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Resource is not available:
VirtualRouter
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1]
is unreachable: Unable to apply ip association, virtual router is not in
the right state
at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3928)
2014-06-23 13:02:21,595 WARN [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Unable to apply ip address
associations for Ntwk[207|Guest|17] as a part of shutdownNetworkRules
2014-06-23 13:02:21,595 WARN [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Failed to cleanup network
id=207 resources as a part of shutdownNetwork
2014-06-23 13:02:21,599 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Sending network shutdown to
VirtualRouter
2014-06-23 13:02:21,601 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Stopping router
VM[DomainRouter|r-27-VM]
2014-06-23 13:02:21,604 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Stopped called on
VM[DomainRouter|r-27-VM] but the state is Expunging
2014-06-23 13:02:21,606 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Network id=207 is shutdown
successfully, cleaning up corresponding resources now.
2014-06-23 13:02:21,654 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Lock is released for network
Ntwk[207|Guest|17] as a part of network shutdown
2014-06-23 13:02:22,045 WARN [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-16:ctx-011048ba ctx-55af37fa) Unable to complete destroy of
the network due to element: VirtualRouter
After this logs I changed VR status in 'vm_instance' made it 'stopped' &
tried to delete the router..but it has thrown exception,
2014-06-23 13:16:51,878 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-18:ctx-b2b5bc61 ctx-82406597) VM state transitted from
:Stopped to Expunging with event: ExpungeOperationvm's original host id:
null new host id: 6 host id before state transition: 6
2014-06-23 13:16:51,878 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-18:ctx-b2b5bc61 ctx-82406597) Destroying vm
VM[DomainRouter|r-27-VM]
2014-06-23 13:16:51,878 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-18:ctx-b2b5bc61 ctx-82406597) Cleaning up NICS
2014-06-23 13:16:51,878 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-18:ctx-b2b5bc61 ctx-82406597) Cleaning network for vm: 27
2014-06-23 13:16:51,881 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-18:ctx-b2b5bc61 ctx-82406597) Cleaning up hypervisor data
structures (ex. SRs in XenServer) for managed storage
2014-06-23 13:16:51,900 ERROR [c.c.a.ApiAsyncJobDispatcher]
(Job-Executor-18:ctx-b2b5bc61) Unexpected exception while executing
org.apache.cloudstack.api.command.admin.router.DestroyRouterCmd
java.lang.NullPointerException
at
org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.disconnectVolumesFromHost(VolumeOrchestrator.java:885)
at
com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:513)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2014-06-23 13:16:51,912 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-18:ctx-b2b5bc61) Complete async job-174, jobStatus: FAILED,
resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530}
2014-06-23 13:16:51,966 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-18:ctx-b2b5bc61) Done executing
org.apache.cloudstack.api.command.admin.router.DestroyRouterCmd for job-174
2014-06-23 13:16:51,989 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Job-Executor-18:ctx-b2b5bc61) Remove job-174 from job monitoring
Regards,
Tejas
On Mon, Jun 23, 2014 at 12:52 PM, Tejas Gadaria <refond.g...@gmail.com>
wrote:
> Hi,
>
> Should I update the state in "networks" table in cloud db ?
>
> Regards,
> Tejas
>
>
> On Mon, Jun 23, 2014 at 12:46 PM, Suresh Sadhu <suresh.sa...@citrix.com>
> wrote:
>
>>
>> HI,
>>
>> It looks your network in stuck in transition state(implementing) as log
>> clearly says it should either implemented or allocate .
>>
>> You can update the network state in db as implemented and perform again
>> network restart with cleanup option.
>>
>> -sadhu
>>
>>
>> -----Original Message-----
>> From: Tejas Gadaria [mailto:refond.g...@gmail.com]
>> Sent: 23 June 2014 12:10
>> To: dev@cloudstack.apache.org
>> Cc: us...@cloudstack.apache.org
>> Subject: Re: Unable to delete virtual router
>>
>> Hi,
>>
>> After restarting the Network it says, "Network is not in the right state
>> to be restarted. Correct states are: Implemented, Setup"
>>
>>
>> 2014-06-23 12:04:29,112 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
>> Timer:ctx-993ca065) Completed acquiring hosts for clusters not owned by
>> any management server
>> 2014-06-23 12:04:29,269 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-15:null) Ping from 3
>> 2014-06-23 12:04:32,755 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-393:ctx-d0977fbe) Ping from 1(swiftproxy-xen1)
>> 2014-06-23 12:04:33,101 DEBUG [c.c.a.ApiServlet]
>> (catalina-exec-11:ctx-dc604cdc) ===START=== 192.168.160.192 -- GET
>>
>> command=restartNetwork&cleanup=true&id=099ff395-9196-43ab-a1e2-cd368dbe3c75&cleanup=true&response=json&sessionkey=ecSt5VOMl2wgsZtnAYbNEfbh8%2Bk%3D&_=1403505273057
>> 2014-06-23 12:04:33,127 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-12:null) SeqA 4-91008: Processing Seq 4-91008: {
>> Cmd , MgmtId: -1, via: 4, Ver: v1, Flags: 11,
>> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":3,"_loadInfo":"{\n
>> \"connections\": []\n}","wait":0}}] }
>> 2014-06-23 12:04:33,291 DEBUG [c.c.a.m.AgentManagerImpl]
>> (AgentManager-Handler-12:null) SeqA 4-91008: Sending Seq 4-91008: { Ans:
>> ,
>> MgmtId: 64342918221735, via: 4, Ver: v1, Flags: 100010,
>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
>> 2014-06-23 12:04:33,404 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (catalina-exec-11:ctx-dc604cdc ctx-6b97cb9e) submit async job-167, details:
>> AsyncJobVO {id:167, userId: 2, accountId: 2, instanceType: None,
>> instanceId: null, cmd:
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd, cmdInfo:
>>
>> {"response":"json","id":"099ff395-9196-43ab-a1e2-cd368dbe3c75","sessionkey":"ecSt5VOMl2wgsZtnAYbNEfbh8+k\u003d","cleanup":"true","cmdEventType":"NETWORK.RESTART","ctxUserId":"2","httpmethod":"GET","_":"1403505273057","ctxAccountId":"2","ctxStartEventId":"605"},
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 64342918221735, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: null}
>> 2014-06-23 12:04:33,405 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> (Job-Executor-11:ctx-587064a4) Add job-167 into job monitoring
>> 2014-06-23 12:04:33,406 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (Job-Executor-11:ctx-587064a4) Executing AsyncJobVO {id:167, userId: 2,
>> accountId: 2, instanceType: None, instanceId: null, cmd:
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd, cmdInfo:
>>
>> {"response":"json","id":"099ff395-9196-43ab-a1e2-cd368dbe3c75","sessionkey":"ecSt5VOMl2wgsZtnAYbNEfbh8+k\u003d","cleanup":"true","cmdEventType":"NETWORK.RESTART","ctxUserId":"2","httpmethod":"GET","_":"1403505273057","ctxAccountId":"2","ctxStartEventId":"605"},
>> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
>> result: null, initMsid: 64342918221735, completeMsid: null, lastUpdated:
>> null, lastPolled: null, created: null}
>> 2014-06-23 12:04:33,407 DEBUG [c.c.a.ApiServlet]
>> (catalina-exec-11:ctx-dc604cdc ctx-6b97cb9e) ===END=== 192.168.160.192 --
>> GET
>>
>> command=restartNetwork&cleanup=true&id=099ff395-9196-43ab-a1e2-cd368dbe3c75&cleanup=true&response=json&sessionkey=ecSt5VOMl2wgsZtnAYbNEfbh8%2Bk%3D&_=1403505273057
>> 2014-06-23 12:04:33,538 ERROR [c.c.a.ApiAsyncJobDispatcher]
>> (Job-Executor-11:ctx-587064a4) Unexpected exception while executing
>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd
>> com.cloud.exception.InvalidParameterValueException: Network is not in the
>> right state to be restarted. Correct states are: Implemented, Setup
>> at
>>
>> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1832)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>>
>> Regards,
>> Tejas
>>
>>
>> On Mon, Jun 23, 2014 at 12:03 PM, Tejas Gadaria <refond.g...@gmail.com>
>> wrote:
>>
>> > I am using CS 4.3.
>> >
>> > Before restarting the Guest network, Network state : Implementing I
>> > will post logs once i will restart the Network.
>> >
>> > Regards,
>> > Tejas
>> >
>> >
>> > On Mon, Jun 23, 2014 at 10:18 AM, Suresh Sadhu
>> > <suresh.sa...@citrix.com>
>> > wrote:
>> >
>> >> Try restart the network with cleanup option .it will recreate the
>> router.
>> >> By the way, Which version of cloudstack you are using?
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Tejas Gadaria [mailto:refond.g...@gmail.com]
>> >> Sent: 21 June 2014 22:45
>> >> To: us...@cloudstack.apache.org; dev@cloudstack.apache.org
>> >> Subject: Re: Unable to delete virtual router
>> >>
>> >> Hi,
>> >>
>> >> I found this bug still open
>> >> https://issues.apache.org/jira/browse/CLOUDSTACK-1007
>> >>
>> >> Is there any way around to create vm in same Network, with same VLAN
>> ID.
>> >> OR Destroy existing Network.
>> >> Right now I am using shared network offering to create guest network.
>> >>
>> >> Regards,
>> >> Tejas
>> >>
>> >>
>> >>
>> >> On Sat, Jun 21, 2014 at 5:32 PM, Tejas Gadaria
>> >> <refond.g...@gmail.com>
>> >> wrote:
>> >>
>> >> > I found the log on management server,
>> >> >
>> >> > 2014-06-21 17:25:28,750 DEBUG [c.c.h.d.HostDaoImpl]
>> >> > (ClusteredAgentManager
>> >> > Timer:ctx-bc6cacf1) Completed acquiring hosts for clusters not
>> >> > owned by any management server
>> >> > 2014-06-21 17:25:31,809 DEBUG [c.c.a.ApiServlet]
>> >> > (catalina-exec-19:ctx-631429c1) ===START=== 192.168.160.192 -- GET
>> >> > command=deleteNetwork&id=099ff395-9196-43ab-a1e2-cd368dbe3c75&respo
>> >> > nse
>> >> > =json&sessionkey=svge6Qrqq9nR4yPm9TYMn1UgvSo%3D&_=1403351731747
>> >> > 2014-06-21 17:25:31,946 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > (catalina-exec-19:ctx-631429c1 ctx-190760ce) submit async job-160,
>> >> details:
>> >> > AsyncJobVO {id:160, userId: 2, accountId: 2, instanceType: None,
>> >> > instanceId: null, cmd:
>> >> > org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd,
>> >> cmdInfo:
>> >> > {"id":"099ff395-9196-43ab-a1e2-cd368dbe3c75","response":"json","ses
>> >> > sio
>> >> > nkey":"svge6Qrqq9nR4yPm9TYMn1UgvSo\u003d","cmdEventType":"NETWORK.D
>> >> > ELE
>> >> > TE","ctxUserId":"2","httpmethod":"GET","_":"1403351731747","ctxAcco
>> >> > unt
>> >> > Id":"2","ctxStartEventId":"562"},
>> >> > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode:
>> >> > 0,
>> >> > result: null, initMsid: 64342918221735, completeMsid: null,
>> lastUpdated:
>> >> > null, lastPolled: null, created: null}
>> >> > 2014-06-21 17:25:31,947 INFO [o.a.c.f.j.i.AsyncJobMonitor]
>> >> > (Job-Executor-7:ctx-a9e7a416) Add job-160 into job monitoring
>> >> > 2014-06-21 17:25:31,948 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > (Job-Executor-7:ctx-a9e7a416) Executing AsyncJobVO {id:160, userId:
>> >> > 2,
>> >> > accountId: 2, instanceType: None, instanceId: null, cmd:
>> >> > org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd,
>> >> cmdInfo:
>> >> > {"id":"099ff395-9196-43ab-a1e2-cd368dbe3c75","response":"json","ses
>> >> > sio
>> >> > nkey":"svge6Qrqq9nR4yPm9TYMn1UgvSo\u003d","cmdEventType":"NETWORK.D
>> >> > ELE
>> >> > TE","ctxUserId":"2","httpmethod":"GET","_":"1403351731747","ctxAcco
>> >> > unt
>> >> > Id":"2","ctxStartEventId":"562"},
>> >> > cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode:
>> >> > 0,
>> >> > result: null, initMsid: 64342918221735, completeMsid: null,
>> lastUpdated:
>> >> > null, lastPolled: null, created: null}
>> >> > 2014-06-21 17:25:31,949 DEBUG [c.c.a.ApiServlet]
>> >> > (catalina-exec-19:ctx-631429c1 ctx-190760ce) ===END===
>> >> > 192.168.160.192 -- GET
>> >> > command=deleteNetwork&id=099ff395-9196-43ab-a1e2-cd368dbe3c75&respo
>> >> > nse
>> >> > =json&sessionkey=svge6Qrqq9nR4yPm9TYMn1UgvSo%3D&_=1403351731747
>> >> > 2014-06-21 17:25:32,041 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> >> > (Job-Executor-7:ctx-a9e7a416 ctx-190760ce) Network is not
>> implemented:
>> >> > Ntwk[207|Guest|17]
>> >> > 2014-06-21 17:25:32,044 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>> >> > (Job-Executor-7:ctx-a9e7a416 ctx-190760ce) Network is not not in
>> >> > the correct state to be destroyed: Implementing
>> >> > 2014-06-21 17:25:32,100 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > (Job-Executor-7:ctx-a9e7a416) Complete async job-160, jobStatus:
>> >> > FAILED,
>> >> > resultCode: 530, result:
>> >> >
>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
>> >> > [],"errorcode":530,"errortext":"Failed
>> >> > to delete network"}
>> >> > 2014-06-21 17:25:32,152 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> >> > (Job-Executor-7:ctx-a9e7a416) Done executing
>> >> > org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd for
>> >> > job-160
>> >> >
>> >> >
>> >> > Regards,
>> >> > Tejas
>> >> >
>> >> >
>> >> >
>> >> > On Sat, Jun 21, 2014 at 5:18 PM, Tejas Gadaria
>> >> > <refond.g...@gmail.com>
>> >> > wrote:
>> >> >
>> >> >> Hi,
>> >> >>
>> >> >> I have created Redundant virtual router.
>> >> >> Right now no vms are running on that, but I am not to delete VR.
>> >> >>
>> >> >>
>> >> >
>> >>
>> >
>> >
>>
>
>
