[
https://issues.apache.org/jira/browse/CLOUDSTACK-84?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456993#comment-13456993
]
Rohit Yadav commented on CLOUDSTACK-84:
---------------------------------------
The problem is the user/account gets deleted first and when the router is
deleted, it gets PermissionDeniedException (and fails to cleanup the db entry)
which is graciously buried in the logs.
When any router related API is evoked, listRouter for example, the router in
the DB is in stopped state but it does not exist on the host (it fails as it
tries to retrieve the owner information which also does not exist, hence
NullPointException).
Commentary on the process:
A. When user project is deleted, foreach instances;
0. Instance is stopped.
1. hypervisor helper fails to destroy VLAN as the network still has active VIF:
2012-09-17 18:14:20,302 DEBUG
[network.router.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:null) Found 3 routers.
2012-09-17 18:14:22,029 INFO [xen.resource.XenServer56Resource]
(DirectAgent-28:null) Catch com.xensource.xenapi.Types$VifInUse: failed to
destory VLAN eth0 on host 88441d4c-892f-4f05-8ef0-7d446810f3a0 due to Network
has active VIFs
2. resources are released, vm state transitioned to expunging/destroyed.
3. NICs are cleaned, NIC removed.
4. Root disk expunged.
B. Account gets deleted.
C. Network for that account gets deletion request. Finds all routers, sends
command to shutdown. RVM goes into stopped state.
0. Hypervisor fails to destroy VLAN:
2012-09-17 18:14:41,373 INFO [xen.resource.XenServer56Resource]
(DirectAgent-38:null) Catch com.xensource.xenapi.Types$InvalidValue: failed to
destory VLAN eth0 on host 88441d4c-892f-4f05-8ef0-7d446810f3a0 due to The value
given is invalid
2012-09-17 18:14:41,603 INFO [xen.resource.XenServer56Resource]
(DirectAgent-38:null) Catch com.xensource.xenapi.Types$VifInUse: failed to
destory VLAN eth0 on host 88441d4c-892f-4f05-8ef0-7d446810f3a0 due to Network
has active VIFs
1. Tries to release resource for the router. Network shuts down successfully.
2. Attempts to destroy the router and fails since CS has already deleted the
account information and hence the command from a deleted account/user to delete
the VirtualRouter is denied and the exception is thrown:
2012-09-17 18:14:41,801 WARN [cloud.network.NetworkManagerImpl]
(Job-Executor-1:job-27) Unable to complete destroy of the network due to
element: VirtualRouter
com.cloud.exception.PermissionDeniedException: Acct[3-user1] does not have
permission to operate with resource VM[DomainRouter|r-4-TEST]
> Getting Null Pointer Excpetion while executing listRouters command after
> deleting a user project.
> -------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-84
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-84
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: Management sever : Rhel 6.3
> Setup : Advanced
> Host : XenServer 6.0.2
> Build Details :-
> Git Revision: 03df2fa9dd45c938f72cd1866044b09d1b0cc978
> Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git
> Reporter: Abhinav Roy
> Assignee: Rohit Yadav
> Fix For: pre-4.0.0
>
> Attachments: api-server.log, Cloudstack-84.jpg, management-server.log
>
>
> Steps to reproduce :
> -------------------------
> 1. Deploy a CS advanced networking setup.
> 2. create a sub-domain 'Domain1' under 'ROOT' domain.
> 3. Create 2 users - user1 and user2 under 'Domain1'.
> 4. login as user1 and create a project. Add user2 to the project.
> 5. Create 2 VMs , one as user1 and the other as user2.
> 6. Login as user1(project admin) and delete the project.
> Expected Behaviour :
> ------------------------------
> The project deletion along with deletion of all the resources associated with
> the project should be successful and no error should be seen.
> Observed Behaviour :
> ------------------------------
> 1. Got this Exception during project deletion :
> 2012-09-12 17:22:30,545 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-26:job-40)
> Attempting to destroy router 82012-09-12 17:22:30,547 WARN
> [cloud.network.NetworkManagerImpl] (Job-Executor-26:job-40) Unable to
> complete destroy of the network due to element:
> VirtualRoutercom.cloud.exception.PermissionDeniedException: Acct[6-user4]
> does not have permission to operate with resource VM[DomainRouter|r-8-VM]
> at com.cloud.acl.DomainChecker.checkAccess(DomainChecker.java:128)
> at com.cloud.user.AccountManagerImpl.checkAccess(AccountManagerImpl.java:353)
> at
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.destroyRouter(VirtualNetworkApplianceManagerImpl.java:381)
> at
> com.cloud.network.element.VirtualRouterElement.destroy(VirtualRouterElement.java:641)
> at
> com.cloud.network.NetworkManagerImpl.destroyNetwork(NetworkManagerImpl.java:3554)
> at
> com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at
> com.cloud.user.AccountManagerImpl.cleanupAccount(AccountManagerImpl.java:587)
> at
> com.cloud.user.AccountManagerImpl.deleteAccount(AccountManagerImpl.java:475)
> at
> com.cloud.projects.ProjectManagerImpl.cleanupProject(ProjectManagerImpl.java:305)
> at
> com.cloud.projects.ProjectManagerImpl.deleteProject(ProjectManagerImpl.java:286)
> at
> com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at
> com.cloud.projects.ProjectManagerImpl.deleteProject(ProjectManagerImpl.java:265)
> at
> com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:36)
> at
> com.cloud.api.commands.DeleteProjectCmd.execute(DeleteProjectCmd.java:69)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138) at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:449)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)2012-09-12 17:22:30,548 WARN
> [cloud.user.AccountManagerImpl] (Job-Executor-26:job-40) Unable to destroy
> network Ntwk[206|Guest|8] as a part of account id=8 cleanup.2012-09-12
> 17:22:30,548 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-26:job-40)
> Deleting vpcs for account 8
> 2012-09-12 17:22:30,550 DEBUG [cloud.user.AccountManagerImpl]
> (Job-Executor-26:job-40) Deleting site-to-site VPN customer gateways for
> account 8
> 2012-09-12 17:22:30,551 INFO [cloud.user.AccountManagerImpl]
> (Job-Executor-26:job-40) Cleanup for account 8 is needed.
> 2. After the deletion was done, when I tried to list the virtual routers, i
> got the following Null pointer exception
> Exception while executing ListRoutersCmd:
> java.lang.NullPointerException
> at
> com.cloud.api.ApiResponseHelper.populateOwner(ApiResponseHelper.java:3362)
> at
> com.cloud.api.ApiResponseHelper.createDomainRouterResponse(ApiResponseHelper.java:1771)
> at
> com.cloud.api.commands.ListRoutersCmd.execute(ListRoutersCmd.java:136)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:543)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:422)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:304)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:63)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 3. In UI also we are not able to see the routers on the virtual routers page.
> 4. The issue persists even after restarting Management Server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
