[
https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457495#comment-13457495
]
edison su commented on CLOUDSTACK-127:
--------------------------------------
On one of our demo machines, yas the same issue. The root cause is the file
system of router vm is in read-only state. Destroy/recreate router solves the
problem.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0
> fails with "ResourceUnavailableException: Resource [DataCenter:1] is
> unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision:
> 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL:
> https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the
> VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl]
> (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18)
> Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId:
> 160940036302157, via: 1, Ver: v1, Flags: 100001,
> [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}]
> }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId:
> 160940036302157, via: 1, Ver: v1, Flags: 100001,
> [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}]
> }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase]
> (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null)
> Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver:
> v1, Flags: 0,
> [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}]
> }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId:
> 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl]
> (Job-Executor-7:job-18) Unable to apply the load balancer config because
> resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is
> unreachable: Unable to apply ip association on router
> at
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at
> com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at
> com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at
> com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at
> com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at
> com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at
> com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at
> com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at
> com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at
> com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher]
> (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified
> loadbalancerruleid for vms [6]
> at
> com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at
> com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at
> com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530,
> result: Error Code: 530 Error text: Failed to add specified
> loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl]
> (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
