An update on this.
I finally succeeded in getting my hosts out of alert state by reverting to an
earlier version of the kernel (see other thread). Unfortunately when it came
up I realized I had installed the 4.3 systemvm template and not the 4.4 one so
I just reverted to my 4.1.1 installation and did the upgrade to 4.3 which
appears to work. I am seeing some errors in the log which I’ll post to a
separate thread. I’ll wait to test 4.4 when it is released.
With regards to the db.properties file. I looked into it in more detail and
the problem was that the file had somehow gotten re-ordered in a random fashion
which is why I could not make sense out of the diff during the install. Not
sure how that happened but my concern mentioned below is probably a non-issue.
regards,
-Carlos
On Jun 30, 2014, at 5:11 PM, Carlos Reátegui <create...@gmail.com> wrote:
> I set encryption to none in db.properties and updated the passwords in
> host_details to unencrypted versions so I could make progress.
>
> I don’t know what exactly the problem was but this is probably something that
> needs better testing. I’m pretty sure I had all the encryption stuff correct
> in the db.properties file but could not get it to work.
>
> It would be nice if there was a specialized merging utility for the
> db.properties given the change in the organization of the file. I am
> guessing if the file had not been reorganized it would have been more obvious
> how to merge the 2 and I may have avoided this issue.
>
> Now my hosts come up in an alert state and I’m not sure where to go from
> here. Please note I am not using bridge mode because I wanted to to a 4 nic
> bridge which bridge does not allow (only 2 nics). This was working fine in
> 4.1 so hopefully this is not a requirement for 4.4. I am not using security
> groups which was my understanding is what requires bridge networking:
>
> The error in the log is this:
> 2014-06-30 14:06:50,073 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-1:ctx-35941dc7) Failed to configure brige firewall
> 2014-06-30 14:06:50,073 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-1:ctx-35941dc7) Check host 172.30.45.32 for CSP is installed or
> not and check network mode for bridge
> 2014-06-30 14:06:50,074 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-1:ctx-35941dc7) Seq 2-6232418934327345153: Response Received:
> 2014-06-30 14:06:50,075 DEBUG [c.c.a.t.Request] (DirectAgent-1:ctx-35941dc7)
> Seq 2-6232418934327345153: Processing: { Ans: , MgmtId: 233845174730253,
> via: 2, Ver: v1, Flags: 110, [{"com.cloud.agent.api
> .SetupAnswer":{"_reconnect":true,"result":false,"details":"Failed to
> configure brige firewall","wait":0}}] }
> 2014-06-30 14:06:50,075 DEBUG [c.c.a.t.Request]
> (AgentTaskPool-2:ctx-b360d1bb) Seq 2-6232418934327345153: Received: { Ans: ,
> MgmtId: 233845174730253, via: 2, Ver: v1, Flags: 110, { SetupAnswer } }
> 2014-06-30 14:06:50,076 DEBUG [c.c.a.m.AgentAttache]
> (DirectAgent-1:ctx-35941dc7) Seq 2-6232418934327345153: No more commands found
> 2014-06-30 14:06:50,076 WARN [c.c.h.x.d.XcpServerDiscoverer]
> (AgentTaskPool-2:ctx-b360d1bb) Unable to setup agent 2 due to Failed to
> configure brige firewall
> 2014-06-30 14:06:50,079 INFO [c.c.u.e.CSExceptionErrorCode]
> (AgentTaskPool-2:ctx-b360d1bb) Could not find exception:
> com.cloud.exception.ConnectionException in error code list for exceptions
> 2014-06-30 14:06:50,079 WARN [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Monitor XcpServerDiscoverer says there is an
> error in the connect process for 2 due to Reinitialize agent after se
> tup.
> 2014-06-30 14:06:50,079 INFO [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Host 2 is disconnecting with event
> AgentDisconnected
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) The next status of agent 2is Alert, current
> status is Connecting
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Deregistering link for 2 with state Alert
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Remove Agent : 2
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.DirectAgentAttache]
> (AgentTaskPool-2:ctx-b360d1bb) Processing disconnect 2(srvengxen02)
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.hypervisor.hyperv.discoverer.HypervServerDiscoverer
> 2014-06-30 14:06:50,085 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.network.security.SecurityGroupListener
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.vm.ClusteredVirtualMachineManagerImpl
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.storage.secondary.SecondaryStorageListener
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.deploy.DeploymentPlanningManagerImpl
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.storage.listener.StoragePoolMonitor
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.storage.download.DownloadListener
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.network.SshKeysDistriMonitor
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.network.router.VirtualNetworkApplianceManagerImpl
> 2014-06-30 14:06:50,086 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.consoleproxy.ConsoleProxyListener
> 2014-06-30 14:06:50,091 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.network.SshKeysDistriMonitor
> 2014-06-30 14:06:50,091 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl
> 2014-06-30 14:06:50,091 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.storage.LocalStoragePoolListener
> 2014-06-30 14:06:50,091 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.storage.upload.UploadListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.capacity.StorageCapacityListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.capacity.ComputeCapacityListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.n.NetworkUsageManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Disconnected called on 2 with status Alert
> 2014-06-30 14:06:50,093 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Sending Disconnect to listener:
> com.cloud.agent.manager.AgentManagerImpl$BehindOnPingListener
> 2014-06-30 14:06:50,093 DEBUG [c.c.h.Status] (AgentTaskPool-2:ctx-b360d1bb)
> Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host
> id = 2, name = srvengxen02]
> 2014-06-30 14:06:50,102 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Notifying other nodes of to disconnect
> 2014-06-30 14:06:50,108 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-2:ctx-59808af2) Copying
> /usr/share/cloudstack-management/webapps/client/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xenserver60/../../../../network/domr//router_proxy.sh
> to /opt/cloud/bin on 172.30.45.32 with permission 0755
> 2014-06-30 14:06:50,108 DEBUG [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-2:ctx-59808af2) Unable to create destination path:
> /opt/cloud/bin on 172.30.45.32 but trying anyway
> 2014-06-30 14:06:50,110 WARN [c.c.r.ResourceManagerImpl]
> (AgentTaskPool-2:ctx-b360d1bb) Unable to connect due to
> com.cloud.exception.ConnectionException: Reinitialize agent after setup.
> at
> com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer.processConnect(XcpServerDiscoverer.java:656)
> at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:514)
> at
> com.cloud.agent.manager.AgentManagerImpl.handleDirectConnectAgent(AgentManagerImpl.java:1427)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1765)
> at
> com.cloud.resource.ResourceManagerImpl.createHostAndAgent(ResourceManagerImpl.java:1891)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>
>
>
>
> On Jun 30, 2014, at 1:57 PM, Carlos Reátegui <create...@gmail.com> wrote:
>
>> Making a little progress but still stuck…
>>
>> I realized that when I did the upgrade it had asked me if to keep the old
>> dp.properties or use the new one. The structure of the file seemed
>> different enough and I did not recall using anything but the defaults so I
>> went ahead and told it to use the new one. Seems this was not the right
>> thing to do.
>>
>> I have updated the password/ecryption settings to match the old file but it
>> is still not working. Now I am getting stuck here:
>>
>> 2014-06-30 13:50:32,139 DEBUG [c.c.s.d.VMTemplateDaoImpl] (main:null) Found
>> parameter routing unique name null
>> 2014-06-30 13:50:32,139 DEBUG [c.c.s.d.VMTemplateDaoImpl] (main:null) Use
>> console proxy template : routing
>> 2014-06-30 13:50:32,143 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = HostPodDaoImpl status = STATUS_ALIVE eternal = false
>> overflowToDisk = false maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0
>> memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds =
>> 300 persistence = none diskExpiryThreadIntervalSeconds = 120
>> cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
>> hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound
>> = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,157 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = DedicatedResourceDaoImpl status = STATUS_ALIVE eternal =
>> false overflowToDisk = false maxEntriesLocalHeap = 30 maxEntriesLocalDisk =
>> 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 3600 timeToIdleSeconds
>> = 300 persistence = none diskExpiryThreadIntervalSeconds = 120
>> cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
>> hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound
>> = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,168 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = HypervisorCapabilitiesDaoImpl status = STATUS_ALIVE
>> eternal = false overflowToDisk = false maxEntriesLocalHeap = 100
>> maxEntriesLocalDisk = 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds =
>> 600 timeToIdleSeconds = 300 persistence = none
>> diskExpiryThreadIntervalSeconds = 120 cacheEventListeners:
>> net.sf.ehcache.statistics.LiveCacheStatisticsWrapper hitCount = 0
>> memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound = 0
>> missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,175 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = UserDaoImpl status = STATUS_ALIVE eternal = false
>> overflowToDisk = false maxEntriesLocalHeap = 5000 maxEntriesLocalDisk = 0
>> memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 300 timeToIdleSeconds =
>> 300 persistence = none diskExpiryThreadIntervalSeconds = 120
>> cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
>> hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound
>> = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,180 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = ServiceOfferingDaoImpl status = STATUS_ALIVE eternal =
>> false overflowToDisk = false maxEntriesLocalHeap = 50 maxEntriesLocalDisk =
>> 0 memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds
>> = 300 persistence = none diskExpiryThreadIntervalSeconds = 120
>> cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
>> hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound
>> = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,187 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = DataCenterDaoImpl status = STATUS_ALIVE eternal = false
>> overflowToDisk = false maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0
>> memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds =
>> 300 persistence = none diskExpiryThreadIntervalSeconds = 120
>> cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
>> hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound
>> = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,188 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = Ip Alloc status = STATUS_ALIVE eternal = false
>> overflowToDisk = false maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0
>> memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds =
>> 300 persistence = none diskExpiryThreadIntervalSeconds = 120
>> cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
>> hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound
>> = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,189 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = vnet Alloc status = STATUS_ALIVE eternal = false
>> overflowToDisk = false maxEntriesLocalHeap = 50 maxEntriesLocalDisk = 0
>> memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 600 timeToIdleSeconds =
>> 300 persistence = none diskExpiryThreadIntervalSeconds = 120
>> cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
>> hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound
>> = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,198 INFO [c.c.u.d.GenericDaoBase] (main:null) Cache
>> created: [ name = VlanDaoImpl status = STATUS_ALIVE eternal = false
>> overflowToDisk = false maxEntriesLocalHeap = 30 maxEntriesLocalDisk = 0
>> memoryStoreEvictionPolicy = LRU timeToLiveSeconds = 3600 timeToIdleSeconds =
>> 300 persistence = none diskExpiryThreadIntervalSeconds = 120
>> cacheEventListeners: net.sf.ehcache.statistics.LiveCacheStatisticsWrapper
>> hitCount = 0 memoryStoreHitCount = 0 diskStoreHitCount = 0 missCountNotFound
>> = 0 missCountExpired = 0 maxBytesLocalHeap = 0 overflowToOffHeap = false
>> maxBytesLocalOffHeap = 0 maxBytesLocalDisk = 0 pinned = false ]
>> 2014-06-30 13:50:32,232 DEBUG [c.c.u.c.DBEncryptionUtil] (main:null) Error
>> while decrypting: true
>>
>> The key is still the default password and I have decrypted all the ENC
>> parameters from the db.properties file and they seem ok. What am I missing?
>>
>> thanks,
>> Carlos
>>
>>
>> On Jun 30, 2014, at 1:16 PM, Carlos Reátegui <create...@gmail.com> wrote:
>>
>>> I found the comments in:
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-3990 useful but how do I
>>> find out the database key so that I can set the pw.
>>>
>>> Also in looking at my previous backups for the host_details table it seems
>>> like the password entry changes on a regular basis.
>>>
>>> Is there something the keeps updating the db key and re-ecrypts the host
>>> passwords?
>>>
>>> On Jun 30, 2014, at 1:01 PM, Carlos Reátegui <create...@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I am having problems bringing my system back up. I have not checked the
>>>> credentials of my hosts but the upgraded management server is unable to
>>>> connect to them. Where is the password stored?
>>>>
>>>> thanks.
>>>> Carlos
>>>>
>>>>
>>>> 2014-06-30 12:55:59,277 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>>>> (ClusteredAgentManager Timer:ctx-060c8ace) Loading directly connected host
>>>> 1(srvengxen01)
>>>> 2014-06-30 12:56:04,394 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
>>>> (LBHealthCheck-1:ctx-c6869648) LB HealthCheck Manager is running and
>>>> getting the updates from LB providers and updating service status
>>>> 2014-06-30 12:56:04,428 DEBUG [c.c.n.l.LBHealthCheckManagerImpl]
>>>> (LBHealthCheck-1:ctx-c6869648) LB HealthCheck Manager is running and
>>>> getting the updates from LB providers and updating service status
>>>> 2014-06-30 12:56:06,844 DEBUG [c.c.h.x.r.XenServerConnectionPool]
>>>> (ClusteredAgentManager Timer:ctx-060c8ace) Unable to create master
>>>> connection to host(172.30.45.31) , due to The credentials given by the
>>>> user are incorrect, so access has been denied, and you have not been
>>>> issued a session handle.
>>>> 2014-06-30 12:56:06,848 DEBUG [c.c.h.Status] (ClusteredAgentManager
>>>> Timer:ctx-060c8ace) Transition:[Resource state = Enabled, Agent event =
>>>> AgentDisconnected, Host id = 1, name = srvengxen01]
>>>> 2014-06-30 12:56:06,862 WARN [c.c.a.m.ClusteredAgentManagerImpl]
>>>> (ClusteredAgentManager Timer:ctx-060c8ace) can not load directly
>>>> connected host 1(srvengxen01) due to
>>>> com.cloud.utils.exception.CloudRuntimeException: Unable to create master
>>>> connection to host(172.30.45.31) , due to The credentials given by the
>>>> user are incorrect, so access has been denied, and you have not been
>>>> issued a session handle.
>>>> at
>>>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.getConnect(XenServerConnectionPool.java:168)
>>>> at
>>>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.CheckXenHostInfo(CitrixResourceBase.java:5722)
>>>> at
>>>> com.cloud.hypervisor.xen.resource.CitrixResourceBase.configure(CitrixResourceBase.java:5705)
>>>> at
>>>> com.cloud.resource.DiscovererBase.reloadResource(DiscovererBase.java:157)
>>>> at
>>>> com.cloud.agent.manager.AgentManagerImpl.loadDirectlyConnectedHost(AgentManagerImpl.java:672)
>>>> at
>>>> com.cloud.agent.manager.ClusteredAgentManagerImpl.scanDirectAgentToLoad(ClusteredAgentManagerImpl.java:218)
>>>> at
>>>> com.cloud.agent.manager.ClusteredAgentManagerImpl.runDirectAgentScanTimerTask(ClusteredAgentManagerImpl.java:184)
>>>> at
>>>> com.cloud.agent.manager.ClusteredAgentManagerImpl.access$100(ClusteredAgentManagerImpl.java:98)
>>>> at
>>>> com.cloud.agent.manager.ClusteredAgentManagerImpl$DirectAgentScanTimerTask.runInContext(ClusteredAgentManagerImpl.java:234)
>>>> at
>>>> org.apache.cloudstack.managed.context.ManagedContextTimerTask$1.runInContext(ManagedContextTimerTask.java:30)
>>>> at
>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>> at
>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>> at
>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>> at
>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>> at
>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>> at
>>>> org.apache.cloudstack.managed.context.ManagedContextTimerTask.run(ManagedContextTimerTask.java:27)
>>>> at java.util.TimerThread.mainLoop(Timer.java:555)
>>>> at java.util.TimerThread.run(Timer.java:505)
>>>> Caused by: The credentials given by the user are incorrect, so access has
>>>> been denied, and you have not been issued a session handle.
>>>> at com.xensource.xenapi.Types.checkResponse(Types.java:322)
>>>> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
>>>> at com.xensource.xenapi.Session.loginWithPassword(Session.java:537)
>>>> at
>>>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.loginWithPassword(XenServerConnectionPool.java:321)
>>>> at
>>>> com.cloud.hypervisor.xen.resource.XenServerConnectionPool.getConnect(XenServerConnectionPool.java:154)
>>>> ... 17 more
>>>> 2014-06-30 12:56:06,864 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>>>> (ClusteredAgentManager Timer:ctx-060c8ace) Loading directly connected host
>>>> 2(srvengxen02)
>>>> 2014-06-30 12:56:09,225 DEBUG [c.c.s.StatsCollector]
>>>> (StatsCollector-1:ctx-8458e286) HostStatsCollector is running...
>>>> 2014-06-30 12:56:09,226 DEBUG [c.c.s.StatsCollector]
>>>> (StatsCollector-2:ctx-aa245eed) VmStatsCollector is running...
>>>> 2014-06-30 12:56:09,227 DEBUG [c.c.s.StatsCollector]
>>>> (StatsCollector-3:ctx-19894fa1) StorageCollector is running...
>>>> 2014-06-30 12:56:09,230 DEBUG [c.c.s.StatsCollector]
>>>> (StatsCollector-4:ctx-d66c71fb) AutoScaling Monitor is running...
>>>>
>>>>
>>>>
>>>> On Jun 30, 2014, at 9:54 AM, Carlos Reátegui <create...@gmail.com> wrote:
>>>>
>>>>> Hi Sudha,
>>>>> Thanks for checking in. I was out for the weekend and just getting back
>>>>> to this now.
>>>>>
>>>>> My main question at this point is if it is ok for me to kill the system
>>>>> vms with the xe vm-shutdown command since the script provided by
>>>>> cloudstack does not work with ubuntu.
>>>>>
>>>>> Also it would be great if someone could have a look at my logs to see if
>>>>> they look normal. I am seeing a lot of HA-Worker messages but I do not
>>>>> have an HA deployment (unless this is the thread that keeps the system
>>>>> vas running).
>>>>>
>>>>> thanks,
>>>>> Carlos
>>>>>
>>>>>
>>>>>
>>>>> On Jun 29, 2014, at 11:51 PM, Sudha Ponnaganti
>>>>> <sudha.ponnaga...@citrix.com> wrote:
>>>>>
>>>>>> Hi Carlos,
>>>>>>
>>>>>> Were you able to resolve the following? Was your upgrade successful?
>>>>>>
>>>>>> Thanks
>>>>>> /sudha
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Carlos Reátegui [mailto:create...@gmail.com]
>>>>>> Sent: Friday, June 27, 2014 8:55 PM
>>>>>> To: CloudStack-Users
>>>>>> Cc: dev@cloudstack.apache.org
>>>>>> Subject: 4.4 upgrade issues
>>>>>>
>>>>>> I am trying out the upgrade instructions from
>>>>>> http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.3/rnotes.html#upgrade-from-4-1-x-to-4-3
>>>>>> but going to 4.4 built from source today.
>>>>>>
>>>>>> My setup: XenServer 6.0.2 Hosts, Management Server on Ubuntu 12.04,
>>>>>> Primary and Secondary on NFS, Basic Network, no security groups
>>>>>>
>>>>>> -----
>>>>>> Notes on the docs:
>>>>>>
>>>>>> 8.4 - 8.6: This is only for hosts that use the cloudstack agent. Does
>>>>>> not apply to KVM. In general this whole section does not do a good job
>>>>>> of explaining what is on the MS vs the Hosts.
>>>>>>
>>>>>> 13: This fails on ubuntu because: cloudstack-sysvmadm sources
>>>>>> /etc/rc.d/init.d/functions which does not exist on ubuntu/debian systems.
>>>>>>
>>>>>> 14: Copy vhf-util from where? Also the path
>>>>>> /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver does not
>>>>>> exist on the hosts so I am assuming this is on the MS, however the MS
>>>>>> already has it since it is an upgrade and was put there by the original
>>>>>> install. Or is this a new version that needs to be grabbed from
>>>>>> somewhere?
>>>>>>
>>>>>> Other: earlier versions like 4.1 worked with JDK 1.6 current releases
>>>>>> require 1.7 but the Upgrade doc does not mention that.
>>>>>>
>>>>>> --
>>>>>> Issues:
>>>>>>
>>>>>> Saw the following in catalina.out, not sure if it is an issues:
>>>>>> Jun 27, 2014 5:28:42 PM org.apache.catalina.loader.WebappClassLoader
>>>>>> validateJarFile
>>>>>> INFO:
>>>>>> validateJarFile(/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/servlet-api-2.5-20081211.jar)
>>>>>> - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class:
>>>>>> javax/servlet/Servlet.class Jun 27, 2014 5:28:42 PM
>>>>>> org.apache.catalina.loader.WebappClassLoader validateJarFile
>>>>>> INFO:
>>>>>> validateJarFile(/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/tomcat-embed-core-7.0.30.jar)
>>>>>> - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class:
>>>>>> javax/servlet/Servlet.class
>>>>>>
>>>>>> Since the above script in step 13 did not work is it ok to do "xe
>>>>>> vm-shutdown vm=." on each of the system vms? Will CloudStack notice
>>>>>> they are ton and start new ones?
>>>>>>
>>>>>> Here are my log files (please note I stopped the service prior to
>>>>>> capturing these logs in case you are wondering):
>>>>>> Management server log:
>>>>>> https://www.dropbox.com/s/7xhkutt8e724il1/management-server.log
>>>>>> Catalina log:
>>>>>> https://www.dropbox.com/s/f45ypkbazhkogyj/catalina.2014-06-27.log
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
