Abhinav Roy created CLOUDSTACK-248:
--------------------------------------
Summary: After upgrading from CS-3.0.2 to ASF 4.0 the KVM Host
ends up in disconnected state.
Key: CLOUDSTACK-248
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-248
Project: CloudStack
Issue Type: Bug
Components: Hypervisor Controller, Install and Setup, KVM
Affects Versions: pre-4.0.0
Environment: MS : Rhel 6.2
HOST : KVM (Rhel 6.2)
Reporter: Abhinav Roy
Priority: Blocker
Fix For: pre-4.0.0
Executed the upgrade from CS 3.0.2 to ASF 4.0 using the build below
http://jenkins.cloudstack.org/job/build-4.0-rhel63/lastSuccessfulBuild/artifact/CloudStack-oss-4.0.0-356.tar.bz2,
following observations were made
===========================================
1. After the upgrade we have
-----------------------------------------
Installed:
cloud-scripts.x86_64 0:4.0.0-0.356.el6.4.0
Replaced/Removed:
cloud-agent-scripts.x86_64 0:3.0.2-1.el6
Now, due to this we don't have all the scripts in the
/usr/lib64/cloud/common/scripts folder :
[root@burnank CloudStack-oss-4.0.0-356]# ls /usr/lib64/cloud/common/scripts/vm/
hypervisor
So, to get all the scripts we try to install cloud-scripts package again, but
since it is already installed, it can't be installed again :
[root@burnank CloudStack-oss-4.0.0-356]# yum install
cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: ftp.iitm.ac.in
* extras: mirrors.sin3.sg.voxel.net
* updates: ftp.iitm.ac.in
Setting up Install Process
Examining cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm:
cloud-scripts-4.0.0-0.356.el6.4.0.x86_64
cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm: does not update installed
package.
Error: Nothing to do.
So, we go to step 2.
2. Uninstall cloud-scripts and then install again
------------------------------------------------
Now, uninstalling cloud-scripts uninstalls 4 other packages as dependencies
:
Removed:
cloud-scripts.x86_64 0:4.0.0-0.356.el6.4.0
Dependency Removed:
cloud-client.x86_64 0:4.0.0-0.356.el6.4.0 cloud-client-ui.x86_64
0:4.0.0-0.356.el6.4.0 cloud-server.x86_64 0:4.0.0-0.356.el6.4.0
cloud-setup.x86_64 0:4.0.0-0.356.el6.4.0
Install cloud-scripts and 4 other packages which got removed
Now , we can see all the scripts present
[root@burnank CloudStack-oss-4.0.0-356]# ls
/usr/lib64/cloud/common/scripts/vm/systemvm/
id_rsa.cloud injectkeys.sh
3. We do the same on our KVM host.
------------------------------------------------
Here, while uninstalling cloud-scripts , cloud-agent also gets removed so,
we install both of them again.
4. Now, the upgrade process is over, so we start the cloud-agent and
cloud-management services.
5. The upgrade goes fine, there are no errors.
Post Upgrade issues :
=================================
1. The host is in disconnected state, it is not being recognied , reconnecting
gives the following exception :
2012-10-03 15:41:27,118 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-4:job-13) Executing com.cloud.api.commands.ReconnectHostCmd for
job-13
2012-10-03 15:41:27,125 INFO [agent.manager.AgentManagerImpl]
(Job-Executor-4:job-13) Unable to disconnect host because it is not in the
correct state: host=1; Status=Disconnected
2012-10-03 15:41:27,126 WARN [api.commands.ReconnectHostCmd]
(Job-Executor-4:job-13) Exception:
com.cloud.api.ServerApiException
at
com.cloud.api.commands.ReconnectHostCmd.execute(ReconnectHostCmd.java:108) at
com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138) at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-10-03 15:41:27,127 WARN [cloud.api.ApiDispatcher] (Job-Executor-4:job-13)
class com.cloud.api.ServerApiException : null
2012-10-03 15:41:27,127 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-4:job-13) Complete async job-13, jobStatus: 2, resultCode: 530,
result: Error Code: 534 Error text: null
2012-10-03 15:41:32,180 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-24:null) Async job-13 completed
2. The VMs can not be instantiated as the host is not in UP state. To reconnect
the host I tried
[root@rajesh-kvm3 ~]# cloud-setup-agent
Welcome to the CloudStack Agent Setup:
Please input the Management Server
Hostname/IP-Address:[localhost]10.102.125.218
Please input the Zone Id:[default]
Please input the Pod Id:[default]
Please input the Cluster Id:[default]
Please choose which network used to create VM:[cloudbr0]
Starting to configure your system:
Configure Cgroup ... [OK]
Configure SElinux ... [OK]
Configure Network ... [OK]
Configure Libvirt ... [OK]
Configure Firewall ... [OK]
Configure Nfs ... [OK]
Configure cloudAgent ... [OK]
CloudStack Agent setup is done!
But this didn't reconnect the host, it added the same host as the new one while
the same host was also present in disconnected state.
But again the VMs were not getting deployed as the host was not recognising the
secondary storage vm etc.
---------------------------------------------------------------------------------------------------------
So because of the above mentioned issues we need to first of all get our
packaging right, the install/uninstall of the packages should happen in such a
way that the user doesn't need to install/remove any package manually. Just
using the upgrade utility in the install.sh script should be enough, both in
the case of management server and the KVM host.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
