[
https://issues.apache.org/jira/browse/CLOUDSTACK-209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhinav Roy reopened CLOUDSTACK-209:
------------------------------------
Re-opening the issue due to following observations made during upgrade done
using the latest build
http://jenkins.cloudstack.org/job/build-4.0-rhel63/lastSuccessfulBuild/artifact/CloudStack-oss-4.0.0-356.tar.bz2
===========================================
1. After the upgrade we have
-----------------------------------------
Installed:
cloud-scripts.x86_64 0:4.0.0-0.356.el6.4.0
Replaced/Removed:
cloud-agent-scripts.x86_64 0:3.0.2-1.el6
Now, due to this we don't have all the scripts in the
/usr/lib64/cloud/common/scripts folder :
[root@burnank CloudStack-oss-4.0.0-356]# ls /usr/lib64/cloud/common/scripts/vm/
hypervisor
So, to get all the scripts we try to install cloud-scripts package again, but
since it is already installed, it can't be installed again :
[root@burnank CloudStack-oss-4.0.0-356]# yum install
cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: ftp.iitm.ac.in
* extras: mirrors.sin3.sg.voxel.net
* updates: ftp.iitm.ac.in
Setting up Install Process
Examining cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm:
cloud-scripts-4.0.0-0.356.el6.4.0.x86_64
cloud-scripts-4.0.0-0.356.el6.4.0.x86_64.rpm: does not update installed package.
Error: Nothing to do.
So, we go to step 2.
2. Uninstall cloud-scripts and then install again
------------------------------------------------
Now, uninstalling cloud-scripts uninstalls 4 other packages as dependencies
:
Removed:
cloud-scripts.x86_64 0:4.0.0-0.356.el6.4.0
Dependency Removed:
cloud-client.x86_64 0:4.0.0-0.356.el6.4.0 cloud-client-ui.x86_64
0:4.0.0-0.356.el6.4.0 cloud-server.x86_64 0:4.0.0-0.356.el6.4.0
cloud-setup.x86_64 0:4.0.0-0.356.el6.4.0
Install cloud-scripts and 4 other packages which got removed
Now , we can see all the scripts present
[root@burnank CloudStack-oss-4.0.0-356]# ls
/usr/lib64/cloud/common/scripts/vm/systemvm/
id_rsa.cloud injectkeys.sh
3. We do the same on our KVM host.
------------------------------------------------
Here, while uninstalling cloud-scripts , cloud-agent also gets removed so,
we install both of them again.
4. Now, the upgrade process is over, so we start the cloud-agent and
cloud-management services.
5. The upgrade goes fine, there are no errors.
Post Upgrade issues :
=================================
1. The host is in disconnected state, it is not being recognied , reconnecting
gives the following exception :
2012-10-03 15:41:27,118 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-4:job-13) Executing com.cloud.api.commands.ReconnectHostCmd for
job-13
2012-10-03 15:41:27,125 INFO [agent.manager.AgentManagerImpl]
(Job-Executor-4:job-13) Unable to disconnect host because it is not in the
correct state: host=1; Status=Disconnected
2012-10-03 15:41:27,126 WARN [api.commands.ReconnectHostCmd]
(Job-Executor-4:job-13) Exception:
com.cloud.api.ServerApiException
at
com.cloud.api.commands.ReconnectHostCmd.execute(ReconnectHostCmd.java:108)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138) at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-10-03 15:41:27,127 WARN [cloud.api.ApiDispatcher] (Job-Executor-4:job-13)
class com.cloud.api.ServerApiException : null
2012-10-03 15:41:27,127 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-4:job-13) Complete async job-13, jobStatus: 2, resultCode: 530,
result: Error Code: 534 Error text: null
2012-10-03 15:41:32,180 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-24:null) Async job-13 completed
2. The VMs can not be instantiated as the host is not in UP state. To reconnect
the host I tried
[root@rajesh-kvm3 ~]# cloud-setup-agent
Welcome to the CloudStack Agent Setup:
Please input the Management Server Hostname/IP-Address:[localhost]10.102.125.218
Please input the Zone Id:[default]
Please input the Pod Id:[default]
Please input the Cluster Id:[default]
Please choose which network used to create VM:[cloudbr0]
Starting to configure your system:
Configure Cgroup ... [OK]
Configure SElinux ... [OK]
Configure Network ... [OK]
Configure Libvirt ... [OK]
Configure Firewall ... [OK]
Configure Nfs ... [OK]
Configure cloudAgent ... [OK]
CloudStack Agent setup is done!
But this didn't reconnect the host, it added the same host as the new one while
the same host was also present in disconnected state.
But again the VMs were not getting deployed as the host was not recognising the
secondary storage vm etc.
---------------------------------------------------------------------------------------------------------
So because of the above mentioned issues we need to first of all get our
packaging right, the install/uninstall of the packages should happen in such a
way that the user doesn't need to install/remove any package manually. Just
using the upgrade utility in the install.sh script should be enough, both in
the case of management server and the KVM host.
> Upgrade from CS-3.0.2 to ASF 4.0 fails with
> com.cloud.utils.exception.CloudRuntimeException: Unable to find key inject
> script scripts/vm/systemvm/injectkeys.sh
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-209
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-209
> Project: CloudStack
> Issue Type: Bug
> Components: Install and Setup, Management Server
> Affects Versions: pre-4.0.0
> Environment: MS : Rhel 6.2
> HOST : KVM ( Rhel 6.2)
> BUILDS :
> CS-3.0.2 - CloudStack-3.0.2-1-rhel6.2.tar.gz
> ASF 4.0 - CloudStack-oss-4.0.0-187.tar.bz2
> Reporter: Abhinav Roy
> Assignee: Abhinav Roy
> Priority: Blocker
> Fix For: pre-4.0.0
>
> Attachments: agent.log, api-server.log, management-server.log,
> SQL_09_26.dmp
>
>
> Steps :
> ==============================
> 1. Deploy an advanced zone setup with CS-3.0.2 and KVM host (Rhel 6.2)
> 2. Create a VM instance.
> 3. Stop Management server.
> 4. Upgrade to ASF 4.0
> 5. Stop agent services on the host.
> 6. Upgrade to ASF 4.0
> 7. Start agent on the host.
> 8. Start management server services.
> Expected Behaviour :
> ==============================
> The upgrade should happen smoothly without any error.
> Observed Behaviour :
> ==============================
> 1. After executing above steps, following exception is seen in the logs
> 2012-09-26 17:44:41,571 DEBUG [utils.script.Script] (main:null) Looking
> for scripts/vm/systemvm/injectkeys.sh in
> /var/lib/cloud/management/./scripts/vm/systemvm/injectkeys.sh
> 2012-09-26 17:44:41,571 DEBUG [utils.script.Script] (main:null) Looking for
> scripts/vm/systemvm/injectkeys.sh in
> /var/lib/cloud/management/./scripts/vm/systemvm/injectkeys.sh
> 2012-09-26 17:44:41,571 WARN [utils.script.Script] (main:null) Unable to
> find script scripts/vm/systemvm/injectkeys.sh
> 2012-09-26 17:44:41,574 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in the classpath
> 2012-09-26 17:44:41,574 DEBUG [utils.script.Script] (main:null) System
> resource: null
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Current
> binaries reside at /usr/share/java
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/share/java/vms/systemvm.iso
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/share/java/vms/systemvm.iso
> 2012-09-26 17:44:41,575 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/share/vms/systemvm.iso
> 2012-09-26 17:44:41,576 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/vms/systemvm.iso
> 2012-09-26 17:44:41,576 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /vms/systemvm.iso
> 2012-09-26 17:44:41,576 DEBUG [utils.script.Script] (main:null) Searching in
> environment.properties
> 2012-09-26 17:44:41,576 DEBUG [utils.script.Script] (main:null)
> environment.properties says scripts should be in /usr/lib64/cloud/common
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/lib64/cloud/common/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/lib64/cloud/common/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/lib64/cloud/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/lib64/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /usr/vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /vms/systemvm.iso
> 2012-09-26 17:44:41,578 DEBUG [utils.script.Script] (main:null) Searching in
> the current directory
> 2012-09-26 17:44:41,579 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /var/lib/cloud/management/./vms/systemvm.iso
> 2012-09-26 17:44:41,579 DEBUG [utils.script.Script] (main:null) Looking for
> vms/systemvm.iso in /var/lib/cloud/management/./vms/systemvm.iso
> 2012-09-26 17:44:41,579 WARN [utils.script.Script] (main:null) Unable to
> find script vms/systemvm.iso
> 2012-09-26 17:44:41,582 ERROR [cloud.servlet.CloudStartupServlet] (main:null)
> Exception starting management server
> com.cloud.utils.exception.CloudRuntimeException: Unable to find key inject
> script scripts/vm/systemvm/injectkeys.sh
> at
> com.cloud.server.ConfigurationServerImpl.updateKeyPairs(ConfigurationServerImpl.java:675)
> at
> com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at
> com.cloud.server.ConfigurationServerImpl.persistDefaultValues(ConfigurationServerImpl.java:265)
> at
> com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at
> com.cloud.servlet.CloudStartupServlet.init(CloudStartupServlet.java:47)
> at javax.servlet.GenericServlet.init(GenericServlet.java:212)
> at
> org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
> at
> org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:993)
> at
> org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4187)
> at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4496)
> at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
> at
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
> at
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
> at
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1041)
> at
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:964)
> at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
> at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
> at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
> at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
> at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
> at
> org.apache.catalina.core.StandardService.start(StandardService.java:516)
> at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:593)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> 2. This exception I feel is because of the change in script paths recently
> and is independent of the host hypervisor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
