[ https://issues.apache.org/jira/browse/CLOUDSTACK-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587885#comment-13587885 ]
Venkata Siva Vijayendra Bhamidipati commented on CLOUDSTACK-1228: ----------------------------------------------------------------- Sangeetha just spoke with me regarding this bug. She pointed out that packaging scripts being used here in the QA teams have been changed recently to use /var/cloudstack/ instead of /var/lib/cloud/. On Kiran's server, I do see: [root@KiranRhel63-3 vijay]# ls /var/cloudstack/mnt/VM/ 7614406590488.32f8324 [root@KiranRhel63-3 vijay]# So it looks like something needs to be additionally changed in the QA packaging scripts. Sangeetha has a workaround for this - she will update this bug with the workaround. I'm pasting the stack trace she ran into with ssh keypairs below - but I don't see a similar issue in Krian's server's logs - probably, Kiran missed modifying a part of the scripts. 013-02-26 11:52:53,705 INFO [cloud.server.ConfigurationServerImpl] (Timer-1:null) Stored SSL keystore to database. 2013-02-26 11:52:53,707 INFO [cloud.server.ConfigurationServerImpl] (Timer-1:null) Processing updateKeyPairs 2013-02-26 11:52:53,708 INFO [cloud.server.ConfigurationServerImpl] (Timer-1:null) Systemvm keypairs not found in database. Need to store them in the database 2013-02-26 11:52:53,708 DEBUG [utils.script.Script] (Timer-1:null) Executing: /bin/bash -c if [ -f /var/lib/cloud/management/.ssh/id_rsa ]; then rm -f /var/lib/cloud/management/.ssh/id_rsa; fi; ssh-keygen -t rsa -N '' -f /var/lib/cloud/management/.ssh/id_rsa -q 2013-02-26 11:52:54,335 DEBUG [utils.script.Script] (Timer-1:null) Exit value is 1 2013-02-26 11:52:54,336 DEBUG [utils.script.Script] (Timer-1:null) Could not create directory '/var/lib/cloud/management/.ssh'.open /var/lib/cloud/management/.ssh/id_rsa failed: No such file or directory.Saving the key failed: /var/lib/cloud/management/.ssh/id_rsa. 2013-02-26 11:52:54,337 ERROR [cloud.server.ConfigurationServerImpl] (Timer-1:null) Cannot read the private key file java.io.FileNotFoundException: /var/lib/cloud/management/.ssh/id_rsa (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(FileInputStream.java:137) at com.cloud.server.ConfigurationServerImpl.updateKeyPairs(ConfigurationServerImpl.java:617) at com.cloud.server.ConfigurationServerImpl.persistDefaultValues(ConfigurationServerImpl.java:275) at com.cloud.server.ConfigurationServerImpl.configure(ConfigurationServerImpl.java:144) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80) at com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:37) at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) at $Proxy233.configure(Unknown Source) at com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(ComponentContext.java:110) at com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:50) at java.util.TimerThread.mainLoop(Timer.java:534) at java.util.TimerThread.run(Timer.java:484) > Unable to Create System Vm's in the VMware Hypervisor setup. > ------------------------------------------------------------ > > Key: CLOUDSTACK-1228 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1228 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Install and Setup > Affects Versions: 4.1.0 > Reporter: Kiran Koneti > Assignee: Venkata Siva Vijayendra Bhamidipati > Priority: Blocker > Fix For: 4.1.0 > > Attachments: catalina.2013-02-11.zip, catalina.zip, > management-server-18-2-2013.zip, management-server.zip > > > Created a Advanced Zone setup using the Latest 4.1 build > (http://repo-ccp.citrix.com/releases/ASF/rhel/6.3/4.1/CloudStack-non-OSS-11-rhel6.3.tar.gz) > While trying to create the system VM's error messages are obserevd. > Attaching the Management server logs. > Error message Observed: > Executing copyTemplateFromSecondaryToPrimary. secondaryStorage: > nfs://10.147.28.7/export/home/kiran/s, templatePathAtSecondaryStorage: > template/tmpl/1/8/, templateName: routing-8 > 2013-02-11 23:03:41,530 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.47bfef59 > 2013-02-11 23:03:41,531 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.648a617f > 2013-02-11 23:03:41,531 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.2257f5b0 > 2013-02-11 23:03:41,532 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.3e22e9af > 2013-02-11 23:03:41,533 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.1e46e9bc > 2013-02-11 23:03:41,534 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.49fee4a0 > 2013-02-11 23:03:41,535 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.c1b6a94 > 2013-02-11 23:03:41,535 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.5604ba42 > 2013-02-11 23:03:41,536 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.3632e0de > 2013-02-11 23:03:41,537 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount: > /var/lib/cloud/mnt/VM/7313725325399.22475368 > 2013-02-11 23:03:41,537 WARN [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create a mount point > 2013-02-11 23:03:41,538 ERROR [vmware.manager.VmwareManagerImpl] > (DirectAgent-2:10.147.40.18) Unable to create mount point for > nfs://10.147.28.7/export/home/kiran/s > 2013-02-11 23:03:41,538 INFO [vmware.manager.VmwareStorageManagerImpl] > (DirectAgent-2:10.147.40.18) Secondary storage mount point: /mnt/sec > 2013-02-11 23:03:41,538 INFO [vmware.manager.VmwareStorageManagerImpl] > (DirectAgent-2:10.147.40.18) Executing command: tar --no-same-owner -xf > /mnt/sec/template/tmpl/1/8/routing-8.ova > 2013-02-11 23:03:41,539 DEBUG [vmware.manager.VmwareStorageManagerImpl] > (DirectAgent-2:10.147.40.18) Executing: tar --no-same-owner -xf > /mnt/sec/template/tmpl/1/8/routing-8.ova > 2013-02-11 23:03:41,603 WARN [vmware.manager.VmwareStorageManagerImpl] > (DirectAgent-2:10.147.40.18) Exception: tar --no-same-owner -xf > /mnt/sec/template/tmpl/1/8/routing-8.ova > java.io.IOException: Cannot run program "tar" (in directory > "/mnt/sec/template/tmpl/1/8"): java.io.IOException: error=2, No such file or > directory > at java.lang.ProcessBuilder.start(ProcessBuilder.java:475) > at com.cloud.utils.script.Script.execute(Script.java:183) > at com.cloud.utils.script.Script.execute(Script.java:161) > at > com.cloud.hypervisor.vmware.manager.VmwareStorageManagerImpl.copyTemplateFromSecondaryToPrimary(VmwareStorageManagerImpl.java:443) > at > com.cloud.hypervisor.vmware.manager.VmwareStorageManagerImpl.execute(VmwareStorageManagerImpl.java:138) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3587) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:401) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > Caused by: java.io.IOException: java.io.IOException: error=2, No such file or > directory > at java.lang.UNIXProcess.<init>(UNIXProcess.java:164) > at java.lang.ProcessImpl.start(ProcessImpl.java:81) > at java.lang.ProcessBuilder.start(ProcessBuilder.java:468) > ... 15 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira