harikrishna-patnala commented on PR #11435: URL: https://github.com/apache/cloudstack/pull/11435#issuecomment-3183417438
> @harikrishna-patnala hitting the following exceptions > > ``` > 2025-08-13 07:17:42,492 DEBUG [c.c.u.d.Upgrade42010to42100] (main:[]) (logid:) Updating System Vm template IDs > 2025-08-13 07:17:42,698 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Updating System Vm template IDs > 2025-08-13 07:17:42,706 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Updating SystemVM Template Guest OS [Debian GNU/Linux 12 (64-bit)] id > 2025-08-13 07:17:42,744 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Updating System VM template for hypervisor: LXC > 2025-08-13 07:17:42,749 WARN [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Cannot upgrade 4.20.2 system VM template for hypervisor: LXC as it is not used, not failing upgrade > 2025-08-13 07:17:42,787 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Updating System VM template for hypervisor: VMware > 2025-08-13 07:17:42,789 WARN [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Cannot upgrade 4.20.2 system VM template for hypervisor: VMware as it is not used, not failing upgrade > 2025-08-13 07:17:42,797 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Updating System VM template for hypervisor: KVM, arch: x86_64 > 2025-08-13 07:17:42,799 INFO [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Grabbing lock to register templates. > 2025-08-13 07:17:46,262 DEBUG [c.c.u.s.Script] (main:[]) (logid:) Executing command [/bin/bash -c sudo mount -t nfs 10.0.32.4:/acs/secondary/ref-trl-9218-k-Mol8-harikrishna-patnala/ref-trl-9218-k-Mol8-harikrishna-patnala-sec1 /tmp/tmp9555288554741205640 ]. > 2025-08-13 07:17:46,440 DEBUG [c.c.u.s.Script] (main:[]) (logid:) Successfully executed process [4532] for command [/bin/bash -c sudo mount -t nfs 10.0.32.4:/acs/secondary/ref-trl-9218-k-Mol8-harikrishna-patnala/ref-trl-9218-k-Mol8-harikrishna-patnala-sec1 /tmp/tmp9555288554741205640 ]. > 2025-08-13 07:17:46,568 DEBUG [c.c.u.s.Script] (main:[]) (logid:) Looking for setup-sysvm-tmplt in the classpath > 2025-08-13 07:17:46,569 DEBUG [c.c.u.s.Script] (main:[]) (logid:) System resource: null > 2025-08-13 07:17:46,569 DEBUG [c.c.u.s.Script] (main:[]) (logid:) Classpath resource: file:/usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt > 2025-08-13 07:17:46,570 DEBUG [c.c.u.s.Script] (main:[]) (logid:) Absolute path = /usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt > 2025-08-13 07:17:46,570 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Executing command [/usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt -u 74c1f164-dbb1-4c1b-82b9-fbb334d8b235 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2 -h kvm -d /tmp/tmp9555288554741205640/template/tmpl/1/203 ]. > 2025-08-13 07:17:46,711 WARN [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Execution of process [4543] for command [/usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt -u 74c1f164-dbb1-4c1b-82b9-fbb334d8b235 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2 -h kvm -d /tmp/tmp9555288554741205640/template/tmpl/1/203 ] failed. > 2025-08-13 07:17:46,712 DEBUG [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Exit value of process [4543] for command [/usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt -u 74c1f164-dbb1-4c1b-82b9-fbb334d8b235 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2 -h kvm -d /tmp/tmp9555288554741205640/template/tmpl/1/203 ] is [2]. > 2025-08-13 07:17:46,713 WARN [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Process [4543] for command [/usr/share/cloudstack-common/scripts/storage/secondary/setup-sysvm-tmplt -u 74c1f164-dbb1-4c1b-82b9-fbb334d8b235 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2 -h kvm -d /tmp/tmp9555288554741205640/template/tmpl/1/203 ] encountered the error: [+ [[ ! -u 74c1f164-dbb1-4c1b-82b9-fbb334d8b235 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2 -h kvm -d /tmp/tmp9555288554741205640/template/tmpl/1/203 =~ ^-.+ ]]+ OPTERR=0+ DISKSPACE=2120000+ getopts h:f:d:u::# OPTION+ case $OPTION in+ uflag=1+ uuid=74c1f164-dbb1-4c1b-82b9-fbb334d8b235+ getopts h:f:d:u::# OPTION+ case $OPTION in+ fflag=1+ tmpltimg=/usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2+ getopts h:f:d:u::# OPTION+ case $OPTION in+ hyper=kvm+ getopts h:f:d:u::# OPTION + case $OPTION in+ destdir=/tmp/tmp9555288554741205640/template/tmpl/1/203+ getopts h:f:d:u::# OPTION+ [[ kvm == \k\v\m ]]+ ext=qcow2++ which qemu-img+ qemuimgcmd=/usr/bin/qemu-img+ localfile=74c1f164-dbb1-4c1b-82b9-fbb334d8b235.qcow2+ sudo mkdir -p /tmp/tmp9555288554741205640/template/tmpl/1/203+ [[ 0 -ne 0 ]]+ [[ -f /tmp/tmp9555288554741205640/template/tmpl/1/203/template.properties ]]+ failed 2 'Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203'+ local returnval=2+ local 'returnmsg=Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203'+ [[ -z Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203 ]]+ echo -e Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203+ [[ 2 -eq 0 ]]+ echo 'Installation failed'Installation failed+ exit 2]. > 2025-08-13 07:17:46,716 ERROR [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) failed to create template: + [[ ! -u 74c1f164-dbb1-4c1b-82b9-fbb334d8b235 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2 -h kvm -d /tmp/tmp9555288554741205640/template/tmpl/1/203 =~ ^-.+ ]]+ OPTERR=0+ DISKSPACE=2120000+ getopts h:f:d:u::# OPTION+ case $OPTION in+ uflag=1+ uuid=74c1f164-dbb1-4c1b-82b9-fbb334d8b235+ getopts h:f:d:u::# OPTION+ case $OPTION in+ fflag=1+ tmpltimg=/usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2+ getopts h:f:d:u::# OPTION+ case $OPTION in+ hyper=kvm+ getopts h:f:d:u::# OPTION+ case $OPTION in+ destdir=/tmp/tmp9555288554741205640/template/tmpl/1/203+ getopts h:f:d:u::# OPTION+ [[ kvm == \k\v\m ]]+ ext=qcow2++ which qemu-img+ qemuimgcmd=/usr/bin/qemu-img+ localfile=74c1f164-dbb1-4c1b-82b9-fbb334d8b235.qcow2+ sudo mkdir -p /tmp/tmp9555288554741205640/template/tmpl/1/203+ [[ 0 -ne 0 ]]+ [[ -f /tmp/tmp9555288554741205640/template/tmpl/1/203/template.properties ]]+ failed 2 'Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203'+ local returnval=2+ local 'returnmsg=Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203'+ [[ -z Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203 ]]+ echo -e Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203+ [[ 2 -eq 0 ]]+ echo 'Installation failed'Installation failed+ exit 2 > 2025-08-13 07:17:46,717 ERROR [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Failed to register template for hypervisor: KVM com.cloud.utils.exception.CloudRuntimeException: failed to create template: + [[ ! -u 74c1f164-dbb1-4c1b-82b9-fbb334d8b235 -f /usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2 -h kvm -d /tmp/tmp9555288554741205640/template/tmpl/1/203 =~ ^-.+ ]]+ OPTERR=0+ DISKSPACE=2120000+ getopts h:f:d:u::# OPTION+ case $OPTION in+ uflag=1+ uuid=74c1f164-dbb1-4c1b-82b9-fbb334d8b235+ getopts h:f:d:u::# OPTION+ case $OPTION in+ fflag=1+ tmpltimg=/usr/share/cloudstack-management/templates/systemvm/systemvmtemplate-4.20.2-x86_64-kvm.qcow2.bz2+ getopts h:f:d:u::# OPTION+ case $OPTION in+ hyper=kvm+ getopts h:f:d:u::# OPTION+ case $OPTION in+ destdir=/tmp/tmp9555288554741205640/template/tmpl/1/203+ getopts h:f:d:u::# OPTION+ [[ kvm == \k\v\m ]]+ ext=qcow2++ which qemu-img+ qemuimgcmd=/usr/bin/qemu-img+ localfile=74c1f164- dbb1-4c1b-82b9-fbb334d8b235.qcow2+ sudo mkdir -p /tmp/tmp9555288554741205640/template/tmpl/1/203+ [[ 0 -ne 0 ]]+ [[ -f /tmp/tmp9555288554741205640/template/tmpl/1/203/template.properties ]]+ failed 2 'Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203'+ local returnval=2+ local 'returnmsg=Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203'+ [[ -z Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203 ]]+ echo -e Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203+ [[ 2 -eq 0 ]]+ echo 'Installation failed'Installation failed+ exit 2 > at com.cloud.upgrade.SystemVmTemplateRegistration.setupTemplate(SystemVmTemplateRegistration.java:701) > at com.cloud.upgrade.SystemVmTemplateRegistration.performTemplateRegistrationOperations(SystemVmTemplateRegistration.java:728) > at com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplateForNonExistingEntries(SystemVmTemplateRegistration.java:755) > at com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplatesForZone(SystemVmTemplateRegistration.java:943) > at com.cloud.upgrade.SystemVmTemplateRegistration$4.doInTransactionWithoutResult(SystemVmTemplateRegistration.java:969) > at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:48) > at com.cloud.utils.db.Transaction.execute(Transaction.java:38) > at com.cloud.utils.db.Transaction.execute(Transaction.java:45) > at com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplates(SystemVmTemplateRegistration.java:958) > at com.cloud.upgrade.SystemVmTemplateRegistration.registerOrUpdateSystemVmTemplate(SystemVmTemplateRegistration.java:1037) > at com.cloud.upgrade.SystemVmTemplateRegistration$5.doInTransactionWithoutResult(SystemVmTemplateRegistration.java:1075) > at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:48) > at com.cloud.utils.db.Transaction.execute(Transaction.java:38) > at com.cloud.utils.db.Transaction.execute(Transaction.java:45) > at com.cloud.upgrade.SystemVmTemplateRegistration.updateSystemVmTemplates(SystemVmTemplateRegistration.java:1063) > at com.cloud.upgrade.dao.Upgrade42010to42100.updateSystemVmTemplates(Upgrade42010to42100.java:90) > at com.cloud.upgrade.DatabaseUpgradeChecker.updateSystemVmTemplates(DatabaseUpgradeChecker.java:291) > at com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:310) > at com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:483) > at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:62) > at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:52) > at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) > at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) > at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) > at java.base/java.lang.Iterable.forEach(Iterable.java:75) > at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) > at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) > at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:937) > at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586) > at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:174) > at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:143) > at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:274) > at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:279) > at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:262) > at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:131) > at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79) > at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37) > at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:71) > at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:58) > at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:62) > at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:53) > at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073) > at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) > at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002) > at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765) > at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) > at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) > at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) > at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916) > at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) > at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) > at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) > at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) > at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > at org.eclipse.jetty.server.Server.start(Server.java:423) > at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) > at org.eclipse.jetty.server.Server.doStart(Server.java:387) > at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > at org.apache.cloudstack.ServerDaemon.start(ServerDaemon.java:207) > at org.apache.cloudstack.ServerDaemon.main(ServerDaemon.java:117) > > 2025-08-13 07:17:46,718 INFO [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Unmounting store > 2025-08-13 07:17:46,718 DEBUG [c.c.u.s.Script] (main:[]) (logid:) Executing command [/bin/bash -c sudo umount /tmp/tmp9555288554741205640 ]. > 2025-08-13 07:17:46,977 DEBUG [c.c.u.s.Script] (main:[]) (logid:) Successfully executed process [4550] for command [/bin/bash -c sudo umount /tmp/tmp9555288554741205640 ]. > 2025-08-13 07:17:46,989 DEBUG [c.c.u.d.T.Transaction] (main:[]) (logid:) Rolling back the transaction: Time = 4490 Name = Upgrade; called by -TransactionLegacy.rollback:905-TransactionLegacy.removeUpTo:848-TransactionLegacy.close:672-Transaction.execute:36-Transaction.execute:45-SystemVmTemplateRegistration.registerTemplates:958-SystemVmTemplateRegistration.registerOrUpdateSystemVmTemplate:1037-SystemVmTemplateRegistration$5.doInTransactionWithoutResult:1075-TransactionCallbackNoReturn.doInTransaction:25-Transaction$2.doInTransaction:48-Transaction.execute:38-Transaction.execute:45 > 2025-08-13 07:17:47,005 ERROR [c.c.u.SystemVmTemplateRegistration] (main:[]) (logid:) Exception while registering/updating system VM templates for hypervisors in metadata com.cloud.utils.exception.CloudRuntimeException: Failed to register 4.20.2 templates for hypervisors: [kvm-x86_64]. Cannot upgrade system VMs > at com.cloud.upgrade.SystemVmTemplateRegistration.registerOrUpdateSystemVmTemplate(SystemVmTemplateRegistration.java:1040) > at com.cloud.upgrade.SystemVmTemplateRegistration$5.doInTransactionWithoutResult(SystemVmTemplateRegistration.java:1075) > at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:48) > at com.cloud.utils.db.Transaction.execute(Transaction.java:38) > at com.cloud.utils.db.Transaction.execute(Transaction.java:45) > at com.cloud.upgrade.SystemVmTemplateRegistration.updateSystemVmTemplates(SystemVmTemplateRegistration.java:1063) > at com.cloud.upgrade.dao.Upgrade42010to42100.updateSystemVmTemplates(Upgrade42010to42100.java:90) > at com.cloud.upgrade.DatabaseUpgradeChecker.updateSystemVmTemplates(DatabaseUpgradeChecker.java:291) > at com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:310) > at com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:483) > > > ``` @kiranchavala I see this error "'Data already exists at destination /tmp/tmp9555288554741205640/template/tmpl/1/203'" based on this, I think the environment seems to have reused for upgrade testing where one of the attempts was successful. Please delete this folder in the secondary storage and try upgrade again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
