Hi Jeremy,
You can enable trace logs to check the states. You can also query the database
to check which resources are not ready.
Here are some queries you can use:
SELECT uuid, name, store_id, state FROM template_view WHERE state NOT IN
('Ready', 'Allocated', 'Destroying', 'Destroyed', 'Failed') AND removed IS NULL;
SELECT uuid, name, store_id, store_state FROM snapshot_view WHERE store_state
NOT IN ('Ready', 'Allocated', 'Destroying', 'Destroyed', 'Failed') AND removed
IS NULL;
SELECT volumes.uuid, volumes.name, volume_store_ref.store_id,
volume_store_ref.state
FROM volumes
INNER JOIN volume_store_ref ON volumes.id = volume_store_ref.volume_id
WHERE volumes.removed IS NULL AND volume_store_ref.state NOT IN ('Ready',
'Allocated', 'Destroying', 'Destroyed', 'Failed');
________________________________
From: Jeremy Hansen
Sent: Saturday, January 27, 2024 5:06 AM
To: Vivek Kumar via users
Subject: Migrating secondary storage
I’m trying to migrate to new secondary storage. I’m receiving this error:
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]:
com.cloud.utils.exception.CloudRuntimeException: Complete migration failed as
there are data objects which are not Ready - i.e, they may be in Migrating,
creating, copying, etc. states
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.engine.orchestration.DataMigrationUtility.checkIfCompleteMigrationPossible(DataMigrationUtility.java:122)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.engine.orchestration.StorageOrchestrator.migrateData(StorageOrchestrator.java:149)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
com.cloud.storage.ImageStoreServiceImpl.migrateData(ImageStoreServiceImpl.java:157)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
com.sun.proxy.$Proxy386.migrateData(Unknown Source)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.api.command.admin.storage.MigrateSecondaryStorageDataCmd.execute(MigrateSecondaryStorageDataCmd.java:100)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:172)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:112)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:654)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:602)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Jan 26 23:33:25 dell1.fr1.clx.corp java[1787564]: at
java.base/java.lang.Thread.run(Thread.java:829)
Yet, I see nothing in a creating or migrating state. Where would I find this
in the db? Maybe something is out of sync due to issues I was having. What
can I do about this?
Thanks
-jeremy
