[
https://issues.apache.org/jira/browse/CLOUDSTACK-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Sorensen updated CLOUDSTACK-1056:
----------------------------------------
Description:
KVM creates this KVMHA directory in the template/tmpl/1/3 directory on
secondary storage, where the system vm is prepared. It's some sort of heartbeat
cluster thingy for KVM. This seems to be causing the S3 upload to break:
SSVM log:
20:20:58,779 ERROR NfsSecondaryStorageResource:426 - Failed to upload template
id 3
com.amazonaws.AmazonClientException: Unable to calculate MD5 hash:
/mnt/SecStorage/1cf3b9fa-3024-329a-991a-7bdf7b3f14f7/template/tmpl/1/3/KVMHA
(Is a directory)
at
com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1012)
at
com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:963)
at com.cloud.utils.S3Utils.putDirectory(S3Utils.java:282)
at
com.cloud.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:396)
at
com.cloud.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:210)
at com.cloud.agent.Agent.processRequest(Agent.java:525)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.FileNotFoundException:
/mnt/SecStorage/1cf3b9fa-3024-329a-991a-7bdf7b3f14f7/template/tmpl/1/3/KVMHA
(Is a directory)
Management Server log:
INFO [cloud.template.S3SyncTask] (s3-template-sync-1:) Uploading template
SystemVM Template (KVM) (id: 3) to S3.
ERROR [storage.s3.S3ManagerImpl] (s3-template-sync-1:) Failed to upload
template id 3 for zone id 1 to S3.
com.cloud.utils.exception.CloudRuntimeException: Failed to upload template id 3
to S3 from secondary storage due to Failed to upload template id 3.
at com.cloud.storage.s3.S3ManagerImpl$3.call(S3ManagerImpl.java:622)
at com.cloud.storage.s3.S3ManagerImpl$3.call(S3ManagerImpl.java:606)
at com.cloud.utils.db.GlobalLock.executeWithLock(GlobalLock.java:225)
at
com.cloud.utils.db.GlobalLock.executeWithNoWaitLock(GlobalLock.java:240)
at
com.cloud.storage.s3.S3ManagerImpl.uploadTemplateToS3FromSecondaryStorage(S3ManagerImpl.java:605)
at com.cloud.template.S3SyncTask.run(S3SyncTask.java:80)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
was:
KVM creates this KVMHA directory in the template/tmpl/1/3 directory on
secondary storage, where the system vm is prepared. It's some sort of heartbeat
cluster thingy for KVM. This seems to be causing the S3 upload to break:
20:20:58,779 ERROR NfsSecondaryStorageResource:426 - Failed to upload template
id 3
com.amazonaws.AmazonClientException: Unable to calculate MD5 hash:
/mnt/SecStorage/1cf3b9fa-3024-329a-991a-7bdf7b3f14f7/template/tmpl/1/3/KVMHA
(Is a directory)
at
com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1012)
at
com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:963)
at com.cloud.utils.S3Utils.putDirectory(S3Utils.java:282)
at
com.cloud.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:396)
at
com.cloud.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:210)
at com.cloud.agent.Agent.processRequest(Agent.java:525)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
at com.cloud.utils.nio.Task.run(Task.java:83)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.FileNotFoundException:
/mnt/SecStorage/1cf3b9fa-3024-329a-991a-7bdf7b3f14f7/template/tmpl/1/3/KVMHA
(Is a directory)
> S3 secondary storage fails to upload systemvm template due to KVMHA directory
> -----------------------------------------------------------------------------
>
> Key: CLOUDSTACK-1056
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1056
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: KVM, Storage Controller
> Affects Versions: 4.1.0
> Environment: KVM advanced zone
> Reporter: Marcus Sorensen
> Assignee: John Burwell
> Fix For: 4.1.0
>
>
> KVM creates this KVMHA directory in the template/tmpl/1/3 directory on
> secondary storage, where the system vm is prepared. It's some sort of
> heartbeat cluster thingy for KVM. This seems to be causing the S3 upload to
> break:
> SSVM log:
> 20:20:58,779 ERROR NfsSecondaryStorageResource:426 - Failed to upload
> template id 3
> com.amazonaws.AmazonClientException: Unable to calculate MD5 hash:
> /mnt/SecStorage/1cf3b9fa-3024-329a-991a-7bdf7b3f14f7/template/tmpl/1/3/KVMHA
> (Is a directory)
> at
> com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1012)
> at
> com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:963)
> at com.cloud.utils.S3Utils.putDirectory(S3Utils.java:282)
> at
> com.cloud.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:396)
> at
> com.cloud.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:210)
> at com.cloud.agent.Agent.processRequest(Agent.java:525)
> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
> at com.cloud.utils.nio.Task.run(Task.java:83)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.FileNotFoundException:
> /mnt/SecStorage/1cf3b9fa-3024-329a-991a-7bdf7b3f14f7/template/tmpl/1/3/KVMHA
> (Is a directory)
> Management Server log:
> INFO [cloud.template.S3SyncTask] (s3-template-sync-1:) Uploading template
> SystemVM Template (KVM) (id: 3) to S3.
> ERROR [storage.s3.S3ManagerImpl] (s3-template-sync-1:) Failed to upload
> template id 3 for zone id 1 to S3.
> com.cloud.utils.exception.CloudRuntimeException: Failed to upload template id
> 3 to S3 from secondary storage due to Failed to upload template id 3.
> at com.cloud.storage.s3.S3ManagerImpl$3.call(S3ManagerImpl.java:622)
> at com.cloud.storage.s3.S3ManagerImpl$3.call(S3ManagerImpl.java:606)
> at com.cloud.utils.db.GlobalLock.executeWithLock(GlobalLock.java:225)
> at
> com.cloud.utils.db.GlobalLock.executeWithNoWaitLock(GlobalLock.java:240)
> at
> com.cloud.storage.s3.S3ManagerImpl.uploadTemplateToS3FromSecondaryStorage(S3ManagerImpl.java:605)
> at com.cloud.template.S3SyncTask.run(S3SyncTask.java:80)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
