Can you mount the secondary storage from your KVM host? On Oct 8, 2013 4:01 PM, "Mike Tutkowski" <mike.tutkow...@solidfire.com> wrote:
> Although the host is added to KVM, I do see the following issues in the CS > MS console (any thoughts on this?): > > WARN [c.c.u.d.Merovingian2] (secstorage-1:ctx-c1c573ee) Was unable to > find lock for the key template_spool_ref2 and thread id 2049868806 > INFO [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-c1c573ee) Unable > to contact resource. > com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] > is unreachable: Unable to create > Vol[1|vm=1|ROOT]:com.cloud.utils.exception.CloudRuntimeException: > org.libvirt.LibvirtException: internal error Child process (/bin/mount > 192.168.233.10:/mnt/secondary/template/tmpl/1/3 > /mnt/334b3c4e-764b-362a-be2c-ebe8c490d0a9) status unexpected: exit status 32 > at > org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1027) > at > org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1069) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:830) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:649) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:261) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:693) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1265) > at > com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123) > at > com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50) > at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:101) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:78) > at > com.cloud.vm.SystemVmLoadScanner$1.runInContext(SystemVmLoadScanner.java:71) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > > > On Tue, Oct 8, 2013 at 3:58 PM, Mike Tutkowski < > mike.tutkow...@solidfire.com> wrote: > >> So...got some good news: >> >> I spent a couple hours setting up a KVM environment on Ubuntu 12.04.1 >> from scratch (Installing SSH, Open iSCSI, Java 7, KVM, Git, CloudStack, >> CloudStack DEBs, KVM system template, etc.) and I can now add this KVM host >> to CloudStack (on a related note, no errors in agent.err either). >> >> I have no idea what is messed up with my old KVM install on Ubuntu, but >> the new one works. >> >> That being the case, I can close out the JIRA ticket I logged a while >> back and start integrating your code into mine. >> >> >> On Mon, Oct 7, 2013 at 7:46 PM, Mike Tutkowski < >> mike.tutkow...@solidfire.com> wrote: >> >>> Right...yeah, I didn't mean we'd commit to 4.2, but maybe I should work >>> off of 4.2 since master seems to be un-stable in this regard. >>> >>> I plan to set up a machine in the lab tomorrow with Ubuntu 12.04 from >>> scratch to see if it works when I start clean, but - if it doesn't - I >>> should just use 4.2 for development. >>> >>> >>> On Mon, Oct 7, 2013 at 7:05 PM, Marcus Sorensen <shadow...@gmail.com>wrote: >>> >>>> We can't. This patch will never see 4.2. You can still start working on >>>> your plugin on 4.2, but the change represented by this patch can only go >>>> into master. >>>> On Oct 7, 2013 5:01 PM, "Mike Tutkowski" <mike.tutkow...@solidfire.com> >>>> wrote: >>>> >>>>> So, now that I'm getting back to this, do you think I should just try >>>>> to make this work with 4.2 (like we originally talked about)? >>>>> >>>>> I updated again from master, rebuilt, redeployed DEBs and still get >>>>> this JNA error message: >>>>> >>>>> log4j:WARN No appenders could be found for logger >>>>> (org.apache.commons.httpclient.params.DefaultHttpParams). >>>>> log4j:WARN Please initialize the log4j system properly. >>>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfigfor >>>>> more info. >>>>> java.lang.reflect.InvocationTargetException >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at >>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>>> at >>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>> at java.lang.reflect.Method.invoke(Method.java:606) >>>>> at >>>>> org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243) >>>>> Caused by: java.lang.UnsatisfiedLinkError: Can't obtain >>>>> updateLastError method for class com.sun.jna.Native >>>>> at com.sun.jna.Native.initIDs(Native Method) >>>>> at com.sun.jna.Native.<clinit>(Native.java:139) >>>>> at org.libvirt.jna.Libvirt.<clinit>(Unknown Source) >>>>> at org.libvirt.Library.<clinit>(Unknown Source) >>>>> at org.libvirt.Connect.<init>(Unknown Source) >>>>> at >>>>> com.cloud.hypervisor.kvm.resource.LibvirtConnection.getConnection(LibvirtConnection.java:44) >>>>> at >>>>> com.cloud.hypervisor.kvm.resource.LibvirtConnection.getConnection(LibvirtConnection.java:37) >>>>> at >>>>> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.configure(LibvirtComputingResource.java:733) >>>>> at com.cloud.agent.Agent.<init>(Agent.java:161) >>>>> at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:415) >>>>> at >>>>> com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:370) >>>>> at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:351) >>>>> at com.cloud.agent.AgentShell.start(AgentShell.java:448) >>>>> ... 5 more >>>>> Cannot start daemon >>>>> Service exit with a return value of 5 >>>>> >>>>> >>>>> On Mon, Oct 7, 2013 at 2:31 PM, Mike Tutkowski < >>>>> mike.tutkow...@solidfire.com> wrote: >>>>> >>>>>> Sure, that's a good plan. >>>>>> >>>>>> I'll get to it. >>>>>> >>>>>> >>>>>> On Mon, Oct 7, 2013 at 2:29 PM, Marcus Sorensen >>>>>> <shadow...@gmail.com>wrote: >>>>>> >>>>>>> I know you mentioned you might need some minor changes to it, as >>>>>>> well as other minor changes just for master (attach volume switched to >>>>>>> pool >>>>>>> vs adapter or something). My hope was that you would be able to send an >>>>>>> update that works for your plugin on master, I'll test against existing >>>>>>> libvirtd storage and apply it. >>>>>>> On Oct 7, 2013 1:49 PM, "Mike Tutkowski" < >>>>>>> mike.tutkow...@solidfire.com> wrote: >>>>>>> >>>>>>>> This is an automatically generated e-mail. To reply, visit: >>>>>>>> https://reviews.apache.org/r/14381/ >>>>>>>> >>>>>>>> This looks reasonable to me, Marcus. >>>>>>>> >>>>>>>> When do you think you might start the process of getting this into >>>>>>>> master? >>>>>>>> >>>>>>>> >>>>>>>> - Mike Tutkowski >>>>>>>> >>>>>>>> On September 30th, 2013, 5:14 p.m. UTC, Marcus Sorensen wrote: >>>>>>>> Review request for cloudstack, edison su and Mike Tutkowski. >>>>>>>> By Marcus Sorensen. >>>>>>>> >>>>>>>> *Updated Sept. 30, 2013, 5:14 p.m.* >>>>>>>> *Repository: * cloudstack-git >>>>>>>> Description >>>>>>>> >>>>>>>> With custom storage plugins comes the need to prep the KVM host prior >>>>>>>> to utilizing the disks. e.g. an iscsi initiator needs to log into the >>>>>>>> target and scan for the lun before it can be used on the host. This >>>>>>>> patch is an example I developed against 4.2, minor changes may be >>>>>>>> necessary to apply to master, but I want to share with others who are >>>>>>>> working on storage so they can ensure it works for them. Please tweak >>>>>>>> as you see fit. >>>>>>>> >>>>>>>> MigrateCommand: pass vmTO object so we can see which disks/storage >>>>>>>> pool types belong to the vm when migrating a VM. This facilitates >>>>>>>> being able to call disconnectPhysicalDisksViaVmSpec >>>>>>>> >>>>>>>> VirtualMachineManagerImpl: pass VirtualMachineTO when migrating so >>>>>>>> that we can see which disks belong to the VM and what storage >>>>>>>> pools/adaptors should be used >>>>>>>> >>>>>>>> LibvirtComputingResource: add calls KVMStoragePoolManager's >>>>>>>> connectPhysicalDiskViaVmSpec and disconnectPhysicalDiskViaVmSpec calls >>>>>>>> where appropriate (when starting a vm, migrating a vm). Ensure that we >>>>>>>> create 'raw' format XML disk definitions when the storage format is >>>>>>>> RAW. Move cleanupDisk logic to storage adaptors so that each adaptor >>>>>>>> type can clean up its disks in is own way. >>>>>>>> >>>>>>>> KVMStoragePoolManager: add connectPhysicalDisk, >>>>>>>> disconnectPhysicalDisk, connectPhysicalDiskViaVmSpec, >>>>>>>> disconnectPhysicalDiskViaVmSpec, disconnectPhysicalDiskByPath. These >>>>>>>> all call the specific StorageAdaptor's connectPhysicalDisk, >>>>>>>> disconnectPhysicalDisk, or disconnectPhysicalDiskByPath calls. >>>>>>>> >>>>>>>> KVMStorageProcessor: Call connectPhysicalDisk/disconnectPhysicalDisk >>>>>>>> on the storage adaptor. Whether or not this is implemented is up to >>>>>>>> the storage adaptor. >>>>>>>> >>>>>>>> LibvirtStorageAdaptor: implement dummy >>>>>>>> connectPhysicalDisk/disconnectPhysicalDisk, move cleanupDisk logic >>>>>>>> from LibvirtComputingResource to disconnectPhysicalDiskByPath >>>>>>>> >>>>>>>> StorageAdaptor: define >>>>>>>> connectPhysicalDisk/disconnectPhysicalDisk/disconnectPhysicalDiskByPath >>>>>>>> in the interface >>>>>>>> >>>>>>>> >>>>>>>> Testing >>>>>>>> >>>>>>>> Basic testing with my storage adaptor >>>>>>>> >>>>>>>> Diffs >>>>>>>> >>>>>>>> - core/src/com/cloud/agent/api/MigrateCommand.java (5042b8c) >>>>>>>> - >>>>>>>> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java >>>>>>>> (3ee811f) >>>>>>>> - >>>>>>>> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStoragePoolManager.java >>>>>>>> (e09c9ba) >>>>>>>> - >>>>>>>> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java >>>>>>>> (c69f9b0) >>>>>>>> - >>>>>>>> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java >>>>>>>> (123a9f1) >>>>>>>> - >>>>>>>> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/StorageAdaptor.java >>>>>>>> (4956d8d) >>>>>>>> - server/src/com/cloud/vm/VirtualMachineManagerImpl.java >>>>>>>> (d46bbb0) >>>>>>>> >>>>>>>> View Diff <https://reviews.apache.org/r/14381/diff/> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Mike Tutkowski* >>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>> e: mike.tutkow...@solidfire.com >>>>>> o: 303.746.7302 >>>>>> Advancing the way the world uses the >>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>>> *™* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Mike Tutkowski* >>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>> e: mike.tutkow...@solidfire.com >>>>> o: 303.746.7302 >>>>> Advancing the way the world uses the >>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>> *™* >>>>> >>>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkow...@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the >>> cloud<http://solidfire.com/solution/overview/?video=play> >>> *™* >>> >> >> >> >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkow...@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the >> cloud<http://solidfire.com/solution/overview/?video=play> >> *™* >> > > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *™* >
