Sure. The majority of it only affects people who are on your storage
anyway. Perhaps you can post a patch and I can run it through the
simulator to verify that the minor change to the existing code hasn't
broken the standard storages. I don't think it is since I've
thoroughly tested the code I posted, but I know there were some
additional changes.
On Wed, Oct 23, 2013 at 10:35 PM, Mike Tutkowski
<mike.tutkow...@solidfire.com> wrote:
> OK, Marcus, I made the change to detect my volumes and it seems to work just
> fine.
>
> Perhaps another day of testing and we can check this code in. What do you
> think?
>
>
> On Wed, Oct 23, 2013 at 9:14 PM, Mike Tutkowski
> <mike.tutkow...@solidfire.com> wrote:
>>
>> Thanks, Marcus...I hadn't read that note, but that makes sense.
>>
>> Yes, that must be the root disk for the VM. I can put in code, as you
>> recommend, to handle only my volumes.
>>
>>
>> On Wed, Oct 23, 2013 at 5:37 PM, Marcus Sorensen <shadow...@gmail.com>
>> wrote:
>>>
>>> It should be sending the path info for each disk per the XML of the
>>> VM... so it will send all disks regardless of whether or not your
>>> adaptor manages that disk, and it's up to your adaptor to ignore any
>>> that aren't managed by it. There should be notes to that effect in the
>>> code near the disconnectPhysicalDisk interface in StorageAdaptor:
>>>
>>> // given local path to file/device (per Libvirt XML), 1) check
>>> that device is
>>> // handled by your adaptor, return false if not. 2) clean up
>>> device, return true
>>> public boolean disconnectPhysicalDiskByPath(String localPath);
>>>
>>> Since we only have XML disk definitions when we stop or migrate a VM,
>>> we have to try all adaptors against all defined disks. So in your
>>> disconnectPhysicalDisk you might do something like check that the path
>>> starts with '/dev/disk/by-path' and contains 'iscs-iqn' (maybe there's
>>> some way that's more robust like checking the full path against a lun
>>> listing or something). If it doesn't match, then your
>>> disconnectPhysicalDisk just does nothing.
>>>
>>> I assume this is a root disk or some other local storage disk. If it's
>>> not, then your VM XML is messed up somehow.
>>>
>>> On Wed, Oct 23, 2013 at 5:01 PM, Mike Tutkowski
>>> <mike.tutkow...@solidfire.com> wrote:
>>> > I found the problem.
>>> >
>>> > disconnectPhysicalDiskByPath is being passed in (in my situation) the
>>> > following:
>>> >
>>> > /var/lib/libvirt/images/9887d511-8dc7-4cb4-96f9-01230fe4bbb6
>>> >
>>> > Due to the name of the method, my code was expecting data such as the
>>> > following:
>>> >
>>> >
>>> > /dev/disk/by-path/ip-192.168.233.10:3260-iscsi-iqn.2012-03.com.solidfire:volume1-lun-0
>>> >
>>> > Was it intentional to send the data into this method in the current
>>> > way?
>>> >
>>> >
>>> > On Wed, Oct 23, 2013 at 1:57 PM, Mike Tutkowski
>>> > <mike.tutkow...@solidfire.com> wrote:
>>> >>
>>> >> You know, I forgot we supposed to be doing that! :) Multi-tasking too
>>> >> much
>>> >> today, I guess.
>>> >>
>>> >> Anyways, it must not be working because I still had a hypervisor
>>> >> connection after I shut down the VM.
>>> >>
>>> >> Let me investigate.
>>> >>
>>> >>
>>> >> On Wed, Oct 23, 2013 at 1:48 PM, Marcus Sorensen <shadow...@gmail.com>
>>> >> wrote:
>>> >>>
>>> >>> Are we not disconnecting when we stop the vm? There's a method for
>>> >>> it, we
>>> >>> should be. disconnectPhysicalDiskViaVmSpec
>>> >>>
>>> >>> On Oct 23, 2013 1:28 PM, "Mike Tutkowski"
>>> >>> <mike.tutkow...@solidfire.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> I see one problem for us now, Marcus.
>>> >>>>
>>> >>>> * You have a running VM that you attach a volume to.
>>> >>>> * You stop the VM.
>>> >>>> * You detach the volume.
>>> >>>> * You start up the VM.
>>> >>>>
>>> >>>> The VM will not be connected to the volume (which is good), but the
>>> >>>> hypervisor will still be connected to the volume.
>>> >>>>
>>> >>>> It would be great if we actually sent a command to the last host ID
>>> >>>> of
>>> >>>> the stopped VM when detaching a volume (to have the hypervisor
>>> >>>> disconnect
>>> >>>> from the volume).
>>> >>>>
>>> >>>> What do you think about that?
>>> >>>>
>>> >>>>
>>> >>>> On Wed, Oct 23, 2013 at 1:15 PM, Mike Tutkowski
>>> >>>> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>
>>> >>>>> OK, whatever way you prefer then, Marcus (createVdb first or
>>> >>>>> second).
>>> >>>>>
>>> >>>>> If I leave createVdb first and return 0, it does seem to work.
>>> >>>>>
>>> >>>>>
>>> >>>>> On Wed, Oct 23, 2013 at 11:13 AM, Marcus Sorensen
>>> >>>>> <shadow...@gmail.com>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>> I think we could flip-flop these two lines if necessary:
>>> >>>>>>
>>> >>>>>> createVbd(conn, vmSpec, vmName, vm);
>>> >>>>>>
>>> >>>>>> _storagePoolMgr.connectPhysicalDisksViaVmSpec(vmSpec);
>>> >>>>>>
>>> >>>>>> I haven't actually tried it though. But in general I don't see the
>>> >>>>>> Libvirt DiskDef using size at all, which is what createVbd does
>>> >>>>>> (creates XML definitions for disks to attach to the VM
>>> >>>>>> definition). It
>>> >>>>>> just takes the device at it's native advertised size when it
>>> >>>>>> actually
>>> >>>>>> goes to use it.
>>> >>>>>>
>>> >>>>>> On Wed, Oct 23, 2013 at 10:52 AM, Mike Tutkowski
>>> >>>>>> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> > Little problem that I wanted to get your take on, Marcus.
>>> >>>>>> >
>>> >>>>>> > When a VM is being started, we call createVdb before calling
>>> >>>>>> > connectPhysicalDisksViaVmSpec.
>>> >>>>>> >
>>> >>>>>> > The problem is that createVdb calls getPhysicalDisk and my
>>> >>>>>> > volume
>>> >>>>>> > has not
>>> >>>>>> > yet been connected because connectPhysicalDisksViaVmSpec has not
>>> >>>>>> > yet
>>> >>>>>> > been
>>> >>>>>> > called.
>>> >>>>>> >
>>> >>>>>> > When I try to read up the size of the disk to populate a
>>> >>>>>> > PhysicalDisk, I get
>>> >>>>>> > an error, of course, because the path does not yet exist.
>>> >>>>>> >
>>> >>>>>> > I could populate a 0 for the size of the physical disk and then
>>> >>>>>> > the
>>> >>>>>> > next
>>> >>>>>> > time getPhysicalDisk is called, it should be filled in with a
>>> >>>>>> > proper
>>> >>>>>> > size.
>>> >>>>>> >
>>> >>>>>> > Do you see a problem with that approach?
>>> >>>>>> >
>>> >>>>>> >
>>> >>>>>> > On Tue, Oct 22, 2013 at 6:40 PM, Marcus Sorensen
>>> >>>>>> > <shadow...@gmail.com>
>>> >>>>>> > wrote:
>>> >>>>>> >>
>>> >>>>>> >> That's right. All should be well.
>>> >>>>>> >>
>>> >>>>>> >> On Oct 22, 2013 6:03 PM, "Mike Tutkowski"
>>> >>>>>> >> <mike.tutkow...@solidfire.com>
>>> >>>>>> >> wrote:
>>> >>>>>> >>>
>>> >>>>>> >>> Looks like we disconnect physical disks when the VM is
>>> >>>>>> >>> stopped.
>>> >>>>>> >>>
>>> >>>>>> >>> I didn't see that before.
>>> >>>>>> >>>
>>> >>>>>> >>> I suppose that means the disks are physically disconnected
>>> >>>>>> >>> when
>>> >>>>>> >>> the VM is
>>> >>>>>> >>> stopped, but the CloudStack DB still has the VM associated
>>> >>>>>> >>> with
>>> >>>>>> >>> the disks
>>> >>>>>> >>> for the next time the VM may be started up (unless someone
>>> >>>>>> >>> does a
>>> >>>>>> >>> disconnect
>>> >>>>>> >>> while the VM is in the Stopped State).
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>> On Tue, Oct 22, 2013 at 4:19 PM, Mike Tutkowski
>>> >>>>>> >>> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> >>>>
>>> >>>>>> >>>> Hey Marcus,
>>> >>>>>> >>>>
>>> >>>>>> >>>> Quick question for you related to attaching/detaching volumes
>>> >>>>>> >>>> when the
>>> >>>>>> >>>> VM is in the Stopped State.
>>> >>>>>> >>>>
>>> >>>>>> >>>> If I detach a volume from a VM that is in the Stopped State,
>>> >>>>>> >>>> the
>>> >>>>>> >>>> DB
>>> >>>>>> >>>> seems to get updated, but I don't see a command going to the
>>> >>>>>> >>>> KVM
>>> >>>>>> >>>> hypervisor
>>> >>>>>> >>>> that leads to the removal of the iSCSI target.
>>> >>>>>> >>>>
>>> >>>>>> >>>> It seems the iSCSI target is only removed the next time the
>>> >>>>>> >>>> VM is
>>> >>>>>> >>>> started.
>>> >>>>>> >>>>
>>> >>>>>> >>>> Do you know if this is true?
>>> >>>>>> >>>>
>>> >>>>>> >>>> If it is, I'm concerned that the volume could be attached to
>>> >>>>>> >>>> another VM
>>> >>>>>> >>>> before the Stopped VM is re-started and when the Stopped VM
>>> >>>>>> >>>> gets
>>> >>>>>> >>>> restarted
>>> >>>>>> >>>> that it would disconnect the iSCSI volume from underneath the
>>> >>>>>> >>>> VM
>>> >>>>>> >>>> that now
>>> >>>>>> >>>> has the volume attached.
>>> >>>>>> >>>>
>>> >>>>>> >>>> I still want to perform some tests on this, but am first
>>> >>>>>> >>>> trying
>>> >>>>>> >>>> to get a
>>> >>>>>> >>>> VM to start after I've attached a volume to it when it was in
>>> >>>>>> >>>> the
>>> >>>>>> >>>> Stopped
>>> >>>>>> >>>> State.
>>> >>>>>> >>>>
>>> >>>>>> >>>> Thanks,
>>> >>>>>> >>>> Mike
>>> >>>>>> >>>>
>>> >>>>>> >>>>
>>> >>>>>> >>>> On Mon, Oct 21, 2013 at 10:57 PM, Mike Tutkowski
>>> >>>>>> >>>> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> Thanks for that info, Marcus.
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> By the way, I wanted to see if I could attach my volume to a
>>> >>>>>> >>>>> VM
>>> >>>>>> >>>>> in the
>>> >>>>>> >>>>> Stopped State.
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> The attach logic didn't trigger any exceptions; however,
>>> >>>>>> >>>>> when I
>>> >>>>>> >>>>> started
>>> >>>>>> >>>>> the VM, I received an Insufficient Capacity exception.
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> If I detach the volume and then start the VM, the VM starts
>>> >>>>>> >>>>> just
>>> >>>>>> >>>>> fine.
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> I noticed a problem here (in StoragePoolHostDaoImpl):
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> @Override
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> public StoragePoolHostVO findByPoolHost(long poolId,
>>> >>>>>> >>>>> long
>>> >>>>>> >>>>> hostId) {
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> SearchCriteria<StoragePoolHostVO> sc =
>>> >>>>>> >>>>> PoolHostSearch.create();
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> sc.setParameters("pool_id", poolId);
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> sc.setParameters("host_id", hostId);
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> return findOneIncludingRemovedBy(sc);
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> }
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> The findOneIncludingRemovedBy method returns null (the
>>> >>>>>> >>>>> poolId is
>>> >>>>>> >>>>> my
>>> >>>>>> >>>>> storage pool's ID and the hostId is the expected host ID).
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> I'm not sure what this method is trying to do.
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> I looked in the storage_pool_host_ref table (is that the
>>> >>>>>> >>>>> correct
>>> >>>>>> >>>>> table?) and it only has one row, which maps the local
>>> >>>>>> >>>>> storage
>>> >>>>>> >>>>> pool of the
>>> >>>>>> >>>>> KVM host to the KVM host (which explains why no match is
>>> >>>>>> >>>>> found
>>> >>>>>> >>>>> for my
>>> >>>>>> >>>>> situation).
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> Do you understand what this logic is trying to do?
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> Thanks!
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> On Mon, Oct 21, 2013 at 8:08 PM, Marcus Sorensen
>>> >>>>>> >>>>> <shadow...@gmail.com>
>>> >>>>>> >>>>> wrote:
>>> >>>>>> >>>>>>
>>> >>>>>> >>>>>> Do you have the capability to clone the root disk? Normally
>>> >>>>>> >>>>>> the
>>> >>>>>> >>>>>> template is installed to primary, and then cloned for each
>>> >>>>>> >>>>>> root
>>> >>>>>> >>>>>> disk.
>>> >>>>>> >>>>>> In some cases (such as CLVM), this isn't efficient and so
>>> >>>>>> >>>>>> the
>>> >>>>>> >>>>>> template
>>> >>>>>> >>>>>> is copied fresh to populate each root disk.
>>> >>>>>> >>>>>>
>>> >>>>>> >>>>>> I'm actually not 100% sure how this works in the new code.
>>> >>>>>> >>>>>> It
>>> >>>>>> >>>>>> used to
>>> >>>>>> >>>>>> be handled by copyPhysicalDisk in the storage adaptor,
>>> >>>>>> >>>>>> called
>>> >>>>>> >>>>>> by
>>> >>>>>> >>>>>> copyTemplateToPrimaryStorage, which runs on the agent. It
>>> >>>>>> >>>>>> would
>>> >>>>>> >>>>>> pass
>>> >>>>>> >>>>>> template/secondary storage info, and the destination
>>> >>>>>> >>>>>> volume/primary
>>> >>>>>> >>>>>> storage info, and copyPhysicalDisk would do the work of
>>> >>>>>> >>>>>> installing the
>>> >>>>>> >>>>>> image to the destination. Then subsequent root disks would
>>> >>>>>> >>>>>> be
>>> >>>>>> >>>>>> cloned
>>> >>>>>> >>>>>> in CreateCommand by calling createDiskFromTemplate.
>>> >>>>>> >>>>>>
>>> >>>>>> >>>>>> In master it looks like this was moved to
>>> >>>>>> >>>>>> KVMStorageProcessor
>>> >>>>>> >>>>>> 'cloneVolumeFromBaseTemplate', although I think this just
>>> >>>>>> >>>>>> takes
>>> >>>>>> >>>>>> over
>>> >>>>>> >>>>>> as default, and there's something in your storage driver
>>> >>>>>> >>>>>> that
>>> >>>>>> >>>>>> should
>>> >>>>>> >>>>>> be capable of cloning templates on the mgmt server side.
>>> >>>>>> >>>>>> I'm
>>> >>>>>> >>>>>> less sure
>>> >>>>>> >>>>>> about how the template gets to primary storage in the first
>>> >>>>>> >>>>>> place, I
>>> >>>>>> >>>>>> assume copyTemplateToPrimaryStorage in KVMStorageProcessor
>>> >>>>>> >>>>>> calling
>>> >>>>>> >>>>>> copyPhysicalDisk in your adaptor. It's a bit tough for me
>>> >>>>>> >>>>>> to
>>> >>>>>> >>>>>> tell
>>> >>>>>> >>>>>> since our earlier storage adaptor did everything on the
>>> >>>>>> >>>>>> host it
>>> >>>>>> >>>>>> mostly
>>> >>>>>> >>>>>> just worked with the default stuff.
>>> >>>>>> >>>>>>
>>> >>>>>> >>>>>> On Mon, Oct 21, 2013 at 4:49 PM, Mike Tutkowski
>>> >>>>>> >>>>>> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> >>>>>> > Hey Marcus,
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> > So...now that this works well for data disks, I was
>>> >>>>>> >>>>>> > wondering
>>> >>>>>> >>>>>> > what
>>> >>>>>> >>>>>> > might be
>>> >>>>>> >>>>>> > involved in getting this process to work for root disks.
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> > Can you point me in the right direction as far as what
>>> >>>>>> >>>>>> > gets
>>> >>>>>> >>>>>> > invoked
>>> >>>>>> >>>>>> > when a
>>> >>>>>> >>>>>> > VM is being created on KVM (so that its root disk can be
>>> >>>>>> >>>>>> > created and
>>> >>>>>> >>>>>> > the
>>> >>>>>> >>>>>> > necessary template laid down or ISO installed)?
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> > Thanks!
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> > On Mon, Oct 21, 2013 at 1:14 PM, Mike Tutkowski
>>> >>>>>> >>>>>> > <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> Hey Marcus,
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> Just wanted to let you know the branch of mine that has
>>> >>>>>> >>>>>> >> your
>>> >>>>>> >>>>>> >> code
>>> >>>>>> >>>>>> >> and mine
>>> >>>>>> >>>>>> >> appears to work well with regards to attaching a data
>>> >>>>>> >>>>>> >> disk
>>> >>>>>> >>>>>> >> to a
>>> >>>>>> >>>>>> >> running VM:
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> fdisk -l from hypervisor:
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> http://i.imgur.com/NkP5fo0.png
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> fdisk -l from within VM:
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> http://i.imgur.com/8YwiiC7.png
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> I plan to do more testing on this over the coming days.
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> If all goes well, perhaps we can check this code in by
>>> >>>>>> >>>>>> >> the
>>> >>>>>> >>>>>> >> end of
>>> >>>>>> >>>>>> >> the
>>> >>>>>> >>>>>> >> week?
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> Talk to you later,
>>> >>>>>> >>>>>> >> Mike
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> On Sun, Oct 20, 2013 at 10:23 PM, Mike Tutkowski
>>> >>>>>> >>>>>> >> <mike.tutkow...@solidfire.com> wrote:
>>> >>>>>> >>>>>> >>>
>>> >>>>>> >>>>>> >>> Don't ask me, but it works now (I've been having this
>>> >>>>>> >>>>>> >>> trouble
>>> >>>>>> >>>>>> >>> quite a
>>> >>>>>> >>>>>> >>> while today).
>>> >>>>>> >>>>>> >>>
>>> >>>>>> >>>>>> >>> I guess the trick is to send you an e-mail. :)
>>> >>>>>> >>>>>> >>>
>>> >>>>>> >>>>>> >>>
>>> >>>>>> >>>>>> >>> On Sun, Oct 20, 2013 at 10:05 PM, Marcus Sorensen
>>> >>>>>> >>>>>> >>> <shadow...@gmail.com>
>>> >>>>>> >>>>>> >>> wrote:
>>> >>>>>> >>>>>> >>>>
>>> >>>>>> >>>>>> >>>> Did you create a service offering that uses local
>>> >>>>>> >>>>>> >>>> storage,
>>> >>>>>> >>>>>> >>>> or add
>>> >>>>>> >>>>>> >>>> a
>>> >>>>>> >>>>>> >>>> shared primary storage? By default there is no storage
>>> >>>>>> >>>>>> >>>> that
>>> >>>>>> >>>>>> >>>> matches the
>>> >>>>>> >>>>>> >>>> built in offerings.
>>> >>>>>> >>>>>> >>>>
>>> >>>>>> >>>>>> >>>> On Oct 20, 2013 9:39 PM, "Mike Tutkowski"
>>> >>>>>> >>>>>> >>>> <mike.tutkow...@solidfire.com>
>>> >>>>>> >>>>>> >>>> wrote:
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> Hey Marcus,
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> So, I went back to the branch of mine that has your
>>> >>>>>> >>>>>> >>>>> code
>>> >>>>>> >>>>>> >>>>> and
>>> >>>>>> >>>>>> >>>>> mine and
>>> >>>>>> >>>>>> >>>>> was able to create a new CloudStack install from
>>> >>>>>> >>>>>> >>>>> scratch
>>> >>>>>> >>>>>> >>>>> with it
>>> >>>>>> >>>>>> >>>>> (once
>>> >>>>>> >>>>>> >>>>> again, after manually deleting what was in
>>> >>>>>> >>>>>> >>>>> /var/lib/libvirt/images to the
>>> >>>>>> >>>>>> >>>>> get system VMs to start).
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> Anyways, my system VMs are running now and I tried to
>>> >>>>>> >>>>>> >>>>> kick off a
>>> >>>>>> >>>>>> >>>>> VM
>>> >>>>>> >>>>>> >>>>> using the CentOS 6.3 image you provided me a while
>>> >>>>>> >>>>>> >>>>> back.
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> The virtual router has a Status of Running; however,
>>> >>>>>> >>>>>> >>>>> my
>>> >>>>>> >>>>>> >>>>> VM fails
>>> >>>>>> >>>>>> >>>>> to
>>> >>>>>> >>>>>> >>>>> start (with the generic message of Insufficient
>>> >>>>>> >>>>>> >>>>> Capacity).
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> I've not seen this exception before (related to the
>>> >>>>>> >>>>>> >>>>> VR).
>>> >>>>>> >>>>>> >>>>> Do you
>>> >>>>>> >>>>>> >>>>> have
>>> >>>>>> >>>>>> >>>>> any insight into this?:
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.exception.ResourceUnavailableException:
>>> >>>>>> >>>>>> >>>>> Resource
>>> >>>>>> >>>>>> >>>>> [Pod:1] is
>>> >>>>>> >>>>>> >>>>> unreachable: Unable to apply userdata and password
>>> >>>>>> >>>>>> >>>>> entry
>>> >>>>>> >>>>>> >>>>> on
>>> >>>>>> >>>>>> >>>>> router
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3793)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyUserData(VirtualNetworkApplianceManagerImpl.java:3017)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.network.element.VirtualRouterElement.addPasswordAndUserdata(VirtualRouterElement.java:933)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElement(NetworkOrchestrator.java:1172)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(NetworkOrchestrator.java:1288)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1224)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:826)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:508)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3338)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2919)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2905)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:421)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$1.runInContext(AsyncJobManagerImpl.java:532)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>> >>>>>> >>>>>> >>>>> at
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>> >>>>>> >>>>>> >>>>> at java.lang.Thread.run(Thread.java:724)
>>> >>>>>> >>>>>> >>>>>
>>> >>>>>> >>>>>> >>>>> Thanks!
>>> >>>>>> >>>>>> >>>
>>> >>>>>> >>>>>> >>>
>>> >>>>>> >>>>>> >>>
>>> >>>>>> >>>>>> >>>
>>> >>>>>> >>>>>> >>> --
>>> >>>>>> >>>>>> >>> Mike Tutkowski
>>> >>>>>> >>>>>> >>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> >>>>>> >>> e: mike.tutkow...@solidfire.com
>>> >>>>>> >>>>>> >>> o: 303.746.7302
>>> >>>>>> >>>>>> >>> Advancing the way the world uses the cloud™
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >>
>>> >>>>>> >>>>>> >> --
>>> >>>>>> >>>>>> >> Mike Tutkowski
>>> >>>>>> >>>>>> >> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> >>>>>> >> e: mike.tutkow...@solidfire.com
>>> >>>>>> >>>>>> >> o: 303.746.7302
>>> >>>>>> >>>>>> >> Advancing the way the world uses the cloud™
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> >
>>> >>>>>> >>>>>> > --
>>> >>>>>> >>>>>> > Mike Tutkowski
>>> >>>>>> >>>>>> > Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> >>>>>> > e: mike.tutkow...@solidfire.com
>>> >>>>>> >>>>>> > o: 303.746.7302
>>> >>>>>> >>>>>> > Advancing the way the world uses the cloud™
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> --
>>> >>>>>> >>>>> Mike Tutkowski
>>> >>>>>> >>>>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> >>>>> e: mike.tutkow...@solidfire.com
>>> >>>>>> >>>>> o: 303.746.7302
>>> >>>>>> >>>>> Advancing the way the world uses the cloud™
>>> >>>>>> >>>>
>>> >>>>>> >>>>
>>> >>>>>> >>>>
>>> >>>>>> >>>>
>>> >>>>>> >>>> --
>>> >>>>>> >>>> Mike Tutkowski
>>> >>>>>> >>>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> >>>> e: mike.tutkow...@solidfire.com
>>> >>>>>> >>>> o: 303.746.7302
>>> >>>>>> >>>> Advancing the way the world uses the cloud™
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>> --
>>> >>>>>> >>> Mike Tutkowski
>>> >>>>>> >>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> >>> e: mike.tutkow...@solidfire.com
>>> >>>>>> >>> o: 303.746.7302
>>> >>>>>> >>> Advancing the way the world uses the cloud™
>>> >>>>>> >
>>> >>>>>> >
>>> >>>>>> >
>>> >>>>>> >
>>> >>>>>> > --
>>> >>>>>> > Mike Tutkowski
>>> >>>>>> > Senior CloudStack Developer, SolidFire Inc.
>>> >>>>>> > e: mike.tutkow...@solidfire.com
>>> >>>>>> > o: 303.746.7302
>>> >>>>>> > Advancing the way the world uses the cloud™
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> Mike Tutkowski
>>> >>>>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>>> e: mike.tutkow...@solidfire.com
>>> >>>>> o: 303.746.7302
>>> >>>>> Advancing the way the world uses the cloud™
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> Mike Tutkowski
>>> >>>> Senior CloudStack Developer, SolidFire Inc.
>>> >>>> e: mike.tutkow...@solidfire.com
>>> >>>> o: 303.746.7302
>>> >>>> Advancing the way the world uses the cloud™
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Mike Tutkowski
>>> >> Senior CloudStack Developer, SolidFire Inc.
>>> >> e: mike.tutkow...@solidfire.com
>>> >> o: 303.746.7302
>>> >> Advancing the way the world uses the cloud™
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Mike Tutkowski
>>> > Senior CloudStack Developer, SolidFire Inc.
>>> > e: mike.tutkow...@solidfire.com
>>> > o: 303.746.7302
>>> > Advancing the way the world uses the cloud™
>>
>>
>>
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud™
>
>
>
>
> --
> Mike Tutkowski
> Senior CloudStack Developer, SolidFire Inc.
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud™
