James,
I will be on IRC later tonight, we can try to debug this.
-Sebastien
On Sep 30, 2012, at 9:18 PM, James Martin wrote:
> in my devcloud.cfg file (attached) I have:
>
> "pods": [
> {
> "endip": "10.0.2.220",
> "name": "test00",
> "startip": "10.0.2.200",
> "guestIpRanges": [
> {
> "startip": "10.0.2.100",
> "endip": "10.0.2.199",
> "netmask": "255.255.255.0",
> "gateway": "10.0.2.2"
> }
> ],
> "netmask": "255.255.255.0",
> "clusters": [
> {
> "clustername": "test000",
> "hypervisor": "XenServer",
> "hosts": [
> {
> "username": "root",
> "url": "http://10.0.2.15/";,
> "password": "password"
> }
> ],
> "clustertype": "CloudManaged",
> "primaryStorages": [
> {
> "url":
> "nfs://10.0.2.15:/opt/storage/primary",
> "name": "PS0"
> } ]
> }
> ],
> "gateway": "10.0.2.2"
> }
> ],
>
>
> On Sun, Sep 30, 2012 at 3:08 PM, Kelceydamage@bbits <kel...@bbits.ca> wrote:
>> Seems odd but you could be short IPs for the POD pool.
>>
>> Sent from my iPhone
>>
>> On Sep 30, 2012, at 11:58 AM, "Kelceydamage@bbits" <kel...@bbits.ca> wrote:
>>
>>> The database check the IP allocation and network pools alongside the
>>> storage pool
>>>
>>> Sent from my iPhone
>>>
>>> On Sep 30, 2012, at 11:56 AM, James Martin <jmar...@basho.com> wrote:
>>>
>>>> Primary storage and Secondary storage are good according to the GUI.
>>>> Is that what you are referring to?
>>>>
>>>> James
>>>>
>>>> On Sun, Sep 30, 2012 at 2:48 PM, Kelceydamage@bbits <kel...@bbits.ca>
>>>> wrote:
>>>>> Just winging it here but for scope interface errors check that your pools
>>>>> are not completely allocated.
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Sep 30, 2012, at 11:21 AM, James Martin <jmar...@basho.com> wrote:
>>>>>
>>>>>> I think I fixed one problem but solved another.. I had to add
>>>>>>
>>>>>> {
>>>>>> "name":"system.vm.use.local.storage",
>>>>>> "value":"false"
>>>>>> },
>>>>>>
>>>>>> to the devcloud.cfg and no longer receive that error.
>>>>>>
>>>>>> My SSVMs and CPVMs still aren't starting. I've attached a big log
>>>>>> that should capture the context of the errors.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> James
>>>>>>
>>>>>> On Sun, Sep 30, 2012 at 11:54 AM, Ahmad Emneina
>>>>>> <ahmad.emne...@citrix.com> wrote:
>>>>>>> Hey James, can you post more logs. I'm interested to see the logs above
>>>>>>> what was pasted in the previous email. I think the real reason hides
>>>>>>> there...
>>>>>>>
>>>>>>> On 9/30/12 8:50 AM, "James Martin" <jmar...@basho.com> wrote:
>>>>>>>
>>>>>>>> This was because I did have the nfs service subscribing to
>>>>>>>> /etc/exports in the puppet recipe.
>>>>>>>>
>>>>>>>> I'm back to the same error as before:
>>>>>>>>
>>>>>>>>
>>>>>>>> 2012-09-30 10:16:49,695 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>>>>>> (consoleproxy-1:null) VM state transitted from :Starting to Stopped
>>>>>>>> with event: OperationFailedvm's original host id: null new host id:
>>>>>>>> null host id before state transition: null
>>>>>>>> 2012-09-30 10:16:49,703 WARN
>>>>>>>> [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null)
>>>>>>>> Exception while trying to start console proxy
>>>>>>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>>>>>>> create a deployment for VM[ConsoleProxy|v-2-TEST]Scope=interface
>>>>>>>> com.cloud.dc.DataCenter; id=1
>>>>>>>> at
>>>>>>>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerI
>>>>>>>> mpl.java:734)
>>>>>>>> at
>>>>>>>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.jav
>>>>>>>> a:472)
>>>>>>>> at
>>>>>>>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.jav
>>>>>>>> a:465)
>>>>>>>> at
>>>>>>>> com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyMana
>>>>>>>> gerImpl.java:624)
>>>>>>>> at
>>>>>>>> com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyM
>>>>>>>> anagerImpl.java:1154)
>>>>>>>> at
>>>>>>>> com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyMana
>>>>>>>> gerImpl.java:1961)
>>>>>>>> at
>>>>>>>> com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyMana
>>>>>>>> gerImpl.java:170)
>>>>>>>> at
>>>>>>>> com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:113)
>>>>>>>> at
>>>>>>>> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:34)
>>>>>>>> at
>>>>>>>> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:83)
>>>>>>>> at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:73)
>>>>>>>> at
>>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>>>>> at
>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acces
>>>>>>>> s$201(ScheduledThreadPoolExecutor.java:165)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(S
>>>>>>>> cheduledThreadPoolExecutor.java:267)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>>>>>>>> 1110)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>>>>>>>> :603)
>>>>>>>> at java.lang.Thread.run(Thread.java:679)
>>>>>>>> 2012-09-30 10:16:49,730 DEBUG [cloud.storage.StorageManagerImpl]
>>>>>>>> (secstorage-1:null) Expunging Vol[1|vm=1|ROOT]
>>>>>>>> 2012-09-30 10:16:49,742 DEBUG [cloud.storage.StorageManagerImpl]
>>>>>>>> (secstorage-1:null) Marking volume that was never created as
>>>>>>>> destroyed: Vol[1|vm=1|ROOT]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Sep 30, 2012 at 11:28 AM, James Martin <jmar...@basho.com>
>>>>>>>> wrote:
>>>>>>>>> My SSVM never starts (or the CPVM for the matter). In the GUI I have
>>>>>>>>>
>>>>>>>>> Zones: 1
>>>>>>>>> Pods: 1
>>>>>>>>> Clusters: 1
>>>>>>>>> Hosts : 1
>>>>>>>>> Primary Storage : 0
>>>>>>>>> Secondary Storage : 1
>>>>>>>>> System VMs : 0
>>>>>>>>> Virtual Routers : 0
>>>>>>>>>
>>>>>>>>> Also, These messages keep repeating in the logs:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2012-09-30 09:51:01,012 DEBUG [storage.snapshot.SnapshotSchedulerImpl]
>>>>>>>>> (SnapshotPollTask:null) Snapshot scheduler.poll is being called at
>>>>>>>>> 2012-09-30 09:51:01 GMT
>>>>>>>>> 2012-09-30 09:51:01,013 DEBUG [storage.snapshot.SnapshotSchedulerImpl]
>>>>>>>>> (SnapshotPollTask:null) Got 0 snapshots to be executed at 2012-09-30
>>>>>>>>> 09:51:01 GMT
>>>>>>>>> 2012-09-30 09:51:01,055 DEBUG
>>>>>>>>> [cloud.network.ExternalLoadBalancerUsageManagerImpl]
>>>>>>>>> (ExternalNetworkMonitor-1:null) External load balancer devices stats
>>>>>>>>> collector is running...
>>>>>>>>> 2012-09-30 09:51:01,106 DEBUG
>>>>>>>>> [network.router.VirtualNetworkApplianceManagerImpl]
>>>>>>>>> (RouterMonitor-1:null) Found 0 running routers.
>>>>>>>>> 2012-09-30 09:51:01,120 DEBUG
>>>>>>>>> [network.router.VirtualNetworkApplianceManagerImpl]
>>>>>>>>> (RouterStatusMonitor-1:null) Found 0 routers.
>>>>>>>>> 2012-09-30 09:51:15,064 DEBUG [cloud.server.StatsCollector]
>>>>>>>>> (StatsCollector-1:null) StorageCollector is running...
>>>>>>>>> 2012-09-30 09:51:15,066 DEBUG [cloud.server.StatsCollector]
>>>>>>>>> (StatsCollector-1:null) There is no secondary storage VM for secondary
>>>>>>>>> storage host nfs://10.0.2.15:/opt/storage/secondary
>>>>>>>>> 2012-09-30 09:51:15,072 DEBUG [cloud.server.StatsCollector]
>>>>>>>>> (StatsCollector-2:null) VmStatsCollector is running...
>>>>>>>>> 2012-09-30 09:51:15,943 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>>> (DirectAgent-1:null) Ping from 1
>>>>>>>>> 2012-09-30 09:51:16,706 DEBUG [cloud.server.StatsCollector]
>>>>>>>>> (StatsCollector-1:null) HostStatsCollector is running...
>>>>>>>>> 2012-09-30 09:51:16,714 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>>> (DirectAgent-14:null) Seq 1-164560912: Executing request
>>>>>>>>> 2012-09-30 09:51:16,874 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>>> (DirectAgent-14:null) Seq 1-164560912: Response Received:
>>>>>>>>> 2012-09-30 09:51:16,875 DEBUG [agent.transport.Request]
>>>>>>>>> (StatsCollector-1:null) Seq 1-164560912: Received: { Ans: , MgmtId:
>>>>>>>>> 8796761203466, via: 1, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>>>>>>>>> 2012-09-30 09:51:18,640 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>>> (DirectAgent-31:null) Seq 1-164560898: Executing request
>>>>>>>>> 2012-09-30 09:51:18,797 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>>> (DirectAgent-31:null) Seq 1-164560898: Response Received:
>>>>>>>>> 2012-09-30 09:51:18,798 DEBUG [agent.transport.Request]
>>>>>>>>> (DirectAgent-31:null) Seq 1-164560898: Processing: { Ans: , MgmtId:
>>>>>>>>> 8796761203466, via: 1, Ver: v1, Flags: 10,
>>>>>>>>>
>>>>>>>>> [{"ClusterSyncAnswer":{"_clusterId":1,"_newStates":{},"_isExecuted":false
>>>>>>>>> ,"result":true,"wait":0}}]
>>>>>>>>> }
>>>>>>>>> 2012-09-30 09:51:30,279 DEBUG
>>>>>>>>> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
>>>>>>>>> Primary storage is not ready, wait until it is ready to launch
>>>>>>>>> secondary storage vm
>>>>>>>>> 2012-09-30 09:51:30,279 DEBUG
>>>>>>>>> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
>>>>>>>>> Zone 1 is not ready to launch secondary storage VM yet
>>>>>>>>> 2012-09-30 09:51:30,568 DEBUG
>>>>>>>>> [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null)
>>>>>>>>> Skip capacity scan due to there is no Primary Storage UPintenance mode
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> James
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Æ
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> <log.txt>
