I actually did upgrade from 4.1.1 to 4.2, but had to roll back.
http://comments.gmane.org/gmane.comp.apache.cloudstack.user/7387

After the roll back, that's when my company moved to a new office and
changed networks. In the middle of me trying to get 4.1.1 back up again
after the move, I did upgrade to 4.2 (again) to see if I would have more
luck this time. So I am actually running on 4.2 now. I still cannot deploy
my system VMs however, despite following these instructions:

http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conquering-the-cloudstack-4-2-dragon-kvm/

I actually performed that procedure several times thinking that perhaps I
missed something. The very first time I ran it, I could not umount /mnt
because I'm self contained, and my /export/primary storage is mounted there
as well. So... I realized that those mounting/umounting steps are merely to
gain access to the /export/secondary/Template/tmpl/1/3/ files, which are
local to my box. So the second time I performed the procedure, I simply
renamed the *.qcow2 file to match what was in my database.

mv 83f8aec6-9446-4fe5-bc9e-258d25c9348a.qcow2
5c331a9a-8d48-4e53-b9fa-b5e7655b9175.qcow2

Restarted my entire server (probably overkill, but I'm desperate) but that
still didn't fix it. I still cannot start system VMs.

I created a clean management-server.log file and found that it now thinks I
have 3 orphaned management hosts.

2013-10-29 14:48:41,679 DEBUG [utils.db.ConnectionConcierge]
(Cluster-Heartbeat-1:null) Registering a database connection for
ClusterManagerHeartBeat2
2013-10-29 14:48:41,691 INFO  [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) Add orphan management server msid found in host
table to initial clustering notification, orphan msid: 279278805450800
2013-10-29 14:48:41,691 INFO  [utils.component.ComponentContext]
(main:null) Configuring
com.cloud.bridge.service.controller.s3.ServiceProvider_EnhancerByCloudStack_f278348c
2013-10-29 14:48:41,691 INFO  [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) Add orphan management server msid found in host
table to initial clustering notification, orphan msid: 279278805451095
2013-10-29 14:48:41,691 INFO  [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) Add orphan management server msid found in host
table to initial clustering notification, orphan msid: 279278805451190
2013-10-29 14:48:41,691 INFO  [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) Found 3 inactive management server node based on
timestamp
2013-10-29 14:48:41,691 INFO  [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) management server node msid: 279278805450800,
name: null, service ip: orphan, version: null
2013-10-29 14:48:41,691 INFO  [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) management server node msid: 279278805451095,
name: null, service ip: orphan, version: null
2013-10-29 14:48:41,691 INFO  [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) management server node msid: 279278805451190,
name: null, service ip: orphan, version: null
2013-10-29 14:48:41,691 INFO  [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) Trying to connect to orphan
2013-10-29 14:48:41,693 ERROR [cloud.cluster.ClusterManagerImpl]
(Cluster-Heartbeat-1:null) Unexpected exception in cluster heartbeat
java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:127)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:640)
at
com.cloud.cluster.ClusterManagerImpl.pingManagementNode(ClusterManagerImpl.java:1405)
at
com.cloud.cluster.ClusterManagerImpl.initPeerScan(ClusterManagerImpl.java:1036)
at
com.cloud.cluster.ClusterManagerImpl.access$1100(ClusterManagerImpl.java:95)
at com.cloud.cluster.ClusterManagerImpl$4.run(ClusterManagerImpl.java:786)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
2013-10-29 14:48:41,699 DEBUG [utils.db.ConnectionConcierge]
(Cluster-Heartbeat-1:null) Registering a database connection for
ClusterManagerHeartBeat2


I kept looking through the logs and found this error repeated several times:


2013-10-29 14:49:40,072 INFO  [cloud.vm.VirtualMachineManagerImpl]
(consoleproxy-1:null) Insufficient capacity
com.cloud.exception.InsufficientAddressCapacityException: Insufficient
address capacityScope=interface com.cloud.dc.Pod; id=1
at
com.cloud.network.NetworkManagerImpl.fetchNewPublicIp(NetworkManagerImpl.java:472)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
com.cloud.network.NetworkManagerImpl.assignPublicIpAddress(NetworkManagerImpl.java:376)
at
com.cloud.network.guru.DirectPodBasedNetworkGuru.getIp(DirectPodBasedNetworkGuru.java:186)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
com.cloud.network.guru.DirectPodBasedNetworkGuru.reserve(DirectPodBasedNetworkGuru.java:153)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
com.cloud.network.NetworkManagerImpl.prepareNic(NetworkManagerImpl.java:2157)
at
com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2127)
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:887)
at
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
at
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:571)
at
com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556)
at
com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:928)
at
com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1672)
at
com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157)
at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111)
at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)


I've checked my Zone, Pod, Cluster, Network settings, CIDRs, Management IP
Range, Guest IP Range, etc... a ton of times now, and cannot find out where
the problem may be.

Since the problem began when we moved the self-contained server from one
network to another, and now the errors suggest that I do not have enough
IPs (and I'm assuming it means I do not have enough public IPs based on the
stack trace "at
com.cloud.network.NetworkManagerImpl.fetchNewPublicIp(NetworkManagerImpl.java:472)"),
what table(s) do I need to look at and potentially clear out to fix this
issue?

-Adam


Best Regards,



Adam Scarcella


On Tue, Oct 29, 2013 at 11:59 AM, Andrija Panic <[email protected]>wrote:

> my bad for secondary storage -  there is no maintance mode for that :)
>
> I was just refering to line:
>
> host=10.97.38.10 - which is hopefully new IP address :)
>
> Since you got Server capactity issues, I suggest folowing, as temp
> solution:
>
> Check CPU alocation, RAM alocation, and increase overprovisioning factor at
> POD level (at least this is in 4.2, or only in global config as was in
> 4.0.0), so if you are running out of CPU/RAM resopurces, you might be able
> to fix this this way.
>
> I had this exact type of problem (SSVM not starting), when upgraded from
> 4,0 to 4.2,because of bad SSVM template  - I had to download
>
> http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2(previously
> I had very old, which came with Cs 4.0.0) - but if you did not
> upgrade your CS, then I suppose this is not the case...
>
> Best
>
>
>
>
>
> On 29 October 2013 16:42, Adam <[email protected]> wrote:
>
> > Hi Andrija,
> >
> > Thanks for your reply.
> >
> > Here is the output of my agent.properties file (which looks correct to
> me):
> >
> > [root@cs-east-prod1 ~]# cat /etc/cloudstack/agent/agent.properties
> > #Storage
> > #Tue Oct 29 10:15:49 GMT-05:00 2013
> > guest.network.device=cloudbr0
> > workers=5
> > private.network.device=cloudbr0
> > port=8250
> > resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource
> > pod=1
> > zone=1
> > guid=db24e08f-ac6e-3974-b223-274d49af52d0
> > cluster=1
> > public.network.device=cloudbr0
> > local.storage.uuid=296f0404-9658-4269-bc24-64b1247d5e3f
> > domr.scripts.dir=scripts/network/domr/kvm
> > host=10.97.38.10
> > LibvirtComputingResource.id=10
> >
> > [root@cs-east-prod1 ~]# ifconfig cloudbr0
> > cloudbr0  Link encap:Ethernet  HWaddr AC:16:2D:84:D4:24
> >           inet addr:10.97.38.10  Bcast:10.97.38.255  Mask:255.255.255.0
> >           inet6 addr: fe80::ae16:2dff:fe84:d424/64 Scope:Link
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:10216 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:10343 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:1605339 (1.5 MiB)  TX bytes:24242332 (23.1 MiB)
> >
> >
> > I have 4.5 TB free on my 5.7 TB RAID so I haven't even come close to the
> > "pool.storage.allocated.capacity.disablethreshold" or
> > "pool.storage.capacity.disablethreshold" of "0.85"
> >
> > [root@cs-east-prod1 ~]# df -kh
> > Filesystem            Size  Used Avail Use% Mounted on
> > /dev/mapper/vg_cloudstackuse-lv_root
> >                       5.7T  921G  4.5T  17% /
> > tmpfs                  64G   72K   64G   1% /dev/shm
> > /dev/sda1             485M  118M  342M  26% /boot
> > 10.97.38.10:/export/primary
> >                       5.7T  921G  4.5T  17%
> > /mnt/910365e6-a298-37b8-904f-2b49198c8286
> > [root@cs-east-prod1 ~]# cd /export/
> > [root@cs-east-prod1 export]# ls -l
> > total 8
> > drwxrwxrwx. 3 root root 4096 Oct 27 19:18 primary
> > drwxrwxrwx. 5 root root 4096 Jul 16 11:27 secondary
> > [root@cs-east-prod1 export]# du -sh *
> > 238G    primary
> > 110G    secondary
> > [root@cs-east-prod1 export]#
> >
> > I've manually destroyed the SSVM several times and yes CS does attempt to
> > create a new one, but the same issue occurs. Regardless of whether I
> > destroy the SSVM or if CS does it, it just keeps looping.
> >
> > I'm not exactly sure how to check if my secondary storage is in
> Maintenance
> > Mode. I know I can enable Maintenance Mode of my Primary Storage, but how
> > to check for the Secondary Storage?
> >
> > Best Regards,
> >
> >
> >
> > Adam Scarcella
> >
> >
> > On Tue, Oct 29, 2013 at 11:16 AM, Andrija Panic <[email protected]
> > >wrote:
> >
> > > I suppose, you can recheck the network, and also check the AGENT
> > > configuration file, make changes
> > > there: /etc/cloudstack/agent/agent.properties
> > >
> > > From logs:
> > >
> > > InsufficientServerCapacityException: Unable to create
> > > > a deployment for VM.....
> > >
> > > I had this kind of error, when my storage pools were disabled once, and
> > > also when I have reached "disable capacity treshold" - my storage pool
> > was
> > > over 85% full.
> > >
> > > So check if you secondaru storge is in maintance mode, or similar...
> > >
> > > Also, destroying SSVM will force CS into making new one...
> > >
> > >
> > > On 29 October 2013 16:09, Adam <[email protected]> wrote:
> > >
> > > > Hi Geoff,
> > > >
> > > > Thanks for the reply, I was able to manually mount the secondary
> > storage
> > > > NFS:
> > > >
> > > > [root@cs-east-prod1 secondaryStorage]# hostname
> > > > cs-east-prod1
> > > > [root@cs-east-prod1 secondaryStorage]# hostname --fqdn
> > > > cs-east-prod1.na.tibco.com
> > > > [root@cs-east-prod1 secondaryStorage]#
> > > > [root@cs-east-prod1 ~]# mkdir /secondaryStorage
> > > > [root@cs-east-prod1 ~]# mount -t nfs cs-east-prod1:/export/secondary
> > > > /secondaryStorage
> > > > [root@cs-east-prod1 ~]# cd /
> > > > [root@cs-east-prod1 /]# ls -l
> > > > total 120
> > > > dr-xr-xr-x.   2 root root  4096 Oct 23 04:48 bin
> > > > dr-xr-xr-x.   5 root root  3072 Oct 22 21:10 boot
> > > > drwxr-xr-x.  10 root root  4096 Jun 25 10:23 cgroup
> > > > drwxr-xr-x.  19 root root  3960 Oct 29 10:38 dev
> > > > drwxr-xr-x. 121 root root 12288 Oct 29 11:07 etc
> > > > drwxrwxrwx.   4 root root  4096 Jul  3 20:10 export
> > > > drwxr-xr-x.   3 root root  4096 Jul  3 19:14 home
> > > > lrwxrwxrwx.   1 root root    25 Aug 28 15:46 Installers ->
> > > > /var/www/html/Installers/
> > > > lrwxrwxrwx.   1 root root    19 Jul  5 20:58 ISOs ->
> > /var/www/html/ISOs/
> > > > dr-xr-xr-x.  11 root root  4096 Jul  3 19:02 lib
> > > > dr-xr-xr-x.  10 root root 12288 Oct 28 03:33 lib64
> > > > drwx------.   2 root root 16384 Jul  3 18:44 lost+found
> > > > drwxr-xr-x.   3 root root  4096 Oct 27 12:05 media
> > > > drwxr-xr-x.   2 root root     0 Oct 29 10:08 misc
> > > > drwxr-xr-x.  39 root root  4096 Sep  6 12:07 mnt
> > > > drwxr-xr-x.   2 root root     0 Oct 29 10:08 net
> > > > drwxr-xr-x.   3 root root  4096 Jul  3 19:05 opt
> > > > drwxr-xr-x.   2 root root  4096 Oct 27 10:06 primarymount
> > > > dr-xr-xr-x. 557 root root     0 Oct 29 10:08 proc
> > > > dr-xr-x---.  35 root root  4096 Oct 29 10:09 root
> > > > dr-xr-xr-x.   2 root root 12288 Oct 28 03:33 sbin
> > > > drwxr-xr-x.   2 root root  4096 Oct 27 10:07 secondarymount
> > > > drwxrwxrwx.   5 root root  4096 Jul 16 11:27 secondaryStorage
> > > > drwxr-xr-x.   7 root root     0 Oct 29 10:08 selinux
> > > > drwxr-xr-x.   2 root root  4096 Sep 23  2011 srv
> > > > drwxr-xr-x.  13 root root     0 Oct 29 10:08 sys
> > > > drwxrwxrwt.  16 root root  4096 Oct 29 11:07 tmp
> > > > drwxr-xr-x.  13 root root  4096 Jul  3 18:55 usr
> > > > drwxr-xr-x.  23 root root  4096 Jul  3 19:45 var
> > > > [root@cs-east-prod1 /]# cd secondaryStorage/
> > > > [root@cs-east-prod1 secondaryStorage]# ls -l
> > > > total 12
> > > > drwxrwxrwx. 2 root root 4096 Jul 16 11:27 snapshots
> > > > drwxrwxrwx. 3 root root 4096 Jul 16 11:10 template
> > > > drwxrwxrwx. 3 root root 4096 Jul 22 09:38 volumes
> > > > [root@cs-east-prod1 secondaryStorage]#
> > > >
> > > >
> > > >
> > > > Best Regards,
> > > >
> > > >
> > > >
> > > > Adam Scarcella
> > > >
> > > >
> > > > On Tue, Oct 29, 2013 at 10:54 AM, Geoff Higginbottom <
> > > > [email protected]> wrote:
> > > >
> > > > > Adam,
> > > > >
> > > > > System VMs looping and failing to create is often caused by the
> Host
> > > > being
> > > > > unable to access Sec Storage.
> > > > >
> > > > > Do a manual mount of the Sec Storage NFS on the Host to confirm it
> > can
> > > > > access it.
> > > > >
> > > > > Regards
> > > > >
> > > > > Geoff Higginbottom
> > > > >
> > > > > D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
> > > > >
> > > > > [email protected]
> > > > >
> > > > > -----Original Message-----
> > > > > From: Adam [mailto:[email protected]]
> > > > > Sent: 29 October 2013 14:44
> > > > > To: [email protected]
> > > > > Subject: Re: How to recover entire self contained CloudStack after
> > > major
> > > > > network changes?
> > > > >
> > > > > I exported the db as a *.sql file, did search/replace and that
> seemed
> > > to
> > > > > help a lot, but still not 100% out of the woods. Now I'm hitting
> the
> > > > issue
> > > > > where the System VMs (SecondaryStorageVm & ConsoleProxy) do not
> > start.
> > > > They
> > > > > just keep looping from creation to being expunged.
> > > > >
> > > > > My Zone is enabled, Pod is enabled, Cluster is enabled, Host is Up,
> > > > > Primary Storage is online:
> > > > >
> > > > > [root@cs-east-prod1 ~]# df -kh
> > > > > Filesystem            Size  Used Avail Use% Mounted on
> > > > > /dev/mapper/vg_cloudstackuse-lv_root
> > > > >                       5.7T  921G  4.5T  17% /
> > > > > tmpfs                  64G   72K   64G   1% /dev/shm
> > > > > /dev/sda1             485M  118M  342M  26% /boot
> > > > > 10.97.38.10:/export/primary
> > > > >                       5.7T  921G  4.5T  17%
> > > > > /mnt/910365e6-a298-37b8-904f-2b49198c8286
> > > > >
> > > > > Here is an excerpt from my most recent log:
> > > > >
> > > > > 2013-10-29 09:22:23,389 DEBUG
> > > > > [network.router.VirtualNetworkApplianceManagerImpl]
> > > > > (RouterStatusMonitor-1:null) Found 0 routers to update status.
> > > > > 2013-10-29 09:22:23,391 DEBUG
> > > > > [network.router.VirtualNetworkApplianceManagerImpl]
> > > > > (RouterStatusMonitor-1:null) Found 0 networks to update RvR status.
> > > > > 2013-10-29 09:22:23,441 DEBUG
> > > > > [network.router.VirtualNetworkApplianceManagerImpl]
> > > > > (RouterStatusMonitor-1:null) Found 0 routers to update status.
> > > > > 2013-10-29 09:22:23,443 DEBUG
> > > > > [network.router.VirtualNetworkApplianceManagerImpl]
> > > > > (RouterStatusMonitor-1:null) Found 0 networks to update RvR status.
> > > > > 2013-10-29 09:22:26,872 DEBUG [agent.manager.AgentManagerImpl]
> > > > > (AgentManager-Handler-5:null) Ping from 10
> > > > > 2013-10-29 09:22:38,509 DEBUG [cloud.server.StatsCollector]
> > > > > (StatsCollector-2:null) VmStatsCollector is running...
> > > > > 2013-10-29 09:22:38,708 DEBUG [cloud.server.StatsCollector]
> > > > > (StatsCollector-3:null) StorageCollector is running...
> > > > > 2013-10-29 09:22:38,716 INFO
> > >  [storage.endpoint.DefaultEndPointSelector]
> > > > > (StatsCollector-3:null) No running ssvm is found, so command will
> be
> > > sent
> > > > > to LocalHostEndPoint
> > > > > 2013-10-29 09:22:44,969 DEBUG [cloud.server.StatsCollector]
> > > > > (StatsCollector-1:null) HostStatsCollector is running...
> > > > > 2013-10-29 09:22:45,587 DEBUG [agent.transport.Request]
> > > > > (StatsCollector-1:null) Seq 10-768344083: Received:  { Ans: ,
> MgmtId:
> > > > > 86691596708691, via: 10, Ver: v1, Flags: 10, { GetHostStatsAnswer
> } }
> > > > > 2013-10-29 09:22:51,802 DEBUG
> > > > > [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> > > Zone
> > > > 1
> > > > > is ready to launch secondary storage VM
> > > > > 2013-10-29 09:22:51,807 INFO
> > > > >  [cloud.secstorage.PremiumSecondaryStorageManagerImpl]
> > > > (secstorage-1:null)
> > > > > No running secondary storage vms found in datacenter id=1, starting
> > one
> > > > > 2013-10-29 09:22:51,814 INFO
> > > > >  [storage.secondary.SecondaryStorageManagerImpl]
> (secstorage-1:null)
> > No
> > > > > stopped secondary storage vm is available, need to allocate a new
> > > > secondary
> > > > > storage vm
> > > > > 2013-10-29 09:22:51,817 DEBUG
> > > > > [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)
> > > > Assign
> > > > > secondary storage vm from a newly started instance for request from
> > > data
> > > > > center : 1
> > > > > 2013-10-29 09:22:51,837 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Found existing network configuration for
> offering
> > > > > [Network Offering
> > [6-Guest-DefaultSharedNetworkOfferingWithSGService]:
> > > > > Ntwk[204|Guest|6]
> > > > > 2013-10-29 09:22:51,837 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Releasing lock for
> > > > > Acct[6f5c60f0-ee29-11e2-b62c-ac162d84d424-system]
> > > > > 2013-10-29 09:22:51,842 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Found existing network configuration for
> offering
> > > > > [Network Offering [3-Control-System-Control-Network]:
> > > Ntwk[202|Control|3]
> > > > > 2013-10-29 09:22:51,842 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Releasing lock for
> > > > > Acct[6f5c60f0-ee29-11e2-b62c-ac162d84d424-system]
> > > > > 2013-10-29 09:22:51,846 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Found existing network configuration for
> offering
> > > > > [Network Offering [2-Management-System-Management-Network]:
> > > > > Ntwk[201|Management|2]
> > > > > 2013-10-29 09:22:51,847 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Releasing lock for
> > > > > Acct[6f5c60f0-ee29-11e2-b62c-ac162d84d424-system]
> > > > > 2013-10-29 09:22:51,851 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Found existing network configuration for
> offering
> > > > > [Network Offering [4-Storage-System-Storage-Network]:
> > > Ntwk[203|Storage|4]
> > > > > 2013-10-29 09:22:51,851 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Releasing lock for
> > > > > Acct[6f5c60f0-ee29-11e2-b62c-ac162d84d424-system]
> > > > > 2013-10-29 09:22:51,860 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (secstorage-1:null) Allocating entries for VM:
> > > > > VM[SecondaryStorageVm|s-4930-VM]
> > > > > 2013-10-29 09:22:51,865 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (secstorage-1:null) Allocating nics for
> > > VM[SecondaryStorageVm|s-4930-VM]
> > > > > 2013-10-29 09:22:51,866 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Allocating nic for vm
> > > > VM[SecondaryStorageVm|s-4930-VM]
> > > > > in network Ntwk[204|Guest|6] with requested profile
> > > > > NicProfile[0-0-null-null-null
> > > > > 2013-10-29 09:22:51,881 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Allocating nic for vm
> > > > VM[SecondaryStorageVm|s-4930-VM]
> > > > > in network Ntwk[202|Control|3] with requested profile null
> > > > > 2013-10-29 09:22:51,890 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Allocating nic for vm
> > > > VM[SecondaryStorageVm|s-4930-VM]
> > > > > in network Ntwk[201|Management|2] with requested profile null
> > > > > 2013-10-29 09:22:51,903 DEBUG [cloud.network.NetworkManagerImpl]
> > > > > (secstorage-1:null) Allocating nic for vm
> > > > VM[SecondaryStorageVm|s-4930-VM]
> > > > > in network Ntwk[203|Storage|4] with requested profile null
> > > > > 2013-10-29 09:22:51,915 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (secstorage-1:null) Allocating disks for
> > > VM[SecondaryStorageVm|s-4930-VM]
> > > > > 2013-10-29 09:22:51,922 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (secstorage-1:null) Allocation completed for VM:
> > > > > VM[SecondaryStorageVm|s-4930-VM]
> > > > > 2013-10-29 09:22:51,925 DEBUG
> > > > [cloud.alert.SecondaryStorageVmAlertAdapter]
> > > > > (secstorage-1:null) received secondary storage vm alert
> > > > > 2013-10-29 09:22:51,925 DEBUG
> > > > [cloud.alert.SecondaryStorageVmAlertAdapter]
> > > > > (secstorage-1:null) New secondary storage vm created, zone:
> > > > EastProdZone1,
> > > > > secStorageVm: s-4930-VM, public IP: null, private IP: null
> > > > > 2013-10-29 09:22:51,942 DEBUG [cloud.capacity.CapacityManagerImpl]
> > > > > (secstorage-1:null) VM state transitted from :Stopped to Starting
> > with
> > > > > event: StartRequestedvm's original host id: null new host id: null
> > host
> > > > id
> > > > > before state transition: null
> > > > > 2013-10-29 09:22:51,942 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (secstorage-1:null) Successfully transitioned to start state for
> > > > > VM[SecondaryStorageVm|s-4930-VM] reservation id =
> > > > > 3b2435a0-9e32-4777-a42f-a766a375bc1b
> > > > > 2013-10-29 09:22:51,944 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (secstorage-1:null) Trying to deploy VM, vm has dcId: 1 and podId:
> > null
> > > > > 2013-10-29 09:22:51,944 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (secstorage-1:null) Deploy avoids pods: null, clusters: null,
> hosts:
> > > null
> > > > > 2013-10-29 09:22:51,945 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (secstorage-1:null) Deploy avoids pods: null, clusters: null,
> hosts:
> > > null
> > > > > 2013-10-29 09:22:51,946 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (secstorage-1:null) DeploymentPlanner allocation algorithm:
> > > > >
> > com.cloud.deploy.FirstFitPlanner_EnhancerByCloudStack_d4b92e44@713baeaa
> > > > > 2013-10-29 09:22:51,946 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (secstorage-1:null) Trying to allocate a host and storage pools
> from
> > > > dc:1,
> > > > > pod:null,cluster:null, requested cpu: 500, requested ram: 268435456
> > > > > 2013-10-29 09:22:51,946 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (secstorage-1:null) Is ROOT volume READY (pool already allocated)?:
> > No
> > > > > 2013-10-29 09:22:51,946 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (secstorage-1:null) Searching all possible resources under this
> > Zone: 1
> > > > > 2013-10-29 09:22:51,946 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (secstorage-1:null) Listing clusters in order of aggregate
> capacity,
> > > that
> > > > > have (atleast one host with) enough CPU and RAM capacity under this
> > > > Zone: 1
> > > > > 2013-10-29 09:22:51,958 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (secstorage-1:null) Checking resources in Cluster: 1 under Pod: 1
> > > > > 2013-10-29 09:22:51,958 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (secstorage-1:FirstFitRoutingAllocator) Looking for hosts in dc: 1
> > >  pod:1
> > > > >  cluster:1
> > > > > 2013-10-29 09:22:51,962 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (secstorage-1:FirstFitRoutingAllocator) FirstFitAllocator has 0
> hosts
> > > to
> > > > > check for allocation: []
> > > > > 2013-10-29 09:22:51,964 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (secstorage-1:FirstFitRoutingAllocator) Found 0 hosts for
> allocation
> > > > after
> > > > > prioritization: []
> > > > > 2013-10-29 09:22:51,964 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (secstorage-1:FirstFitRoutingAllocator) Looking for speed=500Mhz,
> > > Ram=256
> > > > > 2013-10-29 09:22:51,964 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (secstorage-1:FirstFitRoutingAllocator) Host Allocator returning 0
> > > > > suitable hosts
> > > > > 2013-10-29 09:22:51,965 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (secstorage-1:null) No suitable hosts found
> > > > > 2013-10-29 09:22:51,965 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (secstorage-1:null) No suitable hosts found under this Cluster: 1
> > > > > 2013-10-29 09:22:51,968 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (secstorage-1:null) Could not find suitable Deployment Destination
> > for
> > > > > this VM under any clusters, returning.
> > > > > 2013-10-29 09:22:51,968 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (secstorage-1:null) Searching all possible resources under this
> > Zone: 1
> > > > > 2013-10-29 09:22:51,969 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (secstorage-1:null) Listing clusters in order of aggregate
> capacity,
> > > that
> > > > > have (atleast one host with) enough CPU and RAM capacity under this
> > > > Zone: 1
> > > > > 2013-10-29 09:22:51,973 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (secstorage-1:null) Removing from the clusterId list these clusters
> > > from
> > > > > avoid set: [1]
> > > > > 2013-10-29 09:22:51,973 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (secstorage-1:null) No clusters found after removing disabled
> > clusters
> > > > and
> > > > > clusters in avoid list, returning.
> > > > > 2013-10-29 09:22:51,981 DEBUG [cloud.capacity.CapacityManagerImpl]
> > > > > (secstorage-1:null) VM state transitted from :Starting to Stopped
> > with
> > > > > event: OperationFailedvm's original host id: null new host id: null
> > > host
> > > > > id before state transition: null
> > > > > 2013-10-29 09:22:51,985 WARN
> > > > >  [storage.secondary.SecondaryStorageManagerImpl]
> (secstorage-1:null)
> > > > > Exception while trying to start secondary storage vm
> > > > > com.cloud.exception.InsufficientServerCapacityException: Unable to
> > > create
> > > > > a deployment for VM[SecondaryStorageVm|s-4930-VM]Scope=interface
> > > > > com.cloud.dc.DataCenter; id=1
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:842)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:571)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:267)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:696)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1300)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50)
> > > > >         at
> > > > >
> > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104)
> > > > >         at
> > > > >
> > >
> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
> > > > >         at
> > > > >
> > >
> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
> > > > >         at
> > > > > com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
> > > > >         at
> > > > >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > > > >         at
> > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > > >         at java.lang.Thread.run(Thread.java:744)
> > > > > 2013-10-29 09:22:51,987 INFO
> > > > >  [storage.secondary.SecondaryStorageManagerImpl]
> (secstorage-1:null)
> > > > > Unable to start secondary storage vm for standby capacity,
> > secStorageVm
> > > > vm
> > > > > Id :
> > > > > 4930, will recycle it and start a new one
> > > > > 2013-10-29 09:22:51,987 INFO
> > > > >  [cloud.secstorage.PremiumSecondaryStorageManagerImpl]
> > > > (secstorage-1:null)
> > > > > Primary secondary storage is not even started, wait until next turn
> > > > > 2013-10-29 09:22:52,009 DEBUG
> > > > [cloud.consoleproxy.ConsoleProxyManagerImpl]
> > > > > (consoleproxy-1:null) Zone 1 is ready to launch console proxy
> > > > > 2013-10-29 09:22:52,009 DEBUG
> > > > [cloud.consoleproxy.ConsoleProxyManagerImpl]
> > > > > (consoleproxy-1:null) Expand console proxy standby capacity for
> zone
> > > > > EastProdZone1
> > > > > 2013-10-29 09:22:52,012 INFO
> > > >  [cloud.consoleproxy.ConsoleProxyManagerImpl]
> > > > > (consoleproxy-1:null) Found a stopped console proxy, bring it up to
> > > > > running pool. proxy vm id : 4271
> > > > > 2013-10-29 09:22:52,020 DEBUG [cloud.capacity.CapacityManagerImpl]
> > > > > (consoleproxy-1:null) VM state transitted from :Stopped to Starting
> > > with
> > > > > event: StartRequestedvm's original host id: null new host id: null
> > host
> > > > id
> > > > > before state transition: null
> > > > > 2013-10-29 09:22:52,020 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (consoleproxy-1:null) Successfully transitioned to start state for
> > > > > VM[ConsoleProxy|v-4271-VM] reservation id =
> > > > > 11861d83-fc09-4b25-8c07-bcb6ad8e7f54
> > > > > 2013-10-29 09:22:52,021 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (consoleproxy-1:null) Trying to deploy VM, vm has dcId: 1 and
> podId:
> > > null
> > > > > 2013-10-29 09:22:52,022 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> > > > > (consoleproxy-1:null) Deploy avoids pods: null, clusters: null,
> > hosts:
> > > > null
> > > > > 2013-10-29 09:22:52,023 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (consoleproxy-1:null) Deploy avoids pods: null, clusters: null,
> > hosts:
> > > > null
> > > > > 2013-10-29 09:22:52,024 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (consoleproxy-1:null) DeploymentPlanner allocation algorithm:
> > > > >
> > com.cloud.deploy.FirstFitPlanner_EnhancerByCloudStack_d4b92e44@713baeaa
> > > > > 2013-10-29 09:22:52,024 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (consoleproxy-1:null) Trying to allocate a host and storage pools
> > from
> > > > > dc:1, pod:null,cluster:null, requested cpu: 500, requested ram:
> > > > 1073741824
> > > > > 2013-10-29 09:22:52,024 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (consoleproxy-1:null) Is ROOT volume READY (pool already
> allocated)?:
> > > No
> > > > > 2013-10-29 09:22:52,024 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (consoleproxy-1:null) Searching all possible resources under this
> > > Zone: 1
> > > > > 2013-10-29 09:22:52,024 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (consoleproxy-1:null) Listing clusters in order of aggregate
> > capacity,
> > > > > that have (atleast one host with) enough CPU and RAM capacity under
> > > this
> > > > > Zone: 1
> > > > > 2013-10-29 09:22:52,031 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (consoleproxy-1:null) Checking resources in Cluster: 1 under Pod: 1
> > > > > 2013-10-29 09:22:52,031 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (consoleproxy-1:FirstFitRoutingAllocator) Looking for hosts in dc:
> 1
> > > >  pod:1
> > > > >  cluster:1
> > > > > 2013-10-29 09:22:52,033 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (consoleproxy-1:FirstFitRoutingAllocator) FirstFitAllocator has 0
> > hosts
> > > > to
> > > > > check for allocation: []
> > > > > 2013-10-29 09:22:52,034 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (consoleproxy-1:FirstFitRoutingAllocator) Found 0 hosts for
> > allocation
> > > > > after prioritization: []
> > > > > 2013-10-29 09:22:52,034 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (consoleproxy-1:FirstFitRoutingAllocator) Looking for speed=500Mhz,
> > > > > Ram=1024
> > > > > 2013-10-29 09:22:52,034 DEBUG [allocator.impl.FirstFitAllocator]
> > > > > (consoleproxy-1:FirstFitRoutingAllocator) Host Allocator returning
> 0
> > > > > suitable hosts
> > > > > 2013-10-29 09:22:52,034 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (consoleproxy-1:null) No suitable hosts found
> > > > > 2013-10-29 09:22:52,034 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (consoleproxy-1:null) No suitable hosts found under this Cluster: 1
> > > > > 2013-10-29 09:22:52,036 DEBUG
> > > > [cloud.deploy.DeploymentPlanningManagerImpl]
> > > > > (consoleproxy-1:null) Could not find suitable Deployment
> Destination
> > > for
> > > > > this VM under any clusters, returning.
> > > > > 2013-10-29 09:22:52,036 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (consoleproxy-1:null) Searching all possible resources under this
> > > Zone: 1
> > > > > 2013-10-29 09:22:52,037 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (consoleproxy-1:null) Listing clusters in order of aggregate
> > capacity,
> > > > > that have (atleast one host with) enough CPU and RAM capacity under
> > > this
> > > > > Zone: 1
> > > > > 2013-10-29 09:22:52,039 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (consoleproxy-1:null) Removing from the clusterId list these
> clusters
> > > > from
> > > > > avoid set: [1]
> > > > > 2013-10-29 09:22:52,039 DEBUG [cloud.deploy.FirstFitPlanner]
> > > > > (consoleproxy-1:null) No clusters found after removing disabled
> > > clusters
> > > > > and clusters in avoid list, returning.
> > > > > 2013-10-29 09:22:52,043 DEBUG [cloud.capacity.CapacityManagerImpl]
> > > > > (consoleproxy-1:null) VM state transitted from :Starting to Stopped
> > > with
> > > > > event: OperationFailedvm's original host id: null new host id: null
> > > host
> > > > > id before state transition: null
> > > > > 2013-10-29 09:22:52,045 WARN
> > > >  [cloud.consoleproxy.ConsoleProxyManagerImpl]
> > > > > (consoleproxy-1:null) Exception while trying to start console proxy
> > > > > com.cloud.exception.InsufficientServerCapacityException: Unable to
> > > create
> > > > > a deployment for VM[ConsoleProxy|v-4271-VM]Scope=interface
> > > > > com.cloud.dc.DataCenter; id=1
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:842)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:571)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:928)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1672)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157)
> > > > >         at
> > > > >
> > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111)
> > > > >         at
> > > > >
> > >
> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
> > > > >         at
> > > > >
> > >
> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
> > > > >         at
> > > > > com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
> > > > >         at
> > > > >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > > > >         at
> > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > > >         at java.lang.Thread.run(Thread.java:744)
> > > > >
> > > > >
> > > > > What am I missing? How can I get these two system VMs online?
> > > > >
> > > > > Best Regards,
> > > > >
> > > > >
> > > > >
> > > > > Adam Scarcella
> > > > >
> > > > >
> > > > > On Mon, Oct 28, 2013 at 8:07 PM, Adam <[email protected]>
> > > wrote:
> > > > >
> > > > > > It's worth a shot. I've gone through the db manually, and changed
> > all
> > > > > > the places where I could see the old IPs, but your way is much
> more
> > > > > efficient.
> > > > > > Doing it now and will report back soon. Thanks.
> > > > > >
> > > > > > Best Regards,
> > > > > >
> > > > > >
> > > > > >
> > > > > > Adam Scarcella
> > > > > >
> > > > > >
> > > > > > On Mon, Oct 28, 2013 at 6:53 PM, Andrija Panic <
> > > > [email protected]
> > > > > >wrote:
> > > > > >
> > > > > >> one of the crazy tries:
> > > > > >> backup your database, search for all IP addresses inside SQL
> file,
> > > > > >> map old to new IP addresses (old1 --> new1, old2 -->new2...) and
> > > then
> > > > > >> do search/replace each old one with new one.
> > > > > >> This is for netowrking/ip addresses (guest, public,etc), same
> for
> > > NFS
> > > > > >> serves, same for NFS host, etc...
> > > > > >>
> > > > > >> Not sure if that is going to make you UP again, but...
> > > > > >>
> > > > > >>
> > > > > >> On 28 October 2013 22:59, Adam <[email protected]>
> wrote:
> > > > > >>
> > > > > >> > Hi All,
> > > > > >> >
> > > > > >> > I know what I'm about to ask may seem like crazy talk, but I
> > have
> > > a
> > > > > >> > situation at work where our CloudStack "box" was moved to
> > another
> > > > > >> network.
> > > > > >> > This single server is a self-contained cloud with everything
> on
> > > one
> > > > > box.
> > > > > >> > I'm running CS 4.1.1 on CentOS 6.4 x86_64 with KVM and NFS for
> > > > > >> > primary
> > > > > >> and
> > > > > >> > secondary storage mounts. It's a very basic configuration, but
> > my
> > > > > >> company
> > > > > >> > moved from one town to another, and the IT dept decided to
> move
> > > our
> > > > > >> server
> > > > > >> > to a different switch and subnet without asking. So now
> > literally
> > > > > >> > every single IP address of any importance has been changed.
> > > > > >> >
> > > > > >> > I've been trying in vain to get this CS up and running again,
> > but
> > > > > >> > it
> > > > > >> seems
> > > > > >> > like everything has changed including the guids, or at least
> > that
> > > > > >> > seems
> > > > > >> to
> > > > > >> > be the case since I cannot match up guids from the database to
> > > > > >> > guids on
> > > > > >> the
> > > > > >> > primary/secondary NFS.
> > > > > >> >
> > > > > >> > I'm sure if each piece were separate and moved one piece at a
> > > time,
> > > > > >> > it would be fairly straightforward, but since this box is
> > > > > >> > self-contained
> > > > > >> and
> > > > > >> > everything changed at the same time, it is proving to be very
> > > > > difficult.
> > > > > >> >
> > > > > >> > Does anyone have any suggestions for me to attempt to recover
> my
> > > CS?
> > > > > >> >
> > > > > >> > Best Regards,
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >> > Adam Scarcella
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >>
> > > > > >> Andrija Panić
> > > > > >> --------------------------------------
> > > > > >>   http://admintweets.com
> > > > > >> --------------------------------------
> > > > > >>
> > > > > >
> > > > > >
> > > > > This email and any attachments to it may be confidential and are
> > > intended
> > > > > solely for the use of the individual to whom it is addressed. Any
> > views
> > > > or
> > > > > opinions expressed are solely those of the author and do not
> > > necessarily
> > > > > represent those of Shape Blue Ltd or related companies. If you are
> > not
> > > > the
> > > > > intended recipient of this email, you must neither take any action
> > > based
> > > > > upon its contents, nor copy or show it to anyone. Please contact
> the
> > > > sender
> > > > > if you believe you have received this email in error. Shape Blue
> Ltd
> > > is a
> > > > > company incorporated in England & Wales. ShapeBlue Services India
> LLP
> > > is
> > > > a
> > > > > company incorporated in India and is operated under license from
> > Shape
> > > > Blue
> > > > > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
> in
> > > > Brasil
> > > > > and is operated under license from Shape Blue Ltd. ShapeBlue is a
> > > > > registered trademark.
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > > --------------------------------------
> > >   http://admintweets.com
> > > --------------------------------------
> > >
> >
>
>
>
> --
>
> Andrija Panić
> --------------------------------------
>   http://admintweets.com
> --------------------------------------
>

Reply via email to