Hi Nitin, pls check out http://pastebin.com/AfU4Vs91
nfs-common failed during VM startup, but it later seems to have stabilized. ssvm-check.sh chokes with DNS not available, as expected. iptables is wide open. Like I mentioned in my earlier message, the ISOs are now being downloaded… so rebooting the XS / SSVM might have done the trick after modifying secstorage.allowed.internal.sites Thanks! David La Motta Technical Marketing Engineer Citrix Solutions NetApp 919.476.5042 dlamo...@netapp.com<mailto:dlamo...@netapp.com> On May 31, 2013, at 1:17 AM, Prasanna Santhanam <t...@apache.org<mailto:t...@apache.org>> wrote: On Thu, May 30, 2013 at 09:34:00PM +0000, La Motta, David wrote: Nothing in cloud.out--I have some old messages which don't make me feel that great about things, but a tail -f on the file keeps it static when I attempt to add the ISO. The old messages are: 18:47:27,229 WARN NioConnection:110 - Unable to connect to remote: is there a server running on port 8250 This guy here - he tries to reach out to port 8250 on the management server to tell him he's good and ready. Have you allowed your managemetn server port 8250 open? What's the iptables -L -nvx listing on your management server? The ssvm-check.sh script does a few checks out of which 8250 agent connectivity is one. Did that show ERROR or succeed? Can you paste the output of ssvm-check.sh on a pastebin? I have tried secstorage.allowed.internal.sites with 172.20.39.0/24,10.63.172.0/24 and 172.20.39.0 and 172.20.39.51 to no avail. The SSVM can ping 172.20.39.51 without a prob. Now something strange that I just noticed. I can wget from SSVM to my internal web server and pull down whatever (actually, the CS login page) from http://172.20.39.51:8080 but I get a connection refused from http://172.20.39.51:80 *sigh* David La Motta Technical Marketing Engineer Citrix Solutions NetApp 919.476.5042 dlamo...@netapp.com<mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com> On May 30, 2013, at 4:09 PM, Ahmad Emneina <aemne...@gmail.com<mailto:aemne...@gmail.com><mailto:aemne...@gmail.com>> wrote: can you check the output of /var/log/cloud/cloud.out when adding the iso. Also if youre adding the iso from an internal web server, make sure you have secstorage.allowed.internal.sites set to the internal ip or range of where you host those images. On Thu, May 30, 2013 at 12:25 PM, La Motta, David <david.lamo...@netapp.com<mailto:david.lamo...@netapp.com><mailto:david.lamo...@netapp.com>>wrote: Ok, so I took a gander at some of the suggestions on that page, and though I feel I am close? I am not there yet. Just so you all have a bit of context how my environment is setup: I am running MS in a CentOS 6.4 VM, inside XS 61. Cluster XS is another 6.1 VM MS has a public and private (vlan) IP address vlan itself can't access outside world (though MS can via the public IP) When I add my ISO, the MS console still continues to spit out: WARN [storage.download.DownloadMonitorImpl] (1292666621@qtp-1329125626-10:) There is no secondary storage VM for secondary storage host nfs:// 172.20.39.74/SnapCreator_secondary Both SSVM and proxy VM are running in XS, I can ssh to SSVM and execute commands per the troubleshooting guide. Things that I have done since the series of messages on this thread: 1. Blew away the DB 2. Started with a clean Cluster XS. 3. MS iptables disabled 4. Added secstorage.allowed.internal.sites value to have my vlan CIDR ( 172.20.39.0/24) and public network CIDR (10.63.172.0/24) 5. Changed host value for the MS to be the vlan IP address instead of the public address 6. Ran ssvm-check.sh and it stops short at not being able to hit DNS (expected, because the vlan can't connect to the 'net) 7. Ran service cloud status and it came back fine, everything seems to be running a-ok. 8. Primary and secondary storage remain intact on the storage controller. Setup zone, pod, cluster, host, primary & secondary storage. System VMs are up and running. DB says: mysql> select name,status,type from host; +------------------------------------------+--------+------------------+ | name | status | type | +------------------------------------------+--------+------------------+ | XenServer-CS | Up | Routing | | nfs://172.20.39.74/SnapCreator_secondary | Alert | SecondaryStorage | +------------------------------------------+--------+------------------+ 2 rows in set (0.00 sec) mysql> select url,download_state,error_str from template_host_ref; +-----------------------------------------------------------------------------+----------------+-----------+ | url | download_state | error_str | +-----------------------------------------------------------------------------+----------------+-----------+ | http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2| DOWNLOADED | NULL | | http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2 | DOWNLOADED | NULL | | http://download.cloud.com/templates/burbank/burbank-systemvm-08012012.ova | DOWNLOADED | NULL | | http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2| DOWNLOADED | NULL | | http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2 | DOWNLOADED | NULL | | http://172.20.39.51/software/linux/ubuntu/ubuntu-12.04.1-desktop-i386.iso | NOT_DOWNLOADED | NULL | +-----------------------------------------------------------------------------+----------------+-----------+ 6 rows in set (0.00 sec) root@s-1-VM:/# tail -20 /var/log/cloud.log Thu May 30 18:02:28 UTC 2013 Enable service cloud = 1 Thu May 30 18:39:21 UTC 2013 Executing cloud-early-config Thu May 30 18:39:22 UTC 2013 Detected that we are running inside xen-domU guest Thu May 30 18:39:24 UTC 2013 Setting up secondary storage system vm Thu May 30 18:39:24 UTC 2013 checking that eth0 has IP Thu May 30 18:39:25 UTC 2013 checking that eth1 has IP Thu May 30 18:39:25 UTC 2013 checking that eth2 has IP Thu May 30 18:39:34 UTC 2013 checking that eth3 has IP Thu May 30 18:39:34 UTC 2013 Successfully setup storage network with STORAGE_IP:172.20.39.6, STORAGE_NETMASK:255.255.255.0, STORAGE_CIDR: Thu May 30 18:39:34 UTC 2013 Setting up apache web server Thu May 30 18:39:34 UTC 2013 cloud: disable rp_filter Thu May 30 18:39:34 UTC 2013 disable rpfilter Thu May 30 18:39:34 UTC 2013 cloud: enable_fwding = 0 Thu May 30 18:39:34 UTC 2013 enable_fwding = 0 Thu May 30 18:39:34 UTC 2013 Enable service haproxy = 0 Thu May 30 18:39:34 UTC 2013 Enable service dnsmasq = 0 Thu May 30 18:39:34 UTC 2013 Enable service cloud-passwd-srvr = 0 Thu May 30 18:39:35 UTC 2013 Enable service cloud = 1 Thu May 30 18:47:35 UTC 2013 User stops cloud.com<http://cloud.com><http://cloud.com><http://cloud.com> service Thu May 30 19:04:13 UTC 2013 User stops cloud.com<http://cloud.com><http://cloud.com><http://cloud.com> service (I ran a service cloud restart a couple of times, per troubleshooting instructions) I am at a loss, but happy to try anything else? David La Motta Technical Marketing Engineer Citrix Solutions NetApp 919.476.5042 dlamo...@netapp.com<mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com> On May 30, 2013, at 9:44 AM, Nitin Mehta <nitin.me...@citrix.com<mailto:nitin.me...@citrix.com><mailto:nitin.me...@citrix.com><mailto: nitin.me...@citrix.com<mailto:nitin.me...@citrix.com><mailto:nitin.me...@citrix.com>>> wrote: David - Can you please follow https://cwiki.apache.org/CLOUDSTACK/ssvm-troubleshooting.html to troubleshoot the issues ? Let us know how it goes ? On 30/05/13 7:02 PM, "La Motta, David" <david.lamo...@netapp.com<mailto:david.lamo...@netapp.com><mailto:david.lamo...@netapp.com><mailto: david.lamo...@netapp.com<mailto:david.lamo...@netapp.com><mailto:david.lamo...@netapp.com>>> wrote: Hi Edison? The table is empty, which would indicate the error message I see on the console is accurate. However, that I see the VM running in XenServer and I have it listed in the Infrastructure area in CS doesn't make much sense. David La Motta Technical Marketing Engineer Citrix Solutions NetApp 919.476.5042 dlamo...@netapp.com<mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com On May 29, 2013, at 5:41 PM, Edison Su <edison...@citrix.com<mailto:edison...@citrix.com><mailto:edison...@citrix.com><mailto:edison...@citrix.com><mailto: edison...@citrix.com<mailto:edison...@citrix.com><mailto:edison...@citrix.com>>> wrote: Can you try: select status from host where type = 'SecondaryStorageVM'; in db? Make sure the agent in ssvm can connect back to mgt serer. -----Original Message----- From: La Motta, David [mailto:david.lamo...@netapp.com<http://netapp.com><http://netapp.com><http://netapp.com>< http://netapp.com>] Sent: Wednesday, May 29, 2013 2:28 PM To: <dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org><mailto:dev@cloudstack.apache.org><mailto:dev@cloudstack.apache.org><mailto: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org><mailto:dev@cloudstack.apache.org>>> Subject: 4.2 snapshot: setting up ISOs - no go So my 4.2 instance is up and running, system VMs are running (secondary storage VM and console proxy VM), primary and secondary storage are up on NFS... all things considered and at a glance, things look good. However, when I try to add an ISO so that I can provision a VM, I am getting: WARN [storage.download.DownloadMonitorImpl] (1018141210@qtp- 2013781391-38:) There is no secondary storage VM for secondary storage host nfs://172.20.39.74/SnapCreator_secondary Which is throwing me off, because the secondary storage VM seems to be up and happy. I can see it on XenCenter alive and well, too. When I go through the ISO registration wizard, registration seems to be successful in the UI, since the ISOs get added to the templates table--except I get the warning in the console above. If I try to add another secondary storage, I get: ERROR [cloud.api.ApiServer] (1260872222@qtp-2013781391-40:) unhandled exception executing api command: addSecondaryStorage com.cloud.utils.exception.CloudRuntimeException: Cannot transit agent status with event AgentDisconnected for host 9, mangement server id is 271423992217301,Unable to transition to a new state from Creating via AgentDisconnected at com.cloud.agent.manager.AgentManagerImpl.agentStatusTransitTo(Agent ManagerImpl.java:1436) When in doubt, reboot, right? So I shutdown my CS mgmt server and XenServer host and tried again. I even added more secondary storage without anything in it (the first NFS volume had the system VM template on it). Still, I get the same behavior as described above: the UI says the ISO was added, but the log file warns about the lack of a secondary storage VM. If I attempt to go through the instance provisioning wizard, long and behold, the ISO is not listed in the "My ISOs" tab. There is also something strange that I've noticed (slightly off topic): if I have primary storage allocated and resize the underlying volume on the storage controller, CS doesn't see the new size. Is there any way to force that to take place from the UI? Clicking on refresh did nothing about it. Thanks! David La Motta Technical Marketing Engineer - Citrix NetApp 919.476.5042 dlamo...@netapp.com<mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com><mailto:dlamo...@netapp.com <mailto:dlamo...@netapp.com -- Prasanna., ------------------------ Powered by BigRock.com<http://BigRock.com>
