Hi Meng, No problem, ubuntu -> Meng looks like an issue, I think it would be worth looking at this to ensure mounting works properly. Can you try the mount command I gave you?
Also: ping 10.244.18.55 traceroute 10.244.18.55 Very bizarre error message seeing as you said ping responds. Marty On Fri, Aug 23, 2013 at 8:54 PM, Han,Meng <meng...@ufl.edu> wrote: > Hi Marty, > > Thanks for the fast reply! > > > > On Fri, 23 Aug 2013 20:26:07 +0100, Marty Sweet wrote: > >> Thanks, what setup do you have in the cloudstack interface? Can you >> provide >> a screenshot of the secondary storage page (under Infrastructure)? >> > > I attached the screenshot. > > >> Could you also try the following on another host (not the management or >> SSVM): >> sudo -i >> mkdir mounttest >> sudo mount 10.244.18.55:/secondary mounttest >> Can you write into the directory mounttest and see the files? >> > > I tried mount from another host(10.244.18.122), it failed. The following > is the output: > > root@ubuntu:~# mount -t nfs -v 10.244.18.55:/secondary test > mount.nfs: timeout set for Fri Aug 23 15:52:59 2013 > mount.nfs: trying text-based options 'vers=4,addr=10.244.18.55,** > clientaddr=10.244.18.122' > mount.nfs: mount(2): No route to host > mount.nfs: trying text-based options 'vers=4,addr=10.244.18.55,** > clientaddr=10.244.18.122' > mount.nfs: mount(2): No route to host > mount.nfs: trying text-based options 'vers=4,addr=10.244.18.55,** > clientaddr=10.244.18.122' > mount.nfs: mount(2): No route to host > mount.nfs: trying text-based options 'vers=4,addr=10.244.18.55,** > clientaddr=10.244.18.122' > mount.nfs: mount(2): No route to host > mount.nfs: trying text-based options 'vers=4,addr=10.244.18.55,** > clientaddr=10.244.18.122' > mount.nfs: mount(2): No route to host > connection timed out. > > This host can ping the nfs server. > > [root@meng Downloads]# rpcinfo -p > program vers proto port service > 100000 4 tcp 111 portmapper > 100000 3 tcp 111 portmapper > 100000 2 tcp 111 portmapper > 100000 4 udp 111 portmapper > 100000 3 udp 111 portmapper > 100000 2 udp 111 portmapper > 100011 1 udp 875 rquotad > 100011 2 udp 875 rquotad > 100011 1 tcp 875 rquotad > 100011 2 tcp 875 rquotad > 100005 1 udp 892 mountd > 100005 1 tcp 892 mountd > 100005 2 udp 892 mountd > 100005 2 tcp 892 mountd > 100005 3 udp 892 mountd > 100005 3 tcp 892 mountd > 100003 2 tcp 2049 nfs > 100003 3 tcp 2049 nfs > 100003 4 tcp 2049 nfs > 100227 2 tcp 2049 nfs_acl > 100227 3 tcp 2049 nfs_acl > 100003 2 udp 2049 nfs > 100003 3 udp 2049 nfs > 100003 4 udp 2049 nfs > 100227 2 udp 2049 nfs_acl > 100227 3 udp 2049 nfs_acl > 100021 1 udp 32769 nlockmgr > 100021 3 udp 32769 nlockmgr > 100021 4 udp 32769 nlockmgr > 100021 1 tcp 32803 nlockmgr > 100021 3 tcp 32803 nlockmgr > 100021 4 tcp 32803 nlockmgr > > > > > And finally, on your management server: >> cat /etc/mtab >> > > [root@meng Downloads]# cat /etc/mtab > /dev/mapper/vg_meng-lv_root / ext4 rw 0 0 > proc /proc proc rw 0 0 > sysfs /sys sysfs rw 0 0 > devpts /dev/pts devpts rw,gid=5,mode=620 0 0 > tmpfs /dev/shm tmpfs rw,rootcontext="system_u:**object_r:tmpfs_t:s0" 0 0 > /dev/sda1 /boot ext4 rw 0 0 > /dev/mapper/vg_meng-lv_home /home ext4 rw 0 0 > none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0 > sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0 > nfsd /proc/fs/nfsd nfsd rw 0 0 > 10.244.18.55:/primary /mnt/82df7a6d-6708-3333-877a-**1cf1f3f5e945 nfs > rw,noac,vers=4,addr=10.244.18.**55,clientaddr=10.244.18.55 0 0 > /dev/sdb1 /media/C03A-B0D6 vfat rw,nosuid,nodev,uhelper=** > udisks,uid=500,gid=500,**shortname=mixed,dmask=0077,**utf8=1,flush 0 0 > > > df -f >> > [root@meng Downloads]# df -h > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/vg_meng-lv_root > 50G 9.0G 38G 20% / > tmpfs 1.8G 656K 1.8G 1% /dev/shm > /dev/sda1 485M 38M 422M 9% /boot > /dev/mapper/vg_meng-lv_home > 241G 1.7G 227G 1% /home > 10.244.18.55:/primary > 50G 9.0G 38G 20% /mnt/82df7a6d-6708-3333-877a-* > *1cf1f3f5e945 > /dev/sdb1 7.8G 2.2G 5.7G 28% /media/C03A-B0D6 > > Thanks again! > > > > > >> Thanks! >> Marty >> >> >> >> On Fri, Aug 23, 2013 at 8:11 PM, Han,Meng <meng...@ufl.edu> wrote: >> >> Hi Marty, >>> >>> Here is the information: >>> >>> [root@meng]#cat /etc/exports >>> /secondary *(rw,async,no_root_squash) >>> /primary *(rw,async,no_root_squash) >>> >>> [root@meng Downloads]# exportfs >>> /secondary <world> >>> /primary <world> >>> >>> [root@meng Downloads]# exportfs -a >>> >>> There is no output for exportfs -a. >>> >>> >>> >>> >>> >>> >>> On Fri, 23 Aug 2013 18:53:51 +0100, Marty Sweet wrote: >>> >>> Hi Meng, >>>> >>>> I would first look at the management server and ignore the SSVM until >>>> you >>>> are sure the NFS share is working. >>>> >>>> Could you post the output of: exportfs -a and cat /etc/exports ? >>>> I can then give more guidance. >>>> >>>> Thanks, >>>> Marty >>>> >>>> >>>> On Fri, Aug 23, 2013 at 5:01 PM, Han,Meng <meng...@ufl.edu> wrote: >>>> >>>> Hi folks, >>>> >>>>> >>>>> I am following the 4.1 quick install guide to deploy cloudstack. Now I >>>>> am >>>>> unable to mount the secondary storage from ssvm. I tried system >>>>> reinstallation many times... >>>>> >>>>> Below is some information about this issue: >>>>> >>>>> root@s-1-VM:~# /usr/local/cloud/systemvm/******ssvm-check.sh >>>>> ==============================******================== >>>>> >>>>> >>>>> First DNS server is 8.8.8.8 >>>>> PING 8.8.8.8 (8.8.8.8): 56 data bytes >>>>> 64 bytes from 8.8.8.8: icmp_seq=0 ttl=47 time=1067.133 ms >>>>> 64 bytes from 8.8.8.8: icmp_seq=1 ttl=47 time=66.248 ms >>>>> --- 8.8.8.8 ping statistics --- >>>>> 2 packets transmitted, 2 packets received, 0% packet loss >>>>> round-trip min/avg/max/stddev = 66.248/566.691/1067.133/500.******443 >>>>> ms >>>>> >>>>> >>>>> Good: Can ping DNS server >>>>> ==============================******================== >>>>> Good: DNS resolves download.cloud.com >>>>> ==============================******================== >>>>> >>>>> >>>>> ERROR: NFS is not currently mounted >>>>> Try manually mounting from inside the VM >>>>> NFS server is 169.254.3.46 >>>>> PING 169.254.3.46 (169.254.3.46): 56 data bytes >>>>> 64 bytes from 169.254.3.46: icmp_seq=0 ttl=64 time=0.042 ms >>>>> 64 bytes from 169.254.3.46: icmp_seq=1 ttl=64 time=0.054 ms >>>>> --- 169.254.3.46 ping statistics --- >>>>> 2 packets transmitted, 2 packets received, 0% packet loss >>>>> round-trip min/avg/max/stddev = 0.042/0.048/0.054/0.000 ms >>>>> Good: Can ping NFS server >>>>> ==============================******================== >>>>> >>>>> >>>>> Management server is 10.244.18.55. Checking connectivity. >>>>> Good: Can connect to management server port 8250 >>>>> ==============================******================== >>>>> >>>>> >>>>> Good: Java process is running >>>>> ==============================******================== >>>>> >>>>> >>>>> Tests Complete. Look for ERROR or WARNING above. >>>>> >>>>> The NFS server address listed above is actually the address of ssvm. >>>>> Could >>>>> someone tell me why the ssvm address is there instead of the nfs >>>>> server? >>>>> I >>>>> am using the management server as the nfs server, its ip is >>>>> 10.244.18.55 >>>>> >>>>> The contents of /var/cache/cloud/cmdline is: >>>>> template=domP type=secstorage host=10.244.18.55 port=8250 name=s-1-VM >>>>> zone=1 pod=1 guid=s-1-VM resource=com.cloud.storage.***** >>>>> *resource.**** >>>>> >>>>> NfsSecondaryStorageResource >>>>> >>>>> instance=SecStorage sslcopy=true role=templateProcessor mtu=1500 >>>>> eth2ip=10.244.18.56 eth2mask=255.255.255.0 gateway=10.244.18.1 >>>>> eth0ip=169.254.3.46 eth0mask=255.255.0.0 eth1ip=10.244.18.47 >>>>> eth1mask=255.255.255.0 mgmtcidr=10.244.18.0/24 localgw=10.244.18.1 >>>>> private.network.device=eth1 eth3ip=10.244.18.49 eth3mask=255.255.255.0 >>>>> storageip=10.244.18.49 storagenetmask=255.255.255.0 >>>>> storagegateway=10.244.18.1 internaldns1=8.8.8.8 internaldns2=8.8.4.4 >>>>> dns1=8.8.8.8 dns2=8.8.4.4 >>>>> >>>>> The firewall rule on the nfs server is : >>>>> -A INPUT -p tcp -m tcp --dport 49152:49216 -j ACCEPT >>>>> -A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT >>>>> -A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT >>>>> -A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT >>>>> -A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT >>>>> -A INPUT -p tcp -m tcp --dport 5900:6100 -j ACCEPT >>>>> -A INPUT -p tcp -m tcp --dport 16509 -j ACCEPT >>>>> -A INPUT -p tcp -m tcp --dport 9090 -j ACCEPT >>>>> -A INPUT -p tcp -m tcp --dport 8250 -j ACCEPT >>>>> -A INPUT -p tcp -m tcp --dport 7080 -j ACCEPT >>>>> -A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT >>>>> -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT >>>>> -A INPUT -p icmp -j ACCEPT >>>>> -A INPUT -i lo -j ACCEPT >>>>> -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT >>>>> -A INPUT -j REJECT --reject-with icmp-host-prohibited >>>>> -A INPUT -s 10.244.18.0/24 -p udp -m state --state NEW -m udp --dport >>>>> 111 >>>>> -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p tcp -m state --state NEW -m tcp --dport >>>>> 111 >>>>> -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p tcp -m state --state NEW -m tcp --dport >>>>> 2049 -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p tcp -m state --state NEW -m tcp --dport >>>>> 32803 -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p udp -m state --state NEW -m udp --dport >>>>> 32769 -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p tcp -m state --state NEW -m tcp --dport >>>>> 892 >>>>> -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p udp -m state --state NEW -m udp --dport >>>>> 892 >>>>> -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p tcp -m state --state NEW -m tcp --dport >>>>> 875 >>>>> -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p udp -m state --state NEW -m udp --dport >>>>> 875 >>>>> -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p tcp -m state --state NEW -m tcp --dport >>>>> 662 >>>>> -j ACCEPT >>>>> -A INPUT -s 10.244.18.0/24 -p udp -m state --state NEW -m udp --dport >>>>> 662 >>>>> -j ACCEPT >>>>> -A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state >>>>> RELATED,ESTABLISHED -j ACCEPT >>>>> -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT >>>>> >>>>> The network information is: >>>>> >>>>> cloud0 Link encap:Ethernet HWaddr FE:00:A9:FE:01:A8 >>>>> inet addr:169.254.0.1 Bcast:169.254.255.255 >>>>> Mask:255.255.0.0 >>>>> inet6 addr: fe80::94da:98ff:fefd:ffa/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:364 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:562 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:0 >>>>> RX bytes:43632 (42.6 KiB) TX bytes:52405 (51.1 KiB) >>>>> >>>>> cloudbr0 Link encap:Ethernet HWaddr 00:23:AE:94:F7:22 >>>>> inet addr:10.244.18.55 Bcast:10.244.18.255 >>>>> Mask:255.255.255.0 >>>>> inet6 addr: fe80::223:aeff:fe94:f722/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:1293010 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:855962 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:0 >>>>> RX bytes:1021904803 (974.5 MiB) TX bytes:240327484 (229.1 >>>>> MiB) >>>>> >>>>> eth0 Link encap:Ethernet HWaddr 00:23:AE:94:F7:22 >>>>> inet6 addr: fe80::223:aeff:fe94:f722/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:2524952 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:1556673 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:1000 >>>>> RX bytes:2862800124 (2.6 GiB) TX bytes:283924007 (270.7 MiB) >>>>> Interrupt:21 Memory:fe6e0000-fe700000 >>>>> >>>>> lo Link encap:Local Loopback >>>>> inet addr:127.0.0.1 Mask:255.0.0.0 >>>>> inet6 addr: ::1/128 Scope:Host >>>>> UP LOOPBACK RUNNING MTU:16436 Metric:1 >>>>> RX packets:130588685 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:130588685 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:0 >>>>> RX bytes:444681794053 (414.1 GiB) TX bytes:444681794053 >>>>> (414.1 >>>>> GiB) >>>>> >>>>> virbr0 Link encap:Ethernet HWaddr 52:54:00:7C:47:2C >>>>> inet addr:192.168.122.1 Bcast:192.168.122.255 >>>>> Mask:255.255.255.0 >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:1 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:0 >>>>> RX bytes:0 (0.0 b) TX bytes:85 (85.0 b) >>>>> >>>>> vnet0 Link encap:Ethernet HWaddr FE:00:A9:FE:03:2E >>>>> inet6 addr: fe80::fc00:a9ff:fefe:32e/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:347 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:3086 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:500 >>>>> RX bytes:47385 (46.2 KiB) TX bytes:218921 (213.7 KiB) >>>>> >>>>> vnet1 Link encap:Ethernet HWaddr FE:25:C6:00:00:03 >>>>> inet6 addr: fe80::fc25:c6ff:fe00:3/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:14481 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:285518 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:500 >>>>> RX bytes:1442681 (1.3 MiB) TX bytes:28710317 (27.3 MiB) >>>>> >>>>> vnet2 Link encap:Ethernet HWaddr FE:E9:D6:00:00:0B >>>>> inet6 addr: fe80::fce9:d6ff:fe00:b/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:219253 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:831307 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:500 >>>>> RX bytes:12892318 (12.2 MiB) TX bytes:816300221 (778.4 MiB) >>>>> >>>>> vnet3 Link encap:Ethernet HWaddr FE:29:F2:00:00:05 >>>>> inet6 addr: fe80::fc29:f2ff:fe00:5/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:26 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:271065 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:500 >>>>> RX bytes:1332 (1.3 KiB) TX bytes:27089428 (25.8 MiB) >>>>> >>>>> vnet4 Link encap:Ethernet HWaddr FE:00:A9:FE:01:A8 >>>>> inet6 addr: fe80::fc00:a9ff:fefe:1a8/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:17 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:2572 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:500 >>>>> RX bytes:1343 (1.3 KiB) TX bytes:169551 (165.5 KiB) >>>>> >>>>> vnet5 Link encap:Ethernet HWaddr FE:93:CA:00:00:07 >>>>> inet6 addr: fe80::fc93:caff:fe00:7/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:82974 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:353899 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:500 >>>>> RX bytes:9191158 (8.7 MiB) TX bytes:35429714 (33.7 MiB) >>>>> >>>>> vnet6 Link encap:Ethernet HWaddr FE:72:0C:00:00:0E >>>>> inet6 addr: fe80::fc72:cff:fe00:e/64 Scope:Link >>>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>>> RX packets:9 errors:0 dropped:0 overruns:0 frame:0 >>>>> TX packets:270934 errors:0 dropped:0 overruns:0 carrier:0 >>>>> collisions:0 txqueuelen:500 >>>>> RX bytes:778 (778.0 b) TX bytes:27076384 (25.8 MiB) >>>>> >>>>> The contents of /etc/sysconfig/network-******scripts/ifcfg-eth0 is: >>>>> DEVICE=eth0 >>>>> TYPE=Ethernet >>>>> UUID=7ba41ef9-9868-432d-866d-******b4dbe18d9ba5 >>>>> >>>>> >>>>> ONBOOT=yes >>>>> NM_CONTROLLED=no >>>>> BOOTPROTO=none >>>>> DEFROUTE=yes >>>>> IPV4_FAILURE_FATAL=yes >>>>> IPV6INIT=no >>>>> NAME="System eth0" >>>>> HWADDR=00:23:AE:94:F7:22 >>>>> PEERDNS=yes >>>>> PEERROUTES=yes >>>>> IPADDR=10.244.18.55 >>>>> NETMASK=255.255.255.0 >>>>> GATEWAY=10.244.18.1 >>>>> DNS1=8.8.8.8 >>>>> DNS2=8.8.4.4 >>>>> BRIDGE=cloudbr0 >>>>> >>>>> Now on the management server side, I see the following error on one of >>>>> my >>>>> templates: >>>>> Failed post download script: bunzip2: I/O or other error, bailing out. >>>>> Possible reason follows.bunzip2: No space left on devicefailed to >>>>> uncompress /mnt/SecStorage/9db5f864-def4-** >>>>> ****30f9-8e25-2f5215113de8/** >>>>> template/tmpl/1/4/******dnld1953772300383463570tmp_ >>>>> >>>>> >>>>> >>>>> The secondary storage is giving me a alert about low availability. I >>>>> tried >>>>> manually mount the nfs share inside the ssvm, the mount gives me an >>>>> input/output error. But I can mount the nfs share from the management >>>>> server. >>>>> >>>>> Could someone give me some light what is wrong here? I reinstalled my >>>>> OS >>>>> many times, this error shows every time. >>>>> >>>>> Thanks you all. >>>>> >>>>> Best Regards, >>>>> Meng >>>>> >>>>> >>>>> >>>>> >>>
