Re: [URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Indra Pramana
Hi Marcus, Good day to you, and thank you for your e-mail. We will try to do what you have suggested and will see if we can fix the problem. I am looking into increasing the filesystem of the KVM hosts to see if it can resolve the problem for now. Thank you very much for your help! Cheers. O

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Marcus Sorensen
You'll have to go back further in the log then, and/or go to the agent log, because the issue of the ghost mount is due to the filesystem being full, and if that's your secondary storage I'm not sure why it would be written to from the kvm host unless you were doing backups or snapshots. On Thu, O

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Indra Pramana
Hi Marcus, It happens randomly, mounting fails, and then / gets filled. I suspect this occurs when the mounting fails, any files which is supposed to be transferred to the mount point (e.g. templates used to create VMs) will fill-up the / partition of the KVM host instead. This doesn't happen in 4

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Marcus Sorensen
It sort of looks like the out of space triggered the issue. Libvirt shows 2013-10-03 15:38:57.414+: 2710: error : virCommandWait:2348 : internal error Child process (/bin/umount /mnt/5230667e-9c58-3ff6-983c-5fc2a72df669) unexpected exit status 32: error writing /etc/mtab.tmp: No space left on

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Indra Pramana
Hi Marcus and all, I also find some strange and interesting error messages from the libvirt logs: http://pastebin.com/5ByfNpAf Looking forward to your reply, thank you. Cheers. On Fri, Oct 4, 2013 at 1:38 AM, Indra Pramana wrote: > Hi Marcus, > > Good day to you, and thank you for your e-m

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Indra Pramana
Hi Dean, Good day to you, and thank you for your e-mail. The current setting (value) of storage.overprovisioning.factor is 2. Do you think it matters? I can't remember whether the value was the default value, or is it something that we changed earlier. Do you think changing this to 1 will help?

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Indra Pramana
Hi Marcus, Good day to you, and thank you for your e-mail. See my reply inline. On Fri, Oct 4, 2013 at 12:29 AM, Marcus Sorensen wrote: > Are you using the release artifacts, or your own 4.2 build? > [Indra:] We are using the release artifacts from below repo since we are using Ubuntu: deb htt

Re: [URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Marcus Sorensen
Are you using the release artifacts, or your own 4.2 build? When you rebooted the host, did the problem go away or come back the same? You may want to look at 'virsh pool-list' to see if libvirt is mounting/registering the secondary storage. Is this happening on multiple hosts, the same way? You

[URGENT] Mounting issues from KVM hosts to secondary storage

2013-10-03 Thread Indra Pramana
Dear all, We face a major problem after upgrading to 4.2.0. Mounting from KVM hosts to secondary storage seems to fail, every time a new VM instance is created, it will use up the / (root) partition of the KVM hosts instead. Here is the df result: root@hv-kvm-02:/home/indra# df Filesystem