Re: [ceph-users] Ceph + Xen - RBD io hang

2013-10-28 Thread John Morris
The suspicious line in /var/log/debug (see the pastebin below) and that 'blkid' was the culprit keeping the device open looked like juicy clues: kernel: vbd vbd-51728: 16 Device in use; refusing to close Search results: https://www.redhat.com/archives/libguestfs/2012-February/msg00023.html

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-10-28 Thread John Morris
On Behalf Of John Morris >> Sent: Tuesday, 29 October 2013 6:01 AM >> To: ceph-users@lists.ceph.com >> Subject: [ceph-users] Ceph + Xen - RBD io hang >> >> I'm encountering a problem with RBD-backed Xen. During a VM boot, >> pygrub attaches the VM's root V

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-10-28 Thread James Harper
is > Sent: Tuesday, 29 October 2013 6:01 AM > To: ceph-users@lists.ceph.com > Subject: [ceph-users] Ceph + Xen - RBD io hang > > I'm encountering a problem with RBD-backed Xen. During a VM boot, > pygrub attaches the VM's root VDI to dom0. This hangs with these > message

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-10-28 Thread John Morris
Hi John, On 10/28/2013 08:55 PM, John Wilkins wrote: > John, > > I've never installed anything on Scientific Linux. SL6 is binary compatible with other EL6 varieties. This host uses the same repos as CentOS for Dave Scott's xenserver-core technology preview: EPEL6, Dave's XenServer + Ceph tech

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-10-28 Thread John Wilkins
John, I've never installed anything on Scientific Linux. Are you sure that QEMU has RBD support? I have some wip-doc text, which I'm going to move around shortly. You can see the yum install requirements here: http://ceph.com/docs/wip-doc-install/install/yum-priorities/ http://ceph.com/docs/wip-

[ceph-users] Ceph + Xen - RBD io hang

2013-10-28 Thread John Morris
I'm encountering a problem with RBD-backed Xen. During a VM boot, pygrub attaches the VM's root VDI to dom0. This hangs with these messages in the debug log: Oct 27 21:19:59 xen27 kernel: vbd vbd-51728: 16 Device in use; refusing to close Oct 27 21:19:59 xen27 xenopsd-xenlight: [xenops] wait

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-08-28 Thread Oliver Daudey
Hi, On wo, 2013-08-28 at 10:10 +0200, Sylvain Munaut wrote: > Hi, > > > I am doing some experimentation with Ceph and Xen (on the same host) and I'm > > experiencing some problems with the rbd device that I'm using as the block > > device. My environment is: > > AFAIK running the OSD and the ke

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-08-28 Thread Olivier Bonvalet
Le mercredi 28 août 2013 à 10:07 +0200, Sylvain Munaut a écrit : > Hi, > > > I use Ceph 0.61.8 and Xen 4.2.2 (Debian) in production, and can't use > > kernel 3.10.* on dom0, which hang very soon. But it's visible in kernel > > logs of the dom0, not the domU. > > Weird. I'm using 3.10.0 without i

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-08-28 Thread Sylvain Munaut
Hi, > I am doing some experimentation with Ceph and Xen (on the same host) and I'm > experiencing some problems with the rbd device that I'm using as the block > device. My environment is: AFAIK running the OSD and the kernel driver inside the same host is not recommended at all ... Personally

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-08-28 Thread Sylvain Munaut
Hi, > I use Ceph 0.61.8 and Xen 4.2.2 (Debian) in production, and can't use > kernel 3.10.* on dom0, which hang very soon. But it's visible in kernel > logs of the dom0, not the domU. Weird. I'm using 3.10.0 without issue here. What's the issue you're hitting ? Cheers, Sylvain _

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-08-28 Thread James Dingwall
Sage Weil wrote: Can you post the contents of the hung task warning so we can see where it is stuck? The messages in the domU are: [35284.544137] INFO: task dd:4903 blocked for more than 120 seconds. [35284.544148] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [3528

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-08-27 Thread Sage Weil
Hi James, Can you post the contents of the hung task warning so we can see where it is stuck? Thanks! sage On Tue, 27 Aug 2013, James Dingwall wrote: > Hi, > > I am doing some experimentation with Ceph and Xen (on the same host) and I'm > experiencing some problems with the rbd device that I

Re: [ceph-users] Ceph + Xen - RBD io hang

2013-08-27 Thread Olivier Bonvalet
Hi, I use Ceph 0.61.8 and Xen 4.2.2 (Debian) in production, and can't use kernel 3.10.* on dom0, which hang very soon. But it's visible in kernel logs of the dom0, not the domU. Anyway, you should probably re-try with kernel 3.9.11 for the dom0 (I also use 3.10.9 in domU). Olivier Le mardi 27 a

[ceph-users] Ceph + Xen - RBD io hang

2013-08-27 Thread James Dingwall
Hi, I am doing some experimentation with Ceph and Xen (on the same host) and I'm experiencing some problems with the rbd device that I'm using as the block device. My environment is: 2 node Ceph 0.67.2 cluster, 4x OSD (btrfs) and 1x mon Xen 4.3.0 Kernel 3.10.9 The domU I'm trying to build i