Re: [ceph-users] Need suggestions for using ceph as reliable block storage

2016-01-03 Thread Robert LeBlanc
I could see that if you have size and min_size equal. Can you provide some details about your set up? The peering souks be pretty fast and if min_size < size then writes can happen without recovery. Also if you are using KVM, I suggest using librbd instead of KRBD. If something funky happens with

Re: [ceph-users] Need suggestions for using ceph as reliable block storage

2016-01-03 Thread Kalyana sundaram
As far as I read as soon as a osd is marked down writes won't recover because pgs have to be peered and the object has to be recovered before being written. We got kernel hung task timeout on a bunch of vms when a ceph node was taken down. On Jan 4, 2016 11:04 AM, "Robert LeBlanc" wrote: > I'm no

Re: [ceph-users] Need suggestions for using ceph as reliable block storage

2016-01-03 Thread Robert LeBlanc
I'm not sure what you mean by transparent? Does the IO hang forever when a node goes down? If an OSD is taken down gracefully then there should be minimal disruption of traffic. If you yank the network or power cables, it can take 30 seconds before the cluster considers it down to mark it bad. Ro