Hi, I have an openstack cluster which runs on ceph . I tried running hadoop inside VM's and I noticed that map tasks take long time to complete with time and finally it fails. RDB read/writes are getting slower with time. Is it because of too many objects in ceph per volume?
I have 8 node cluster with 24* 1TB disk for each node. master : mon slave1: 1 osd per disk, ie 23 slave 2: 1 osd per disk ie 23 ..... ..... slave 7 : 1 osd per isk ie 23 replication factor :2 pg nums in default pool: 128 In openstack,I have 14 instances. 14 5TB volumes are created and each one is attached to an instance.I am using default stripe settings. rpd -p volumes info volume-1 size 5000GB in 1280000 objects order 22(4096kb objects) 1.I couldn't find the documentation for stripe settings which can be used for volume creation in openstack. Can it be exposed through any configuration files? . (http://ceph.com/docs/master/rbd/rbd-openstack/). Like 64MB default block size in hdfs, how do we set layout for the objects,?.Can we change it after volume creation ?. Will this affect performance for huge I/O applications like MapReduce? 2. How can RBD caching improve the performance? 3. Like in hdfs which gives priority over localized writes, how can we implement the same feature because rbd volumes are striped across the cluster. I am not sure of crush rulesets which can help this situation Can someone give me debug points and ideas related to this?. I have not used cephfs for now .
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com