The apply latency is how long it's taking for the backing filesystem to ack (not sync to disk) writes from the OSD. Either it's getting a lot more writes than the other OSDs (you can check by seeing how many PGs are mapped to each) and then just apply standard local fs debugging techniques to that local node. -Greg
On Monday, March 3, 2014, Chris Kitzmiller <ckitzmil...@hampshire.edu> wrote: > I've got a 3 node cluster where ceph osd perf reports reasonable > fs_apply_latency for 2 out of 3 of my nodes (~30ms). But on the third node > I've got latencies averaging 15000+ms for all OSDs. > > Running ceph 72.2 on Ubuntu 10.13. Each node has 30 HDDs with 6 SSDs for > journals. iperf reports full bidirectional 10Gbps. fio locally on any of > the nodes across all OSDs gives me ~4000MBps with clat's ~60ms. At first I > thought this was an issue with my client running rbd but rados bench is > also very slow for writes. > > ceph osd perf, rados bench, and fio output at: > http://pastebin.com/Kze0AKnr > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com <javascript:;> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Software Engineer #42 @ http://inktank.com | http://ceph.com
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
