Re: [ceph-users] incomplete pg, recovery some data

2015-06-20 Thread Mateusz Skała
Thanks for help, I have solved this problem in other way. After many time spent on importing data to osd where data was in past, always pg was incomplete. I tried to import to one osd, to all osd, always pg have status 'incomplete'. At last I do ceph pg force_create_pg 0.109 and pg was creating

[ceph-users] Peeks on physical drives, iops on drive, ceph performance

2015-06-20 Thread Mateusz Skała
Hi, Looking for better performance on our cluster, we found this blog https://telekomcloud.github.io/ceph/2014/02/26/ceph-performance-analysis_fio _rbd.html In our environment IOPS looks like on this picture of the blog https://telekomcloud.github.io/images/2014-02-26-ceph-performance-analysis_f

Re: [ceph-users] Old vs New pool on same OSDs - Performance Difference

2015-06-20 Thread Nick Fisk
Sorry to dig up this old thread, but I've finally had time to schedule some quiet time on the cluster and perform some more testing. Just to recap, the problem is not related to pools, but it appears to be with RBD's which haven't been accessed in a while. Re-writing over blocks of an old RBD s

Re: [ceph-users] New cluster in unhealthy state

2015-06-20 Thread Nick Fisk
Hi Dave, It can't increase the pgp's because the pg's are still being created. I can see you currently have 2 OSD's down, not 100% certain this is the cause, but you might to try and get them back online or remove them if they no longer exist. Nick From: ceph-users [mailto:ceph-users-bo

Re: [ceph-users] Old vs New pool on same OSDs - Performance Difference

2015-06-20 Thread Somnath Roy
Nick, I think the only IO operation between 'reached_pg' and 'started' is fetching the xttrs. Check the following. 1. Enable debug log 20 (for OSD and filestore) and record the time taken by the function find_object_context/get_object_context/get_snapset_context/getattr (in filestore). 2. If n

Re: [ceph-users] Old vs New pool on same OSDs - Performance Difference

2015-06-20 Thread Nick Fisk
Thanks for your response Somnath, Responses inline > -Original Message- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of > Somnath Roy > Sent: 20 June 2015 17:52 > To: Nick Fisk > Cc: ceph-users@lists.ceph.com > Subject: Re: [ceph-users] Old vs New pool on same OS

Re: [ceph-users] Old vs New pool on same OSDs - Performance Difference

2015-06-20 Thread Somnath Roy
What release you are using ? filestore_xattr_use_omap is deprecated long back.. Thanks & Regards Somnath -Original Message- From: Nick Fisk [mailto:n...@fisk.me.uk] Sent: Saturday, June 20, 2015 1:51 PM To: Somnath Roy Cc: ceph-users@lists.ceph.com Subject: RE: [ceph-users] Old vs New po