Re: [ceph-users] cephfs rm -rf on directory of 160TB /40M files

On Wed, Apr 6, 2016 at 10:42 PM, Scottix wrote: > I have been running some speed tests in POSIX file operations and I noticed > even just listing files can take a while compared to an attached HDD. I am > wondering is there a reason it takes so long to even just list files. If you're running comp

Re: [ceph-users] cephfs rm -rf on directory of 160TB /40M files

On Wed, Apr 6, 2016 at 2:42 PM, Scottix wrote: > I have been running some speed tests in POSIX file operations and I noticed > even just listing files can take a while compared to an attached HDD. I am > wondering is there a reason it takes so long to even just list files. > > Here is the test I r

Re: [ceph-users] cephfs rm -rf on directory of 160TB /40M files

I have been running some speed tests in POSIX file operations and I noticed even just listing files can take a while compared to an attached HDD. I am wondering is there a reason it takes so long to even just list files. Here is the test I ran time for i in {1..10}; do touch $i; done Interna

Re: [ceph-users] cephfs rm -rf on directory of 160TB /40M files

On Mon, Apr 4, 2016 at 9:55 AM, Gregory Farnum wrote: > Deletes are just slow right now. You can look at the ops in flight on you > client or MDS admin socket to see how far along it is and watch them to see > how long stuff is taking -- I think it's a sync disk commit for each unlink > though so

Re: [ceph-users] cephfs rm -rf on directory of 160TB /40M files

Thanks! Now this slow rm is not our biggest issue anymore (for the moment).. Since this night all our MDSs did crash. I opened a ticket http://tracker.ceph.com/issues/15379 with the stacktraces I got. We aren't able to restart the mds's at all for now.. I stopped the rm, and also increased the

Re: [ceph-users] cephfs rm -rf on directory of 160TB /40M files

On Tue, Apr 5, 2016 at 12:55 AM, Gregory Farnum wrote: > Deletes are just slow right now. You can look at the ops in flight on you > client or MDS admin socket to see how far along it is and watch them to see > how long stuff is taking -- I think it's a sync disk commit for each unlink > though so

Re: [ceph-users] cephfs rm -rf on directory of 160TB /40M files

Deletes are just slow right now. You can look at the ops in flight on you client or MDS admin socket to see how far along it is and watch them to see how long stuff is taking -- I think it's a sync disk commit for each unlink though so at 40M it's going to be a good looong while. :/ -Greg On Monda