I'm quite clear now. > this is my test setup - that's why I'm trying to break it and fix it (best way to learn)
Thanks for your feedback!! Kinjo On Sun, Jul 5, 2015 at 8:51 PM, Jacek Jarosiewicz < jjarosiew...@supermedia.pl> wrote: > Well, the docs say that when your osds get full you should add another osd > - and the cluster should redistribute data by it self: > > > http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-osd/#no-free-drive-space > > this is my test setup - that's why I'm trying to break it and fix it (best > way to learn) - so I intentionally filled the cache pool and saw this > problem twice - when I have full or near full osds the cluster becomes a > little less responsive and I get the "blocked requests" warning and some > clients hang.. I tried to delete some files manually before but I think I > deleted too many or something else went wrong, because the last time I > could not get the cluster working and had to reinitialize all the osds from > scratch. This time it worked - but I'm worried and wondering that maybe > there is a better way to go about this problem.. > > I was thinking that maybe - if I upped the near full and full ratio - the > warning would go away and maybe I would be able to flush the cache pool. > But that's only a solution for the cache pool - I'd rather not touch the > normal data on the cold storage.. > > Cheers, > J > > On 07/05/2015 01:27 PM, Shinobu Kinjo wrote: > >> That's good! >> >> So was the root cause is because the osd was full? What's your thought >> about that? >> Was there any reason to delete any files? >> >> Kinjo >> >> >> On Sun, Jul 5, 2015 at 6:51 PM, Jacek Jarosiewicz >> <jjarosiew...@supermedia.pl <mailto:jjarosiew...@supermedia.pl>> wrote: >> >> ok, I got it working... >> >> first i manually deleted some files from the full osd, set the flag >> noout and restarted the osd daemon. >> >> then i waited a while for the cluster to backfill pgs, and after >> that the rados -p cache cache-try-flush-evict-all command went OK. >> >> I'm wondering though, because this happened to me before (hanging >> blocked request when osd's are near full), is there a better way >> maybe to solve this problem? >> >> J >> >> -- >> Life w/ Linux <http://i-shinobu.hatenablog.com/> >> > > > -- > Jacek Jarosiewicz > Administrator Systemów Informatycznych > > > ---------------------------------------------------------------------------------------- > SUPERMEDIA Sp. z o.o. z siedzibą w Warszawie > ul. Senatorska 13/15, 00-075 Warszawa > Sąd Rejonowy dla m.st.Warszawy, XII Wydział Gospodarczy Krajowego Rejestru > Sądowego, > nr KRS 0000029537; kapitał zakładowy 42.756.000 zł > NIP: 957-05-49-503 > Adres korespondencyjny: ul. Jubilerska 10, 04-190 Warszawa > > > ---------------------------------------------------------------------------------------- > SUPERMEDIA -> http://www.supermedia.pl > dostep do internetu - hosting - kolokacja - lacza - telefonia > -- Life w/ Linux <http://i-shinobu.hatenablog.com/>
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
