From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Stuart Harland Sent: 10 January 2017 11:58 To: Wido den Hollander <w...@42on.com> Cc: ceph new <ceph-users@lists.ceph.com>; n...@fisk.me.uk Subject: Re: [ceph-users] Write back cache removal
Yes Wido, you are correct. There is a RBD pool in the cluster, but is not currently running with a cache attached. The Pool I’m trying to manage here is only used by Librados to write objects directly to the pool as opposed to any of the other niceties that ceph provides. Specifically I ran: `ceph osd tier cache-mode <hot-storage> forward` which returned `Error EPERM: 'forward' is not a well-supported cache mode and may corrupt your data. pass --yes-i-really-mean-it to force.` Currently we are running 10.2.5. I suspect that it’s fine in our use case, however given the sparsity of the documentation I didn’t like to assume anything. Regards Stuart Yep, sorry, I got this post mixed up with the one from Daznis yesterday who was using RBD’s. I think that warning was introduced as some bugs were found that corrupted some users data after frequently switching between writeback and forward modes. As it is very rarely used mode and so wasn’t worth the testing I believe the decision was taken to just implement the warning. If you are using it as part of removing a cache tier and you have already flushed the tier, then I believe it should be fine to use. Another way would probably be to set the min promote thresholds to higher than your hit set counts, this will abuse the tiering logic but should also stop anything getting promoted into your cache tier. On 10 Jan 2017, at 09:52, Wido den Hollander <w...@42on.com <mailto:w...@42on.com> > wrote: Op 10 januari 2017 om 9:52 schreef Nick Fisk <n...@fisk.me.uk <mailto:n...@fisk.me.uk> >: -----Original Message----- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Wido den Hollander Sent: 10 January 2017 07:54 To: ceph new <ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com> >; Stuart Harland <s.harl...@livelinktechnology.net <mailto:s.harl...@livelinktechnology.net> > Subject: Re: [ceph-users] Write back cache removal Op 9 januari 2017 om 13:02 schreef Stuart Harland <s.harl...@livelinktechnology.net <mailto:s.harl...@livelinktechnology.net> >: Hi, We’ve been operating a ceph storage system storing files using librados (using a replicated pool on rust disks). We implemented a cache over the top of this with SSDs, however we now want to turn this off. The documentation suggests setting the cache mode to forward before draining the pool, however the ceph management controller spits out an error about this saying that it is unsupported and hence dangerous. What version of Ceph are you running? And can you paste the exact command and the output? Wido Hi Wido, I think this has been discussed before and looks like it might be a current limitation. Not sure if it's on anybody's radar to fix. https://www.mail-archive.com/ceph-users@lists.ceph.com/msg24472.html Might be, but afaik they are using their own application which writes to RADOS using librados, not RBD. Is that correct Stuart? Wido Nick The thing is I cannot really locate any documentation as to why it’s considered unsupported and under what conditions it is expected to fail: I have read a passing comment about EC pools having data corruption, but we are using replicated pools. Is this something that is safe to do? Otherwise I have noted the read proxy mode of cache tiers which is documented as a mechanism to transition from write back to disabled, however the documentation is even sparser on this than forward mode. Would this be a better approach if there is some unsupported behaviour in the forward mode cache option? Any thoughts would be appreciated - we really cannot afford to corrupt the data, and I really do not want to have to do some manual software based eviction on this data. regards Stuart − Stuart Harland: Infrastructure Engineer Email: s.harl...@livelinktechnology.net <mailto:s.harl...@livelinktechnology.net> <mailto:s.harl...@livelinktechnology.net> LiveLink Technology Ltd McCormack House 56A East Street Havant PO9 1BS IMPORTANT: The information transmitted in this e-mail is intended only for the person or entity to whom it is addressed and may contain confidential and/or privileged information. If you are not the intended recipient of this message, please do not read, copy, use or disclose this communication and notify the sender immediately. Any review, retransmission, dissemination or other use of, or taking any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. Any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of LiveLink. This e-mail message has been checked for the presence of computer viruses. However, LiveLink is not able to accept liability for any damage caused by this e-mail. _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
