>> >> Is it possible to create a three level cache tier? Searching documentation >> and archives suggests that I’m not the first one to ask about >> it, but I can’t tell if it is supported yet. > > At this time, this is not possible. I'm afraid I'm not aware of any short > term plans for this to be implemented. Not sure if anybody has any more info > around plans for this?
It sure would be nice to stack ram (RBD cache) to NVMe to SSD to Spinning disk. We have mixed workloads, some data is intensely used and some just sits there doing nothing. Also, we allow the public to provision resources so if we create multiple pools everyone will just pick the cheapest and complain it’s slow. I do see some nifty tricks where we could make the second tier a SSD/Platter combo by setting affinity to 0 on the spinning disks and pooling SSD and Spinning drives, but I think this will create a write bottleneck when data is passed from the cache to the storage. It could push us into a scenario where we have to keep a much larger NVMe pools than we would otherwise need. But, it is documented. At the least we could push replicated data to spinning disk and hit the SSD for all read activity. The only problem is write commitment on data that gets pushed into the storage pool won’t happen until replicas are created. That makes the spinning disk the slowest common denominator. I’ve toyed with the idea of matching one SSD to one SAS, using another caching mechanism that presents Ceph with one block devices that’s a mix of the two, but now I’m getting so far away from documented practices that I’m on my own if it has problems in production. I’ve got a few hundred servers to play with having different configs, but I want to minimize my trial and error by following the path everyone else has and learn from those mistakes. It wastes a lot of time to build this out in hardware only to find out it won’t play nice or has some problem that wasn’t anticipated. Having a large pool of spinning disks helps us manage large volumes of data that doesn’t do very much. Rob _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
