Hi Anthony, >They do for sure. Note that when using these for WAL+DB offload, conventional >wisdom is to not exceed 5:1 for SATA SSDs. 36:1 as you propose would likely >yield *worse* performance than leaving WAL+DB colocated with the main OSD data
Thanks for the info, that's helpful to know. >I might consider constructing your CephFS with the metadata and first data >pools on 3x replicated pools constrained to the SSD OSDs and a second data >pool added for the HDDs, replicated or EC as you see fit. Then use the layout I will explore this path. >Remember that Ceph does replication for you. This strategy is effectively >RAID on top of RAID. It would overload your SATA SSDs, resulting in slow >RADOS and MDS ops. Your Prom /Grafana would show false alarms. My >Cephalocon presentation from 2024 demonstrates this dynamic, and I’ve seen it >on community systems. >If mirrored, each of those SSDs would see the writes from 36 OSDs. Which would >really be ugly. The main motivation for RAID1 is having more resilience for the OS, so in case one SSD fails, the system stays online. >* Download SST from Solidigm and ensure the SSDs have up to date firmware. Or >if Dell, DSU, etc. Noted, thank you! >what I would do is mirror ~500 GiB for the OS, with minimal partioning and no >swap, and use the balance on *each* SSD for an unmirrored OSD. I tried that with my HBA, but it didn't work, it seems that MD is a better idea anyway. I will go ahead and try that. One thing that popped up in my mind, if I do this, the reamining unused space, e.g., /dev/sdX2 in both disks, will be *partitions*, and not *disks*. Does Ceph have a problem with using partitions for OSDs? >I think you’d not have a good experience, see my compromise suggestion above. >Remind me how many chassis total? As a certain chassis count, the importance >of mirrored boot (and network bonding) diminishes. We have 13x chassis. I am already convinced to not have OSDs on VDs :-) >Does it show that the BBU is a supercap vs a battery? It looks like supercap: Cachevault_Info : =============== -------------------- Property Value -------------------- Type CVPM05 Temperature 24 C State Optimal -------------------- ... GasGaugeStatus : ============== ------------------------------ Property Value ------------------------------ Pack Energy 136 J Capacitance 100 % Remaining Reserve Space 0 ------------------------------ >When I see RAID HBAs I either just set passthrough and ignore the RoC I think I'll do just that and just use MD for the small RAID volume for the OS. Thank you, Gustavo _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
