I have a stage cluster with 4 HDDs and an SSD in each host.  I have an EC
profile that specifically chooses HDDs for placement.  Also several Replica
pools that write to either HDD or SSD.  This has all worked well for a
while.  When I updated the Tunables to Jewel on the cluster, all of a
sudden the data for the EC profile started placing it's data on the SSDs
and filling them up.  Setting the CRUSH Tunables back to Hammer reverts
this change and all is well again.

The odd part is that it's not like it's choosing to mix the data on HDDs
and SSDs, it just moves the data to all SSDs and off of the HDDs.  Has
anyone else experienced this or know what is causing it to choose to place
the EC PGs on the wrong device-class?

[1] This is the rule in question.


[1]
{
    "rule_id": 2,
    "rule_name": "local-stage.rgw.buckets.data",
    "ruleset": 2,
    "type": 3,
    "min_size": 3,
    "max_size": 5,
    "steps": [
        {
            "op": "set_chooseleaf_tries",
            "num": 5
        },
        {
            "op": "set_choose_tries",
            "num": 100
        },
        {
            "op": "take",
            "item": -24,
            "item_name": "default~hdd"
        },
        {
            "op": "chooseleaf_indep",
            "num": 0,
            "type": "host"
        },
        {
            "op": "emit"
        }
    ]
}
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to