[ceph-users] Re: Full OSD with almost empty pools

[Adam Prycki]

Hi,

This crush rule is not in use on any pool
Every crush rule in use has configured class

ceph osd dump | grep pool


[
    {
        "rule_id": 0,
        "rule_name": "replicated_rule",
        "type": 1,
        "steps": [
            {
                "op": "take",
                "item": -1,
                "item_name": "default"
            },
            {
                "op": "chooseleaf_firstn",
                "num": 0,
                "type": "host"
            },
            {
                "op": "emit"
            }
        ]
    },
    {
        "rule_id": 2,
        "rule_name": "replicated_rack_hdd",
        "type": 1,
        "steps": [
            {
                "op": "take",
                "item": -170,
                "item_name": "default~hdd"
            },
            {
                "op": "chooseleaf_firstn",
                "num": 0,
                "type": "rack"
            },
            {
                "op": "emit"
            }
        ]
    },
    {
        "rule_id": 3,
        "rule_name": "cephfs.arm-vol.data",
        "type": 3,
        "steps": [
            {
                "op": "set_chooseleaf_tries",
                "num": 5
            },
            {
                "op": "set_choose_tries",
                "num": 100
            },
            {
                "op": "take",
                "item": -170,
                "item_name": "default~hdd"
            },
            {
                "op": "chooseleaf_indep",
                "num": 0,
                "type": "rack"
            },
            {
                "op": "emit"
            }
        ]
    },
    {
        "rule_id": 4,
        "rule_name": "replicated_rack_ssd",
        "type": 1,
        "steps": [
            {
                "op": "take",
                "item": -340,
                "item_name": "default~ssd"
            },
            {
                "op": "chooseleaf_firstn",
                "num": 0,
                "type": "rack"
            },
            {
                "op": "emit"
            }
        ]
    }
]

Best regards
Adam Prycki


W dniu 5.05.2025 o 19:48, Anthony D'Atri pisze:

pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 2 object_hash rjenkins pg_num 15 pgp_num 7 pg_num_target 1 pgp_num_target 1 autoscale_mode on last_change 1763264 lfor 0/1763261/1763259 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 420.00 pool 2 'cephfs.arm-vol.meta' replicated size 3 min_size 2 crush_rule 4 object_hash rjenkins pg_num 4096 pgp_num 4096 autoscale_mode on last_change 1752586 lfor 0/445075/445073 flags hashpspool,nearfull,bulk stripe_width 0 pg_autoscale_bias 1 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.51 pool 4 'cephfs.arm-vol.data' erasure profile ec_k6_m2_rack_hdd size 8 min_size 7 crush_rule 3 object_hash rjenkins pg_num 16384 pgp_num 16384 autoscale_mode on last_change 3427 lfor 0/0/3233 flags hashpspool,ec_overwrites,bulk stripe_width 24576 application cephfs pool 24 '.rgw.root' replicated size 3 min_size 2 crush_rule 2 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 1092266 lfor 0/0/1092257 flags hashpspool stripe_width 0 application rgw read_balance_score 75.00 pool 25 'bst-ro-backup.rgw.log' replicated size 3 min_size 2 crush_rule 2 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 1092436 lfor 0/0/1092257 flags hashpspool stripe_width 0 application rgw read_balance_score 74.25 pool 26 'bst-ro-backup.rgw.control' replicated size 3 min_size 2 crush_rule 2 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 1092266 lfor 0/0/1092259 flags hashpspool stripe_width 0 application rgw read_balance_score 72.75 pool 27 'bst-ro-backup.rgw.meta' replicated size 3 min_size 2 crush_rule 2 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 1092436 lfor 0/0/1092259 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw read_balance_score 73.50 pool 32 '.nfs' replicated size 3 min_size 2 crush_rule 4 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 1763299 lfor 0/0/1092261 flags hashpspool,nearfull stripe_width 0 application nfs read_balance_score 10.02

On May 5, 2025, at 10:58 AM, Adam Prycki <apry...@man.poznan.pl> wrote:

Every crush rules has device class

No device class here.


[
     {
         "rule_id": 0,
         "rule_name": "replicated_rule",
         "type": 1,
         "steps": [
             {
                 "op": "take",
                 "item": -1,
                 "item_name": "default"
             },
             {
                 "op": "chooseleaf_firstn",
                 "num": 0,
                 "type": "host"
             },
             {
                 "op": "emit"
             }
         ]
     },


_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

smime.p7s
Description: Kryptograficzna sygnatura S/MIME

_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io