[ceph-users] Re: About scrub and deep-scrub

2024-10-09 Thread xadhoom76
Hi, you could work on threshold , it depends on the average nodes load. set something like this ceph tell 'osd.*' injectargs --osd_max_scrubs=2 --osd_scrub_load_threshold=6 and ceph tell 'osd.*' injectargs '--osd-deep-scrub-interval 1209600' ceph tell 'osd.*' injectargs '--osd-scrub-max-interv

[ceph-users] Re: About scrub and deep-scrub

2024-10-08 Thread Phong Tran Thanh
Hi Daniel Thank your advide Vào Th 2, 7 thg 10, 2024 vào lúc 17:10 Daniel Baumann < daniel.baum...@bfh.ch> đã viết: > Hi, > > On 10/7/24 09:27, Phong Tran Thanh wrote: > > How about the disable scrub and deep-scrub > > neither scrubbing nor deep-scrubbing should be disabled, it is an > integra

[ceph-users] Re: About scrub and deep-scrub

2024-10-07 Thread Phong Tran Thanh
Yes, I understand. Many thank Eugen Vào Th 2, 7 thg 10, 2024 vào lúc 16:29 Eugen Block đã viết: > Well, if you have more PGs, the deep-scrubs per PG finish faster, but > you have more PGs to scrub. But that way you can stretch the > deep-scrub interval. As I said, I can not recommend to increa

[ceph-users] Re: About scrub and deep-scrub

2024-10-07 Thread Eugen Block
Well, if you have more PGs, the deep-scrubs per PG finish faster, but you have more PGs to scrub. But that way you can stretch the deep-scrub interval. As I said, I can not recommend to increase them in general. Overloading OSDs with too many PGs can have a negative effect as well. If you c

[ceph-users] Re: About scrub and deep-scrub

2024-10-07 Thread Phong Tran Thanh
Hi Eugen Adding PG to the cluster only helps to reduce PG size and reduce scrub time is right? >150 PG and <200 in osd i think it's good number My cluster I/O 2-3GB/s real time in 24h IOPS 2K. If choosing to increase PG is it a good choice? Vào Th 2, 7 thg 10, 2024 vào lúc 15:49 Eugen Block đ

[ceph-users] Re: About scrub and deep-scrub

2024-10-07 Thread Eugen Block
So your PGs for pool 52 have a size of around 320 GB, that is quite a lot and not surprising that deep-scrubs take a long time. At the same time, your PGs per OSD are already > 150. We had a similar situation on a customer cluster this year as well, also with 12 TB drives. We decided to inc

[ceph-users] Re: About scrub and deep-scrub

2024-10-07 Thread Phong Tran Thanh
Hi Eugen Can you see and give me some advice, number of PG and PG size.. ID CLASS WEIGHTREWEIGHT SIZE RAW USE DATA OMAP META AVAIL%USE VAR PGS STATUS 2hdd 10.98349 1.0 11 TiB 4.5 TiB 4.2 TiB 5.0 MiB 12 GiB 6.5 TiB 40.77 1.00 182 up 17

[ceph-users] Re: About scrub and deep-scrub

2024-10-07 Thread Eugen Block
Hi, disabling scrubbing in general is bad idea, because you won't notice any data corruption except when it might be too late. But you can fine tune scrubbing, for example increase the interval to allow fewer scrubs to finish in a longer interval. Or if the client load is mainly during busi

[ceph-users] Re: About scrub and deep-scrub

2024-10-07 Thread Daniel Baumann
Hi, On 10/7/24 09:27, Phong Tran Thanh wrote: > How about the disable scrub and deep-scrub neither scrubbing nor deep-scrubbing should be disabled, it is an integral part of ensuring data consistency and data availability. if you disable it, ceph will not know when/if data on the disks/ssds has