[ceph-users] Re: 3 OSDs can not be started after a server reboot - rocksdb Corruption

2022-03-15 Thread Igor Fedotov
Hi Sebastian, I don't think you have got tons of corrupted objects. The tricky thing about the bug is that corruption might occur if new allocation occurred in a pretty short period only:  when OSD is starting but haven't applied deferred writes yet. This mostly applies to Bluefs/RocksDB perfo

[ceph-users] Replication problems on multi-sites configuration

2022-03-15 Thread Gilles Mocellin
Hello Cephers, We've just configured multi-site between an existing Octopus cluster and a new one in another Datacenter. On our biggest bucket (replic_cfn_prod/cfb : 1.4 M objects, 670 GB) we have many errors like that on the new site : 2022-03-15T10:21:00.800+0100 7f9834750700 1 == st

[ceph-users] Re: Replication problems on multi-sites configuration

2022-03-15 Thread Gilles Mocellin
Hum, it seems I hit that bug : https://tracker.ceph.com/issues/50785 Merged in Pacific, not backported to Octopus, at least not linked to this issue. One more reason to migrate to Pacific... Le 2022-03-15 10:43, Gilles Mocellin a écrit : Hello Cephers, We've just configured multi-site betwee

[ceph-users] 17 OSDs down simultaneously from past_interval assert

2022-03-15 Thread Robert LeBlanc
We had a host that had hung this morning and after restarting it 17 OSDs across the cluster crashed with a past_interval assert. The weird thing is that the OSD complaining about the PG isn't in the acting, up or acting_recovery_backfill sets. This is 14.2.22 with BlueStore. It appears that it's t

[ceph-users] Re: 17 OSDs down simultaneously from past_interval assert

2022-03-15 Thread Robert LeBlanc
TLDR; Old crusty PGs that could be deleted without consequence. Since it looked like we had a full set of OSDs providing the PG, I went ahead and removed the PG from the OSD that was having trouble and the OSD booted fine. We were able to do this for all 17 OSDs and the cluster is now online again

[ceph-users] Ceph User + Dev Monthly March Meetup

2022-03-15 Thread Neha Ojha
Hi everyone, This month's Ceph User + Dev Monthly meetup is on March 17, 14:00-15:00 UTC (note the time change!). Please add topics you'd like to discuss in the agenda: https://pad.ceph.com/p/ceph-user-dev-monthly-minutes. Hope to see you there! Thanks, Neha

[ceph-users] Re: rbd namespace create - operation not supported

2022-03-15 Thread Kai Stian Olstad
On 11.03.2022 14:04, Ilya Dryomov wrote: On Fri, Mar 11, 2022 at 8:04 AM Kai Stian Olstad wrote: Isn't namespace supported with erasure encoded pools? RBD images can't be created in EC pools, so attempting to create RBD namespaces there is pointless. The way to store RBD image data in an EC