Re: [ceph-users] ceph 12.2.9 release

2018-11-07 Thread Thomas White
One of the Ceph clusters my team manages is on 12.2.9 on a Proxmox environment seems to be running fine with simple x3 replication and RBD. Would be interesting to know what issues have been encountered so far. All our OSDs are simple filestore at present and our path to 12.2.9 was 10.2.7 -> 10.2.1

[ceph-users] All shards of PG missing object and inconsistent

2018-09-21 Thread Thomas White
Hi all, I have recently performed a few tasks, namely purging several buckets from our RGWs and added additional hosts into Ceph causing some data movement for a rebalance. As this is now almost completed, I kicked off some deep scrubs and one PG is now returning the following information:

Re: [ceph-users] SAN or DAS for Production ceph

2018-08-28 Thread Thomas White
Hi James, I can see where some of the confusion has arisen, hopefully I can put at least some of it to rest. In the Tumblr post from Yahoo, the keyword to look out for is “nodes”, which is distinct from individual hard drives which in Ceph is an OSD in most cases. So you would have multiple

Re: [ceph-users] Inconsistent PG could not be repaired

2018-08-14 Thread Thomas White
Hi Arvydas, The error seems to suggest this is not an issue with your object data, but the expected object digest data. I am unable to access where I stored my very hacky diagnosis process for this, but our eventual fix was to locate the bucket or files affected and then rename an object wit

Re: [ceph-users] Ceph upgrade Jewel to Luminous

2018-08-14 Thread Thomas White
Hi Jaime, Upgrading directly should not be a problem. It is usually recommended to go to the latest minor release before upgrading major versions, but my own migration from 10.2.10 to 12.2.5 went seamlessly and I can’t see of any technical limitation which would hinder or prevent this proces

Re: [ceph-users] limited disk slots - should I ran OS on SD card ?

2018-08-13 Thread Thomas White
Hi Steven, Just to somewhat clarify my previous post, I mention OSDs in the sense that the OS is installed on the OSD server using the SD card, I would absolutely recommend against using SD cards as the actual OSD media. This of course misses another point, which is for the Mons or other suc

[ceph-users] Bluestore OSD Segfaults (12.2.5/12.2.7)

2018-08-07 Thread Thomas White
Hi all, We have recently begun switching over to Bluestore on our Ceph cluster, currently on 12.2.7. We first began encountering segfaults on Bluestore during 12.2.5, but strangely these segfaults apply exclusively to our SSD pools and not the PCIE/HDD disks. We upgraded to 12.2.7 last week to

[ceph-users] Reset Object ACLs in RGW

2018-08-02 Thread Thomas White
Hi all, At present I have a cluster with a user on the RGW who has lost access to many of his files. The bucket has the correct ACL to be accessed by the account and so with their access and secret key many items can be listed, but are unable to be downloaded. Is there a way of using the rados