[ceph-users] Re: fixing future rctimes

2021-05-07 Thread David Rivera
Hi guys, Did anyone ever figure out how to fix rctime? I had a directory that was robocopied from a windows host that contained files with modified times in the future. Now the directory tree up to the root will not update rctime. Thanks, David ___ ceph

[ceph-users] Re: orch upgrade mgr starts too slow and is terminated?

2021-05-07 Thread Kai Börnert
Hi, thanks for your explanation. I see the 16.2.3 build is published, but (of course) I cannot update to it, as the version I currently use does still have this bug. Is there some workaround/hack I can use to upgrade? So far I tried to directly specify the image names and go via --ceph-versi

[ceph-users] Re: orch upgrade mgr starts too slow and is terminated?

2021-05-07 Thread Robert Sander
Am 07.05.21 um 10:42 schrieb Kai Börnert: > Hi, thanks for your explanation. > > I see the 16.2.3 build is published, but (of course) I cannot update to > it, as the version I currently use does still have this bug. > > Is there some workaround/hack I can use to upgrade? I had success with stopp

[ceph-users] Re: orch upgrade mgr starts too slow and is terminated?

2021-05-07 Thread Kai Börnert
Thanks, this did work for me, (additionally I had to stop and restart the upgrade after stopping and restarting the first mgr manually, as it was in a failed no stdby mgr state), the cluster is now proceeding to updating the osds :) On 5/7/21 11:28 AM, Robert Sander wrote: Am 07.05.21 um 10

[ceph-users] Re: Stuck OSD service specification - can't remove

2021-05-07 Thread mabi
Hi David, I had a similar issue yesterday where I wanted to remove an OSD on an OSD node which had 2 OSDs so for that I used "ceph orch osd rm" command which completed successfully but after rebooting that OSD node I saw it was still trying to start the systemd service for that OSD and one CPU

[ceph-users] Re: Weird PG Acting Set

2021-05-07 Thread Lazuardi Nasution
Hi, After recreating some related OSDs (3, 71 and 237), now the acting set is normal but the PG is incomplete now and there are slow ops on primary OSD (3). I have tried to make it normal with osd_find_best_info_ignore_history_les way but the PG is still incomplete. On this condition the I/O from

[ceph-users] Re: Stuck OSD service specification - can't remove

2021-05-07 Thread David Orman
Hi, I'm not attempting to remove the OSDs, but instead the service/placement specification. I want the OSDs/data to persist. --force did not work on the service, as noted in the original email. Thank you, David On Fri, May 7, 2021 at 1:36 AM mabi wrote: > > Hi David, > > I had a similar issue y

[ceph-users] [v15.2.11] radosgw / RGW crash at start, Segmentation Fault

2021-05-07 Thread Gilles Mocellin
Hello, Since I upgrade to Ceph Octopus v15.2.11, on Ubuntu 18.04, Radosgw crash straight at start. On Two clusters, one Lab, and some test on a production cluster, shows the same crash for radosgw. As I don't find any similar bug in the Tracker, neither in this mailing list... Am I alone ?

[ceph-users] Re: [v15.2.11] radosgw / RGW crash at start, Segmentation Fault

2021-05-07 Thread Casey Bodley
this is https://tracker.ceph.com/issues/50218, a radosgw build issue specific to ubuntu bionic that affected all of our releases. the build issue has been resolved, so the next point releases should resolve the crashes On Fri, May 7, 2021 at 10:51 AM Gilles Mocellin wrote: > > Hello, > > Since I

[ceph-users] Re: Weird PG Acting Set

2021-05-07 Thread 胡玮文
在 2021/5/7 下午6:46, Lazuardi Nasution 写道: Hi, After recreating some related OSDs (3, 71 and 237), now the acting set is normal but the PG is incomplete now and there are slow ops on primary OSD (3). I have tried to make it normal Hi Lazuardi, I assume this pg is in a EC 4+2 pool, so you can lo

[ceph-users] Re: [v15.2.11] radosgw / RGW crash at start, Segmentation Fault

2021-05-07 Thread Gilles Mocellin
Thank you for this prompt reply ! Do you know when is planned the next point release ? Le 2021-05-07 16:58, Casey Bodley a écrit : this is https://tracker.ceph.com/issues/50218, a radosgw build issue specific to ubuntu bionic that affected all of our releases. the build issue has been resolved,

[ceph-users] How to trim RGW sync errors

2021-05-07 Thread by morphin
Hello. I was have multisite RGW (14.2.16 nautilus) setup and some of the bucket couldn't finish bucket sync due to overfill buckets, There was different needs and the sync started purpose of migration. I made the secondary zone the master and removed the old master zone from zonegroup. Now I still

[ceph-users] Monitor gets removed from monmap when host down

2021-05-07 Thread contact
Hi everyone, I'm new to ceph, and I'm currently doing some tests with cephadm and a few virtual machines. I've deployed ceph with cephadm on 5 VMs, each one have a 10GB virtual disk attached, and everything is working perfectly. (So 5 osd and 5 monitor in my cluster) However when I turn down a

[ceph-users] Re: Natutilus - not unmapping

2021-05-07 Thread Joe Comeau
Hi Mattias thanks for the info All OSDs active and clean With help we found it is related to this we think: https://tracker.ceph.com/issues/48946 osd_committed is better now I'm not sure that it is fixed so are watching closely currently ceph report |grep "osdmap_.*_committed" report 331

[ceph-users] Building ceph clusters with 8TB SSD drives?

2021-05-07 Thread Matt Larson
Is anyone trying Ceph clusters containing larger (4-8TB) SSD drives? 8TB SSDs are described here ( https://www.anandtech.com/show/16136/qlc-8tb-ssd-review-samsung-870-qvo-sabrent-rocket-q ) and make use QLC NAND flash memory to reach the costs and capacity. Currently, the 8TB Samsung 870 SSD is $8

[ceph-users] Re: x-amz-request-id logging with beast + rgw (ceph 15.2.10/containerized)?

2021-05-07 Thread David Orman
Hi Yuval, We've managed to get an upgrade done with the 16.2.3 release in a testing cluster, and we've been able to implement some of the logging I need via this mechanism, but the logs are emitted only when debug_rgw is set to 20. I don't need to log any of that level of data (we used centralized

[ceph-users] Re: x-amz-request-id logging with beast + rgw (ceph 15.2.10/containerized)?

2021-05-07 Thread Matt Benjamin
Hi David, I think the solution is most likely the ops log. It is called for every op, and has the transaction id. Matt On Fri, May 7, 2021 at 4:58 PM David Orman wrote: > > Hi Yuval, > > We've managed to get an upgrade done with the 16.2.3 release in a > testing cluster, and we've been able to

[ceph-users] Re: Building ceph clusters with 8TB SSD drives?

2021-05-07 Thread Nico Schottelius
Matt Larson writes: > Is anyone trying Ceph clusters containing larger (4-8TB) SSD drives? > > 8TB SSDs are described here ( > https://www.anandtech.com/show/16136/qlc-8tb-ssd-review-samsung-870-qvo-sabrent-rocket-q > ) and make use QLC NAND flash memory to reach the costs and capacity. > Curre

[ceph-users] Re: Upgrade and lost osds Operation not permitted

2021-05-07 Thread Neha Ojha
Thanks for reporting this issue. It has been fixed by https://github.com/ceph/ceph/pull/40845 and will be released in the next pacific point release. Neha On Mon, Apr 19, 2021 at 8:19 AM Behzad Khoshbakhti wrote: > > thanks by commenting the ProtectClock directive, the issue is resolved. > Thank

[ceph-users] Re: x-amz-request-id logging with beast + rgw (ceph 15.2.10/containerized)?

2021-05-07 Thread David Orman
Has anyone figured out an elegant way to emit this from inside cephadm managed/containerized ceph, so it can be handled via the host's journald and processed/shipped? We had gone down that path before, but decided to hold off on the suggestion that the LUA-based scripting might be a better option.

[ceph-users] Re: x-amz-request-id logging with beast + rgw (ceph 15.2.10/containerized)?

2021-05-07 Thread Matt Benjamin
That hasn't been glued up, but likely will be. Something similar for k8s has been discussed. You could certainly use lua to do something custom. I suspect lua to customize the canned ops-log will become an option, too. Matt On Fri, May 7, 2021 at 5:51 PM David Orman wrote: > > Has anyone figu

[ceph-users] Host crash undetected by ceph health check

2021-05-07 Thread Frank Schilder
Dear cephers, today it seems I observed an impossible event for the first time: an OSD host crashed, but the ceph health monitoring did not recognise the crash. Not a single OSD was marked down and IO simply stopped, waiting for the crashed OSDs to respond. All that was reported was slow ops, s

[ceph-users] Re: Building ceph clusters with 8TB SSD drives?

2021-05-07 Thread Frank Schilder
We are using 4TB Kingston DC500M drives for 6+2 EC RBD data pools, 2 OSDs per disk. They deliver great iops, but I think they are TLC, so probably not comparable with QLC drives. I think QLC drives are OK for mostly cold/static data due to their performance drop when running full: https://www.h

[ceph-users] Re: v16.2.2 Pacific released

2021-05-07 Thread Norman.Kern
Hi David, The web page is missing: https://docs.ceph.com/en/latest/docs/master/install/get-packages/ \ SORRY/ \ / \This page does / ] not exist yet.[,'| ] [ / |

[ceph-users] Re: v16.2.2 Pacific released

2021-05-07 Thread kefu chai
On Sat, May 8, 2021 at 10:42 AM Norman.Kern wrote: > > Hi David, > > The web page is missing: > https://docs.ceph.com/en/latest/docs/master/install/get-packages/ probably we should just replace "http" with "https" in > > * For packages, see http://docs.ceph.com/docs/master/install/get-packages/